Welcome to Scaler’s documentation!

Scaler is a lightweight distributed computing Python framework that lets you easily distribute tasks across multiple machines or parallelize on a single machine.

Scaler is designed to be a drop-in replacement for Dask requiring minimal code changes. Scaler’s design focuses on performance, simplicity, reduced overhead, debuggable errors.

Key features include:

• Python’s multiprocessing module like API - e.g. client.map() and client.submit()

• Graph Tasks - submit DAG tasks with complex interdependence

• Monitoring Dashboard - monitor utilization of workers and task completion in real time

• Task Profiling - profile and trace execution of tasks

Content

• Quickstart
  • When to use it
  • Architecture
  • Installation
  • First Look (Code API)
  • Client.map
  • Client.submit
  • Things to Avoid
  • Spinning up Scheduler and Cluster Separately
• Additional Features
  • Scaler Top (Monitoring)
  • Task Profiling
  • Send Object
  • Graph Submission
  • Nested Tasks
  • Dynamically Building Graph Within a Task
  • Client Disconnect and Shutdown
  • Custom Serialization
• Configuration
  • Scheduler Settings
  • Worker Settings
• Examples
  • Basic Usage
  • Applications