`simpler.orchestrator`¶

Generated from the source. For the curated view — which **config keys Worker accepts, CallConfig defaults, and the argument-order footguns — see Python API.

simpler.orchestrator ¶

Orchestrator — DAG builder passed to a Worker submit/run callback.

A thin Python facade over the C++ Orchestrator. The Worker creates one Orchestrator handle at init, retrieves the C++ object via Worker.get_orchestrator(), and passes the handle to the user's orch function::

def my_orch(orch, args, cfg):
    # chip_handle/sub_handle come from Worker.register(...)
    # build the args object yourself; tags drive dependency inference
    a = TaskArgs()
    a.add_tensor(make_tensor_arg(input_tensor),  TensorArgType.INPUT)
    a.add_tensor(make_tensor_arg(output_tensor), TensorArgType.OUTPUT)
    orch.submit_next_level(chip_handle, a, cfg, worker=0)

    sub_args = TaskArgs()
    sub_args.add_tensor(make_tensor_arg(output_tensor), TensorArgType.INPUT)
    orch.submit_sub(sub_handle, sub_args)

handle = w.submit(my_orch, my_args, my_config)
handle.wait()

Scope and submission-close lifecycle is managed by Worker.submit(); completion is managed by its RunHandle. Worker.run() remains the blocking submit(...).wait() compatibility entry point.

Orchestrator ¶

DAG builder. Valid only inside the orch function passed to Worker.run().

Wraps a borrowed reference to the C++ Orchestrator owned by the parent Worker. The Python Worker keeps a strong reference to the parent C++ Worker for the entire orch-fn execution, so the borrowed reference stays valid.

submit_next_level ¶

submit_next_level(callable_handle: Any, args: TaskArgs, config: CallConfig | None = None, *, worker: int)

Submit a NEXT_LEVEL task by registered callable handle.

callable_handle must be returned by Worker.register. Tags inside args drive deps. worker is the exact stable NEXT_LEVEL worker id that runs the task. For L3 chip dispatch, these are the existing chip worker ids.

submit_next_level_group ¶

submit_next_level_group(callable_handle: Any, args_list: list, config: CallConfig | None = None, *, workers: list)

Submit a group of NEXT_LEVEL tasks (N TaskArgs → N worker selections, 1 DAG node).

workers contains the exact stable NEXT_LEVEL worker id for each member. For L3 chip dispatch, these are the existing chip worker ids.

submit_sub ¶

submit_sub(callable_handle: Any, args: TaskArgs | None = None)

Submit a SUB task by registered callable handle.

args may be omitted for a tag-less task (no dependencies, no outputs).

submit_sub_group ¶

submit_sub_group(callable_handle: Any, args_list: list)

Submit a group of SUB tasks (N TaskArgs → N workers, 1 DAG node).

allocate_domain ¶

allocate_domain(*, name: str, workers: Sequence[int], window_size: int, buffers: Sequence[CommBufferSpec] = ()) -> CommDomainHandle

Collectively allocate a fresh CommDomain across workers.

Driven from the orch thread. Dispatches CTRL_ALLOC_DOMAIN to each participating chip in parallel and blocks until all have completed the IPC handshake (HCCL: aclrtMalloc + IPC import; sim: shm + ftruncate). Returns a CommDomainHandle whose contexts[chip_idx] exposes the per-chip ChipDomainContext (device_ctx, local_window_base, buffer_ptrs by name).

name is a local identifier (uniqueness checked against currently-live handles); peers do not need to agree on the string. workers must be a subset of the Worker's device_ids indices; their order defines dense domain ranks. buffers are carved sequentially inside the window in declaration order; their nbytes sum must fit within window_size — this is validated on the orch thread before any chip-side allocation is dispatched, so an oversized request raises ValueError here without leaking a backend allocation.

Use the handle as a context manager for auto-release:

with orch.allocate_domain(name="tp", workers=[0, 1], window_size=4096) as tp:
    for chip_idx in tp.workers:
        orch.submit_next_level(chip_handle, ..., worker=chip_idx)

release_domain ¶

release_domain(handle: CommDomainHandle) -> None

Collective release. Equivalent to handle.release().

create_l3_l2_region ¶

create_l3_l2_region(*, worker_id: int, payload_bytes: int, counter_bytes: int)

Create an L3-L2 communication region on one NEXT_LEVEL chip worker.

create_l3_l2_queue ¶

create_l3_l2_queue(*, worker_id: int, depth: int, input_arena_bytes: int, output_arena_bytes: int)

Create an L3-L2 message queue backed by one L3-L2 communication region.

scope_begin ¶

scope_begin() -> None

Open a nested scope explicitly.

Prefer the scope() context manager, which pairs the end for you. Every scope_begin() must be matched by a scope_end().

scope_end ¶

scope_end() -> None

Close the scope opened by the matching scope_begin().

scope ¶

scope() -> Iterator[Orchestrator]

Open a nested scope for the with block.

Tasks submitted inside the block use a deeper heap ring so they reclaim independently of the outer scope (see Strict-1 in .claude/plans/HIERARCHICAL_RUNTIME_REFACTOR.md).

malloc ¶

malloc(worker_id: int, size: int) -> int

Allocate memory on next-level worker worker_id. Returns a pointer.

This is the single L3 choke for kind4 device memory: Worker.malloc also funnels through here, as does a user's direct orch.malloc. The returned pointer's (worker_id, ptr) provenance is recorded so a later free / copy / kind4 dispatch to the wrong worker is rejected.

free ¶

free(worker_id: int, ptr: int) -> None

Free memory on next-level worker worker_id.

copy_to ¶

copy_to(worker_id: int, dst: int, src: int, size: int) -> None

Copy size bytes from host src to worker dst.

copy_from ¶

copy_from(worker_id: int, dst: int, src: int, size: int) -> None

Copy size bytes from worker src to host dst.

alloc ¶

alloc(shape: Sequence[int], dtype: DataType) -> Tensor

Allocate a runtime-managed intermediate buffer.

Returns a Tensor whose backing memory comes from a per-allocation MAP_SHARED mmap (visible to forked child workers). Lifetime is bound to a synthetic task slot that the Orchestrator treats as the buffer's producer; the buffer is freed when all downstream consumers have completed and the run's scope ends.

Use this for chip-A → chip-B intermediate buffers instead of pre-allocating with torch.share_memory_() — the runtime owns the lifecycle.

simpler.orchestrator¶