`simpler.orchestrator`¶

Generated from the source. For the curated view — which **config keys Worker accepts, CallConfig defaults, and the argument-order footguns — see Python API.

simpler.orchestrator ¶

Orchestrator — DAG builder passed to a Worker submit/run callback.

A thin Python facade over the C++ Orchestrator. The Worker creates one Orchestrator handle at init, retrieves the C++ object via Worker.get_orchestrator(), and passes the handle to the user's orch function::

def my_orch(orch, args, cfg):
    # chip_handle/sub_handle come from Worker.register(...)
    # build the args object yourself as Tensors; tags drive dependency inference
    a = TaskArgs()
    a.add_tensor(input_handle.tensor(shape, dtype),  TensorArgType.INPUT)
    a.add_tensor(output_handle.tensor(shape, dtype), TensorArgType.OUTPUT)
    orch.submit_next_level(chip_handle, a, cfg, worker=0)  # handle from Worker.register(chip_callable)

    sub_args = TaskArgs()
    sub_args.add_tensor(output_handle.tensor(shape, dtype), TensorArgType.INPUT)
    orch.submit_sub(sub_handle, sub_args)

handle = w.submit(my_orch, my_args, my_config)
handle.wait()

Scope and submission-close lifecycle is managed by Worker.submit(); completion is managed by its RunHandle. Worker.run() remains the blocking submit(...).wait() compatibility entry point.

Orchestrator ¶

DAG builder. Valid only inside the orch function passed to Worker.run().

Wraps a borrowed reference to the C++ Orchestrator owned by the parent Worker. The Python Worker keeps a strong reference to the parent C++ Worker for the entire orch-fn execution, so the borrowed reference stays valid.

submit_next_level ¶

submit_next_level(callable_handle: Any, args: TaskArgs, config: CallConfig | None = None, *, worker: int) -> TaskHandle

Submit a NEXT_LEVEL task by registered callable handle.

callable_handle must be returned by Worker.register. Tags inside args drive deps. worker is the exact stable NEXT_LEVEL worker id that runs the task. For L3 chip dispatch, these are the existing chip worker ids. Returns an opaque TaskHandle accepted by TaskArgs.add_dep or TaskArgs.add_dep_wait during the same orchestration run.

submit_next_level_group ¶

submit_next_level_group(callable_handle: Any, args_list: list, config: CallConfig | None = None, *, workers: list) -> TaskHandle

Submit a group of NEXT_LEVEL tasks (N TaskArgs → N worker selections, 1 DAG node).

workers contains the exact stable NEXT_LEVEL worker id for each member. For L3 chip dispatch, these are the existing chip worker ids. When workers is the complete worker-id set of one MPI L3 group, args_list becomes one per-rank mailbox request: MPI rank workers[i] executes only args_list[i]. A single worker id remains directed and is never silently widened to the whole group. Returns one opaque TaskHandle for the group DAG node.

submit_sub ¶

submit_sub(callable_handle: Any, args: TaskArgs | None = None)

Submit a SUB task by registered callable handle.

args may be omitted for a tag-less task (no dependencies, no outputs).

submit_sub_group ¶

submit_sub_group(callable_handle: Any, args_list: list)

Submit a group of SUB tasks (N TaskArgs → N workers, 1 DAG node).

allocate_domain ¶

allocate_domain(*, name: str, workers: Sequence[int], window_size: int, buffers: Sequence[CommBufferSpec] = ()) -> CommDomainHandle

Collectively allocate a fresh CommDomain across workers.

Driven from the orch thread. Dispatches CTRL_ALLOC_DOMAIN to each participating chip in parallel and blocks until all have completed the IPC handshake (HCCL: aclrtMalloc + IPC import; sim: shm + ftruncate). Returns a CommDomainHandle whose contexts[chip_idx] exposes the per-chip ChipDomainContext (device_ctx, local_window_base, buffers by name — each a device VMM_WINDOW Buffer).

name is a local identifier (uniqueness checked against currently-live handles); peers do not need to agree on the string. workers must be a subset of the Worker's device_ids indices; their order defines dense domain ranks. buffers are carved sequentially inside the window in declaration order; their nbytes sum must fit within window_size — this is validated on the orch thread before any chip-side allocation is dispatched, so an oversized request raises ValueError here without leaking a backend allocation.

Use the handle as a context manager for auto-release:

with orch.allocate_domain(name="tp", workers=[0, 1], window_size=4096) as tp:
    for chip_idx in tp.workers:
        orch.submit_next_level(chip_handle, ..., worker=chip_idx)

release_domain ¶

release_domain(handle: CommDomainHandle) -> None

Collective release. Equivalent to handle.release().

allocate_global_domain ¶

allocate_global_domain(*, name: str, members: Sequence[tuple[int, int]], window_size: int, buffers: Sequence[CommBufferSpec] = (), retain_after_run: bool = False) -> GlobalCommDomainHandle

Create a CommDomain across local and/or remote L3 nodes without MPI.

Each member is (l3_worker_id, local_l2_worker_id). The L3 worker may have been registered by Worker.add_worker or Worker.add_remote_worker. L4 collects every L2 export descriptor, sends the complete rank-ordered table back to every L3, and commits only after all L2 imports succeed. retain_after_run=True keeps the domain live after the current DAG drains so a later run can inspect communication results; explicit release or Worker.close() still tears it down.

get_global_domain ¶

get_global_domain(domain_id: int) -> GlobalCommDomainView

Return the committed L3-local view for a domain created by L4.

create_worker_chip_region ¶

create_worker_chip_region(*, worker_id: int, payload_bytes: int, counter_bytes: int)

Create an L3-L2 communication region on one NEXT_LEVEL chip worker.

create_worker_chip_queue ¶

create_worker_chip_queue(*, worker_id: int, depth: int, input_arena_bytes: int, output_arena_bytes: int)

Create an L3-L2 message queue backed by one L3-L2 communication region.

scope_begin ¶

scope_begin() -> None

Open a nested scope explicitly.

Prefer the scope() context manager, which pairs the end for you. Every scope_begin() must be matched by a scope_end().

scope_end ¶

scope_end() -> None

Close the scope opened by the matching scope_begin().

scope ¶

scope() -> Iterator[Orchestrator]

Open a nested scope for the with block.

Tasks submitted inside the block use a deeper heap ring so they reclaim independently of the outer scope (see Strict-1 in .claude/plans/HIERARCHICAL_RUNTIME_REFACTOR.md).

committed_device_memory ¶

committed_device_memory(worker_id: int) -> int

Total device HBM (bytes) committed by next-level worker worker_id's MemoryAllocator.

A query, but one that travels the same chip mailbox as every other command, so it takes the same ordering: read behind a run that is still allocating and the number is a snapshot of neither side.

device_memory_info ¶

device_memory_info(worker_id: int) -> DeviceMemoryInfo

Device-wide ACL_HBM_MEM free/total byte snapshot for worker_id.

alloc_child_tensor ¶

alloc_child_tensor(worker_id: int, shapes: tuple[int, ...], dtype: DataType) -> Buffer

Allocate device memory on next-level worker_id sized for shapes × dtype; returns a DEVICE_MALLOC Buffer. Delegates to Worker.alloc_child_tensor.

free ¶

free(handle: Buffer) -> None

Free a device Buffer (from alloc_child_tensor). Delegates to Worker.free.

copy_to ¶

copy_to(dst: Buffer, src, *, dst_offset: int = 0, src_offset: int = 0, nbytes: int | None = None) -> None

H2D: copy nbytes from src_offset in host src to dst_offset in device dst. Delegates to Worker.copy_to.

copy_from ¶

copy_from(dst, src: Buffer, *, dst_offset: int = 0, src_offset: int = 0, nbytes: int | None = None) -> None

D2H: copy nbytes from src_offset in device src to dst_offset in host dst. Delegates to Worker.copy_from.

alloc ¶

alloc(shape: Sequence[int], dtype: DataType) -> Buffer

Allocate a runtime-managed intermediate buffer; returns a Buffer.

The backing is a MAP_SHARED slab (visible to forked child workers), auto-reclaimed once every downstream consumer has completed and the run's scope ends — no manual free. Name it in a task arg with handle.tensor(shape, dtype): its canonical identity dependency-wires to this alloc's synthetic producer slot (tag it OUTPUT/INOUT on the producer, INPUT on the consumer).

Use this for chip-A → chip-B intermediate buffers instead of pre-allocating with torch.share_memory_() — the runtime owns the lifecycle. Equivalent to worker.alloc_shared_tensor, additionally registered as an L3-L2 orch-comm host buffer so it may back an L3-L2 message-queue payload.

direct_control ¶

direct_control(worker: Any, native_orch: Any, api: str)

Order one command that reaches a child outside any TaskSlot.

malloc, copy_*, domain and region creation, and every remote_* buffer call travel the mailbox rather than the ready queue, so the whole-run FIFO does not sequence them. Two cases, and the reservation is held for the whole call in both — a check that only samples state leaves the command itself outside the decision it just made.

A call issued inside a graph callback belongs to that run and waits for it to hold the FIFO head. A call that belongs to no run is ordered only by being alone: it takes the same serializer submission uses, so no run can be admitted between the check and the command.

simpler.orchestrator¶