`simpler.orchestrator`¶

Generated from the source. For the curated view — which **config keys Worker accepts, CallConfig defaults, and the argument-order footguns — see Python API.

simpler.orchestrator ¶

Orchestrator — DAG builder passed to a Worker submit/run callback.

A thin Python facade over the C++ Orchestrator. The Worker creates one Orchestrator handle at init, retrieves the C++ object via Worker.get_orchestrator(), and passes the handle to the user's orch function::

def my_orch(orch, args, cfg):
    # chip_handle/sub_handle come from Worker.register(...)
    # build the args object yourself as Tensors; tags drive dependency inference
    a = TaskArgs()
    a.add_tensor(input_handle.tensor(shape, dtype),  TensorArgType.INPUT)
    a.add_tensor(output_handle.tensor(shape, dtype), TensorArgType.OUTPUT)
    orch.submit_next_level(chip_handle, a, cfg, worker=0)  # handle from Worker.register(chip_callable)

    sub_args = TaskArgs()
    sub_args.add_tensor(output_handle.tensor(shape, dtype), TensorArgType.INPUT)
    orch.submit_sub(sub_handle, sub_args)

handle = w.submit(my_orch, my_args, my_config)
handle.wait()

Scope and submission-close lifecycle is managed by Worker.submit(); completion is managed by its RunHandle. Worker.run() remains the blocking submit(...).wait() compatibility entry point.

Orchestrator ¶

DAG builder. Valid only inside the orch function passed to Worker.run().

Wraps a borrowed reference to the C++ Orchestrator owned by the parent Worker. The Python Worker keeps a strong reference to the parent C++ Worker for the entire orch-fn execution, so the borrowed reference stays valid.

submit_next_level ¶

submit_next_level(callable_handle: Any, args: TaskArgs, config: CallConfig | None = None, *, worker: int) -> TaskHandle

Submit a NEXT_LEVEL task by registered callable handle.

callable_handle must be returned by Worker.register. Tags inside args drive deps. worker is the exact stable NEXT_LEVEL worker id that runs the task. For L3 chip dispatch, these are the existing chip worker ids. Returns an opaque TaskHandle accepted by TaskArgs.add_dep or TaskArgs.add_dep_wait during the same orchestration run.

submit_next_level_group ¶

submit_next_level_group(callable_handle: Any, args_list: list, config: CallConfig | None = None, *, workers: list) -> TaskHandle

Submit a group of NEXT_LEVEL tasks (N TaskArgs → N worker selections, 1 DAG node).

workers contains the exact stable NEXT_LEVEL worker id for each member. For L3 chip dispatch, these are the existing chip worker ids. When workers is the complete worker-id set of one MPI L3 group, args_list becomes one per-rank mailbox request: MPI rank workers[i] executes only args_list[i]. A single worker id remains directed and is never silently widened to the whole group. Returns one opaque TaskHandle for the group DAG node.

submit_sub ¶

submit_sub(callable_handle: Any, args: TaskArgs | None = None)

Submit a SUB task by registered callable handle.

args may be omitted for a tag-less task (no dependencies, no outputs).

submit_sub_group ¶

submit_sub_group(callable_handle: Any, args_list: list)

Submit a group of SUB tasks (N TaskArgs → N workers, 1 DAG node).

allocate_domain ¶

allocate_domain(*, name: str, members: Sequence | None = None, window_size: int, buffers: Sequence[CommBufferSpec] = (), retain_after_run: bool = False, workers: Sequence[int] | None = None) -> CommDomainHandle

Create a domain using L3 chip ids or L4 (node, chip) members.

Creation is synchronous; L3 control must precede the first task submission. Buffer nbytes must be caller-padded to a multiple of 64 bytes, and window_size must be a multiple of 64 covering their sum. No implicit padding is inserted. workers is the legacy L3 spelling; it cannot be combined with members. Retained domains survive successful runs until explicit release or Worker.close().

get_or_allocate_domain ¶

get_or_allocate_domain(*, name: str, members: Sequence, window_size: int, buffers: Sequence[CommBufferSpec] = (), retain_after_run: bool = False) -> CommDomainHandle

Return the same live handle on an exact configuration match, otherwise create it.

Member order, full buffer definitions, window request, retention and topology must match. A hit neither allocates nor initializes the existing window.

get_domain ¶

get_domain(name: str | CommDomainRef) -> CommDomainHandle

Look up an owned domain by name or ref, or borrow an already-granted domain by ref.

release_domain ¶

release_domain(handle: CommDomainHandle) -> None

Release a domain through its creating Worker's completion fence.

allocate_global_domain ¶

allocate_global_domain(*, name: str, members: Sequence[tuple[int, int]], window_size: int, buffers: Sequence[CommBufferSpec] = (), retain_after_run: bool = False) -> CommDomainHandle

Compatibility spelling of allocate_domain(members=...) for L4.

get_global_domain ¶

get_global_domain(domain_id: int) -> GlobalCommDomainView

Return the committed L3-local view for a domain created by L4.

copy_to_domain ¶

copy_to_domain(handle: CommDomainHandle, domain_rank: int, data: bytes, *, buffer: str | None = None, offset: int = 0) -> None

Synchronously stage bytes; does not wait for queued tasks.

L3 staging precedes task submission and stays within declared buffer slices. L4 keeps its existing window copy path.

copy_from_domain ¶

copy_from_domain(handle: CommDomainHandle, domain_rank: int, nbytes: int, *, buffer: str | None = None, offset: int = 0) -> bytes

Synchronously read bytes with the same range and ordering rules as copy_to_domain.

create_worker_chip_region ¶

create_worker_chip_region(*, worker_id: int, payload_bytes: int, counter_bytes: int)

Create an L3-L2 communication region on one NEXT_LEVEL chip worker.

create_worker_chip_queue ¶

create_worker_chip_queue(*, worker_id: object, depth: object, input_arena_bytes: object, output_arena_bytes: object)

Create an L3-L2 message queue on one NEXT_LEVEL chip worker.

scope_begin ¶

scope_begin() -> None

Open a nested scope explicitly.

Prefer the scope() context manager, which pairs the end for you. Every scope_begin() must be matched by a scope_end().

scope_end ¶

scope_end() -> None

Close the scope opened by the matching scope_begin().

scope ¶

scope() -> Iterator[Orchestrator]

Open a nested scope for the with block.

Tasks submitted inside the block use a deeper heap ring so they reclaim independently of the outer scope (see Strict-1 in .claude/plans/HIERARCHICAL_RUNTIME_REFACTOR.md).

committed_device_memory ¶

committed_device_memory(worker_id: int) -> int

Total device HBM (bytes) committed by next-level worker worker_id's MemoryAllocator.

A query, but one that travels the same chip mailbox as every other command, so it takes the same ordering: read behind a run that is still allocating and the number is a snapshot of neither side.

flush_diagnostics ¶

flush_diagnostics(worker_id: int, timeout_s: float) -> tuple[int, int, int, int]

Publish every diagnostic run next-level worker worker_id has closed.

The barrier behind Worker.flush_diagnostics() and Worker.close(): with cross-run collection on, a run returning no longer means its diagnostic file exists, and this is what says the files up to now do. Raises when a promised file is missing.

timeout_s follows the native protocol unchanged — a negative value means no deadline, and a zero budget on a caller that had one is refused rather than turned into an unbounded wait (LocalMailboxEndpoint::control_dfx_flush). Returns that call's (session_id, watermark_epoch, published, failed).

Travels the same chip control channel as every other command here, so it takes the same admission.

device_memory_info ¶

device_memory_info(worker_id: int) -> DeviceMemoryInfo

Device-wide ACL_HBM_MEM free/total byte snapshot for worker_id.

alloc_child_tensor ¶

alloc_child_tensor(worker_id: int, shapes: tuple[int, ...], dtype: DataType) -> Buffer

Allocate device memory on next-level worker_id sized for shapes × dtype; returns a DEVICE_MALLOC Buffer. Delegates to Worker.alloc_child_tensor.

free ¶

free(handle: Buffer) -> None

Free a device Buffer (from alloc_child_tensor). Delegates to Worker.free.

copy_to ¶

copy_to(dst: Buffer, src, *, dst_offset: int = 0, src_offset: int = 0, nbytes: int | None = None) -> None

H2D: copy nbytes from src_offset in host src to dst_offset in device dst. Delegates to Worker.copy_to.

copy_from ¶

copy_from(dst, src: Buffer, *, dst_offset: int = 0, src_offset: int = 0, nbytes: int | None = None) -> None

D2H: copy nbytes from src_offset in device src to dst_offset in host dst. Delegates to Worker.copy_from.

alloc ¶

alloc(shape: Sequence[int], dtype: DataType) -> Buffer

Allocate a runtime-managed intermediate buffer; returns a Buffer.

The backing is a MAP_SHARED slab (visible to forked child workers), auto-reclaimed once every downstream consumer has completed and the run's scope ends — no manual free. Name it in a task arg with handle.tensor(shape, dtype): its canonical identity dependency-wires to this alloc's synthetic producer slot (tag it OUTPUT/INOUT on the producer, INPUT on the consumer).

Use this for chip-A → chip-B intermediate buffers instead of pre-allocating with torch.share_memory_() — the runtime owns the lifecycle. Equivalent to worker.alloc_shared_tensor, additionally registered as an L3-L2 orch-comm host buffer so it may back an L3-L2 message-queue payload.

direct_control ¶

direct_control(worker: Any, native_orch: Any, api: str)

Order one command that reaches a child outside any TaskSlot.

malloc, copy_*, domain and region creation, and every remote_* buffer call travel the mailbox rather than the ready queue, so the whole-run FIFO does not sequence them. Two cases, and the reservation is held for the whole call in both — a check that only samples state leaves the command itself outside the decision it just made.

A call issued inside a graph callback belongs to that run and waits for it to hold the FIFO head. A call that belongs to no run is ordered only by being alone: it takes the same serializer submission uses, so no run can be admitted between the check and the command.

simpler.orchestrator¶