simpler.task_interface¶
Generated from the source. For the curated view — which **config keys
Worker accepts, CallConfig defaults, and the argument-order footguns — see
Python API.
simpler.task_interface ¶
Public Python API for task_interface nanobind bindings.
Re-exports the canonical C++ types (DataType, ChipTensor, ChipStorageTaskArgs, TaskArgs,
TaskHandle, TensorArgType) plus scalar_to_uint64, and re-exports the address-free Tensor —
the task argument users build — from simpler.buffer. Torch-aware helpers
(make_chip_tensor_arg, torch_dtype_to_datatype) live in simpler_setup.torch_interop —
this module has no torch dependency.
ChipTensor is the chip-only POD the runtime ABI expects, paired with
ChipStorageTaskArgs internally and at the legacy ChipWorker boundary;
it carries a materialized address and never crosses a process boundary.
Usage
from simpler.task_interface import DataType, TaskArgs, Tensor, TensorArgType from simpler_setup.torch_interop import make_chip_tensor_arg
CommDomainRef
dataclass
¶
Creator identity and generation of an already established domain.
RemoteAddressSpace ¶
Bases: IntEnum
How a remote buffer's bytes are reached.
HOST_INLINE carries the payload in the message itself rather than
naming remote memory. REMOTE_WINDOW and UB_LDST are protocol
placeholders: the shipped host_tcp transport uses host-side session buffers.
RemoteBufferHandle ¶
A reference to memory on a remote L3 worker.
Returned by Worker.remote_malloc (an owner handle) or by
Worker.remote_import (an imported handle, told apart by
is_imported). The two are not interchangeable: owner handles are freed
with remote_free, imported ones with remote_release_import.
RemoteTensorRef.host_inline produces a third form, with
address_space of HOST_INLINE: it carries its bytes in the message
and names no remote allocation, so neither release call applies —
remote_free rejects it outright and it is never is_imported.
Construct only through Worker or RemoteTensorRef.host_inline; the
constructor is token-guarded.
worker_id
property
¶
Worker holding this reference — the importer, for an imported handle.
address_space
property
¶
How these bytes are reached; see RemoteAddressSpace.
access_flags
property
¶
Permitted access as a read/write bitmask; an export may only narrow it.
is_imported
property
¶
Whether this came from remote_import rather than remote_malloc.
RemoteBufferExport ¶
Opaque descriptor returned by Worker.remote_export.
The transport fields are intentionally kept private so callers cannot forge or log remote keys by accidentally treating the export as a plain dataclass.
access_flags
property
¶
Access granted here; a subset of the owner handle's flags.
RemoteTensorRef
dataclass
¶
A tensor argument that lives on, or travels to, a remote worker.
host_inline
classmethod
¶
Build a reference whose payload travels inline, naming no remote memory.
payload length must equal the byte size implied by shape and
dtype, and shape entries must be non-negative.
CommBufferSpec
dataclass
¶
A named slice of the per-rank communicator window.
Buffers are placed sequentially inside the window in declaration order,
without implicit padding. nbytes is the caller-padded physical size
and must be a positive multiple of 64 bytes. count may describe a
smaller logical extent. window_size must cover all physical sizes
and must also be a multiple of 64 bytes.
The CommDomainHandle.contexts[chip_idx].buffers dict returned by
Orchestrator.allocate_domain is keyed by CommBufferSpec.name.
ChipDomainContext
dataclass
¶
Per-domain view handed to a chip worker: its rank within the domain and the local slice of the symmetric window.
CommDomainHandle ¶
User-facing handle for one dynamically-allocated CommDomain.
Returned by the L3/L4 Orchestrator.allocate_domain(...) API. Acts as a context
manager: with exit marks the handle for release and prevents
further use; the actual backend free runs after Worker.run has
drained any tasks the orch function submitted using this domain. This
is required because submit_* only enqueues to the DAG — freeing
before drain would create a use-after-free on the chip side.
Lifecycle states::
live — allocated, indexable, can be passed to submit_*
released — release() called; further indexing raises;
backend memory still alive until Worker.run drain
freed — physical reclamation has been confirmed
Most users only see released; the live → released transition
happens at with exit (or explicit release()), and the
released → freed transition is the runtime's job at end-of-run.
Callers must treat handle attributes and context/buffer mappings as read-only
metadata. Reassignment is unsupported; configure through allocation arguments
and end ownership through release(). Buffer contents remain writable.
Tensor submissions register domain use in the current run, including cached
Tensors. Scalar-only use requires explicit lookup. Failed or unconfirmed
reclamation remains tracked and reported with freed=False. Retained
handles may be inspected and released serially after run completion.
ref
property
¶
Return identity; scalar-only use still requires owner lookup in each run.
released
property
¶
True once release() (or with exit) has been called.
Backend memory may still be alive — it is freed by the Worker after
DAG drain at end-of-run. Use this to gate further indexing /
submission, not to assert physical teardown (use freed for that).
freed
property
¶
True once the backend comm_release_domain_windows has executed.
Flips after the owning run's fence and physical release. Releasing a
retained handle after run completion frees it synchronously. An orch_fn
will never observe True for a handle it released within the same run.
member ¶
Return rank-ordered topology and ordinary Buffer slices.
local_context ¶
Return this creator's local chip mapping; L4 creators hold no such mappings.
release ¶
Mark this handle for collective release. Idempotent.
Inside an orch function, this is a non-blocking mark — the actual
backend comm_release_domain_windows runs after
the owning run's completion wait so that tasks already submitted with
this domain's device_ctx see live memory through execution.
Serial release after run completion frees a retained domain immediately.
After this returns, the handle is treated as released for the
user's purposes: __getitem__ raises, repeated release() is
a no-op, and the orch function must not pass it to further
submit_* calls.
GlobalCommDomainView ¶
L3-local imported view and its receiving-node attachment row.
ChipWorker ¶
Unified execution interface wrapping the host runtime C API.
The runtime library and target device are bound once via init() and
cannot be changed.
Public dispatch uses opaque CallableHandle values. Integer execution
slots are private to this wrapper and the runtime ABI.
Usage::
worker = ChipWorker()
worker.init(device_id=0, bins=bins)
handle = worker.register_callable(chip_callable)
worker.run(handle, args=orch_args, config=CallConfig())
worker.unregister_callable(handle)
worker.finalize()
aicpu_dlopen_count
property
¶
Number of distinct callable identities the AICPU has dlopened for.
host_dlopen_count
property
¶
Number of host-side orch SO dlopens (host_build_graph variants).
run_stream_set_create_count
property
¶
Number of AICore run streams the bound runner has created.
launch_depth
property
¶
How many runs this worker may have launched at once, after its request was resolved.
automatic_admission
property
¶
Whether this worker's context sizes its own run concurrency.
admissible_run_resource_sets
property
¶
Run-resource sets this context reports it can hold right now.
A count of identities, never a byte figure: it is one until a run has completed its resource commit, grows with what this context has proved it can hold, and falls when those resources are reclaimed or become unprovable. The granted count on a context that does not size itself.
supports_joined_native_launch
property
¶
Whether one run's native submission may be ordered behind another's right now.
runtime_buffer_addrs
property
¶
Address of each opaque host native-run storage buffer, in slot order.
committed_device_memory
property
¶
Total device HBM (bytes) committed by this chip worker's MemoryAllocator.
init ¶
init(device_id: int, bins: Any, log_level: int | None = None, prewarm_config: CallConfig | None = None, enable_sdma: bool = False, collect_across_runs: bool | None = None, dfx_session: bool | None = None, workspace_budget_bytes: int = 0, manage_workspace: bool = False, requested_pipeline_depth: int = 0)
Attach the calling thread to device_id, load the host runtime
library, and cache platform binaries.
Can only be called once — the runtime and device cannot be changed after init.
Seeds the extension-owned HostLogger state before C++ loads any consumer. Each consumer contains its own logger implementation and receives that state pointer during module init. On sim, C++ retains libcpu_sim_context.so in a process-wide RTLD_GLOBAL registry so host_runtime.so can resolve the PTO simulator hooks.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
device_id
|
int
|
NPU device ID to attach the calling thread to. |
required |
bins
|
Any
|
A |
required |
log_level
|
int | None
|
Threshold (10=DEBUG, 20=INFO, 25=TIMING, 30=WARN,
40=ERROR, 60=NUL). Defaults to a snapshot of the simpler
logger via |
None
|
workspace_budget_bytes
|
int
|
Finite byte budget for this context's workspace regions — the per-slot retained temporary buffer and the three pooled arena regions. 0 (the default) enforces no budget; it does not decide whether those regions have an owner, which the onboard program init entry gives them for every caller. A non-zero value on a module without workspace support raises rather than running unmanaged. Partial accounting: external tensors, run-result and diagnostics regions, code and device ELF, RTS and provider memory are outside it, so it is not a device-wide ceiling. |
0
|
manage_workspace
|
bool
|
Accepted and ignored. Those same four regions are put under one owner by the onboard program init entry, for every caller, so nothing has to request it: a superseded generation is released once its last consumer retires rather than when its replacement is published, a failed growth leaves the previous plan installed, and a failed release is recorded instead of discarded. Kept so existing callers keep working. A simulated backend manages no device workspace and is unchanged. |
False
|
requested_pipeline_depth
|
int
|
How many native run-resource sets this
context may hold at once, or 0 for the standing default of
two — which is what every runtime granted before this
argument existed, so a caller who asks for nothing keeps that
capacity and that footprint however high the runtime's
published maximum goes. The granted count is the smaller of
what is asked and the runtime's supported maximum, and it is
latched here — before any per-slot storage is built or
prewarmed — because nothing can resize those pools
afterwards. |
0
|
collect_across_runs
|
bool | None
|
Let a run's records outlive its own boundary,
so the sealing and the file write happen while the next run
executes instead of at the boundary. Off when neither this nor
Every collection level is served, orchestrator phases (level 4) included: each run's phases go to its own background file. The level a collector serves is the one its device pools were built for — the first collected run's, or the first run after a shape change rebuilt them — so a device-orchestrated run that would need an orchestrator pool an earlier, lower level did not build is refused before anything is allocated, leaving every earlier run's records intact. Profile at the level you want from the first collected run.
What a collector promises is its own rule rather than a file per
run: a PMU run proved to have produced no records writes no
With args dump enabled this also moves that run's argument
content: its payload file and its manifest are finished in
the background, so the payload file may still be growing when a
reader opens it, and the manifest is what publication produces.
Each run owns an exclusive |
None
|
dfx_session
|
bool | None
|
The name this option shipped under, accepted as an alias. Giving both spellings different values is an error. |
None
|
For tests that need to drive the binding directly with arbitrary path
strings (e.g. to assert dlopen failure on /nonexistent/foo.so), call
_ChipWorker.init(...) from _task_interface instead of going
through this wrapper.
flush_diagnostics ¶
Publish every diagnostic run this chip has closed.
A no-op unless init(collect_across_runs=True) let runs be retained.
Raises RuntimeError when a run that was promised a file did not get
one; a published partial carries its verdict in the file and is not a
failure.
finalize ¶
Tear down everything: device resources and runtime library.
Terminal operation — the object cannot be reused after this.
register_callable ¶
Prepare a ChipCallable and return an opaque handle.
The runtime still uses an integer slot internally, but the caller never chooses or observes it.
run ¶
run(handle: CallableHandle, args: TaskArgs | ChipStorageTaskArgs, config: CallConfig | None = None, **kwargs: Any)
Launch a callable previously returned by register_callable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
handle
|
CallableHandle
|
|
required |
args
|
TaskArgs | ChipStorageTaskArgs
|
TaskArgs with scalar values, HOST/NONE controls (HBG), and contiguous HOST/H2D tensors. DEVICE descriptors require Worker(level=2).submit, which validates their live source registration. ChipStorageTaskArgs is accepted for compatibility with callers supplying resolved addresses. |
required |
config
|
CallConfig | None
|
Optional CallConfig. If None, a default is created. |
None
|
**kwargs
|
Any
|
Overrides applied to config (e.g.
|
{}
|
Returns None. Per-stage run timing is emitted as [STRACE] log
markers by the platform — see docs/dfx/host-trace.md.
Keep tensor backing storage alive through return. TaskArgs imports are released after native success; a native-call error retains them and refuses further public runs until finalize() succeeds. This retains mappings, not ownership of the caller's allocation.
unregister_callable ¶
Drop one live callable handle and release its private resources when final.
enable_automatic_admission ¶
Let this worker's device context size its own run concurrency.
Asked once, after init, by a caller that specified no depth and no byte budget. Returns False when the loaded module or the context cannot — a simulated backend, a borrowed context, or a module without the entry — and the worker then admits exactly what it was granted, as it did before.
configure_launch_depth ¶
Ask, before init, for a launch depth.
Depth 1 is the serial path: nothing is ordered behind anything, and no run constructs the boundary that would let it be.
Rejected rather than coerced: int(1.9) is 1 and int(True) is 1, so coercion would
silently pick a device-work capacity the caller did not ask for. bool is an int
subclass, hence the exact type test.
set_exported_device_regions_live ¶
Declare whether this worker's host side still owns exported device regions.
While it does, no run is ordered behind another: those regions are released before the child's device reset.
arena_bank_gm_heap_base ¶
Committed GM heap base of one arena bank, or 0 when uncommitted.
retained_temp_addr ¶
Retained temporary-buffer address for one slot, or 0 when unheld.
comm_init ¶
Initialize a distributed communicator for this rank.
ChipWorker owns ACL bring-up and the aclrtStream internally, so
callers never touch aclInit / aclrtSetDevice / stream
lifetimes. On sim, ACL / stream are not used. Pair with
comm_destroy for teardown.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rank
|
int
|
This process's rank (0-based). |
required |
nranks
|
int
|
Total number of ranks. |
required |
rootinfo_path
|
str
|
Filesystem path used for rank handshake. |
required |
Returns:
| Type | Description |
|---|---|
int
|
Opaque communicator handle (uint64) for the other |
comm_alloc_windows ¶
Allocate per-rank windows. Returns a device CommContext pointer (uint64).
comm_get_local_window_base ¶
Return this rank's local window base address (uint64).
comm_get_window_size ¶
Return the actual per-rank window size in bytes.
comm_derive_context ¶
comm_derive_context(comm_handle: int, rank_ids: list[int], domain_rank: int, window_offset: int, window_size: int) -> int
Derive a domain-local device CommContext from an allocated base communicator.
comm_destroy ¶
Destroy the communicator and release its resources.
device_memory_info ¶
Device-wide ACL_HBM_MEM free/total byte snapshot.
scalar_to_uint64 ¶
Convert a scalar value to uint64.
value can be a Python int, float, bool, a numpy integer scalar, an
IntEnum member, or a ctypes scalar.
The accepted ctypes types are the integer widths (c_int8 ..
c_uint64), c_float, c_double and c_bool, and any subclass
of one of them. A pointer or character type (c_void_p, c_char_p,
c_char, c_wchar) is refused: its buffer holds an address or a
character rather than a number a slot can carry.
A byte-order-qualified variant (c_uint32.__ctype_be__ on a
little-endian host, and its __ctype_le__ counterpart on a big-endian
one) is also refused. Its bytes are stored in the opposite order, so
copying them would encode a different number than the same value written
from orchestration, and to_u64 has no reversed-order form to agree
with.
The encoding matches C++ to_u64() bit for bit, so a slot written from
Python and one written from orchestration read back identically.
A ctypes scalar is read at its own width and zero-extended, never
sign-extended: c_int8(-1) is 0xFF, matching to_u64(int8_t{-1}).
Reading it back with the matching width (scalar<int8_t>) still yields
-1.
Python float values (and numpy.float64, which is itself a float
subclass) are converted to IEEE 754 single precision (32-bit) and their
bit pattern is zero-extended to uint64. This may cause a loss of
precision. A finite value outside single-precision range (1e100)
raises rather than being stored as an infinity; inf and nan pass
through as themselves. For double precision, use ctypes.c_double -- a
bare Python float carries no width, so this is the one encoding that
cannot align with its C++ counterpart, where to_u64(1.5) is a double.
A narrower numpy float (e.g. numpy.float32) is rejected rather than
silently coerced through int() -- wrap it in float(...) first if
that narrowing is what you want.
An integer outside the 64-bit two's-complement range raises rather than truncating to its low 64 bits.