Troubleshooting
Every troubleshooting reference lives in this directory. Start with the device
error-code reference when a run fails, or jump to the environment-specific
guide when the failure happens during build, test setup, or simulation.
Runtime and device failures
Document
What it covers
Device Error Codes Runtime and CANN error classification, first-response triage, minimal reproductions, and links to focused diagnostic guides
Local Runtime Timeouts Local and CI timeout defaults, override variables, and the required ordering for onboard watchdogs
A5 HBG SSBUF Local State Completion-generation mismatch, local-state layout/call-boundary experiments, and the validated workaround
A2/A3 Worker Retirement Why EXITED is not permission to return, and the group-wide CLOSE / GM-release handoff
A2/A3 Scheduling Tails Periodic real-time CPU throttling, normal AICPU thread scheduling, and worker-policy lifetime
a2a3 AICPU Shared-SO Device Fault Diagnosing mass 507899/507018 cascades caused by an AICPU shared-library device fault
Sim CPU Oversubscription Simulation hangs and false timeouts when AICPU and AICore host threads are CPU-starved
Build and test environments
Document
What it covers
DFX and Profiling Timing, traces, dependencies, and resource-usage evidence for runs that complete or make partial progress
Testing Test types, commands, platform selection, and failure-reproduction workflows
Debug a Failed Run User-facing first-response path from the surfaced error to the focused guides in this directory