Distributed Tensor Machine (DTM) #

Language-agnostic intermediate representation (IR) and versioned communication protocol for distributed tensor programs. Unifies data-parallel, tensor/model-parallel, SPMD sharding, and task-graph + tensor-store models under one core.

Field	Value
Spec version	1.0.0
Status	Frozen (v1)
Normative language	RFC 2119
Reference implementation	Common Lisp (lisp/, ASDF system dtm)

Spec and deliverables #

Path	Description
docs/spec/distributed-tensor-machine.md	Canonical formal specification
docs/spec/distributed-tensor-machine.pdf	Built PDF
docs/design/gpu-jit-backend.md	Informative: JIT graphs to CUDA / ROCm
native/dtm_gpu/	GPU Path A: libdtm_gpu (CUDA) + libdtm_gpu_hip (HIP/ROCm)
site/	Static HTML viewer (local preview)

Convenience symlink: grok-design-doc-f5b7cff2.md → the canonical markdown spec.

Build the PDF #

Requires pandoc, XeLaTeX, and DejaVu fonts (see docs/spec/build-pdf.sh).

./docs/spec/build-pdf.sh

Preview the site #

cd site
npm run build   # optional rebuild from markdown
npm start       # http://localhost:3000

End-to-end demo #

Coherent mini-pipeline on the shipped reference (L0 → store/task → L3 checkpoint → mesh resize):

sbcl --script examples/e2e-demo/run-e2e.lisp

See examples/e2e-demo/README.md.

Common Lisp reference (PR2+) #

# End-to-end demo (L0 + store/task + checkpoint + resize)
sbcl --script examples/e2e-demo/run-e2e.lisp

# PR8 L0–L2 suite (gates claim matrix; non-zero on required failure)
sbcl --script tests/conformance/run-suite.lisp

# IR regression (parser, printer, typechecker, module_hash)
sbcl --script tests/conformance/ir/run-ir-tests.lisp

# Protocol + IR vectors (PR2b skeleton + PR8 CollectiveStart/CRC fail-inject)
sbcl --script tests/conformance/run-conformance.lisp

# L0-core multi-device in-process executor (matmul + AllReduce)
sbcl --script tests/conformance/l0/run-l0-tests.lisp

# One shared IR computation across two OS processes (sharded X + TCP AllReduce)
sbcl --script tests/conformance/multiproc/run-multiproc-tests.lisp

# L0-complete (shape/reduce/softmax/collectives) + multi-rank sim (no TCP)
sbcl --script tests/conformance/l0-complete/run-l0-complete-tests.lisp
sbcl --script tests/conformance/multirank/run-multirank-sim-tests.lisp

# L1 multiproc: Hello/JoinMesh membership + DTM-TLV AllReduce + P2P
sbcl --script tests/conformance/l1/run-pr5a-membership-tests.lisp
sbcl --script tests/conformance/l1/run-pr5b-collectives-tests.lisp

# L2: tensor store, task-graph, CAP_QUANT
sbcl --script tests/conformance/store/run-store-tests.lisp
sbcl --script tests/conformance/task-graph/run-task-graph-tests.lisp
sbcl --script tests/conformance/quant/run-quant-tests.lisp

# Frontend API (DDP/SPMD IR + View D) — regression; not a PR8 gate
sbcl --script tests/conformance/frontend/run-frontend-tests.lisp

# PR9a live mesh resize (§8.10) + PR9b L3 resilience
sbcl --script tests/conformance/mesh-resize/run-mesh-resize-tests.lisp
sbcl --script tests/conformance/l3/run-l3-tests.lisp

# GPU Path A L0 (optional claims; build native libs first — see native/dtm_gpu/README.md):
#   cmake -S native/dtm_gpu -B native/dtm_gpu/build -DCMAKE_BUILD_TYPE=Release
#   cmake --build native/dtm_gpu/build -j
export LD_LIBRARY_PATH="$PWD/native/dtm_gpu/build:/usr/local/cuda/lib64:${LD_LIBRARY_PATH:-}"
sbcl --script tests/conformance/gpu/run-cuda-l0-tests.lisp
# HIP/ROCm (separate process; stub fails loud without ROCm device):
sbcl --script tests/conformance/gpu/run-hip-l0-tests.lisp

See lisp/README.md and tests/conformance/matrix.md.

GPU Path A (CUDA Phase 1 + HIP/ROCm Phase 2) #

Piece	Role
native/dtm_gpu/	CMake → libdtm_gpu.so (CUDA) + libdtm_gpu_hip.so (HIP or fail-loud stub)
lisp/src/gpu/	SBCL alien FFI + cuda-collectives / hip-collectives
(make-l0-session mod :backend :cuda)	L0 with cuBLAS matmul + native AllReduce
(make-l0-session mod :backend :hip)	L0 with hipBLAS path (or error if ROCm/stub has no device)

Requesting :cuda or :hip fails loudly if that vendor library cannot load or initialize — no silent fall-through to host or the other vendor. GPU tests are optional and do not gate the CPU PR8 suite.

Implementation roadmap #

Informative PR plan is in the specification (PR1–PR9b). Summary:

1. PR1 — Spec documents (landed)
2. PR2 — IR parser, printer, typechecker (landed; feature/pr2-ir-cl)
3. PR2b — Conformance skeleton (landed; feature/pr2b-conformance-skeleton)
4. PR3 — L0-core executor (landed; feature/pr3-l0-core)
5. PR4 — L0-complete + multi-rank sim (landed; feature/pr4-l0-complete)
6. PR5a/PR5b — L1 membership + wire collectives/P2P (landed)
7. PR6a/b/c — Store, task-graph, CAP_QUANT (landed)
8. PR7 — Frontend API (landed; feature/pr7-frontend-api)
9. PR8 — L0–L2 conformance suite (landed; feature/pr8-conformance-suite)
10. PR9a — Live mesh resize §8.10 (landed; feature/pr9a-mesh-resize)
11. PR9b — L3 checkpoint/cancel/checksum/quota (landed; feature/pr9b-l3-resilience)
12. GPU Path A Phase 1 — CUDA L0 cuBLAS + device AllReduce (landed; feature/gpu-cuda-l0)
13. GPU Path A Phase 2 — HIP/ROCm L0 twin + fail-loud stub (landed; feature/gpu-rocm-l0)

Git tags #

Feature landing tags follow feature/<name>. Spec freeze tags follow semver (v1.0.0). List them with:

git tag -l 'feature/*' 'v*'
git show v1.0.0
git log --oneline --decorate