- OCaml 72%
- C 28%
nox-metal #
Metal GPU matrix multiplication in OCaml, with a CPU fallback everywhere else.
nox-metal binds Apple's
MPSMatrixMultiplication,
the Metal Performance Shaders GEMM. In neural-network inference the matrix
product is where the time goes. Matrices are row-major float32 Bigarrays.
On a machine with no Metal device (another OS, or a Mac without one), gemm
computes the same product on the CPU, so callers do not need two code paths.
# let a = Metal.of_array [| 1.; 2.; 3.; 4. |] (* 2x2 *)
val a : Metal.t = <abstr>
# let b = Metal.of_array [| 5.; 6.; 7.; 8. |] (* 2x2 *)
val b : Metal.t = <abstr>
# Metal.to_array (Metal.gemm ~m:2 ~k:2 ~n:2 a b)
- : float array = [|19.; 22.; 43.; 50.|]
Metal.available () is true when the GPU path is in use, and
Metal.backend () returns Metal or Cpu. METAL_DISABLE=1 forces Cpu.
Two modules sit on top. Metal.Linear scores a dense linear classifier. Its
weights go to the GPU once, in of_model, and logits_batch then scores a
list of feature vectors in one dispatch. The model format itself is in
nox-linear, which has no GPU dependency: a unikernel guest can build and
encode a model, and an agent on a machine with a GPU scores it.
Metal.Compute runs any kernel written in the Metal Shading Language. You
compile the source at run time with compile ~source ~entry and dispatch it
over shared buffers. It has no CPU fallback; with no Metal device, every call
raises Failure.
Installation #
Install with opam:
$ opam install nox-metal
If opam cannot find the package, it may not yet be released in the public
opam-repository. Add the overlay repository, then install it:
$ opam repo add samoht https://tangled.org/gazagnaire.org/opam-overlay.git
$ opam update
$ opam install nox-metal
Building #
The only C stub is lib/metal_stubs.c. On macOS dune builds it as
Objective-C with ARC and links Metal and MetalPerformanceShaders. On
other systems the stub just raises, and the OCaml code never reaches it.
Nothing needs configuring.
Related work #
- Apple's Metal Performance Shaders is the framework underneath. Only its GEMM and a plain kernel dispatch are bound here.
License #
ISC. See LICENSE.md.