Metal GPU matrix multiplication with a CPU fallback
README.md

nox-metal #

Metal GPU matrix multiplication in OCaml, with a CPU fallback everywhere else.

nox-metal binds Apple's MPSMatrixMultiplication, the Metal Performance Shaders GEMM. In neural-network inference the matrix product is where the time goes. Matrices are row-major float32 Bigarrays. On a machine with no Metal device (another OS, or a Mac without one), gemm computes the same product on the CPU, so callers do not need two code paths.

# let a = Metal.of_array [| 1.; 2.; 3.; 4. |]   (* 2x2 *)
val a : Metal.t = <abstr>
# let b = Metal.of_array [| 5.; 6.; 7.; 8. |]   (* 2x2 *)
val b : Metal.t = <abstr>
# Metal.to_array (Metal.gemm ~m:2 ~k:2 ~n:2 a b)
- : float array = [|19.; 22.; 43.; 50.|]

Metal.available () is true when the GPU path is in use, and Metal.backend () returns Metal or Cpu. METAL_DISABLE=1 forces Cpu.

Two modules sit on top. Metal.Linear scores a dense linear classifier. Its weights go to the GPU once, in of_model, and logits_batch then scores a list of feature vectors in one dispatch. The model format itself is in nox-linear, which has no GPU dependency: a unikernel guest can build and encode a model, and an agent on a machine with a GPU scores it.

Metal.Compute runs any kernel written in the Metal Shading Language. You compile the source at run time with compile ~source ~entry and dispatch it over shared buffers. It has no CPU fallback; with no Metal device, every call raises Failure.

Installation #

Install with opam:

$ opam install nox-metal

If opam cannot find the package, it may not yet be released in the public opam-repository. Add the overlay repository, then install it:

$ opam repo add samoht https://tangled.org/gazagnaire.org/opam-overlay.git
$ opam update
$ opam install nox-metal

Building #

The only C stub is lib/metal_stubs.c. On macOS dune builds it as Objective-C with ARC and links Metal and MetalPerformanceShaders. On other systems the stub just raises, and the OCaml code never reaches it. Nothing needs configuring.

License #

ISC. See LICENSE.md.