A FLAC encoder and decoder for Zig, written against RFC 9639, for native FLAC streams and FLAC in Ogg
zig-flac CLAUDE.md
11 kB · 222 lines
Markdown
at main

CLAUDE.md #

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

zig-flac encodes and decodes the Free Lossless Audio Codec. README.md covers the API, the options and how the thing is tested; read the section covering whatever you are changing before changing it.

The standard this project is held to #

Lossless means bit-exact, and the specification is the authority. The specification is RFC 9639, Free Lossless Audio Codec, December 2024. Where behaviour is in question the answer comes out of the RFC and the citation goes in the code -- "(RFC 9639, Section 9.2.7.2)", "(Appendix A.4)" -- rather than out of what some other implementation happens to do. Two habits follow:

  • Cite the section in the comment. Nearly every non-obvious constant here is in a table in the RFC, and a reader who cannot find the table cannot check the constant. The full text is at https://www.rfc-editor.org/rfc/rfc9639.txt and is worth fetching before answering a format question from memory.
  • Test against numbers the RFC printed. Appendix D works three complete files through field by field; src/tests/rfc9639/ holds those files, and the tests that read them assert values the appendix states. A test that asserts what this implementation produced proves only that it has not changed.

Do not propose relaxing losslessness to simplify something. An encoder that cannot represent a signal must fall back to a model that can -- that is what verbatim subframes are for -- and never round.

Commands #

Zig is not on PATH. Everything runs through the flake's devshell:

$ nix develop --command zig build test        # unit tests and the fuzz corpus
$ nix develop --command zig build check       # compile everything, run nothing
$ nix develop --command zig build run -- info song.flac
$ nix develop --command zig fmt --check .
$ nix develop --command reuse lint            # every file needs an SPDX header

One test, by name -- the build has no filter option, so go at the module directly. src/root.zig is the library, and it imports zig-ogg as ogg:

$ nix develop --command zig test --dep ogg -Mroot=src/root.zig \
    -Mogg=zig-pkg/<ogg hash>/src/root.zig --test-filter "wasted bits"

<ogg hash> is the .hash of the ogg dependency in build.zig.zon; any zig build fetches it into zig-pkg/.

Other steps: docs, fuzz, fuzz-run.

Fuzzing #

There are two loops around the same targets in src/tests/fuzz.zig. zig build fuzz --fuzz is Zig's own fuzzer, which follows coverage; a finding prints input saved to '.zig-cache/f/crash' above the report. Its test binary is built with use_llvm = true even in Debug, because the self-hosted backend emits no coverage instrumentation -- a run that ends in pcs_len was zero means that setting has been lost, not that anything was found.

zig build fuzz-run drives the targets from tools/fuzz.zig instead, mutating the corpus rather than following coverage. It is what CI runs, and the easy way to replay one input.

$ nix develop --command zig build fuzz --fuzz=1M
$ nix develop --command zig build fuzz-run -- --seconds 300
$ nix develop --command zig build fuzz-run -- --target decode --seed 12345
$ nix develop --command zig build fuzz-run -- --input fuzz-findings/decode-1.bin

The targets are decode, ogg, metadata and roundtrip.

A finding lands in fuzz-findings/, which is gitignored. Read it, work out what shape it is, and if it is a shape rather than an accident, add it to the corpus at the bottom of src/tests/fuzz.zig where zig build test will run it every time.

Architecture #

The library has no filesystem. A decoder reads from a *std.Io.Reader and an encoder writes to a *std.Io.Writer; opening files is the caller's business, and src/main.zig is the only thing here that does it. Do not add a path parameter to anything under src/ other than main.zig.

It has no std.Io either, and that is why ogg.Encoder.Options.serial_number has no default. RFC 3533 wants an Ogg serial number chosen at random, and in Zig randomness comes from an Io -- which the library deliberately does not take. The caller asks for the bytes. Do not reach for a counter or a clock here to make the option go away: two chained streams sharing a serial number is a malformed file that nothing will notice until it is played.

The Ogg mapping belongs here, not in zig-ogg. src/ogg.zig is RFC 9639 Section 10.1, specified in full at https://xiph.org/flac/ogg_mapping.html, and almost all of it is FLAC knowledge: metadata blocks, frame headers, sample counts. The container library underneath knows only one FLAC fact, the \x7fFLAC magic on a bos page, and it stays that way -- a container that learned what a frame is would have to learn it for Vorbis and Opus too. The dependency arrow is zig-flac -> zig-ogg and never back.

Two seams exist so the codec need not know which container it is in, and both are the only sanctioned way to drive it a frame at a time:

  • Decoder.decodeFrame(reader, offset) decodes one frame from one reader, with everything the decoder accumulates across a stream -- the MD5, the sample count, the bounds -- carried over. Decoder.initFromBlocks builds a decoder around metadata blocks that did not come from the front of a stream.
  • Encoder.samplesUntilFrame() says how many samples the current block still has room for, so a caller that hands over at most that many knows whatever the encoder then wrote is exactly one frame. Options.write_head = false stops it writing the signature and the metadata blocks at all.

Do not make the Ogg layer parse frames back out of a buffer to find their boundaries, and do not have it swap the native encoder's output writer behind its back. Both were tried; the seams are what replaced them.

Everything that touches a sample is i64 integer arithmetic. A side channel is a bit deeper than the frame it is in and a prediction is wider still, and the RFC's own advice is to leave no room for overflow to matter (Appendix A). The one exception is src/lpc.zig, which finds predictors in floating point -- which the RFC explicitly permits, because the coefficients it proposes are quantized to integers before a single residual is computed from them. Keep that boundary where it is: no float ever reaches a sample that gets written.

A hostile stream can make samples grow without bound. A residual sample is bounded by the format; a prediction is not. Decoder checks each restored sample against the depth its subframe codes at, before the next one uses it, which is what keeps every multiplication inside 64 bits. Any new path that reconstructs samples needs the same check -- Bounds in src/decoder.zig is what does it.

The bit reader pulls one byte at a time on purpose. The frame header's CRC and the frame footer's CRC are each defined over exactly the bytes consumed up to a byte-aligned point, so the reader must not read ahead. src/bits.zig says this at the top; a "let's read a machine word at a time" optimization silently breaks both checks.

Costing is exact, not estimated. The encoder's Rice partitioning computes the true cost of every parameter at every partition order in one pass, because the sums involved are additive across partitions (choosePartitioning in src/encoder.zig). Do not replace it with the estimate-and-search-nearby approach other encoders use; it is not faster here and it is not exact.

Metadata blocks keep their bytes when the parse cannot account for them. Block.raw is set when writing the parsed body back would not reproduce the input -- a Vorbis comment block with bytes after its last field, a padding block whose padding is not zeroes. Reading a file and writing it back must never lose a byte, and refusing to play a file over a fault in a tag is worse than either.

Streaminfo is written twice. An encoder cannot know a stream's MD5, its frame sizes or its length until it has seen all of it, so what goes at the head of the file is a placeholder and Encoder.finish leaves the real thing in e.streaminfo for a caller that can seek. The same applies to a reserved seek table. A stream written to a pipe keeps the placeholder, which is legal.

Performance #

zig build benchmark measures encoding and decoding in process; --ablate turns one part of the encoder's search off at a time and reports what it was costing, which is more use than a flat profile for deciding what to change. Always measure with it rather than with the command line tool, which times the filesystem as much as the codec.

Three things worth knowing before optimizing anything here:

  • The output must not change. Every optimization so far has been exact: encode the benchmark's audio at all nine levels before and after, and the sizes must be identical byte for byte. A change that makes the encoder faster and the files bigger is a change to the presets, not an optimization, and belongs in the level table with a measurement beside it.
  • The hot loop is the Rice partition search, not the predictors. Costing a partition runs about four times the work of computing the residual it costs. parameterLimit bounds how far that search has to go and proves why; the residual buffers are i32 rather than i64 because the format bounds them to that and half the width is half the memory traffic. Both were worth 10-19%; going further into that loop was not.
  • --ablate's "every LPC order" row is the argument for lpc_orders_costed. Costing every predictor order buys about 1.7% of size for about 96% more time. The recursion's own error estimate ranks the orders nearly right, so the presets cost the best few in full instead.

Checking against a reference #

There is no reference implementation in this repository, but ffmpeg decodes and encodes FLAC and is the closest thing to hand:

$ zig-flac test some.flac                                    # its own MD5
$ ffmpeg -i mine.flac -f s32le - | cmp - <(ffmpeg -i orig.wav -f s32le -)

The same works for Ogg FLAC, and is worth running in both directions, since ffmpeg both reads and writes the mapping:

$ zig-flac encode --ogg orig.wav mine.oga
$ ffmpeg -i mine.oga -f s32le - | cmp - <(ffmpeg -i orig.wav -f s32le -)
$ ffmpeg -i orig.wav -c:a flac theirs.oga && zig-flac test theirs.oga

Compare audio, not WAVE files: zig-flac decode and ffmpeg write different chunks around the same samples, so cmp on two .wav files reports a difference at byte 5 that means nothing.

Two things to know before drawing conclusions from ffmpeg:

  • Its encoder is not lossless at 32 bits per sample. Given 32-bit PCM it truncates to 24. A size comparison at that depth is comparing different audio.
  • Its compression levels are not the reference encoder's. Its -0 uses stereo decorrelation where flac -0 does not, so -0 is the one preset where a large gap is expected and is not a defect.