# zig-yaml A YAML 1.2.2 parser and emitter for Zig 0.17, with the same shape as `std.json`. The parser is built directly on the grammar of the YAML 1.2.2 specification [1]. It passes every case of the yaml-test-suite [2]. The API documentation is generated from the doc comments. CI publishes it from `main` to , and `zig build docs-serve` serves the same documentation locally. ## Where this lives The repository lives in three places, all carrying the same history. The Forgejo instance at is the one with a web interface: ```console $ git clone https://git.jcollie.dev/jeff/zig-yaml.git ``` It is mirrored on Tangled at . It is also on the Radicle network, with the repository ID ``` rad:z2vTvxxTzoSo75cpepS4VFqUFo8x3 ``` and `rad clone rad:z2vTvxxTzoSo75cpepS4VFqUFo8x3` fetches it from any node that seeds it. Each of the three holds the whole project. ## Adding it to a project ```console $ zig fetch --save git+https://git.jcollie.dev/jeff/zig-yaml.git ``` then, in `build.zig`: ```zig const yaml = b.dependency("zig_yaml", .{ .target = target, .optimize = optimize }); exe.root_module.addImport("yaml", yaml.module("yaml")); ``` `main` needs Zig 0.17.0. For Zig 0.16.0, use the `zig-0.16` branch, which holds the last of zig-yaml to build with it: ```console $ zig fetch --save git+https://git.jcollie.dev/jeff/zig-yaml.git#zig-0.16 ``` The `v0.1.0` tag is the same commit. ## Using it If you know `std.json`, you already know most of this API. ```zig const yaml = @import("yaml"); const Server = struct { host: []const u8, port: u16 = 8080, tags: []const []const u8 = &.{}, }; const source = \\host: example.com \\tags: [web, "prod"] ; const parsed = try yaml.parseFromSlice(Server, gpa, source, .{}); defer parsed.deinit(); // parsed.value.port == 8080 const text = try yaml.Stringify.valueAlloc(gpa, parsed.value, .{}); defer gpa.free(text); // host: example.com // port: 8080 // tags: // - web // - prod ``` This example is also a test in `src/tests.zig`, so it stays correct. ### API overview | `std.json` | zig-yaml | Notes | | --- | --- | --- | | `parseFromSlice`, `…Leaky` | `parseFromSlice`, `…Leaky` | Takes exactly one document. An empty stream reads as a null node. | | — | `parseAllFromSlice`, `…Leaky` | Takes every document of a stream, as `[]T`. | | `parseFromTokenSource` | `parseFromTokenSource` | Takes a `*Scanner` or a `*Reader`. | | `parseFromValue` | `parseFromValue` | | | `innerParse`, `innerParseFromValue` | the same | For use inside parse hooks. | | `Parsed(T)` | `Parsed(T)` | Holds the value and its arena. | | `Value` | `Value` | Mapping keys can be any `Value`, not only strings. | | `Scanner` (tokens) | `Scanner` (events) | `next`, `peek`, `peekNextEventType`, `skipValue`. | | `Reader` | `Reader` | Reads the whole `std.Io.Reader` before parsing; see below. | | `Stringify` | `Stringify` | `value`, `valueAlloc`, and a write stream. | | `fmt` | `fmt` | Use with `{f}`. | ### Customization hooks A type can take over its own parsing or output by declaring any of these functions, the same way it would for `std.json`: ```zig pub fn yamlParse(allocator: Allocator, source: anytype, options: yaml.ParseOptions) !@This() pub fn yamlParseFromValue(allocator: Allocator, source: yaml.Value, options: yaml.ParseOptions) !@This() pub fn yamlStringify(self: @This(), yw: *yaml.Stringify) !void ``` ### How nodes map to Zig types The target type decides how each node is read: - **Scalars are matched by their text, not their style.** `"8080"` parses as a `u16`, the same way `std.json` accepts a number in a string. - **Strings accept any scalar.** `version: 1.10` read into a `[]const u8` gives `"1.10"`, not a float. - **Booleans and numbers use the core schema's spellings.** That includes `True`, `0x1F`, `0o17`, `.inf` and `~`. With `.schema = .json`, only JSON's spellings are accepted. - **Tagged unions** are written as a single-pair mapping, `circle: 1.5`. A variant with a `void` payload can also be written as its bare name. - **Aliases are expanded.** `ParseOptions.max_alias_expansion` caps how many events alias expansion can produce, so a "billion laughs" document is an error rather than running out of memory. When there is no target type, `Value` resolves untagged plain scalars with the schema in `ParseOptions.schema`: core (the default), JSON or failsafe. Explicit tags such as `!!int` and `!!str` are honored. Unknown tags are ignored by default, or rejected with `.unknown_tag_behavior = .@"error"`. ### Output `Stringify` writes block style by default and flow style when `.style = .flow`. It writes each string in the plainest style that reads back as the same string: 1. **Plain**, unless the text could be mistaken for a null, boolean, number or indicator. 2. **Single-quoted**, if the plain form would be misread. 3. **Double-quoted**, when the string needs an escape. 4. **Literal block scalar**, for multi-line strings it can represent exactly. Collection keys are written after a `?` indicator. Formatting is data-only: comments and the original scalar styles are not preserved. ### What this is not **It is not incremental.** The YAML grammar backtracks: whether a line is a mapping key can depend on text after it. No event can be returned until the whole stream has been parsed. `Reader` therefore reads the whole stream first, up to `max_input_len` (64 MiB by default). ## How the parser is built `src/grammar/` holds one file per chapter of the specification. Each numbered production is one function that carries the production's number and text in its doc comment. For example, `[187] l+block-mapping(n)` is `lBlockMapping` in `block.zig`. Functions take the spec's own parameters: the indentation `n`, the context `c` and the chomping `t`. Each returns whether it matched, and restores the parser's state if it did not. Events are emitted where a node begins and ends, and backtracking takes them back. Backtracking alone would make some inputs take exponential time. Two measures keep it polynomial: - **Flow collections are memoized.** In a key context the whole result is stored, events included. Elsewhere only failures are stored. - **Implicit keys are bounded.** While a key attempt runs, the input is narrowed to the spec's 1024-character key limit. Nesting is limited to `max_depth` (512) collections. Deeper input is an `error.NestingTooDeep`, not a stack overflow. ### Where the implementation goes beyond the printed grammar Read as written, the grammar is ambiguous in places. Where it is, the yaml-test-suite decides how it is read. Each case below is commented at its production: - **Optional groups are retried both ways when PEG ordering would get them wrong.** In `[200] s-l+block-collection`, properties followed by a key on the same line (`&a key: value`) belong to the key, not to the mapping. A second property on the next line can belong to that line's key (`!!map` then `&a key: v`). - **Properties followed by JSON-like content are not an empty key.** `[159] ns-flow-yaml-node` does not match properties followed by nothing when JSON-like content comes next, so `{ &a [x]: y }` anchors the sequence. - **A `%YAML` or `%TAG` directive that fails to parse is an error.** `[83]` never falls back to reading it as a reserved directive of the same name. - **The end of the input ends the last line.** It acts as a line break for block scalar chomping and for a trailing line of spaces. - **Both block scalar indicators are optional.** `[163]` is printed without its `?`. - **Block scalars with no content lines take their indentation from their empty lines.** The indentation is the widest leading empty line, so those lines are read as empty rather than as content. - **Directives, anchors and the forbidden markers are handled outside the grammar.** `%TAG` handles, `%YAML` versions and anchors are scoped to their document. The "excluding c-forbidden content" restriction of `[207]` is enforced by treating the first `---` or `...` at the start of a line as the end of the input while a bare document is parsed. ## Testing ```console $ nix develop $ zig build test # unit tests and the whole yaml-test-suite $ zig build test -Dsuite-filter=9KAX # one suite case, with details on failure $ zig build events -- file.yaml # events for a file, in the suite's notation $ zig build fuzz-run -- --seconds 600 # fuzz every target for ten minutes each $ zig build fuzz --fuzz # Zig's own fuzzer, until interrupted ``` The yaml-test-suite [2] is a lazy dependency, so a project that depends on this package never fetches it. The suite runner checks every case three ways: - **Errors and events.** Each invalid input must be rejected, and each valid input must produce exactly the expected event stream. - **JSON.** Where a case has a JSON equivalent, the parsed `Value` must equal it under RFC 8259 [3]. - **Round trip.** Each valid case is written back out in block and flow style and must parse back to an equal `Value`. All 402 cases pass. That covers 94 invalid inputs rejected, 279 JSON comparisons and 308 round trips. `tests/fuzz.zig` holds the fuzzing properties. Two fuzzers drive them. `zig build fuzz --fuzz` is Zig's own, which is guided by coverage; the test binaries are compiled with LLVM because the self-hosted backend emits no coverage instrumentation. `tools/fuzz.zig` is a loop of our own with no coverage, which mutates inputs seeded from the test suite instead. The properties are: - Any input either fails or round-trips. - Any generated `Value` tree round-trips through `Stringify`. - Typed parsing round-trips. A watchdog flags any input that takes too long. CI runs a short campaign on every push. ## Building The development shell in `flake.nix` provides Zig 0.17, `reuse`, `git-pages-cli` and `zon2nix`. `build.zig.zon.nix` is generated from `build.zig.zon`. When a dependency changes, regenerate it with: ```console $ nix develop -c zon2nix --17 --nix=build.zig.zon.nix build.zig.zon ``` CI fetches the Zig dependencies through Nix, from the `zig-deps` flake output, by passing `--system`. ## License MIT. The project follows the [REUSE](https://reuse.software/) specification. ## References cited 1. Ben-Kiki, O., Evans, C., & döt Net, I. (2021, October 1). *YAML Ain’t Markup Language (YAML™) revision 1.2.2*. YAML Language Development Team. 2. YAML Language Development Team. (2022). *YAML Test Suite* (Version data-2022-01-17) [Data set]. GitHub. 3. Bray, T. (Ed.). (2017, December). *The JavaScript Object Notation (JSON) Data Interchange Format* (RFC 8259; STD 90). RFC Editor.