zig-git
Reading and writing git repositories in Zig, without running git.
Star
Fork
Watch
zig-git README.md
16 kB · 297 lines
Markdown
at main

zig-git #

A library for reading git repositories from Zig 0.17 without running git, written in Zig alone: no libgit2, no C zlib, nothing to link. It is meant to answer the questions a history browser asks git — what a commit holds, what a revision range means, what a diff says — with the same answers git gives, and to grow into writing to repositories and talking to remote ones.

It is early. What works so far is the bottom of the core layer:

  • object formats and object IDs, SHA-1 and SHA-256 alike, chosen per repository at run time, and hashing an object to its ID;
  • object headers, read exactly as strictly as git reads them;
  • commits, trees, annotated tags and their author, committer and tagger lines, read without allocating and as forgivingly as git reads them, their oddities included;
  • loose objects: finding, inflating and checking them, whole, header only or streamed, refusing each corruption git refuses and in the same circumstances;
  • packs and their indexes (versions 1 and 2 of the index, versions 2 and 3 of the pack, offsets past 2 GiB), with OFS_DELTA and REF_DELTA chains resolved without recursion and bounded in depth, so that a chain that loops is an error rather than a hang; reading an object's kind and size applies no delta;
  • the multi-pack-index, and incremental chains of them, as a faster way of finding objects in the packs they name;
  • alternates, from objects/info/alternates and from a GIT_ALTERNATE_OBJECT_DIRECTORIES value the caller passes in, nested as deep as git nests them;
  • all of that as one object database, which asks its objects directories, packs and loose objects in the order git can be seen to ask them, passes over a broken copy for the next as git does, and names each corruption it meets;
  • configuration: the file syntax read on bytes, every corner git accepts and every malformed line it refuses at the line it names; the system, global, local and worktree files, GIT_CONFIG_COUNT, GIT_CONFIG_PARAMETERS and -c, in git's order and with every path and environment value passed in; include.path and includeIf with gitdir:, gitdir/i:, onbranch:, worktree: and hasconfig:remote.*.url:, its conditions answered by the caller; booleans, integers with their units, and paths with ~ and %(prefix); and lookups that spell keys as git does;
  • wildmatch, git's glob dialect, as a module of its own for refs, pathspecs and attributes to use, with the corners git check-attr and git tag -l show;
  • discovering a repository as git does: walking up from a starting directory, .git directories and .git files (linked worktrees and submodules, commondir included), bare repositories, GIT_DIR, GIT_WORK_TREE, GIT_COMMON_DIR, GIT_OBJECT_DIRECTORY, GIT_CEILING_DIRECTORIES with its empty-entry and symbolic link rules, filesystem boundaries and GIT_DISCOVERY_ACROSS_FILESYSTEM, core.worktree and core.bare, safe.directory and ownership, safe.bareRepository, and what git requires of a git directory before it trusts one; the environment, the starting directory and the user are all passed in;
  • the repository format: versions 0 and 1 and every extension git 2.56 knows, refused with git's own message where git refuses them, and by name where this library cannot read what git can (extensions.compatObjectFormat, a refStorage payload);
  • Repository, the library's way in: a discovered repository opened with its configuration from every source, config.worktree included, its includeIf conditions answered by the repository itself, and its object database.

zig-git cat-file (-t, -s, -e, -p and <kind> <object>) answers from the library for objects named by full object ID, wherever they are stored, printing what git cat-file prints, its error messages included. zig-git show-index and zig-git verify-pack [-v] [-s] print what git's do, the delta chain histogram included. zig-git config reads configuration as git config does — --get, --get-all, --get-regexp, --list, the get and list subcommands, with --show-origin, --show-scope, -z, --name-only, the types bool, int, bool-or-int and path, and every file option — its output, errors and exit statuses included. It writes nothing, and refuses by name what it does not do: --blob, --url, the color and expiry-date types, and GNU's regular expression extensions and back references. zig-git rev-parse answers the questions about where a repository is — --git-dir, --absolute-git-dir, --git-common-dir, --show-toplevel, --show-prefix, --show-cdup, the --is-* questions, --show-object-format, --show-ref-format, --local-env-vars and --path-format — as git rev-parse does, and refuses revisions by name. Every command finds its repository the way git does, takes -C, --git-dir and --work-tree before its name, and says what git says when there is none, or one it will not use. The differences are deliberate or not yet matched:

  • an object holding fewer bytes than its header says is refused, where git shows whatever its buffer held in their place;
  • a REF_DELTA chain that loops is DeltaChainTooDeep, where git never returns;
  • git never resolves a REF_DELTA whose base is outside its own pack, and nor does the library unless asked to (resolve_external_bases);
  • what git says on standard error of a corrupt delta, of a corrupt pack found by verify-pack, and of a broken pack met on the way to a copy that could be read, is not matched; the answers and exit statuses are;
  • zig-git config expands %(prefix)/ only when given a prefix, since it is not git and has none of its own;
  • a repository with extensions.compatObjectFormat is refused, where git reads it (and crashes when it names the object format itself);
  • safe.bareRepository with no value is refused as a bad setting, where git crashes;
  • a commondir that begins with blank space names no repository, where git names a path part-way through resolving it;
  • --is-shallow-repository asks only whether the shallow file (or GIT_SHALLOW_FILE) is there, where git also reads it;
  • ownership and filesystem boundaries are found with statx, so only on Linux; elsewhere a caller supplies its own answers or discovery says it cannot check.

The API documentation is generated from the doc comments and published at https://jeff.jcollie.page/zig-git/.

What it is for #

grebe, a history browser for git, reads repositories by running git log and git show. zig-git is being written so that it can stop: the plan is a library in layers, each depending only on the ones before it.

  1. Core — the object database (loose objects, packs, the multi-pack index, alternates), the ref store (loose refs, packed-refs, reftable, reflogs), configuration with includes, discovering a repository the way git does, and the index file.
  2. Revision — revision arguments in git's own syntax, and walking the history they choose: ranges, path limiting, the three orderings, mailmap, replace refs, grafts.
  3. Diff — between trees, the index and the working tree, producing patches byte-identical to git's.
  4. Write and transport, later: ref transactions, writing objects, signing, and git's protocol v2 over file://, ssh:// and https://.

Three choices run through all of it, and the decision records in docs/adr/ give the reasons:

  • Agreeing with git is the definition of correct. The tests build their repositories with git and compare answers with it, and patches are to match git's byte for byte.
  • What it does not support, it says so. A revision argument outside the supported set is an error naming it, and a repository using an extension it does not implement is refused, so a caller can fall back to git rather than receive an answer that is nearly right.
  • It is written from git's documentation and behavior, never its source, so that it can be MIT licensed.

The words used for the things in a repository — object ID, object format, object database, ref store, git directory — are defined in GLOSSARY.md.

Building and testing #

Everything is pinned in the Nix flake, Zig and the git the tests compare against included:

nix develop
zig build test --summary all   # every repository test runs with SHA-1 and SHA-256
zig build docs-serve           # the API documentation on http://localhost:8000

CONTRIBUTING.md says more.

Where this lives #

The canonical repository is on my Forgejo instance, and Tangled carries a mirror of it:

git clone https://git.jcollie.dev/jeff/zig-git.git

Report issues and send pull requests to the Tangled mirror. CI and the published documentation come from the Forgejo repository.

License #

MIT; see LICENSES/MIT.txt. The project follows REUSE.

References cited #