shims: nix and cargo #

the model's jobs (run(), job.pipe(), and remote kernels' alike) find two shims first on PATH, each under $ALBEDO_HOME/shims/<needs>/ and prepended only when the program it needs is on the job's PATH (albedo_shims.prepend): shims/nix/nix when nix is, shims/mbx/cargo when mbx is. the kernel writes them where it runs (only .py files travel to a remote kernel), and every nix call a job makes goes through the shim, however deep in a script it is. Before lookup, prepend removes inherited */shims/nix and */shims/mbx directories, then installs only this kernel's shims for real tools on PATH. Nested daemon jobs must not resolve one shim generation through another. there is no nix API for the model to learn: it types nix develop -c cargo test or nix build .#x and the shim makes that safe and fast.

a daemon started with a thin PATH (a GUI or launchd start never ran the user's profile) would see no nix at all, so prepend first looks for each program of FALLBACKS that PATH lacks and appends the first directory holding it (/nix/var/nix/profiles/default/bin, then ~/.nix-profile/bin). it goes last, so it never shadows the user's PATH, and it joins before the shim check so the shim still goes in front. it covers job environments only: shutil.which in a cell reads the kernel process's own PATH.

nix #

why #

a jj workspace other than the colocated one has .jj/ but no .git, so nix reads . as a plain path: flake and copies the whole directory into the store on every evaluation: build output, target/, .jj and all (140 MB per eval measured on a small checkout, ~850 MB for a full albedo one). a git checkout or worktree is fine: nix's git fetcher copies tracked files only.

the program #

shims/nix/nix is sh: with ALBEDO_SHIMS_PYTHON and ALBEDO_SHIMS_LIB set (by prepend, the kernel's python and its priv/python) it runs albedo_nix.py with python -E -s, else, and with ALBEDO_PLAIN_NIX=1, the next nix on PATH. albedo_nix.py is stdlib only and never imported by the kernel. what it does with a command line:

1. reads it with nix's own flag table, nix __dump-cli (50 ms, cached in $ALBEDO_HOME/cache/nix-cli/ per nix binary), so every flag's arity is known. commands it cannot read (a flag or command not in the table) go to nix unchanged, which refuses them itself. ALIASES covers nix's renamed commands the table lists only under their new name (nix shell is env shell, profile install is profile add, dev-shell is develop).
2. nix develop [installable] -c program ... with only output flags (QUIET) runs the program in the cached dev shell (below), with the shims kept first on PATH. any other develop flag (-i, --phase, ...) goes to step 3.
3. every word that names a local flake of a jj workspace without .git is replaced by the workspace's commit ref, following the subcommand's argument labels in the table (installables, installable, flake-url, package, dependency), plus --inputs-from and flake update --flake. a command that names none defaults to . as nix does (not nix eval, not with --expr/--file/--stdin/--all). nix fmt and nix formatter run|build become nix run|build <ref>#formatter.<system> with PRJ_ROOT set to the flake's directory, as nix fmt sets it. the rewritten command is printed to stderr after albedo:, then exec'd.
4. anything else is exec'd unchanged.

finding the real nix #

The shim looks for the next nix on PATH without its own directory (compared with os.path.samefile) and without any directory whose nix is another albedo home's shim (recognized by the script's header line). A job started from an albedo session keeps that session's shim on PATH, so a test or private daemon that writes its own shim puts two homes' shims on one PATH; each skipping only itself would hand the call back and forth without end.

refs and lock files #

flake_ref: jj log -r @ snapshots the working copy, so commit @ holds every edit, and jj git root names the git store. the ref is git+file://<store>?rev=<commit> (&dir= for a flake below the workspace root); a colocated .git is named by its work tree (nix reads it in place; naming the .git directory makes nix clone it into its cache), a bare store as itself. jj keeps every commit under refs/jj/keep/*, so nix finds it. git checkouts and directories outside version control are never rewritten.

nix writes a lock file it had to create or update into the repository a git+file ref names: for a colocated store that is the main checkout's work tree (measured: nix build <ref> created main/flake.lock). so a rewritten command gets --output-lock-file <workspace flake>/flake.lock, which writes where nix would have in a git checkout. that flag covers every flake the command locks (--inputs-from . nixpkgs#hello wrote nixpkgs' empty lock over the workspace's), so a command that locks any other flake gets --no-write-lock-file instead. --commit-lock-file would commit in the main checkout and is refused. nix rewrites the output lock file even when it is unchanged; nix-formatted locks keep their content.

dev shells #

nix print-dev-env --profile <cache>/<key> <ref> writes the same script nix develop sources; the shim sources it with the shell's own bash (the shell variable from the profile's JSON: macOS's /bin/bash 3.2 cannot run it), which runs the shellHook (its output goes to stderr), removes $NIX_BUILD_TOP, and dumps env -0. what is kept is the difference from the job's environment (delta): variables set, prefixes put before a list the shell kept (PATH, XDG_DATA_DIRS) and variables unset. every variable the script exports is kept even where the capture's own environment already held its value (mkShell env.* values, CC, NIX_CFLAGS_COMPILE, ...): what nix develop -c gives a program does not depend on who ran it, and a capture run inside the very shell it was capturing kept nothing of it once (nix develop -c env showed no CGO_ENABLED, and a cgo link failed). the script's function bodies do not run, so their exports do not count; the paths it composes from the caller's own (it saves them as nix_saved_<name>) stay prefixes; and what the script leaves alone (HOME, TERM) is the caller's. TMPDIR, NIX_BUILD_TOP and the like (VOLATILE) are dropped. applied lays it over each later job's environment. a build prints albedo: building the dev shell for <dir> first; a cached run prints nothing of its own.

the cache is $ALBEDO_HOME/cache/nix-develop/<key>.json, and the profile <key> beside it is a garbage-collector root for the shell's closure. the key hashes the system, the attribute, the flake's place in its checkout, and the content of every tracked SHELL_FILES file (*.nix, flake.lock, rust-toolchain, rust-toolchain.toml; jj file list --ignore-working-copy or git ls-files, a walk outside version control). so an unchanged flake costs one jj file list and no nix call (60 ms), and every checkout of the same repository at the same shell files shares one entry. a build that wrote the lock file is stored under the key after it too. a record another capture shape wrote (it may carry too little) is rebuilt when next wanted, not replayed. the 32 most recently used entries are kept; a flake elsewhere (nixpkgs#hello) is keyed by its name and kept a day. shims building the same key at once wait on one flock.

limits: a dev shell made of other files (built from Cargo.lock or pyproject.toml) is not rebuilt when only those change; a shellHook with side effects (starting a service) runs once per build, not per command; a stale jj workspace (jj workspace update-stale) fails with jj's message. nix-shell and nix-build are other binaries and are not shimmed.

cargo #

shims/mbx/cargo runs mbx (jdx's mr-boxington) in its shim mode, so every checkout shares compiled work through mbx's cache. ALBEDO_NO_MBX=1, or mbx gone from PATH, runs the next cargo. mbx's own mbx setup is not used: it edits the user's rust-analyzer config, and its shim exits 127 when mbx is missing.

what the shim changes for mbx's process, all measured on native/render (a second checkout: 32 s with 55 links bypassed, 2.1 s with 201 hits after):

• SDKROOT is unset. mbx names the macOS linker with xcrun --sdk "$SDKROOT" --show-sdk-version, and xcrun rejects a path such as a nix shell's store SDK, which leaves the linker unnamed and every native link uncached: build scripts, proc macros, binaries, tests, and so every crate depending on a proc macro or a cc-built -sys crate. DEVELOPER_DIR stays, so the same SDK is linked.
• inside a nix shell (ALBEDO_NIX_ENV, which the dev-shell capture sets), CC and CXX are unset when they resolve to the same file as cc and c++ on PATH (the nix cc-wrapper's cc is a symlink to its clang), so a build script's C compiles through mbx and stays portable: an explicit CC makes mbx leave that C alone (cc-compiler-override), and a dev build's objects then record the checkout's paths (synthetic workspace, second checkout: 31 hits and 3 misses with CXX set, 34 and 0 without). a shell whose CC is another compiler (gcc, a cross compiler) keeps it. and HOST_CFLAGS/HOST_CXXFLAGS gain -DALBEDO_NIX_ENV_<digest>. the digest is of the shell's compiler-wrapper variables (NIX_CFLAGS*, NIX_LDFLAGS*, NIX_CC*, NIX_HARDENING_ENABLE), which change C output but are in no mbx key: without it a changed shell was served another checkout's stale object.

never set for cargo here: CARGO_INCREMENTAL, RUSTC_WRAPPER, MBX_CACHE_DIR (one cache is the point), a --target equal to the host, or -C link-arg. what stays per checkout: crates whose build-script output or source names the checkout or target path and their dependents (aws-lc-sys's memcmp probe leaves a path-bearing .dSYM in OUT_DIR under -g; tikv-jemalloc-sys runs autotools there unless JEMALLOC_OVERRIDE names a built library), and a workspace crate being edited. a new toolchain or dev shell is a new key, as it should be.

mbx replaces a checkout's target/ with a symlink into its cache. an ignore rule with a trailing slash (target/) matches directories only, so jj would track that symlink: albedo's own rule is /native/render/target.