Something went wrong. Try again.
Container for the pi harness with my defaults.
Something went wrong. Try again.
Files
- Nix 75%
- TypeScript 25%
README.org
#+title: Sandbox
Declarative NixOS development sandboxes for the pi coding agent. The microVM implementation is experimental. The user reports successful laptop builds and runtime use; the laptop also customizes networking and store sharing.
* Declarative MicroVMs
Import =nixosModules.piSandboxes= on the NixOS host. It uses [[https://github.com/microvm-nix/microvm.nix][microvm.nix]] to declare independent QEMU/KVM VMs. Each has 512 MiB RAM, a =developer= account, a guest-only read-only Nix store image, and its own persistent =/context= disk image. The host builds the VM's NixOS configuration and packages; there is no host Nix daemon socket, host store share, project mount, or inherited host credentials. The guest Nix daemon is not enabled, so building or installing Nix packages inside a VM is not yet supported. The explicitly installed guest baseline is pi, tmux, Git, Jujutsu, BusyBox, OpenSSH and CA certificates; NixOS also supplies system tools. Pi's wrapper supplies its Node runtime, fd and ripgrep. Additional tools can be declared with =extraPackages=.
Configure instances on the host, for example:
#+begin_src nix
{ pkgs, ... }: {
services.piSandboxes.instances = {
pi-dev = {
stateVersion = "25.11";
vsockCid = 100;
authorizedKeys = [ "ssh-ed25519 REPLACE_WITH_YOUR_PUBLIC_KEY" ];
extraPackages = with pkgs; [ just ];
contextSizeMiB = 16384;
};
};
}
#+end_src
Use unique hostnames and VSOCK CIDs for all VMs on the host; the module checks uniqueness only among its own instances. Keys are public SSH keys, not private keys or secrets. A fresh VM starts only when requested; =autostart= and automatic restart on host configuration change are disabled. The 512 MiB limit may be too small for compiles or pi workloads. Host activation installs microvm.nix's system services and host-wide defaults, including KSM and a bridge helper; review upstream's host module before activating. VMs run under its system service, not a user service.
There are no virtual network interfaces by default. SSH over VSOCK is configured using microvm.nix and does not require IP routing; the user reports that login works on the laptop. Guest SSH accepts the specified keys for =developer= only and disallows password and root login. Host service start/stop may require administrator authentication. Access to =/dev/vsock= and QEMU/KVM, socket activation and first login require verification on the target host. VSOCK is a host communication channel, not a replacement for a network isolation review. No internet, forwarding or NAT is configured. Verify Mullvad lockdown for guest traffic, IPv4, IPv6 and DNS before enabling any networking; host-originated services are a separate concern.
Do not use =microvm -s= for authenticated entry: its current upstream implementation disables SSH host-key checking. The host module installs =sandbox <instance>= for declared instances, or =sandbox= with no argument when exactly one exists. It connects as =developer= over VSOCK without SSH agent forwarding. SSH automatically accepts a previously unknown host key and records it in =~/.ssh/<instance>_known_hosts=; it rejects a changed key. Existing enrolled keys continue to be checked. This is trust on first use: the first connection does not independently authenticate the guest, even though it uses VSOCK. Investigate a changed-key warning rather than blindly removing the recorded key. The guest host key is stored on its persistent image.
The host module also creates a system SSH alias for each instance, using the same VSOCK CID and per-instance known-hosts file. With the VM running, =ssh pi-dev= opens a normal guest shell and Emacs can open =/ssh:pi-dev:/context/= via =C-x C-f=. Unlike =sandbox=, this does not attach to tmux. The alias uses standard SSH identities; the corresponding public key must be authorized in the guest. User SSH configuration takes precedence over system defaults and can override the proxy, host-key policy or forwarding settings, so avoid conflicting aliases and check =ssh -G pi-dev= if needed. TRAMP executes remote commands using the guest's installed tools; verify editing on the target host.
=sandbox= attaches to one tmux session per VM, creating it with =pi --continue= in =/context= if needed. Detach with =Ctrl-b= then =d=; pi can continue working while the VM runs. Restarting the VM stops tmux and pi; a later connection resumes the latest saved session for =/context=, not an interrupted process. =/resume= can open a session from another directory, but after a VM restart =--continue= still selects the latest session for =/context=. The command does not start the VM. It advertises =TERM=xterm-256color= to SSH regardless of the calling terminal; terminals that differ from xterm may have display or key issues. Terminal key behavior and survival after SSH logout require runtime verification. The guest tmux configuration enables =set-clipboard on= so pi can copy through OSC 52 to a supporting terminal's clipboard. The user reports this works on the laptop. Any program able to write to a tmux pane can also set the attached terminal's clipboard; this does not itself grant clipboard read access.
Upstream creates a missing =/var/lib/microvms/<name>/context.img= on first start using the declared size. The image is a sparse ext4 filesystem and =contextSizeMiB= does not resize an existing image. If image creation fails partway through, upstream may leave a partial file which later starts will not recreate automatically; stop and investigate rather than deleting it blindly. Stop and inspect if the path already exists: it will be reused as a VM disk. The disk mounts at =/persist=, and its =context= directory is bind-mounted at =/context=; the workspace setup sets that directory's ownership and mode to =developer:users= and 0700. Do not reuse a host filesystem, another VM's image, or any old container state. The instance service requests umask 0077 for newly created images, but verify image and parent permissions; existing files are not chmodded. All upstream-managed VMs use the host =microvm= account, so other VMs and host administrators are not isolated from these image files by file ownership alone.
The image also stores an SSH host private key in =/persist/ssh-host-keys=, outside the developer-writable workspace and protected by root-owned directories. Verify that the image is mounted and ownership is correct before placing secrets in it. Back up the image with the VM stopped or using a consistent snapshot, and test restoration; do not mount it read-write on host and guest simultaneously. Deleting the declaration is not a data cleanup or migration procedure. Existing nspawn and Podman data are not migrated or deleted.
Language runtimes and build tools are not explicitly installed by the guest baseline. Add required tools through =extraPackages= or offload builds to CI. Native compatibility remains to be implemented and verified.
* Optional shared SSH commit signing
This feature is experimental and has not yet been evaluated or tested on the laptop. It is disabled by default. When enabled, every =services.piSandboxes.instances= VM on that host gets the same signing identity; there is no per-VM opt-out or enrollment. The host must also import the agenix NixOS module and have =age.identityPaths= capable of decrypting the encrypted key.
#+begin_src nix
services.piSandboxes.signing = {
enable = true;
encryptedKeyFile = ./secrets/pi-sandbox-signing.age;
# The public key is not secret; omit the comment.
publicKey = "ssh-ed25519 AAAA...";
name = "Example Name";
email = "name@example.com";
};
#+end_src
The public key must be a single Ed25519 key without a comment; trailing spaces or tabs are accepted and removed, including the trailing space emitted by =ssh-keygen -C ""=. The email must be one ordinary email address, without SSH allowed-signers patterns. Only the encrypted private key and its public key belong in a Git-backed flake or the Nix store. Generate a dedicated, passphrase-free signing key locally in a private temporary directory outside the repository and Nix store, encrypt it to the laptop's agenix host recipient and a separately protected recovery recipient, and remove the plaintext temporary copy after verifying recovery. Do not reuse a login key or the VM host key. Register the public key as a *signing* key with each Git provider and associate the chosen email with the provider account. Keep recipient definitions and recovery procedures backed up. Never put plaintext key material, passphrases or credentials in a Nix expression, logs or command arguments.
Agenix decrypts on the host. Before each VM's virtiofsd starts, a root service checks that the decrypted, passphrase-free key matches the configured public key and stages only that key in a per-VM runtime directory. The runtime parent is root-only; the file is owned by host numeric UID 1000 so guest UID 1000 can read it through a dedicated read-only share. This also means a host user with UID 1000 could read it if host directory permissions are weakened. The share does not include the agenix directory or other secrets. Staged copies normally disappear when their service stops or the host reboots. Inspect failures rather than deleting existing paths blindly; a missing or mismatched key prevents the VM's virtiofsd from starting, and guest SSH requires the signing mount. The preparation service requires its instance directory to start root-owned and mode 0700; it changes the export root to 0755 only after staging the key, behind the root-only parent. Verify that systemd resets the runtime directory to 0700 on restart, and verify host UID mapping, runtime directory permissions, manual VM startup and failure behavior on the actual host before relying on this design.
Git defaults to signing commits with SSH; Jujutsu uses its SSH backend and =signing.behavior = "own"=, signing changes authored by its configured user when modified. Both use the configured default name and email and trust this key for that email via =/etc/pi-sandbox-allowed-signers=. These are defaults, not enforcement: user/repository configuration and command-line options can override them. Local verification does not establish trust in other keys. Check each provider's SSH signature and email rules independently.
Any guest developer process can read, copy, use or exfiltrate the private key. A compromise in one VM compromises the shared signing authority of every VM; a signature proves use of the key, not personal review or a particular VM. A read-only share does not prevent copying. To rotate or revoke, stop all affected VMs, revoke the old public key at providers, create and encrypt a new key, update ciphertext and public key together, activate the host and manually restart the VMs. Running guests retain their old staged copy until stopped; activation alone is not rotation. Investigate and remove old trust separately if historical verification policy requires it. Do not touch =context.img=, workspaces or pi credentials during deployment.
* Pi state and declarative configuration
The guest launches pi as =developer= with =PI_CODING_AGENT_DIR=/context/.pi/agent= and sessions under =/context/.pi/sessions=. On launch, a wrapper creates the private agent directory if needed and links =settings.json=, =AGENTS.md= and =extensions= individually to the declarative =/etc/pi/agent= baseline. The stable links follow NixOS generation changes; pi can create its lock files next to them, while the linked Nix store files are read-only. Authentication and other writable agent state remain in the sandbox, not the Nix store or the host user's home. The user reports that pi starts and authentication persists in the laptop VM; bundled instruction and extension discovery have not been separately confirmed.
The bundled =AGENTS.md= commit-message template contains an =Assisted-by=
link to the exact sandbox Git revision used to build it. The standalone
=pi-agent-config= package and both exported guest-module paths receive the
sandbox flake's own =self.rev=, not the consuming laptop's revision. Builds
using dirty or unrevisioned sandbox sources fail with a clear error instead of
claiming a misleading commit link. Commit the sandbox changes before building;
Git-backed flakes also require new files to be tracked. Updating the laptop's
pinned sandbox input and rebuilding the host does not restart running VMs;
restart each affected VM to load the updated generated instructions.
The wrapper never overwrites existing files or links. If a managed name already exists with a different target, or either agent-directory path is a symlink, it exits with an error; inspect and migrate the conflict manually. Existing credentials and sessions are not deleted. Directories are created with mode 0700; an existing agent directory must already be owned by developer and have mode 0700. The wrapper runs only as guest UID 1000 and is not a privileged initialization mechanism. It sets its own agent directory explicitly, even if the environment was changed.
The global settings file remains Nix-managed. Pi's UI may report errors when trying to save settings and may change values in memory for the current session. A developer-owned directory also permits its owner or code running as that user to replace the links; trusted project settings, CLI arguments and alternate agent directories can override global defaults. This is configuration management, not an enforcement or security boundary. Credentials remain readable by pi and code running as developer.
* Maintenance
When integrating this flake into the laptop, make =sandbox.inputs.nixpkgs.follows = "nixpkgs"= to keep package sources aligned. Matching revisions alone do not guarantee shared store paths: package configuration and derivations must also match. =pkgs/pi.nix= maintains a local derivation based on upstream's pinned packaging process, including its schema workspace. Review it against upstream when updating pi; the derivation and nixpkgs changes need evaluation and build validation.
Run static checks directly from the development shell:
#+begin_src sh
nix develop --command bash -c '
for file in flake.nix modules/*.nix pkgs/*.nix pkgs/derivations/*.nix; do
nix-instantiate --parse "$file" >/dev/null || exit
done
nixfmt --check flake.nix modules/*.nix pkgs/*.nix pkgs/derivations/*.nix &&
jq empty agent/settings.json && git diff --check
'
#+end_src
* Tangled CI
=.tangled/workflows/build.yml= runs on pushes to =main= and pull requests targeting =main= using the server's NixOS microVM. It uses =nix-fast-build= from this flake's development shell to build every package in =packages.x86_64-linux=, including newly added outputs; aliases may resolve to the same derivation. It fails on an unexpected runner architecture. Other advertised systems need separate compatible runners and are not covered by this workflow.
On a =main= push, =nix-fast-build= also publishes package output closures to the server's =main= Attic cache, including substituted and upstream-signed paths. A missing publisher configuration or failed upload fails the run. Pull requests and manual runs build without explicit main-cache publication. Native private-repository caching is separate and best-effort. Explicit publication requires the server's trusted microVM guest publisher, Attic bridge and =main= cache. The user reports a successful live Tangled sandbox build and end-to-end Attic flow; this does not separately establish every PR, manual-run or denial scenario. The server grants every trusted standalone microVM run a short-lived main-cache push credential, even when this workflow does not use it; access control is enforced by the server, not the workflow's event check. No token is stored in this repository. A failed run can leave partial uploads.
The new microvm flake input must be added to =flake.lock= by the operator before publishing this sandbox revision, then the laptop's sandbox input must be updated. Git flakes omit untracked files from their source snapshot; track new laptop configuration files before evaluation or builds. The former OCI image, launcher and archive-publishing workflow have been retired; existing images and volumes are left untouched.