S2: Use the VM's hardware config and the env file's bucket (ENG-468) #3

Merged

demi.dev 1w ago

targeting main from stoopkidddd/eng-468-s2-deploy-the-executor-and-connect-it-to-the-mill

Merged opened by and assigned to

demi.dev 1w ago targeting main from stoopkidddd/eng-468-s2-deploy-the-executor-and-connect-it-to-the-mill

Linear: ENG-468

Summary #

These are the repo changes the executor needs before the lead deploys it to spindle-exec-1. The deploy itself (token, env file, nixos-rebuild switch, reboot and LAN checks) happens on the VM and the mill, and its results go in the Linear issue.

executor/hardware-configuration.nix: replaces the placeholder with the VM's real hardware config.
- Taken from the output of nixos-generate-config --show-hardware-config: the initrd modules ahci xhci_pci virtio_pci virtio_blk, kvm-amd and swapDevices = [ ].
- Keeps the bootstrap's GRUB on /dev/vda, growPartition, the ttyS0 console, DHCP and qemuGuest.
- The root filesystem stays at /dev/disk/by-label/nixos with autoResize, rather than the generated by-UUID path. The bootstrap image labels it, so the label is stable.
executor/configuration.nix: the S3 bucket placeholder becomes "".
- The real bucket name (spindle-artifacts-<hash>) goes into /var/lib/secrets/spindle-executor.env as SPINDLE_ARTIFACT_STORES_S3_BUCKET, together with the AWS_* values piped from Railway.
- systemd's EnvironmentFile= overrides Environment=, so the env file's value wins.

Verify #

$ mise run check
/nix/store/bgyd1x38p1rna7774vq9ca1i7ihbj5rb-nixos-system-spindle-exec-1-26.05.20261031.fa13731.drv
check_exit=0
$ nix eval (selected)
{"bucket":"","envFile":"/var/lib/secrets/spindle-executor.env","root":"/dev/disk/by-label/nixos", kvm ⊇ ["kvm-amd"], initrd ⊇ ["ahci","xhci_pci","virtio_pci","virtio_blk"]}

The lane is local-only, and there's no CI yet.

🤖 Generated with Claude Code

Commits 1

S2: Use the VM's hardware config and the env file's bucket (ENG-468)

+9 -4

Comments

v0 1w ago

demi.dev 1w ago

Review round 1 (pneumo-reviewer): APPROVE #

Reviewed at bc0e095 (v0). The change touches 2 files and stays within scope item 1, plus the bucket change that item 3 needs. There are no FIX findings.

Findings #

FILE (often 1 × bad 2 = 2) executor/configuration.nix:25: the comment "the env file overrides this" holds for the systemd unit (EnvironmentFile= overrides Environment=), but not for the module's spindle shell wrapper (nix/modules/spindle.nix:1135-1136). The wrapper sources the env file and then exports Environment, so an empty bucket wins there. Running spindle run by hand on the VM would fail with SPINDLE_MILL_ARTIFACT_STORE="s3" is not configured. The systemd service and the admin subcommands are unaffected. Filed as ENG-482, with a runbook note for ENG-471.

Spec and DoD checks #

Hardware config vs the VM's generated file:
- kept: the qemu-guest import, the initrd modules ahci xhci_pci virtio_pci virtio_blk, kvm-amd and swapDevices = [ ];
- dropped: the empty lists;
- changed: the root disk is matched by label instead of UUID, and hostPlatform is removed because the flake already sets system.
Matches the bootstrap: nixosConfigurations.base and executor evaluate identically for root (by-label, ext4, autoResize, growfs), GRUB /dev/vda, growPartition, ttyS0, systemd initrd, the initrd module set, swap, DHCP and qemuGuest.
Bucket "": the rendered unit has Environment=SPINDLE_ARTIFACT_STORES_S3_BUCKET= and EnvironmentFile=/var/lib/secrets/spindle-executor.env (no - prefix). No assertion fires, and spindle's config validation has no bucket check. If the bucket were missing from the env file, NewStores would skip S3 and the server would fail fast. It would never silently use the wrong bucket.
region = "": falls back to AWS_REGION from the env file.
DoD: hadolint and shellcheck clean; mise run check exits 0 (drv bgyd1x38…, same as the lead's); the mill's docker build gives the same image ID; no secrets in the diff.

Summary #

Verify #

Versions

Review round 1 (pneumo-reviewer): APPROVE #

Findings #

Spec and DoD checks #