A user-data change replaces the instance (user_data_replace_on_change = true, infra/compute.tf), and the root volume goes with it. That holds:
/opt/vibescrobble/data:identities.jsonl(the only record of who wrote a deleted account's scrobbles; its one copy is the daily S3 backup),groups.json(so a shared group link breaks),state.jsonanddenylist.txt.- The
caddy-datavolume, so the TLS certificate is re-issued on every replacement; Let's Encrypt's duplicate-certificate limit makes two replacements in a week a problem. - The
relay-datavolume, with the relay's cursors.
Today a replacement is a manual S3 restore of the identity log (infra/README.md, "Replacing the instance") and a loss of the rest. Suggested fix: a separate aws_ebs_volume mounted at /opt/vibescrobble/data and the docker volumes' root, attached by user data and kept across replacements (prevent_destroy), so a replacement is a reboot's worth of state loss. That also covers the burn store without a backup-and-restore path of its own (see 3mxyzaae7g22l). Needs a planned replacement to land, like any user-data change.
Found while answering #40 (PR 276).
Fixed on main by https://tangled.org/permadeath.com/vibescrobble.com/pulls/334: the data directory and docker volumes live on a kept EBS volume with daily snapshots.