prefect server in zig

webserver segfault (exit 139) on 2026-09-26, not reproduced #1

Open zzstoatzz.io
Open opened by zzstoatzz.io

Prod's webserver container exited with code 139 (SIGSEGV) at 2026-09-26 07:33:56Z, 2 s after starting. That was its 6th restart; it has been stable since. The last line it logged was a Postgres FK violation: update or delete on table "flow" violates foreign key constraint "fk_flow_run__flow_id". At that time a maintenance session was deleting flows (musician-studio cleanup).

Tried on 2026-09-28, no crash:

  • single DELETE /flows/{id} hitting that FK, in Debug and ReleaseFast builds on Postgres (macOS);
  • flows/bulk_delete over 12 FK-failing flows;
  • 40 concurrent single and bulk deletes, 127 FK failures in all.

Since c0992d4, deleting a flow deletes its runs in a transaction, so that FK error no longer happens on this path. The crash's cause is still unknown. Next steps if it recurs: capture kubectl logs --previous, and try the Linux/musl ReleaseFast image under load.

Sign up or Sign in to add to the discussion.