Evaluate whether a shared ATProto query source can supply Typeahead's corpus analytics while preserving current stats semantics. This is a bounded investigation, not a decision to replace the actor store or Tantivy, and no removable server or cost saving has been demonstrated.
Motivation: Mizumi turns lexicons into Arrow schemas and queries Hydrant events through DataFusion. Its current implementation is in-memory event history, not a ready-made current-state or blob-backed service. Original demonstration.
Existing work and source findings #
Inspected local revision 2f32d1e; deployment status was not reverified for this issue.
- #34 already owns request/traffic statistics leaving Turso.
docs/plans/stats-off-turso.mdselects D1 and records implemented dual-write steps. This issue must not restart that decision. src/stats-store.tssends statistics writes to Turso and optionally D1.src/enrichment.tsmaterializes metrics, snapshots, actor deltas and traffic histories into KV.src/handlers/stats.tsalso reads source lists and a recent tail from the database on edge-cache misses. It is not a KV-only page.- Search counts, cache hits, latency, HTTP status and caller traffic originate in this application. An ATProto record stream cannot reconstruct them.
- Actor totals, handle/avatar coverage and exclusions depend on Typeahead's discovery, identity enrichment, account state and moderation policy. Counting profile records is not an equivalent definition. Labeler data and appview aggregates are separate inputs.
Bounded investigation #
- Pick one corpus measure, such as actor/profile coverage, and document its exact population, exclusions and freshness contract from the existing code.
- Check the existing Stream archive and store-change-log work (#31, #40–45) before introducing another collector. Identify which identity/account/moderation inputs a shared source actually provides.
- Compare a fixed source snapshot against Typeahead's authoritative result, including updates, deletes, deactivation, reactivation and missing enrichment. Do not use author timestamps as revision order.
- If SQL helps, materialize results off the serving path. Preserve bounded request work, retention and the existing page contract.
Finish with measured discrepancies, resource cost, and the exact job/table/code path that could be removed. A valid outcome is that the existing D1/embedded-store work already covers the need and no additional service is justified. Keep the active shadow-slot migration intact.