desertthunder.dev
a love letter to tangled (android, iOS, and a search API)
1---
2title: Search
3updated: 2026-03-26
4---
5
6Twisted search is now operationally centered on PostgreSQL, Tap ingest, and a
7small set of rebuild tools.
8
9## Current State
10
11- primary storage: PostgreSQL
12- local default URL: `postgresql://localhost/${USER}_dev?sslmode=disable`
13- production deploy target: `docker-compose.prod.yaml` on a VPS or Coolify host
14- legacy fallback: local SQLite behind `--local`
15- `llama-embeddings` is deployed only as future embedding groundwork
16
17## Goals
18
19- keep search fresh through Tap ingest and targeted backfill
20- preserve the current `/search` API contract
21- make local development and production use the same database family
22- keep rebuild and recovery workflows simple enough to rehearse
23
24## Keyword Search
25
26Implemented with PostgreSQL full-text search:
27
28- weighted `tsvector` over title, author handle, repo name, summary, body, tags
29- `websearch_to_tsquery('simple', ...)`
30- `ts_rank_cd`
31- `ts_headline`
32
33Result scores and snippets may differ from the old SQLite FTS5 implementation.
34
35## Ingest Model
36
371. Tap is the authoritative indexing path.
382. Read-through indexing fills misses from detail fetches.
393. JetStream powers only the bounded activity cache.
404. `backfill`, `enrich`, and `reindex` rebuild the serving dataset.
41
42## Operational Rules
43
44- use explicit collection allowlists in production
45- do not import Turso data as the default migration path
46- treat PostgreSQL backups and restore drills as part of normal operations
47- keep the SQLite path only until the PostgreSQL rollout is fully bedded in
48
49## Next Search Work
50
51- synonym expansion
52- stemming and better tokenizer choices
53- field weight tuning from real queries
54- recency boosts
55- relevance fixtures that assert behavior, not exact score strings