hydrant by ptr.pet · Tangled

very fast at protocol indexer with flexible filtering, xrpc queries, cursor-backed event stream, and more, built on fjall

rust fjall at-protocol atproto indexer

Commits on test-13-fix

wip

committed 6mo ago

[tests] use plc.klbr.net instead of plc.wtf

committed 6mo ago

[ingest] change the unbounded channels to be bounded

committed 6mo ago

[backfill] dont hold store / mst tree over db commit, drop early when we can

committed 6mo ago

[lib] expose backlinks properly, add dids field support

committed 6mo ago

[ingest,stream] dont read from db, use live event

committed 6mo ago

[db] compact manually if we pruned enough events

committed 6mo ago

[deps] update fjall and friends

committed 6mo ago

[docs] update verbiage

committed 6mo ago

[db] make count tracking more durable

committed 6mo ago

[backfill] fix count tracking on resync

committed 6mo ago

[docs] point to docs

committed 6mo ago

[flake] remove conflicting rust deps from devshell override

committed 6mo ago

[docs] update verbiage

committed 6mo ago

[docs] move title headers into title frontmatter

committed 6mo ago

[docs] fix rewrites

committed 6mo ago

[db] improve ephemeral task, add test for the relay ttl to the gc test

committed 6mo ago

[db] print pruned events log at info level

committed 6mo ago

[docs] update links in docs readme

committed 6mo ago

[docs] add building and deployment

committed 6mo ago

[docs] add request body tables, restore config descriptions from

committed 6mo ago

[docs] convert nested lists to headings in api

committed 6mo ago

[flake] set VERBIAGE_TITLE=hydrant, update verbiage input

committed 6mo ago

[ingest] store resync buffer commits with named tags

committed 6mo ago

[docs] update verbiage to latest (redirect

committed 6mo ago

[docs] initial documentation + add verbiage to

committed 6mo ago

[firehose] periodically retry offline hosts (every 30 minutes)

committed 6mo ago

[firehose,lib,api] add is_source_running, shortcircuit requestCrawl if source is already running to prevent task restart

committed 6mo ago

[lib,api] add new host limit for hosts added via requestCrawl

committed 6mo ago

[config] enable ephemeral mode by default in relay mode

committed 6mo ago

[docs] remove gitnexus stuff from agents.md

committed 6mo ago

[all] add indexer_stream feature to decouple the events stream from the indexer

committed 6mo ago

[all] simplify code depending on block storage being enabled

committed 6mo ago

[all] dont thread ephemeral bool through the code, use from app state

committed 6mo ago

[indexer] add config option for only indexing links

committed 6mo ago

[ingest,lib,api] add glob rules for pds tiers, TRUSTED_HOSTS -> TIER_RULES

committed 6mo ago

[api] make sure cids are serialized as text in listRepos

committed 6mo ago

[api] fix index page to show indexer if compiled with indexer

committed 6mo ago

[api] also add _health

committed 6mo ago

[firehose] jitter connection startup when we are spawning persisted / configged firehoses

committed 6mo ago

[firehose] use cancel token instead of aborting task, fix bug where upserted tasks would remove the new task from the map

committed 6mo ago

[ingest] make throttling less strict on firehose errors to better match ref behaviour

committed 6mo ago

[lib,api] fix iterating through repos and getting repo info in relay mode

committed 6mo ago

[docs] remove unnecessary method from safe to proxy endpoints

committed 6mo ago

[api] add index file

committed 6mo ago

[build] add rust toolchain toml and update flake to use nci for devshell

committed 6mo ago

[lib] re-export public api deps in deps

committed 6mo ago

[build] add dockerfile

committed 6mo ago

[docs] better instructions for hosting in the readme

committed 6mo ago

[ingest,lib,api] implement host status tracking for pds' and enforce account limit

committed 6mo ago

[docs] update readme

committed 6mo ago

[api] actually reject banned pds in requestCrawl

committed 7mo ago

[firehose] only reset backoff to 0 if the stream has been healthy for over a minute

committed 7mo ago

[firehose] send pings to hosts

committed 7mo ago

[firehose] add jitter on throttles / backoffs

committed 7mo ago

[firehose] print better stream closed errors

committed 7mo ago

[firehose] treat every upgrade error as throttle reason

committed 7mo ago

[firehose] 4xx is also throttle worthy now, also handle rustls AlertReceived better

committed 7mo ago

[firehose,crawler] treat 1xx,2xx,3xx,5xx status codes as server error (aside from crawler, where 500 and 2xx is allowed)

committed 7mo ago

[firehose] respond to pings, better treat close messages

committed 7mo ago