hydrant by ptr.pet · Tangled

very fast at protocol indexer with flexible filtering, xrpc queries, cursor-backed event stream, and more, built on fjall

rust fjall at-protocol atproto indexer

Commits on fix-counts

[db] refactor how repo counts are handled

committed 4mo ago

[backfill/sparse] optimize, make auto the default

committed 4mo ago

[backfill/sparse] increase chunk size to 100

committed 4mo ago

[backfill] initial sparse repo backfill implementation

committed 4mo ago

[relay] make repos start as synced instead of 'backfilling'

committed 4mo ago

[ingest] default tooBig to false when desering commits

committed 4mo ago

[relay] introduce a negative cache for bad did/source-host pairs so we dont have to refresh did doc every single time, skip getRepoStatus check

committed 4mo ago

[ingest] add more stats around signature verifs and message handling

committed 4mo ago

[ingest] add stats for relay worker shards

committed 4mo ago

[ingest] add perf stats tracking for firehose connections

committed 4mo ago

[relay] improve how host connections with stale cursors get treated, hopefully fix a nasty bug with message deduping that caused connections to stall, more debugging

committed 4mo ago

[api] implement get record by uri

committed 4mo ago

[api] add resolveMiniDoc aliases

committed 4mo ago

[scripts] add scripts for validating failing pds' against another relay and general host status comparisons

committed 4mo ago

[lib,api] include pds metadata in /firehouse/source(s) if the host is a pds

committed 5mo ago

[lib,api] return all known sources from /firehose/sources, allow filtering, allow querying single host using /firehose/source

committed 5mo ago

[lib] user-keyspace feature for letting users open keyspaces in hydrant db

committed 5mo ago

[stream] improve the way we handle consumer backpressure, remove unnecessary fixed pause

committed 5mo ago

[jetstream] add live cutover like the rest of the streams

committed 5mo ago

[jetstream] stage backfill commits also

committed 5mo ago

[jetstream] introduce jetstream compatible endpoint for indexer and relay

committed 5mo ago

[relay] fix pds account counts to only those known by hydrant, make them consistent

committed 5mo ago

[tests] improve test times, split and rename some tests

committed 5mo ago

[build] fix nix build

committed 5mo ago

[tests] use cargo artifact paths

committed 5mo ago

[ingest] add sharded mailbox

committed 5mo ago

[hydrant] clippy

committed 5mo ago

[db] use drop_range for ephemeral gc

committed 5mo ago

[api] websocket pong & better ping, round

committed 5mo ago

[lib,api] chunk streaming on replay, add a live buffer, chunk ephmeral gc deletes

committed 5mo ago

Binding multiple listen

committed 5mo ago

[tests] use plc.klbr.net instead of plc.wtf

committed 6mo ago

[ingest] change the unbounded channels to be bounded

committed 6mo ago

[backfill] dont hold store / mst tree over db commit, drop early when we can

committed 6mo ago

[lib] expose backlinks properly, add dids field support

committed 6mo ago

[ingest,stream] dont read from db, use live event

committed 6mo ago

[db] compact manually if we pruned enough events

committed 6mo ago

[deps] update fjall and friends

committed 6mo ago

[docs] update verbiage

committed 6mo ago

[db] make count tracking more durable

committed 6mo ago

[backfill] fix count tracking on resync

committed 6mo ago

[docs] point to docs

committed 6mo ago

[flake] remove conflicting rust deps from devshell override

committed 6mo ago

[docs] update verbiage

committed 6mo ago

[docs] move title headers into title frontmatter

committed 6mo ago

[docs] fix rewrites

committed 6mo ago

[db] improve ephemeral task, add test for the relay ttl to the gc test

committed 6mo ago

[db] print pruned events log at info level

committed 6mo ago

[docs] update links in docs readme

committed 6mo ago

[docs] add building and deployment

committed 6mo ago

[docs] add request body tables, restore config descriptions from

committed 6mo ago

[docs] convert nested lists to headings in api

committed 6mo ago

[flake] set VERBIAGE_TITLE=hydrant, update verbiage input

committed 6mo ago

[ingest] store resync buffer commits with named tags

committed 6mo ago

[docs] update verbiage to latest (redirect

committed 6mo ago

[docs] initial documentation + add verbiage to

committed 6mo ago

[firehose] periodically retry offline hosts (every 30 minutes)

committed 6mo ago

[firehose,lib,api] add is_source_running, shortcircuit requestCrawl if source is already running to prevent task restart

committed 6mo ago

[lib,api] add new host limit for hosts added via requestCrawl

committed 6mo ago

[config] enable ephemeral mode by default in relay mode

committed 6mo ago