mlf-lexicon-fetcher#

ATProto Lexicon fetcher with DNS resolution and HTTP client.

Resolves lexicon NSIDs to DIDs via DNS TXT records and fetches lexicon JSON from ATProto repositories.

Features#

DNS Resolution: Resolves NSIDs to DIDs using DNS TXT records (RFC spec)
HTTP Fetching: Fetches lexicons from ATProto repositories via XRPC
Pattern Matching: Supports wildcard patterns (.* and ._)
Network Optimization: Groups similar NSIDs to reduce HTTP requests
Lockfile Support: Fetches from known DIDs, bypassing DNS resolution
Testable: Mock DNS and HTTP clients for testing

Usage#

Basic Fetching#

use mlf_lexicon_fetcher::ProductionLexiconFetcher;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create a production fetcher (real DNS + HTTP)
    let fetcher = ProductionLexiconFetcher::production().await?;

    // Fetch a single lexicon with metadata
    let result = fetcher.fetch_with_metadata("app.bsky.feed.post").await?;

    for lexicon in result.lexicons {
        println!("NSID: {}", lexicon.nsid);
        println!("DID: {}", lexicon.did);
        println!("Lexicon: {}", serde_json::to_string_pretty(&lexicon.lexicon)?);
    }

    Ok(())
}

Pattern Matching#

// Fetch all lexicons under a namespace
let result = fetcher.fetch_with_metadata("app.bsky.feed.*").await?;
println!("Fetched {} lexicons", result.lexicons.len());

// Fetch direct children only
let result = fetcher.fetch_with_metadata("app.bsky._").await?;

Optimized Batch Fetching#

use mlf_lexicon_fetcher::ProductionLexiconFetcher;

// Fetch many NSIDs with automatic optimization
let nsids = vec![
    "app.bsky.actor.profile".to_string(),
    "app.bsky.actor.defs".to_string(),
    "app.bsky.feed.post".to_string(),
    "app.bsky.feed.like".to_string(),
];

// Automatically groups into patterns like "app.bsky.actor.*" and "app.bsky.feed.*"
let results = fetcher.fetch_many_optimized(&nsids).await?;

Lockfile Mode (Skip DNS)#

// When you already have the DID (e.g., from a lockfile)
let result = fetcher.fetch_from_did_with_metadata(
    "did:plc:abc123",
    "app.bsky.feed.post"
).await?;

Advanced: Pattern Optimization#

use std::collections::HashSet;
use mlf_lexicon_fetcher::optimize_fetch_patterns;

let nsids: HashSet<String> = vec![
    "app.bsky.actor.foo".to_string(),
    "app.bsky.actor.bar".to_string(),
    "app.bsky.feed.post".to_string(),
].into_iter().collect();

// Returns: ["app.bsky.actor.*", "app.bsky.feed.post"]
let optimized = optimize_fetch_patterns(&nsids);

API#

Main Types#

ProductionLexiconFetcher - Ready-to-use fetcher with real DNS and HTTP
LexiconFetcher<D, H> - Generic fetcher (for custom DNS/HTTP implementations)
FetchResult - Contains fetched lexicons with metadata (NSID, DID, JSON)

Key Methods#

fetch_with_metadata(nsid) - Fetch single/pattern, returns metadata
fetch_many(nsids) - Fetch multiple NSIDs sequentially
fetch_many_optimized(nsids) - Fetch multiple NSIDs with pattern optimization
fetch_from_did_with_metadata(did, nsid) - Skip DNS, fetch from known DID

Utilities#

optimize_fetch_patterns(nsids) - Group NSIDs into minimal patterns
parse_nsid(nsid) - Parse NSID into authority and name segments
construct_dns_name(authority, name) - Build DNS TXT record name

Testing#

Use mock implementations for testing:

use mlf_lexicon_fetcher::{LexiconFetcher, MockDnsResolver, MockHttpClient};

let mut dns = MockDnsResolver::new();
dns.add_record("app.bsky", "feed", "did:plc:test".to_string());

let mut http = MockHttpClient::new();
http.add_lexicon(
    "app.bsky.feed.post".to_string(),
    serde_json::json!({"lexicon": 1, "id": "app.bsky.feed.post"})
);

let fetcher = LexiconFetcher::new(dns, http);

License#

MIT

Configure Feed