Monorepo for Tangled tangled.org

AppView pull ingester skips specific CLI-created pull records #576

Closed onev.cat
Closed opened by onev.cat

Hi! We are seeing a case where some CLI-created sh.tangled.repo.pull records exist in PDS/Constellation, but AppView never assigns them a /pulls/N page. Could someone with AppView logs/DB access help check what the pull ingester did for these records?

Repository:

Records that are present in PDS/Constellation but not projected by AppView:

  • at://did:plc:kl2ejrmz5zmxnno3ll4luz76/sh.tangled.repo.pull/3mm2mljaqtw22
  • at://did:plc:kl2ejrmz5zmxnno3ll4luz76/sh.tangled.repo.pull/3mm2ojezxm622

Both records target the same repo DID and use branch-based pull requests:

  • target.repo = did:plc:vglueuiqgjehmkqpjloszu2q
  • target.branch = main
  • source.repo = did:plc:vglueuiqgjehmkqpjloszu2q
  • first source branch: fix/appview-number-resolution
  • second source branch: e2e-identifier-fix-copy-20260518-0028

The patch blob for 3mm2mljaqtw22 is:

  • CID: bafkreiairtrk7ow634jlwe7ykioy5gckn2nomlupbhsz46lszodwvvb7zi
  • size: 13123
  • MIME: application/gzip

We verified from the same PDS endpoint that the blob downloads with HTTP 200, content-length matches, gzip -t passes, and the expanded patch applies cleanly with git apply --check against main.

For comparison, these CLI-created pull records on the same repo did project correctly:

Also, a web-created PR for the same patch appears as:

That web-created PR uses the same patch blob CID as 3mm2mljaqtw22, but it was created through AppView, so it may have gone through the direct web create path that writes AppView DB rows immediately rather than relying on the firehose pull ingester.

Could you please check AppView logs for handler=ingestPull around these rkeys?

  • did=did:plc:kl2ejrmz5zmxnno3ll4luz76 rkey=3mm2mljaqtw22
  • did=did:plc:kl2ejrmz5zmxnno3ll4luz76 rkey=3mm2ojezxm622

The main question is whether ingestPull returned an error and advanced the cursor anyway. If so, the error message should tell us whether the failure is in blob download, gzip extraction, patch parsing, validation, or DB insertion.

Thanks!

thanks for opening this issue, seems like a bug on our end, will investigate.

hitting this from did:plc:kg5hlfyrdjmtiyymo57n7dpg today (2026-05-27). same shape as the original report.

filed two upstream PRs to tangled.org/@tangled.org/core via patch-mode (sh.tangled.repo.pull, gzipped multi-commit git format-patch blob, source omitted):

  • PR 1 rkey 3mmtrv25anv22 (2 commits, 9.5 KB gz, blob bafkreicpqombqzvggh7vx32dfihwfecd7rypyr2x4otdcaglgja37ij3gy) — landed in ~60s as tangled.org/tangled.org/core/pulls/1614.
  • PR 2 rkey 3mmtrzllv7j22 (11 commits, 22 KB gz, blob bafkreidzjxtfxe4t5f4cjiyqb72vgtmutcowkog5tbi76kt5ilivftscgy) — silent skip on ingest, never assigned a pull number.

verified on our side:

  • record present on PDS (goat record ls shows both rkeys)
  • blob fetchable via com.atproto.sync.getBlob — HTTP 200, sha256 matches, valid gzip, ~95 KB uncompressed mbox with 11 well-formed format-patch entries
  • patchutil.ExtractPatches on the literal mbox parses all 11 patches cleanly (sha + title + fragments correct), so it's not a parse failure
  • appview/validator/pull.go::ValidatePull is permissive for format-patch (skips title/body checks); record satisfies its required fields
  • same DID landed PR 1 fine 3 minutes before PR 2; other DIDs' pulls have ingested in the same window (e.g., #1615 from a different DID)

four probes attempted, all silent-dropped:

  1. original create at 15:09:21Z (rkey 3mmtrzllv7j22)
  2. goat record update at 19:25:43Z (same rkey, new createdAt)
  3. second goat record update at 19:49:38Z (same rkey, new createdAt)
  4. delete + republish under fresh rkey 3mmuc2vdmhi22 at ~19:57Z, blob CID unchanged (content-addressed re-upload to the same CID)

so the "fresh rkey" workaround from this thread did not unstick it for us — five separate Jetstream events for what is effectively the same content, all consumed by the cursor without producing an appview row.

appview/ingester.go reads cleanly on our local clone; we cannot see your deployed Warn logs. if failed to ingest record, skipping is firing for our rkeys, the message body would narrow this. happy to publish any other diagnostic record / mint a no-op variant if it helps.

(PR 1 is live and standalone-mergeable: https://tangled.org/tangled.org/core/pulls/1614.)

this should be fixed now!

Sign up or Sign in to add to the discussion.