A minimal AT Protocol Personal Data Server written in JavaScript.
atproto pds

content-encoding: gzip is preserved on authenticated app.bsky.* proxy responses even though the body is already uncompressed #3

open opened by alice.mosphere.at edited

I ran https://tangled.org/alice.mosphere.at/atproto-smoke against pds.js and it ran into this issue. With apologies for the LLMese below, I double-checked that this is a real issue.


Authenticated app.bsky.* endpoints appear to return HTTP 200 with content-encoding: gzip, but the response body bytes are plain JSON rather than gzip data.

This reproduces on a Node deployment behind Caddy, but also reproduces directly against the origin, so it does not look like a TLS/reverse-proxy issue.

Observed behavior

  • Unauthenticated requests to the same endpoints return normal JSON and do not show this problem.
  • Authenticated requests fail in clients that honor Content-Encoding:
    • curl --compressed fails with incorrect header check
    • Node fetch() fails with Z_DATA_ERROR incorrect header check
    • browsers show ERR_CONTENT_DECODING_FAILED
  • In our browser smoke test, this breaks authenticated author-feed/profile flows, so a freshly created post never renders on the author profile.

Affected authenticated endpoints

  • /xrpc/app.bsky.unspecced.getConfig
  • /xrpc/app.bsky.feed.getAuthorFeed
  • /xrpc/app.bsky.actor.getSuggestions
  • /xrpc/app.bsky.graph.getSuggestedFollowsByActor

Direct repro

Authenticated origin requests fail even when hitting the Node server directly:

curl --compressed -H "Authorization: Bearer "
'http://127.0.0.1:7106/xrpc/app.bsky.feed.getAuthorFeed?actor=did:plc:kkplfohmly3evn5t6m2vm6ta&limit=5'

That returns HTTP 200 with content-encoding: gzip, but decompression fails.

I also inspected the raw origin response bytes for an authenticated app.bsky.unspecced.getConfig call. The header says content-encoding: gzip, but the body starts with plain JSON bytes like:

{"checkEmailConfirme...

Likely cause

This looks consistent with the proxy path rebuilding a response from an already-decoded upstream body while preserving the original Content-Encoding header.

Relevant code:

  • packages/core/src/pds.js:193
  • packages/core/src/pds.js:184
  • packages/node/src/index.js:269
  • packages/node/src/index.js:273

In particular:

  • proxyToService() copies response.headers into a new Headers
  • then returns new Response(response.body, { headers: responseHeaders, ... })
  • the Node adapter forwards those headers/body verbatim to the client

If the upstream fetch runtime has already decoded the gzip body but left Content-Encoding: gzip in response.headers, that would produce exactly this mismatch.

sign up or login to add to the discussion
Labels

None yet.

assignee

None yet.

Participants 1
AT URI
at://did:plc:by3jhwdqgbtrcc7q4tkkv3cf/sh.tangled.repo.issue/3mhgf2racrc22