# Working with the GhostOff codebase

This doc describes the architecture of the migration tool and how the pieces fit together.

## Entry point

`src/index.ts` is the CLI entry point. It:

1. Loads configuration from `.env` / CLI flags (`src/config.ts`).
2. Loads the previous run state (`src/state.ts`).
3. Authentices with the atproto PDS (`src/atproto.ts`).
4. Fetches all public posts from Ghost (`src/ghost.ts`).
5. For each post, downloads images, builds blocks, validates the document, writes records, and saves state.

## Configuration

`src/config.ts` uses `commander` and `dotenv`. Any CLI flag overrides its corresponding environment variable. Required values are validated eagerly; missing values throw before any network calls are made.

## Ghost client

`src/ghost.ts` is a thin wrapper around the Ghost Content API. It fetches `/settings` and paginates `/posts` with `include=authors,tags&formats=html,plaintext&limit=100`.

## Asset pipeline

`src/assets.ts`:

- Resolves relative image URLs against the Ghost base URL.
- Downloads images to `ghostoff-export/assets/` with a SHA-256 hash filename, skipping already-cached files.
- Uses `sharp` to resize/compress images over ~900 KB.
- Uploads processed images via the raw `com.atproto.repo.uploadBlob` XRPC endpoint.
- Blob references are cached in the state file so reruns don't re-upload identical images.

## HTML transformation

`src/html-to-offprint.ts` parses each post's HTML with `linkedom` and converts the body into an `app.offprint.content` item array. Because `linkedom`'s `parseHTML()` treats fragments oddly, the HTML is wrapped in `<!DOCTYPE html><html><body>...</body></html>` before parsing.

`src/facets.ts` handles inline formatting. It recursively walks inline nodes, builds plaintext segments, and computes UTF-8 byte-offset facets for bold, italic, links, code, etc.

## atproto layer

`src/atproto.ts` wraps:

- PasswordSession login/resume.
- Blob upload (raw `fetch` because atcute's client is JSON-oriented).
- `com.atproto.repo.createRecord` and `com.atproto.repo.putRecord` for documents and articles.
- Runtime validation of `site.standard.document` against `@atcute/standard-site`.

## State and idempotency

`src/state.ts` reads/writes `ghostoff-state.json`. It maps each Ghost post `uuid` to the created document/article AT-URIs and CIDs. On reruns, existing records are updated via `putRecord` with `swapRecord` set to the old CID.

The state file also caches uploaded image blob references so images are only uploaded once.

## Rate limiting

`src/rate-limit.ts` provides small helpers for parsing `RateLimit-*` and `Retry-After` headers, plus exponential backoff. XRPC calls use `@atcute/client`'s `retryFetchHandler`. Blob uploads have a manual retry loop.

## Type checking and conventions

```bash
npm run typecheck
```

The project uses strict TypeScript, ESM, and Node.js's native `fetch`. The `DOM` lib is included because the tool manipulates HTML elements via `linkedom`.

When adding new block conversions, keep the type definitions in `src/html-to-offprint.ts` close to the conversion logic, and make sure the emitted object has a `$type` discriminator matching the real Offprint lexicon.