Migrating content using the Ghost API to Offprint / Standard Site on atproto
ghostoff docs next-steps.md
3.6 kB · 43 lines
Markdown
at main

What's next for GhostOff #

This is a first working version. The items below are roughly ordered from smaller polish to larger additions.

Content conversion #

  • Better embed support. Bluesky embeds are already converted to block.webEmbed and, when Ghost supplies data-bluesky-uri/data-bluesky-cid, to block.blueskyPost; other embeds (YouTube iframes, GitHub cards, generic oEmbeds) are currently skipped or fall back to text. Extract href from more iframe/src patterns and consider fetching OpenGraph metadata to populate webBookmark blocks.
  • Picture and srcset handling. Ghost can serve responsive images via <picture> and srcset. We currently only use the first <img src>. Prefer the largest available source for upload.
  • Unsupported image formats. .ico files and some SVGs fail in sharp. Either skip them cleanly, convert SVGs to PNG, or detect icons and ignore them entirely.
  • Code block language detection. Ghost stores the language in the <code class="language-foo"> attribute. We extract it, but Ghost sometimes uses different class names; broaden the regex.
  • Reduce warning noise. Block-level elements that contain mixed inline content sometimes produce unsupported element warnings. Some of these are inside paragraphs and should be handled inline rather than as block fallbacks.

CLI and UX #

  • Selective migration. Add flags to filter by tag, date range, or slug, and to limit the number of posts processed.
  • Progress output. Add a progress bar or summary report at the end.
  • JSON/NDJSON export. Allow exporting records to disk without uploading, for manual inspection or external publishing.
  • Resume after failure. If the run stops midway, restarting it should pick up from where it left off based on the state file.
  • Configurable image limits. Expose the blob size target and resize dimensions as CLI flags.

Reliability #

  • Full retry policy for Ghost. The Ghost API can also rate-limit; apply the same retry/backoff logic to src/ghost.ts.
  • Retry failed uploads with state awareness. If a blob upload fails temporarily, don't mark the image as processed in the state file.
  • Validation of Offprint blocks. The atcute validator only validates site.standard.document because it doesn't know about app.offprint schemas. Pull the Offprint schemas from Lexicon Garden and validate app.offprint.content blocks, or at least check byte offsets and required fields in tests.

Tests #

  • Add facet generation tests for nested inline tags and multi-byte UTF-8 characters.
  • Add asset pipeline tests with mocked fetch responses.
  • Add an idempotency test: a second run should call putRecord, not createRecord.

Auth and deployment #

  • OAuth support. App passwords work for one-off migrations, but OAuth would be safer for interactive use and for users on custom PDSs.
  • GitHub Action. Provide a workflow that runs the migration on a schedule so a Ghost publication can stay in sync with Offprint.
  • Docker image. Make the tool runnable without a local Node installation.

Other platforms #

  • Standard.site beyond Offprint. The records already use Standard.site, so they should be readable by Leaflet, pckt, and other readers. Add a mode that skips the Offprint article wrapper for users who only want Standard.site documents.
  • Export to other formats. Use the same converter to emit Markdown, plain HTML, or static-site-friendly JSON.