bates64.com / rfcs
forked from starhaven.dev/rfcs
RFCs for changes to Star Haven
fork atom
rfcs / text / 0001-appview.md
at appview 25 kB view raw view code
bates64.com add appview RFC
7cf7b832

Summary#

Build starhaven.dev as an atproto appview with a community forum and mod browser. Users sign in with any atproto account or sign up for a *.starhaven.dev handle via the Star Haven PDS. Forum threads, replies, and mod listings are atproto records stored in users' repos. The source lives at starhaven.dev/appview.

In this RFC, "end-users" refers to people who play mods (as opposed to modders who create them). Modders are also users of the site, but the distinction matters for the mod browser and patching sections.

Status quo#

Currently, Star Haven is used as a central hub for Paper Mario modding, except for when it is not and breaks off into smaller sub-communities (Modern Paper Mario Modding, WUA, NicTube and Lily's mods, etc.) or when modders migrate to larger platforms (Itch, or formerly romhacking.net).

Star Haven facilitates general chat, private chats, individual mod and game discussion, general modding assistance, mod release notifications and community events.

Some modders have the social capital to promote, release and discuss their mods in their own bespoke communities and have no need for a centralised modding hub. However, such modders are the exception rather than the rule. Initially, most modders bounced around various forums until landing in Star Haven's Discord server, where they continued to develop and release their mods.

Star Haven's Discord Server holds the largest repository of modding advice, through years of Discord chats and real-time assistance from experienced modders. Star Haven also acts as an aggregation service for modders to publish their creations to a wider audience with relative ease, providing either a thread or a bespoke channel dependent on the mod's popularity. On top of that, Star Haven aggregates and hosts modding tools, as well as similar help channels for said tools. Star Haven also uses Discord and Itch.io to host regular jam events, where modders create teams to create themed mods in a certain timeframe.

Modders use GitHub to collaboratively develop their mods, private Discord servers to facilitate development chats and voice-calls, and Itch.io to publish their mods. Offline, modders use a wide variety of tools such as Star Rod, Mamar, Unsimplifier and more.

For players, Star Haven provides an avenue to discuss mods, the Paper Mario games, ask for help on a per-mod or a general basis, or have a general chat. Additionally, a list of mods is available in the Discord, and the (currently broken) mod browser on starhaven.dev/mods.

The community is splintered across multiple technologies and platforms that do not neatly integrate with one another, limiting most modders' reach to a single medium-sized Discord server. That, and everyone quite dislikes Discord.

Motivation#

We add Star Haven Forums (built atop atproto) as a slower-paced complement to the Discord server, replace GitHub with Tangled, and add a mod browser at starhaven.dev. Discord remains the place for real-time chat and voice calls - the forums are for longer-form discussion, modding guides, and announcements that benefit from being permanent and searchable.

Critically, forums get important modding knowledge out of the walled garden that is Discord and onto the open web, where search engines can index it. Years of modding advice currently trapped in Discord messages becomes discoverable by anyone with a web browser - no Discord account required. The best place for agreed-upon knowledge is docs.starhaven.dev, but updating that is out of scope for this RFC. The use of open-source, federated technology also allows other communities to fork the work for their own purposes.

A non-Discord mod browser allows for much greater modder control of how their mods are promoted and advertised, providing descriptions, updates and promotional material all on the same page. Integration with PM-DX (and further tools down the line) enhances the developer experience.

For end-users, a modern, professional mod browser removes the reliance on Discord for viewing, downloading and playing mods.

Why atproto?#

We already run a PDS at pds.starhaven.dev and a knot (git server) at knot.starhaven.dev. Most community members still use GitHub, and some Star Haven infrastructure (e.g. papermario-dx CI) still depends on it, but the aspiration is to self-host community things over time. The question is whether building on atproto is worth it versus a traditional forum with Discord OAuth or similar.

The core benefit is data ownership. Forum posts, mod listings, and profiles are atproto records stored in users' own repos. If starhaven.dev goes down or the maintainer disappears, users still have their data - they can take it to another appview or export it. With a traditional forum, all content lives in the site operator's database. The content may remain accessible, but users have no independent copy and no way to move it. For a small community maintained by volunteers, reducing the bus factor matters.

Beyond that:

  • One identity system - no new accounts to make. A *.starhaven.dev handle works on Bluesky, Tangled, and any future atproto service.
  • We don't have to maintain a user/password system. atproto OAuth handles it.
  • The PDS can support Discord OAuth via tranquil, so Discord users can sign up without learning what atproto is. They get an atproto identity for free.
  • Other communities can run the same appview software against their own PDS and get a fully independent instance with zero data migration needed.

Existing forum software (Discourse, phpBB) doesn't really fit the vibe we want, and we'd need to write a custom auth plugin for atproto OAuth anyway.

Explanation#

Migration from current site#

starhaven.dev is currently a SvelteKit site on GitHub Pages (star-haven/starhaven.dev). It has a homepage and a /mods page that renders a JSON file of mods with download links. The /mods page is broken (500 error). The appview replaces this entirely - we should not regress from the existing functionality, so the homepage and mod listing need to be carried over.

Migration is not about shutting Discord down. Forum-style channels (mod releases, guides, showcase) move to the appview over time, while real-time chat stays on Discord. Optionally, a #forums channel on Discord can post messages for new forum threads via webhook to help with early adoption, though this should be turned off once the forums have enough activity on their own.

How it works#

starhaven.dev is an atproto appview - it subscribes to the firehose, indexes records that match our custom lexicons, and serves a web frontend.

browser  -->  starhaven.dev (appview + web server)  -->  database
                     |  ^
                     |  |  firehose
                     v  |
               relay / PDS

You sign in with atproto OAuth. When you create a forum post, it writes a record to your PDS repo. The appview picks it up from the firehose and indexes it.

Auth#

Signing in#

/login asks for your handle (e.g. luigi.starhaven.dev or someone.bsky.social) and initiates atproto OAuth. Same flow as tangled.sh.

Creating an account#

Users who don't have an atproto account can create one on pds.starhaven.dev. /signup collects an email and a captcha, then calls the PDS admin API (com.atproto.server.createAccount) to create the account. The user picks a handle (*.starhaven.dev) and authenticates via passkey or Discord OAuth (supported by the PDS via tranquil). No passwords.

Routes#

/                               Homepage (community landing page)
/login                          atproto OAuth sign-in
/signup                         Create a *.starhaven.dev account

/mods                           Mod browser (browse/search/filter)
/mods/:game                     Mods for a specific game
/mods/:game/:slug               Mod page (description, media, downloads)
/mods/new                       Upload a new mod listing
/repo.json                      Homebrew App Store / Universal Updater compatible
                                endpoint for on-device mod browsers

/search                         Search across threads, mods, and profiles

/forums                         Forum index (categories/subcategories below)
/forums/:subcategory            Thread list for a subcategory, paginated
/forums/:subcategory/new        New thread form
/forums/thread/:id              Thread view, paginated (?page=n)

/:handle                        Actor profile (avatar, posts, mods)

Mods don't require a source repo - closed-source and legacy mods are welcome. The mod browser is git forge agnostic; modders can use GitHub, Tangled, or anything else. If a mod team happens to use Tangled, the appview can provide tighter integration (linking to repos, showing recent commits), but it is not required.

Lexicons#

Everything under dev.starhaven.*. Record types use key: "tid" unless noted. References to other records use at-uri strings; references that need to pin to an exact version (e.g. to prevent replying to a since-deleted revision) use com.atproto.repo.strongRef.

                        ┌──────────────────┐
                        │  actor.profile   │
                        │  key: self       │
                        └──────────────────┘

┌──────────────────┐    ┌──────────────────┐
│  forum.thread    │◄───│  forum.thread    │
│                  │    │     .post        │
│  title, tags     │    │                  │
│  subcategory     │    │  body            │
└──────────────────┘    └──────────────────┘

┌──────────────────┐    ┌──────────────────┐
│  mod.listing     │◄───│  mod.release     │
│                  │    │                  │
│  has: media[]    │    │  has: deliverable│
│       (PDS blob) │    │       (S3 URL)   │
└──────────────────┘    └──────────────────┘

dev.starhaven.actor.profile              key: "literal:self"
  displayName: string (maxGraphemes: 64)
  description: string (maxGraphemes: 256)
  pronouns: string (maxGraphemes: 20)

dev.starhaven.forum.thread
  title: string (required, maxGraphemes: 300)
  subcategory: string (required, maxLength: 64)
  tags: string[] (maxLength: 8, items maxLength: 64)
  createdAt: datetime (required)

dev.starhaven.forum.thread.post
  thread: at-uri (required)             → dev.starhaven.forum.thread
  body: string (required, maxGraphemes: 30000)
  createdAt: datetime (required)

  The "new thread" form creates both a thread and its initial post. If
  the firehose delivers them out of order, the UI may briefly show a
  thread without a body - this is acceptable.

dev.starhaven.mod.listing
  title: string (required, maxGraphemes: 300)
  slug: string (required, maxLength: 64) URL-safe, globally unique, e.g. "master-quest"
  description: string (required, maxGraphemes: 1000)
  details: string (maxGraphemes: 30000) extended description (markdown)
  game: string (required, maxLength: 32) fixed set, see below
  platform: string (required, maxLength: 32) fixed set, see below
  category: string (maxLength: 64)
  tags: string[] (maxLength: 8, items maxLength: 64)
  license: string (maxLength: 128)
  media: #mediaItem[] (maxLength: 16)   images and videos, ordered
  createdAt: datetime (required)

#mediaItem
  file: blob (required, accept: image/*, video/*, maxSize: 50MB)
  alt: string (maxGraphemes: 1000)      accessibility description

dev.starhaven.mod.release
  mod: strongRef (required)             → dev.starhaven.mod.listing
  version: string (required, maxLength: 32)
  changelog: string (maxGraphemes: 30000)
  deliverable: string (required, format: uri) URL in object store
  createdAt: datetime (required)

Valid game values: 64 (Paper Mario 64), ttyd (TTYD, GCN), ttyd-r (TTYD remake, Switch), spm (Super Paper Mario), ss (Sticker Star), cs (Color Splash), tok (Origami King). Valid platform values: n64, gcn, wii, 3ds, wiiu, switch.

The appview enforces that only the listing author (or contributors listed on the listing, if/when that field is added) can create mod.release records pointing at a given mod.listing.

Mod releases are separate records so they appear individually on the firehose and don't require rewriting the listing on every update. The deliverable URL points to the object store - the type of deliverable (patch file, zip, etc.) depends on the game (see Mod browser below). Media blobs (screenshots, videos) are stored on the PDS like any atproto blob. Mod deliverables are too large for PDS blob storage, so they go in the S3 object store and are referenced by URL.

The game and platform fields are split so the /repo.json endpoint can group mods by console (which the on-device app stores need) while the web mod browser can filter by game. The author field isn't stored in the record - the record's DID identifies the author, and the endpoint generator resolves it to a display name. The Homebrew App Store name field (used for update tracking) is derived from the DID + record TID, which is immutable. Fields like license, details, and changelog (on releases) map directly to their Homebrew App Store equivalents.

Tech#

  • Language: Rust.
  • Frontend: Server-rendered HTML + htmx (like tangled.sh).
  • Database: SQLite (like tangled.sh).
  • Search: SeekStorm (see Search section below).
  • Blob storage: S3-compatible object store, configured via environment variable (endpoint, bucket, credentials). Used for mod deliverables, media (screenshots, videos), and any other uploaded files. The appview doesn't assume a specific provider - any S3-compatible service (MinIO, Tigris, Cloudflare R2, AWS S3, etc.) works.
  • Hosting: kuribo, behind Caddy.
  • Repo: starhaven.dev/appview on tangled.
  • Nix: flake.nix outputs a NixOS module that starhaven.dev/infra (kuribo) consumes as a flake input, plus a devshell for local development.

Forum categories#

Categories and subcategories are hardcoded in the appview source. Changing them is rare enough that a code change and redeploy is fine.

Initial structure (subject to change):

  • About Star Haven
    • Announcements (locked: only admins/moderators can create threads)
    • Suggestions
  • Making Mods
    • Help
    • Collaboration
  • Mod Showcase
    • In Development
    • Released Mods

Threads are tagged with games (64, ttyd, etc.) using the same fixed set as mod.listing.game, so any subcategory can be filtered by game. This avoids duplicating categories per game and scales if new games get modding communities. The subcategory field on forum.thread records stores the subcategory slug, e.g. making-mods/help.

Visual design#

TBD, but the forum area should feel like a 2000s web forum while meeting modern accessibility and mobile standards. Discourse is a good reference for how a modern forum can work - the aesthetic is retro, the UX should not be.

Firehose and backfill#

The appview subscribes to the Bluesky relay's Jetstream for live events.

Jetstream only provides live events and a short replay buffer (a couple of days at most). A fresh database (new developer setup, or recovery from data loss) needs to be seeded from PDSs directly using com.atproto.sync.getRepo, which returns every record in a user's repo.

For comparison, Tangled's appview does not do backfill - it connects to Jetstream with a saved cursor and replays from there. This works for Tangled because its knot servers are the source of truth for git data. For Star Haven, the forum and mod records only exist in user repos, so backfill is necessary.

On first startup with an empty database, the appview:

  1. Lists all repos on each configured PDS via com.atproto.sync.listRepos. The default config includes pds.starhaven.dev. Scanning all of bsky.social (millions of repos) is not practical, so bsky.social users need to have interacted with the appview at least once (e.g. signed in) for their DID to be known. The appview maintains a table of known DIDs and fetches those repos individually.
  2. For each known repo, calls com.atproto.sync.getRepo and processes every dev.starhaven.* record as if it came from the firehose (inserting into both SQLite and the search index).
  3. Saves the current Jetstream cursor.
  4. Switches to live firehose consumption from that cursor.

This backfill runs automatically on first startup.

Search uses SeekStorm, a Rust full-text search crate that embeds in the appview binary. No separate service.

The ingester writes to both SQLite and the SeekStorm index on every firehose event (create, update, delete), so records are searchable immediately.

Indexed content:

  • Forum threads and posts (title, body). Filterable by category, tags.
  • Mod listings (title, description, details). Filterable by game, platform, category.
  • Actor profiles (display name, description).

/search is a single search bar across all record types. Facet filters let you narrow by type (thread, mod, profile), game, platform, or category. Results ranked by BM25F.

SeekStorm has built-in typo tolerance (edit distance 1, so "stikcer star" finds "Sticker Star") and query auto-completion.

Mod browser#

The mod browser is the primary way to discover and download mods. Modders sign in and create a dev.starhaven.mod.listing record with a title, description, media, and one or more versioned releases. The appview indexes these from the firehose like any other record. Mod deliverables (patches, zips, etc.) are uploaded through the appview web UI, which stores them in the S3-compatible object store. For games that need patching, the patching pipeline runs in the browser.

Patching#

Paper Mario mods can't just ship a ready-to-play ROM or ISO - that would require distributing copyrighted game data. Instead, mods are distributed as patches or file overlays that are applied to a user-supplied copy of the original game. The mod browser handles all of this transparently: end-users provide their base ROM/ISO once (stored in the browser's IndexedDB for reuse) and then just click "download" on any mod. No separate patching program required. The patching strategy varies by game:

PM64 (N64) - Mods are distributed as .bps patch files targeting the US ROM (the only supported base for now). The .bps patch is fetched, applied to the base ROM entirely client-side using JavaScript, and the patched ROM is downloaded to the end-user's machine. No game data ever leaves the browser or touches the server.

TTYD (GCN) - Patching strategy TBD. Needs input from existing GameCube modders.

SPM (Wii) - Like PM64, the end-user supplies a base .iso or .wbfs. There are multiple regional revisions (3 US, 2 JP, ~1 PAL) but mods ship a single patch - the tooling uses symbol-based replacement so it handles revision differences automatically. The existing Flipside Mod Manager uses the C libraries wiimms-szs-tools and wiimms-iso-tools to apply patches. Rather than porting these to JavaScript, the patching runs client-side inside CheerpX (WebVM), a browser-based x86 virtual machine. The output is an .iso (WBFS filesystem) that works with both Dolphin and USB loaders on real hardware. As with PM64, no game data leaves the browser.

Modern PM (3DS, Wii U, Switch) - Mods are zip files containing replacement assets that get extracted to console-specific paths (e.g. sdmc:/luma/titles/<title-id>/romfs/ on 3DS). These don't require patching against a base ROM - the zip is the deliverable. Modders upload the zip to starhaven.dev and end-users download it directly, either through the web mod browser or through on-device homebrew app stores (Universal Updater on 3DS, Homebrew App Store on Wii U/Switch) via the /repo.json compatibility endpoint.

/repo.json compatibility endpoint#

The 3DS/Wii U/Switch modding communities already use on-device homebrew app stores that read from a repo.json file listing available mods. starhaven.dev serves a /repo.json endpoint that is compatible with these formats, generated from the indexed dev.starhaven.mod.listing and dev.starhaven.mod.release records. This means existing on-device mod browsers continue to work - they just point at starhaven.dev instead of a GitHub repo.

The 3DS community uses Universal Updater, which reads a .unistore file (a JSON format with a different extension). The Wii U and Switch communities use the Homebrew App Store, which reads repos.json. starhaven.dev generates both formats.

The mapping from lexicon records to these formats:

  • DID + record TID → name (Homebrew App Store package identifier, stable)
  • title → title / unistore info.title
  • DID → resolved to author
  • description → description, details → details
  • license → license
  • category → category / unistore info.category
  • platform → unistore info.console
  • Latest mod.release.version → version / unistore info.version
  • Latest mod.release.changelog → changelog
  • Latest mod.release.deliverable → download URL
  • media → filtered to images only (on-device app stores don't support video)

For the unistore format, the appview also generates per-region install scripts (download, extract, cleanup) using a lookup table of title IDs per game per region (e.g. 00040000000A5E00 for PM:SS USA). This is appview logic, not stored in the lexicon.

Moderation#

Moderation is appview-level: it controls what the site displays, not what exists in user repos. Users' atproto data is theirs regardless of moderation actions.

Roles:

  • Admin - full control. Can manage moderators, edit site config, post in locked categories (e.g. Announcements).
  • Moderator - can remove content from the index, lock/move threads, ban handles, pin/feature content.
  • User - default role for signed-in users.

Moderation state (bans, removed posts, role assignments) is stored in the appview database, not in atproto records. A banned user's records still exist in their repo but are hidden from the site.

Drawbacks#

  • It's a lot of work compared to just deploying Discourse.
  • We'd be maintaining custom forum software forever.
  • The PM modding community is small - might be overkill.
  • atproto custom lexicons are still early days, things might change.
  • atproto is fully public. Modders who want to work in secret (e.g. for mod jams) cannot use Tangled private repos yet. Until atproto adds permissioned data, private work should stay on whatever git forge the modder prefers.
  • If users delete records from their repos, forum threads get holes.

Alternatives#

Discourse/phpBB with atproto auth plugin - still need a custom auth plugin, doesn't give us the retro look without heavy theming, forum data isn't on atproto.

Don't do this - keep everything on Discord. Works for real-time chat but important knowledge stays trapped behind a login wall, unsearchable by search engines and unarchivable.

Future Work#

  • Integration with tangled, so that mod source repos can be linked from listings.
  • Build integration with papermario-dx CI.
  • Mod reviews with a dev.starhaven.mod.review lexicon.
  • Reactions with a dev.starhaven.feed.reaction lexicon (applicable to threads, replies, mods, etc.)
  • Multi-region PM64 patch support via a custom patch format for papermario-dx.
  • Mod collections and events (jams, curated sets like Master Quest, Paper Dank Rave, the Holiday Specials).
  • Additional PDS auth providers beyond passkeys and Discord (e.g. GitHub OAuth).

Unresolved questions#

  • Rich text: facets or Markdown? Markdown is familiar, facets are atproto-native.
  • Thread ordering: chronological or upvote-based?
  • Notifications: atproto has no push for custom lexicons. Email? Bluesky DMs? Just check the site?
  • TTYD (GCN) patching strategy: needs input from existing GameCube modders.
  • Lexicon namespacing: before 1.0, ensure lexicons aren't overly specific to Paper Mario if the platform may open to other modding communities in future.
  • Homepage design: what should it look like? The current one isn't great.