A free, open-source, self-hosted place to write a novel.
Fablewright docs development.md
24 kB · 434 lines
Markdown
at main

Develop Fablewright #

This guide is for people changing the code. It covers setup, the repository, tests, schema changes, and releases. Read AGENTS.md before a change. Read the landmines file for the feature you plan to change. It describes constraints that are not obvious from the code.

Requirements #

  • Node 24 or newer. The app uses node:sqlite, so there is no native SQLite build.
  • pnpm 12.21.0, matching both Dockerfiles.
  • Git.
  • Chromium for the browser tests and Playwright. pnpm test:e2e installs the browser if it is missing.
  • Java and epubcheck only for pnpm check:epub.

Setup #

mkdir -p .agents/skills
pnpm install --frozen-lockfile
cp .env.example .env
pnpm dev

Open http://localhost:5173 and create a disposable account. The first account becomes admin and receives the example project. Use development data only. Do not run tests from a production deployment directory.

The .agents/skills directory is needed by the dependency permissions in pnpm-workspace.yaml. A clean checkout without it can fail installation with ERR_PNPM_NO_AGENT_SKILLS_DIR. Do not remove the permissions or change dependency versions to work around that error.

The development server reads .env. ORIGIN in the example file is http://localhost:5173, the Vite port. The production entry point does not read .env by itself. Use node --env-file=.env server.js, or set the values in the service.

The database defaults to data/dev.db. The watcher ignores databases under the project root, because a save can otherwise look like a source edit and reload the page mid-sentence.

Do not copy a production secret into the checkout. For repeatable local sessions, set a separate BETTER_AUTH_SECRET in the local .env. Do not commit it.

Commands #

Command Purpose
pnpm dev Development server with hot reload.
pnpm build Production build into build/.
node --env-file=.env server.js Run that production build with the chosen environment file. Adjust ORIGIN first.
pnpm check Svelte and TypeScript checks. This is the type-check gate.
pnpm check:watch Repeat the type check after edits.
pnpm lint Prettier checks, followed by ESLint if formatting passes.
pnpm format Write Prettier formatting. This does not run ESLint.
pnpm test:unit Watch the server, browser, and Storybook test projects.
pnpm test:unit --run Run those projects once.
pnpm test:e2e Install Playwright browsers and run the production-build browser suite.
pnpm test Run unit projects once, then the end-to-end suite.
pnpm test:coverage Run Vitest with coverage and write coverage/index.html.
pnpm test:mutation Run TypeScript mutation tests against Node tests.
pnpm storybook Component workbench on port 6006.
pnpm build-storybook Build the static component workbench.
pnpm db:generate Generate a migration from schema changes.
pnpm db:studio Open Drizzle Studio against the configured database. This can change real data.
pnpm check:epub Export the example EPUB and run an available EPUB validator.
pnpm clean Remove generated output, reports, and .test-data/.

Test databases, uploads, and restore-drill records live under .test-data/. Unit tests use .test-data/unit/, and Playwright uses .test-data/e2e/. pnpm clean removes that directory, generated SvelteKit files, build output, and test reports. Never store real instance data under .test-data/.

Cleanup leaves .env, data/, root-level uploads, and root-level drill records alone. The default development uploads stay under data/uploads, beside data/dev.db. Files left at the repository root by older test runs need a separate review before removal.

pnpm preview is a build preview, not the supported deployment entry point. Use server.js to test production request handling, proxy behavior, and the older-build database refusal.

Code map #

Path What lives there
src/routes/ Pages, form actions and API routes. +page.server.ts loads and guards, +page.svelte renders, +server.ts answers an API request.
src/lib/components/ Svelte components for the workspace, library, account, admin and shared controls.
src/lib/server/ Server logic. Access, projects, structure, saves, notes, plot, recovery, search, import, export, auth, mail and the backup-facing helpers.
src/lib/ Shared logic that runs in both places: Markdown, word counts, the save client, preferences, proofreading and the guide renderer.
src/lib/styles/ Design tokens, the product stylesheet and the measure and contrast tests.
src/lib/server/db/ Drizzle schema, the database connection and the generated auth schema.
drizzle/ Applied migrations. The app reads this folder at boot.
scripts/ ops.mjs, the boot-time database state reader and the forwarded-header parser. Plain JavaScript, because server.js loads them before the app.
e2e/ Playwright specs against a production build.
deploy/ The systemd unit and the backup timer.
cloudron/ The Cloudron start, backup and restore hooks.
docs/ Writer, installation, portability, operations, recovery, sharing, troubleshooting, security, and development guides, plus landmines.
PRODUCT.md Product scope and commitments.
DESIGN.md The visual system as the code implements it.
AGENTS.md Rules for an agent working in the repository.
CHANGELOG.md What changed in each release.

The stack is SvelteKit 3 with Svelte 5 runes, TypeScript, Tailwind 4, Vite and adapter-node. SQLite comes from node:sqlite through Drizzle. Authentication is better-auth. The editor is TipTap and ProseMirror. Proofreading is Harper in a Web Worker.

Data model #

A project belongs to one owner. The project_member table grants access for every role, including the owner. A project holds books, a book holds parts and chapters, and a chapter holds scenes. A chapter with matter set is a front or back matter page, not a numbered chapter.

Notes belong to a project and can have a note type. A note type is project-scoped too. A scene-to-note link is a row in scene_note. Storylines belong to a project, and a plot card names a storyline and either a scene or a chapter.

Every scene and note carries a revision. A save names the revision it started from, and a mismatch is a conflict. A plot card records an item version when a save changes it. Scene versions are scene_snapshot, note and card versions are item_version, and project versions are project_checkpoint.

Membership is project_member. A secret link is project_share. Feedback is scene_feedback, scoped to a scene or a chapter. A writing goal is one row per writer and project, and writing_day holds the daily counts.

How a request moves #

  • A page load runs in +page.server.ts, asks src/lib/server/access.ts for the project and the role, loads the data and returns it. A refusal is a 404 with "No such …", so a guessed id cannot confirm that a row exists.
  • A save starts in the editor, goes through src/lib/save-client.ts to /api/scenes/[id], and lands in src/lib/server/save-request.ts. The request carries the revision the writer started from. A stale revision becomes a conflict, not an overwrite. A successful save takes a snapshot when the policy says so, reindexes the scene, and returns the new revision.
  • A note save follows the same shape through /api/notes/[id], with note history instead of scene snapshots.
  • A manuscript move goes through src/lib/server/structure.ts. It re-checks that the node belongs to the project, rewrites the ordinals in one transaction and reindexes the scenes below the node.
  • A project version is a checkpoint. The checkpoint shares the writer's transaction, because a second SQLite transaction on the same connection fails.
  • Auth is better-auth. src/hooks.server.ts sends /api/auth/* through the allowlist in src/lib/server/auth-routes.ts. The allowlist permits the passkey ceremony, the reset link's GET, and a session read. Other auth paths receive a 404. The hook loads the session for every other request. The first account becomes the admin. The sign-up gate is in the user.create.before hook in src/lib/server/auth.ts, and signup_open starts closed, so claiming the instance closes it.
  • Search is FTS5 in search_index, written by src/lib/server/search.ts and nowhere else. A new write path must index its scene or note in the same transaction.

Conventions that affect new code #

Use #lib, not $lib, for library imports. The alias is in package.json, and local imports include .ts. Keep database imports in server modules. A component can import a server type with import type, but importing a runtime constant can pull the database driver into the browser build.

SvelteKit configuration lives in the sveltekit({...}) call in vite.config.ts. Do not create svelte.config.js. Declare new private environment variables in src/env.ts, then import them from $app/env/private. Runtime app modules do not read process.env directly.

Use resolve from $app/paths for internal links and navigation, with the route identifier and parameters. Return Response.json(...) from JSON endpoints. Use the existing access helpers rather than adding route-specific membership queries.

The server derives plain text and word counts from document bodies. Do not trust client-supplied counts. A save must name its base revision and return a conflict for stale prose, not overwrite it silently. Keep a regression test for each new destructive path.

Apply stored prose through the normal body normalization before it reaches TipTap. Read the feature's landmines before changing editor, import, or recovery contracts. Read PRODUCT.md and DESIGN.md before interface work.

Tests #

pnpm test:unit runs three Vitest projects:

  • client: browser specs named *.svelte.spec.ts, in Chromium.
  • server: Node specs named *.spec.ts, against .test-data/unit/test.db.
  • storybook: stories under src/**/*.stories.svelte, through @storybook/addon-vitest.

One server spec:

pnpm exec vitest run --project=server src/lib/server/search.spec.ts

The server project shares one .test-data/unit/test.db. Every spec deletes every project in beforeAll. Build your own rows. fileParallelism: false exists because two files writing one SQLite file lock each other out.

pnpm test:e2e builds the app and runs it on port 4173 against .test-data/e2e/e2e.db. The webServer.command wipes .test-data/e2e/ first, including uploads and drill records. Never point Playwright at data/dev.db. The setup project (e2e/auth.setup.ts) claims the test database once. The chromium project then runs specs one worker at a time. e2e/first-run.e2e.ts runs in an isolated first-run project with its own ephemeral node server.js and throwaway SQLite directory. Mutable specs create isolated projects instead of editing the seeded Example manuscript.

Playwright refuses an occupied port and never reuses a running server. This stops the suite before the database wipe or any test writes. Stop the process on port 4173 before a new run.

For a scene edit, use settled(page, edit, words) with the edit in its callback. It registers the response wait before the edit, requires a successful save with a revision, then waits for the footer to show Saved. Register sceneSaved(page, words) before the edit when the test needs to inspect the page while the save is pending.

Coverage:

pnpm test:coverage

The report covers src/**/*.{ts,svelte} and scripts/**/*.mjs, with thresholds in vite.config.ts. It does not include Playwright.

Mutation testing:

pnpm test:mutation

Stryker mutates src/lib/**/*.ts against the server project only, because its Vitest runner cannot drive browser mode. The report lands in reports/mutation/mutation.html. A surviving change can reveal a missing assertion, but some changes do not alter observable behavior. Stryker runs one test runner because its shared sandbox holds one SQLite database. Do not increase concurrency until each runner has its own database. The sandbox excludes instance data and .env files. To test one file:

pnpm test:mutation --mutate 'src/lib/words.ts'

Storybook:

pnpm storybook
pnpm build-storybook

EPUB:

pnpm check:epub

The script exports the example EPUB. If epubcheck is on PATH, it runs that command. Otherwise it uses Java with EPUBCHECK_JAR when set. Without either, a local run only prints a notice and does not validate the EPUB.

Use CI=true pnpm check:epub when you need absence of the validator to fail the command. There is no CI workflow in this repository to run it for you.

The release check is local: pnpm check, pnpm lint, pnpm test:unit and pnpm test:e2e.

For a non-watching local gate, use pnpm test:unit --run. Run pnpm build-storybook separately after component or story changes. This repository has no CI workflow that substitutes for those local runs.

Test isolation and safety #

Keep data/dev.db separate from .test-data/. The VITEST guard in vite.config.ts sets the test database before SvelteKit generates private environment imports. Removing it can send destructive server specs to the development database.

Use accounts and rows built by each spec. End-to-end helpers in e2e/session.ts open the library, keep openExample read-only, and provide createProject and importMarkdownProject so mutable specs build their own projects.

To run a focused end-to-end check:

pnpm test:e2e e2e/saving.e2e.ts

Make sure that port 4173 is free first. Do not stop an unknown process without identifying it. Reusing an existing test server also reuses its database state.

Read the reports #

Coverage measures executed statements, branches, and functions. It does not prove that assertions check the result. Open coverage/index.html for per-file detail. Playwright results are outside that coverage report.

Mutation testing changes code and asks whether tests detect the change. A surviving change can reveal a weak assertion, but equivalent behavior can survive too. Svelte components and browser projects are outside this mutation report. Run focused mutation tests after changing pure TypeScript logic, rather than expecting a quick whole-repository gate.

For a failed Playwright test, inspect test-results/. The configuration keeps traces on failure. A trace can contain prose and credentials from a test account. Use made-up data and review the trace before sharing.

Change the schema #

  1. Edit src/lib/server/db/schema.ts. If a better-auth plugin adds a table, add that table to src/lib/server/db/auth.schema.ts in the committed shape. The file is generated and committed, and the generator rewrites the whole file with exports, index names and defaults that the app does not use.
  2. Run pnpm db:generate. Commit the new folder and its SQL under drizzle/.
  3. Make sure that the app applies the migration at boot inside one transaction.
  4. If the database state reader changes, update scripts/db-state.d.mts as well.
  5. If the archive shape changes, bump EXPORT_VERSION in src/lib/export-format.ts and keep every earlier version readable.
  6. Record migration and rollback consequences under Unreleased in CHANGELOG.md. Bump release versions together when cutting the release, not for each schema edit.

Do not use drizzle-kit push. It writes the schema with no migration folder and no __drizzle_migrations row, so a fresh instance never receives the change and the state reader cannot see the gap.

Do not edit an applied migration. The runner compares folder names, not hashes, so an edit changes nothing on an existing database and changes what a fresh one receives. Add a new folder instead.

Do not create drizzle/meta/_journal.json. The migration reader throws on it and every startup fails. src/lib/server/db-state.spec.ts asserts its absence.

Change the archive #

manuscript.json carries format and version. Import reads the lossless JSON, not the Markdown inside the archive. The Markdown is a reading copy.

  • Bump EXPORT_VERSION when the shape changes in a way import must understand.
  • Keep every earlier version readable from 1 on.
  • Update the export and import specs, and e2e/library-export-import.e2e.ts.
  • Check that a project archive still keeps images, trash, history and front and back matter placement.

EXPORT_FORMAT and EXPORT_VERSION live in src/lib/export-format.ts, which imports nothing, so a test can read them without opening the database.

Documentation #

  • docs/using-fablewright.md is compiled into the server bundle with ?raw and rendered at /help by src/lib/guide.ts. The renderer handles only the shapes the guide uses: h1 to h3, paragraphs, bullet and numbered lists, bold, italic, inline code, and absolute http(s) links. The four companion guides open their repository pages. Other relative links remain text. Do not add a table, an image, a code fence, a blockquote or an h4 to that guide. The renderer reads italic and bold with asterisks or underscores, so Prettier's rewrite of italics is safe, and it leaves inline code alone, so * and _ inside a code span stay literal.
  • README.md introduces the app, offers quick starts, lists runtime variables, and routes readers to the guides. Keep detailed procedures in their guide.
  • docs/installation.md covers a clean deployment. docs/operations.md covers its later maintenance. docs/import-export.md owns file preparation and portability details. Link to those instead of maintaining several full copies.
  • docs/landmines.md holds the feature traps. Update it when a change adds a trap or makes a line stale.
  • AGENTS.md holds the toolchain, database, auth, access and test rules. Keep it accurate when a change makes it wrong.
  • CHANGELOG.md records a change an operator or a writer can feel, under Unreleased.

Release checklist #

  1. Make the change and its test.

  2. Run pnpm check, pnpm lint, pnpm test:unit and pnpm test:e2e.

  3. Update CHANGELOG.md under Unreleased.

  4. Bump package.json and CloudronManifest.json to the same version. Update the pinned Compose image tag for the release too. Do not claim an image is available until it is published and a clean pull succeeds.

  5. Build and push both images with the commit. The Compose image supports amd64 and arm64. The Cloudron base image is amd64 only, so the Cloudron package stays on amd64:

    VERSION=$(node -p "require('./package.json').version")
    APP_COMMIT=$(git rev-parse --short HEAD)
    
    docker buildx build --platform linux/amd64,linux/arm64 \
      --build-arg APP_COMMIT=$(git rev-parse --short HEAD) \
      -t "katsuricata/fablewright:$VERSION" \
      -t katsuricata/fablewright:latest --push .
    
    docker buildx build --platform linux/amd64 \
      -f Dockerfile.cloudron \
      --build-arg APP_COMMIT=$(git rev-parse --short HEAD) \
      -t "katsuricata/fablewright-cloudron:$VERSION" \
      -t katsuricata/fablewright-cloudron:latest --push .
    
  6. Add the Cloudron version to the catalogue with the image from step 5:

    cloudron versions add --image "katsuricata/fablewright-cloudron:$VERSION"
    

    Commit CloudronVersions.json with the release. The catalogue is served from https://tangled.org/katsuricata.com/Fablewright/raw/main/CloudronVersions.json.

  7. Take a backup of the previous version and test the upgrade on a copy before the live instance. Run a restore drill and inspect its detailed results.

  8. Tag the release and publish the notes. The project home is https://tangled.org/katsuricata.com/Fablewright.

Before a release, test a fresh install as well as an upgrade. Claim the instance, save and reload prose, export and import a project with a note image, and exercise recovery. Make sure that the health endpoint reports the intended build. A successful build alone does not test persistent volumes or proxy headers.

Keep release credentials, registry tokens, production .env, databases, and backups out of the source archive and image context. The runtime must include build/, node_modules/, package.json, server.js, scripts/, and drizzle/. The writer's guide is bundled at build time, so doc changes need a new build to appear in /help.

Writing rules #

  • Comments explain why, never what. Delete commented-out code.
  • Do not put author names, dates, ticket numbers or change histories in code comments.
  • Keep the project voice: plain English, active voice, no em dashes, no generative-AI features. Read the surrounding file before you write a new one.
  • Use the existing patterns for a page, a server module, a component and a test. A new abstraction needs a second reader.