Cross-Platform Sample Storage Plan#

Key Insight: Samples ARE Paintings#

AC already has pixel-sample.mjs which encodes audio samples as RGB pixel data in painting bitmaps. stample.mjs uses this today. Paintings have full infrastructure: upload, CDN, short codes (#k3d), @handle ownership, track-media API, and cross-platform rendering.

Samples use the same # sigil and PNG storage format as paintings, but live in a separate samples MongoDB collection with audio-specific metadata.

Architecture Decision: Shared Sigil, Separate Collection#

Why `#` (not a new sigil like `^`)#

Samples and paintings share the same PNG storage format
A painting can BE a sample (load any image as audio via stample)
A sample IS a painting (the pixel-encoded waveform is visible art)
% and & are URL-unfriendly (% is URL escape, & is query separator)
Keeps the sigil set small: @ people, $ code, # media, * time

Why separate collection (not a flag on paintings)#

Clean querying: "list my samples" vs "list my paintings" without filters
Separate indexes optimized for audio metadata (duration, sampleRate, etc.)
Separate counts/quotas per media type
Future-proof: audio-specific features (waveform preview, BPM detection)

Shared code namespace#

# codes must be unique across BOTH paintings AND samples collections
generateUniqueCode() checks both collections before assigning
Other sigils ($ KidLisp, * clock, tapes) have independent namespaces
The code resolver checks both collections to find which type a #code refers to

MongoDB Schema#

Collection: `samples`#

{
  _id: ObjectId,
  user: "auth0|63effeeb...",      // owner (null for guest)
  slug: "kick-drum",              // user-friendly name
  code: "k3d",                    // unique short code (shared with paintings)
  when: ISODate,                  // upload timestamp
  
  // Sample-specific metadata
  v: 1,                           // pixel-sample encoding version
  sampleRate: 48000,
  sampleLength: 240000,           // exact sample count
  duration: 5.0,                  // seconds
  channels: 1,
  source: "native",               // or "web"
  
  // Standard media fields
  ext: "png",                     // storage format
  width: 256,                     // image dimensions
  height: 313,
}

Indexes#

// Unique code (shared namespace with paintings — enforced at generation time)
await samples.createIndex({ code: 1 }, { unique: true, sparse: true });

// User queries: "list my samples"
await samples.createIndex({ user: 1 });

// Audio-specific queries
await samples.createIndex({ duration: 1 });          // sort by length
await samples.createIndex({ "source": 1 });           // native vs web
await samples.createIndex({ v: 1 });                  // encoding version (for migration)
await samples.createIndex({ when: -1 });              // recent first
await samples.createIndex({ user: 1, slug: 1 }, { unique: true }); // per-user slugs

Encoding Version Contract#

v1 (current — pixel-sample.mjs):
  - 3 audio samples per pixel (R, G, B channels)
  - Float range -1.0..+1.0 mapped to 0..255
  - A channel: 255 (opaque)
  - Width: 256px (configurable)
  - Height: ceil(sampleLength / 3 / width)
  - Mono only
  - The `v` field MUST be stored on every record so decoders know which algorithm to use

Implementation Plan#

Phase 1: Backend — track-media + code generation#

Files to modify:

system/netlify/functions/track-media.mjs
- Add mediaType: "sample" branch for PNG uploads with sample metadata
- Route to samples collection instead of paintings
- Store v, sampleRate, sampleLength, duration, channels, source
system/backend/generate-short-code.mjs
- generateUniqueCode() accepts optional siblingCollections array
- For # codes: checks both paintings AND samples before assigning
- Other types unchanged (single collection check)
system/netlify/functions/painting-code.mjs (or equivalent resolver)
- When resolving #code, check paintings first, then samples
- Return { type: "painting" | "sample", ...record } so the client knows
New: system/netlify/functions/list-samples.mjs (or extend existing)
- GET /api/samples/@handle → list user's samples
- GET /api/samples/@handle/:slug → get specific sample
- Returns CDN URLs + metadata

Phase 2: Native pixel-sample bridge#

Files to create/modify:

fedac/native/pieces/lib/pixel-sample-native.mjs
- Pure-JS encode/decode (no DOM, no Canvas — works in QuickJS)
- Same algorithm as web pixel-sample.mjs
- encodeSampleToBitmap(float32Array, width) → RGBA pixel array
- decodeBitmapToSample(rgbaArray, width, height, sampleLength) → float32[]
PNG write from native
- Option A: Add stb_image_write.h (single-header, ~1KB) for PNG encoding in C
- Option B: Minimal PNG writer in JS (deflate + PNG header — ~100 lines)
- Option C: BMP format (simpler, no compression, server converts to PNG on upload)

Phase 3: Upload from native#

Flow:

Record audio → float32[] → encodeSampleToBitmap → PNG bytes
  → POST /api/track-media { ext:"png", mediaType:"sample", sampleMeta:{v:1,...} }
  → GET presigned URL → PUT PNG to Spaces → #code returned

Files:

fedac/native/pieces/samples.mjs — add upload key (u)
fedac/native/src/js-bindings.c — system.uploadMedia() binding if needed
Or use existing system.fetch() + system.fetchPost() for the API calls

Phase 4: Download to native#