Piece Analytics Plan#

Current State#

• MongoDB Atlas is already set up and used for:

  • @handles - user handles
  • pieces - user-uploaded pieces (via track-media.mjs)
  • paintings - user paintings
  • tapes - recorded tapes
  • moods - user moods
  • logs - system logs
  • chat-system - chat messages

• No tracking exists for system piece hits (the built-in pieces like prompt, line, colors, etc.)

Proposed Schema#

Collection: piece-hits (Aggregate Stats)#

{
  _id: ObjectId,
  piece: "colors",           // piece name (slug)
  type: "system" | "user",   // system pieces vs @handle/piece
  hits: 12345,               // total hit count
  uniqueUsers: 8234,         // unique user count
  firstHit: Date,            // first recorded hit
  lastHit: Date,             // most recent hit
  
  // Rolling daily aggregation (last 30 days)
  daily: {
    "2025-12-31": { hits: 45, unique: 32 },
    "2025-12-30": { hits: 52, unique: 41 },
  }
}

Collection: piece-user-hits (Per-User Stats)#

{
  _id: ObjectId,
  piece: "colors",           // piece name
  user: "auth0|abc123",      // user sub (permanent ID)
  hits: 42,                  // how many times this user hit this piece
  firstHit: Date,
  lastHit: Date,
}
// Compound unique index: { piece: 1, user: 1 }
// Note: handles resolved at query time from @handles collection

Collection: piece-hit-log (Optional Raw Event Log)#

{
  _id: ObjectId,
  piece: "colors",
  user: "auth0|abc123" | null,
  timestamp: Date,
  referrer: "prompt",        // previous piece
  params: ["dark"],          // piece parameters
  sessionId: "xyz789",       // to group session activity
}
// TTL index: auto-delete after 90 days

### Collection: `piece-hit-events` (Optional, for detailed analytics)

```javascript
{
  _id: ObjectId,
  piece: "colors",
  user: "sub_123..." | null,  // null for anonymous
  timestamp: Date,
  referrer: "prompt" | null,  // which piece they came from
  params: ["param1"],         // piece parameters used
  duration: 45000,            // ms spent in piece (optional, tracked on leave)
  device: "mobile" | "desktop"
}

Implementation Options#

Option A: Client-Side Tracking (Lightweight)#

Where: disk.mjs or bios.mjs (client-side)

// In boot() when piece loads
async function trackPieceHit(pieceName) {
  try {
    await fetch('/api/piece-hit', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ 
        piece: pieceName,
        referrer: document.referrer
      })
    });
  } catch (e) { /* silent fail */ }
}

Pros: Accurate user-initiated loads, can track duration
Cons: Can be blocked, adds latency to piece load

Option B: Server-Side Tracking (index.mjs)#

Where: system/netlify/functions/index.mjs

Track every page render request:

// In the handler, after parsing slug
if (statusCode === 200 && slug && !previewOrIcon) {
  // Fire-and-forget hit tracking (don't await)
  trackHit(slug, parsed).catch(e => console.error('Hit tracking failed:', e));
}

Pros: More reliable, no client-side blocking, catches all loads
Cons: Includes bots/crawlers, no duration tracking

Option C: Hybrid (Recommended)#

• Server-side for page view counts (simple increment)
• Client-side for engagement metrics (duration, interactions)

New API Endpoint: /api/piece-hit#

File: system/netlify/functions/piece-hit.mjs

// piece-hit.mjs
import { authorize } from "../../backend/authorization.mjs";
import { connect } from "../../backend/database.mjs";
import { respond } from "../../backend/http.mjs";

export async function handler(event) {
  const database = await connect();
  
  // GET: Return stats for a piece or all pieces
  if (event.httpMethod === "GET") {
    const { piece, top, users } = event.queryStringParameters || {};
    const hitsCol = database.db.collection("piece-hits");
    const userHitsCol = database.db.collection("piece-user-hits");
    const handlesCol = database.db.collection("@handles");
    
    if (piece) {
      const stats = await hitsCol.findOne({ piece });
      
      // Optionally include top users for this piece
      let topUsers = [];
      if (users) {
        const userStats = await userHitsCol
          .find({ piece, user: { $ne: "anonymous" } })
          .sort({ hits: -1 })
          .limit(10)
          .toArray();
        
        // Resolve handles from subs
        for (const u of userStats) {
          const handleDoc = await handlesCol.findOne({ user: u.user });
          topUsers.push({
            handle: handleDoc?._id || null,
            hits: u.hits,
            lastHit: u.lastHit
          });
        }
      }
      
      return respond(200, { 
        ...stats, 
        topUsers: users ? topUsers : undefined 
      });
    }
    
    // Return top pieces overall
    const pieces = await hitsCol
      .find({})
      .sort({ hits: -1 })
      .limit(parseInt(top) || 50)
      .toArray();
    return respond(200, { pieces });
  }
  
  // POST: Record a hit
  if (event.httpMethod === "POST") {
    const { piece, type = "system", referrer, params } = JSON.parse(event.body || "{}");
    if (!piece) return respond(400, { error: "piece required" });
    
    // Try to get user from auth header
    let user = null;
    try {
      user = await authorize(event.headers);
    } catch (e) { /* anonymous hit */ }
    
    const now = new Date();
    const today = now.toISOString().split("T")[0];
    const hitsCol = database.db.collection("piece-hits");
    const userHitsCol = database.db.collection("piece-user-hits");
    
    // 1. Update aggregate stats
    const updateOps = {
      $inc: { 
        hits: 1,
        [`daily.${today}.hits`]: 1
      },
      $set: { lastHit: now, type },
      $setOnInsert: { firstHit: now, uniqueUsers: 0 }
    };
    
    await hitsCol.updateOne({ piece }, updateOps, { upsert: true });
    
    // 2. Update per-user stats (only sub, no handle)
    const userKey = user?.sub || "anonymous";
    const userResult = await userHitsCol.updateOne(
      { piece, user: userKey },
      {
        $inc: { hits: 1 },
        $set: { lastHit: now },
        $setOnInsert: { firstHit: now }
      },
      { upsert: true }
    );
    
    // If this was a new user for this piece, increment uniqueUsers
    if (userResult.upsertedCount > 0 && userKey !== "anonymous") {
      await hitsCol.updateOne(
        { piece },
        { $inc: { uniqueUsers: 1, [`daily.${today}.unique`]: 1 } }
      );
    }
    
    await database.disconnect();
    return respond(200, { success: true });
  }
  
  return respond(405, { error: "Method not allowed" });
}

New API Endpoint: /api/piece-fans#

File: system/netlify/functions/piece-fans.mjs

Get top users ("fans") of a piece:

// piece-fans.mjs - Get users who love a specific piece
import { connect } from "../../backend/database.mjs";
import { respond } from "../../backend/http.mjs";

export async function handler(event) {
  if (event.httpMethod !== "GET") return respond(405);
  
  const { piece, limit = 20 } = event.queryStringParameters || {};
  if (!piece) return respond(400, { error: "piece required" });
  
  const database = await connect();
  const userHitsCol = database.db.collection("piece-user-hits");
  const handlesCol = database.db.collection("@handles");
  
  // Get top users by hits (excluding anonymous)
  const userStats = await userHitsCol
    .find({ piece, user: { $ne: "anonymous" } })
    .sort({ hits: -1 })
    .limit(parseInt(limit))
    .toArray();
  
  // Resolve handles from subs at query time
  const fans = [];
  for (const u of userStats) {
    const handleDoc = await handlesCol.findOne({ user: u.user });
    if (handleDoc) {  // Only include users with handles
      fans.push({
        handle: handleDoc._id,
        hits: u.hits,
        firstHit: u.firstHit,
        lastHit: u.lastHit
      });
    }
  }
  
  await database.disconnect();
  return respond(200, { piece, fans });
}

Using Hit Data in list.mjs#

// In boot(), fetch popular pieces
const hitStats = await fetch('/api/piece-hit').then(r => r.json());
const hitMap = new Map(hitStats.pieces?.map(p => [p.piece, p.hits]) || []);

// Add "🔥 Popular" category sorted by hits
const popularPieces = allItems
  .filter(item => hitMap.get(item.name) > 100)
  .sort((a, b) => (hitMap.get(b.name) || 0) - (hitMap.get(a.name) || 0))
  .slice(0, 20);

Implementation Steps#

1. Create endpoint: piece-hit.mjs with GET/POST + user tracking
2. Create endpoint: piece-fans.mjs to query top users per piece
3. Add server-side tracking: In index.mjs, fire-and-forget POST on each page load
4. Create indexes:
   • piece-hits: { piece: 1 } unique, { hits: -1 } for sorting
   • piece-user-hits: { piece: 1, user: 1 } unique compound, { piece: 1, hits: -1 } for fan queries
   • piece-hit-log: { timestamp: 1 } TTL 90 days (optional)
5. Update list.mjs: Add "🔥 Popular" category using hit data
6. Add to metrics.mjs: Include piece hit stats in /api/metrics
7. Create piece detail view: Show fans/top users for each piece

Example Queries#

// Top 10 most visited pieces
db.collection("piece-hits").find().sort({ hits: -1 }).limit(10)

// Top fans of "colors" piece
db.collection("piece-user-hits")
  .find({ piece: "colors", user: { $ne: "anonymous" } })
  .sort({ hits: -1 })
  .limit(10)

// User's favorite pieces (most visited by a specific user)
db.collection("piece-user-hits")
  .find({ user: "auth0|abc123" })
  .sort({ hits: -1 })
  .limit(10)

// Pieces a user has never visited
const visited = await db.collection("piece-user-hits")
  .find({ user: "auth0|abc123" })
  .project({ piece: 1 }).toArray();
const visitedSet = new Set(visited.map(v => v.piece));
// Then filter allPieces - visitedSet

Privacy Considerations#

• Don't store IP addresses
• User IDs only if logged in (optional)
• No personal data in hit events
• Consider GDPR compliance for EU users
• Add rate limiting to prevent abuse

Estimated Effort#

• Endpoint + Server tracking: 1-2 hours
• Client-side duration tracking: 2-3 hours
• list.mjs integration: 1 hour
• Indexes + testing: 1 hour

Total: ~5-7 hours for full implementation

Questions to Decide#

1. Track duration? Requires client-side tracking on piece leave
2. Track user vs anonymous separately? More complex queries
3. Rolling daily stats? Need cleanup job for old data
4. Bot filtering? Could use User-Agent checking
5. Rate limiting? Prevent spam hits from single client

Created: 2025.12.31