aesthetic.computer
Monorepo for Aesthetic.Computer
aesthetic.computer
1# Oven - Video Processing Service for Aesthetic Computer Tapes
2
3## Overview
4`oven.aesthetic.computer` is a dedicated DigitalOcean Droplet service for processing tape recordings into MP4 videos. This solves the Netlify Function 250MB size limit issue caused by bundling ffmpeg.
5
6## Quick Deployment
7
8### Deploy to Production
9
10```bash
11cd /workspaces/aesthetic-computer/oven
12fish deploy.fish
13```
14
15This automated script will:
161. ✅ Create DigitalOcean droplet (2GB RAM, 2 vCPU, $18/month)
172. ✅ Install Node.js 20, ffmpeg, and Caddy
183. ✅ Deploy oven service code
194. ✅ Configure HTTPS with automatic certificates
205. ✅ Set up systemd service for auto-restart
216. ✅ Configure DNS (oven.aesthetic.computer)
22
23### Post-Deployment Steps
24
251. **Update Netlify Environment:**
26 ```bash
27 # Add to system/.env (for development)
28 OVEN_URL=https://oven.aesthetic.computer
29
30 # Add to Netlify dashboard (for production)
31 # Site settings → Environment variables
32 OVEN_URL=https://oven.aesthetic.computer
33 ```
34
352. **Test the Service:**
36 ```bash
37 # Health check
38 curl https://oven.aesthetic.computer/health
39 # Expected: {"status":"healthy"}
40
41 # View dashboard
42 open https://oven.aesthetic.computer
43 ```
44
453. **Monitor Logs:**
46 ```bash
47 ssh -i ~/.ssh/oven-deploy-key root@DROPLET_IP
48 tail -f /var/log/oven/oven.log
49 ```
50
51## Architecture
52
53### Current Problem
54- Netlify Functions have a 250MB size limit
55- `ffmpeg-static` (50MB) + `ffprobe-static` + dependencies exceed this limit
56- Functions `track-media` and `tape-convert-background` fail to deploy
57
58### Solution
59External video processing service on a dedicated server:
60
61```
62User uploads tape
63 ↓
64aesthetic.computer (Netlify)
65 ↓ POST job to oven
66oven.aesthetic.computer (DigitalOcean Droplet)
67 ↓ Download ZIP to /tmp/
68 ↓ Process video with ffmpeg
69 ↓ Upload MP4 + thumbnail to at-blobs-aesthetic-computer/tapes/
70 ↓ POST callback with Spaces URLs
71 ↓ Clean up /tmp/
72aesthetic.computer receives callback
73 ↓ Download MP4 from Spaces
74 ↓ Upload blob to ATProto
75 ↓ Update MongoDB
76```
77
78## Service Components
79
80### 1. Oven Server (Express.js)
81- **Location**: `/oven/server.mjs`
82- **Port**: 3000 (behind Caddy reverse proxy on 443)
83- **Endpoints**:
84 - `POST /bake` - Accept tape conversion job
85 - `GET /health` - Health check
86 - `GET /status/:jobId` - Check job status (optional)
87
88### 2. Processing Pipeline
89Reuse existing code from `system/backend/tape-to-mp4.mjs`:
901. Download ZIP from `art-aesthetic-computer` to `/tmp/tape-${slug}/`
912. Extract frames and audio
923. Read timing.json
934. Generate thumbnail with Sharp
945. Probe audio duration with ffprobe
956. Convert to MP4 with ffmpeg
967. Upload MP4 to `at-blobs-aesthetic-computer/tapes/${slug}.mp4`
978. Upload thumbnail to `at-blobs-aesthetic-computer/tapes/${slug}-thumb.jpg`
989. POST callback with Spaces URLs
9910. Clean up `/tmp/tape-${slug}/` directory
100
101**Storage Strategy**:
102- **Source ZIPs**: `art-aesthetic-computer/${slug}.zip` (already there)
103- **Processed MP4s**: `at-blobs-aesthetic-computer/tapes/${slug}.mp4`
104- **Thumbnails**: `at-blobs-aesthetic-computer/tapes/${slug}-thumb.jpg`
105- **Temp processing**: `/tmp/tape-${slug}/` (deleted after upload)
106- **Organization**: All ATProto-related blobs in dedicated Space
107
108### 3. Netlify Integration
109Update `system/netlify/functions/track-media.mjs`:
110- Remove inline MP4 conversion
111- Remove ffmpeg-static and ffprobe-static dependencies
112- POST to `https://oven.aesthetic.computer/bake` with:
113 ```json
114 {
115 "mongoId": "...",
116 "slug": "...",
117 "zipUrl": "https://...",
118 "metadata": {...},
119 "callbackUrl": "https://aesthetic.computer/api/tape-bake-complete",
120 "callbackSecret": "shared-secret"
121 }
122 ```
123
124### 4. Callback Webhook
125New Netlify function: `system/netlify/functions/tape-bake-complete.mjs`:
126- Receives JSON callback with Spaces URLs
127- Verifies shared secret for authentication
128- Downloads MP4 from `at-blobs-aesthetic-computer/tapes/${slug}.mp4`
129- Uploads MP4 blob to ATProto PDS
130- Updates MongoDB with completion status, rkey, and MP4 URL
131- Returns 200 on success
132
133**Callback Payload**:
134```json
135{
136 "mongoId": "...",
137 "slug": "...",
138 "mp4Url": "https://at-blobs-aesthetic-computer.sfo3.digitaloceanspaces.com/tapes/${slug}.mp4",
139 "thumbnailUrl": "https://at-blobs-aesthetic-computer.sfo3.digitaloceanspaces.com/tapes/${slug}-thumb.jpg",
140 "secret": "shared-secret"
141}
142```
143
144**Data Flow**:
145- Oven uploads to permanent Spaces storage
146- Webhook downloads from Spaces, uploads to ATProto
147- MP4 remains in Spaces for backup/future use
148- ATProto gets blob, MongoDB gets both URLs and rkey
149
150## Deployment Strategy
151
152### Following Existing Patterns
153Model deployment after `/at` and `/grab`:
154
1551. **Vault Configuration**: `/aesthetic-computer-vault/oven/`
156 - `.env` - Environment variables
157 - `deploy.env` - DigitalOcean deployment config
158
1592. **Deployment Script**: `/oven/deploy.fish`
160 - Loads vault credentials
161 - Creates/updates DigitalOcean Droplet
162 - Installs ffmpeg, Node.js, dependencies
163 - Sets up systemd service
164 - Configures Caddy for HTTPS
165
1663. **DNS Configuration**: Automatic via Cloudflare API
167 - Similar to `/grab/scripts/deploy-with-dns.fish`
168 - Creates A record: `oven.aesthetic.computer` → Droplet IP
169 - Proxied through Cloudflare for DDoS protection
170
171### Infrastructure Requirements
172
173#### DigitalOcean Droplet
174- **Size**: Basic Droplet ($6-12/month)
175 - 1-2 GB RAM (sufficient for 8-second tapes)
176 - 1 vCPU
177 - 25-50 GB SSD
178- **Image**: Ubuntu 24.04 LTS
179- **Region**: SFO3 (same as Spaces)
180- **Firewall**:
181 - Allow 80, 443 (HTTP/HTTPS)
182 - Allow 22 (SSH for deployment)
183
184#### Software Stack
185- **Node.js**: v20+ (via nvm)
186- **ffmpeg**: Latest via apt
187- **ffprobe**: Included with ffmpeg
188- **Caddy**: Automatic HTTPS
189- **PM2** or **systemd**: Process management
190
191### Environment Variables (Vault)
192
193#### `/aesthetic-computer-vault/oven/.env`
194```bash
195# DigitalOcean Spaces - Source ZIPs
196ART_SPACES_KEY=...
197ART_SPACES_SECRET=...
198ART_SPACES_ENDPOINT=https://sfo3.digitaloceanspaces.com
199ART_SPACES_BUCKET=art-aesthetic-computer
200
201# DigitalOcean Spaces - Processed Videos
202AT_BLOBS_SPACES_KEY=...
203AT_BLOBS_SPACES_SECRET=...
204AT_BLOBS_SPACES_ENDPOINT=https://sfo3.digitaloceanspaces.com
205AT_BLOBS_SPACES_BUCKET=at-blobs-aesthetic-computer
206
207# Optional: Custom CDN domain (requires manual Cloudflare CNAME setup)
208# AT_BLOBS_CDN=at-blobs.aesthetic.computer
209# CNAME: at-blobs.aesthetic.computer → at-blobs-aesthetic-computer.sfo3.cdn.digitaloceanspaces.com
210# Enable Cloudflare proxy for CDN caching and DDoS protection
211
212# Webhook callback
213CALLBACK_SECRET=... # Shared secret for webhook authentication
214
215# Service config
216PORT=3000
217NODE_ENV=production
218TEMP_DIR=/tmp # Where to store working files during processing
219```
220
221#### `/aesthetic-computer-vault/oven/deploy.env`
222```bash
223# DigitalOcean API
224DO_TOKEN=...
225
226# Droplet configuration
227DROPLET_NAME=oven-aesthetic-computer
228DROPLET_SIZE=s-1vcpu-1gb
229DROPLET_IMAGE=ubuntu-24-04-x64
230DROPLET_REGION=sfo3
231
232# Cloudflare DNS
233CLOUDFLARE_EMAIL=...
234CLOUDFLARE_API_TOKEN=...
235CLOUDFLARE_ZONE_ID=a23b54e8877a833a1cf8db7765bce3ca
236```
237
238## File Structure
239
240```
241/oven/
242├── README.md # This file (architecture & deployment docs)
243├── package.json # Dependencies
244├── server.mjs # Express server
245├── baker.mjs # Tape processing logic (from tape-to-mp4.mjs)
246├── deploy.fish # Main deployment script
247├── setup.sh # Server provisioning script
248├── oven.service # Systemd service file
249├── Caddyfile # Caddy reverse proxy config
250└── scripts/
251 └── setup-droplet.sh # Initial droplet setup
252
253/system/netlify/functions/
254├── track-media.mjs # Updated: POST to oven instead of inline
255└── tape-bake-complete.mjs # New: Webhook callback handler
256
257/aesthetic-computer-vault/oven/
258├── .env # Service environment variables
259└── deploy.env # Deployment configuration
260
261DigitalOcean Spaces:
262├── art-aesthetic-computer/
263│ └── ${slug}.zip # Source tape recordings
264└── at-blobs-aesthetic-computer/
265 └── tapes/
266 ├── ${slug}.mp4 # Processed videos
267 └── ${slug}-thumb.jpg # Thumbnails
268```
269
270## Implementation Steps
271
272### Phase 1: Oven Service Setup
2731. ✅ Create `/oven` directory structure
2742. Create `server.mjs` with Express endpoints
2753. Extract processing logic to `baker.mjs`
2764. Add health check and status endpoints
2775. Test locally with Docker
278
279### Phase 2: Deployment Automation
2801. Create vault configuration files
2812. Write `deploy.fish` following `/grab` pattern
2823. Create `setup.sh` for server provisioning
2834. Configure Caddy for HTTPS
2845. Set up systemd service
285
286### Phase 3: Netlify Integration
2871. Create `tape-bake-complete.mjs` webhook
2882. Update `track-media.mjs` to POST to oven
2893. Remove ffmpeg-static and ffprobe-static deps
2904. Add oven URL to environment variables
2915. Test end-to-end flow
292
293### Phase 4: Testing & Monitoring
2941. Test anonymous tape uploads
2952. Test authenticated tape uploads
2963. Monitor oven server logs
2974. Set up health check alerts
2985. Document troubleshooting
299
300## Security Considerations
301
302### Authentication
303- Webhook callback uses shared secret
304- Validate callback signature
305- Only accept jobs from known origins
306
307### Rate Limiting
308- Limit concurrent jobs (start with 2-3)
309- Queue additional requests
310- Reject jobs over size/duration limits
311
312### Firewall
313- Restrict SSH to known IPs
314- Only expose 80/443 for web traffic
315- Use Cloudflare proxy for DDoS protection
316
317## Cost Estimation
318
319### Monthly Costs
320- **Droplet**: $6-12/month (Basic/Regular)
321- **Bandwidth**: Included (1TB transfer)
322- **Spaces traffic**: Minimal (already using Spaces)
323- **DNS**: Free (existing Cloudflare account)
324
325**Total**: ~$6-12/month
326
327### Scaling Considerations
328- Single droplet handles ~10-20 concurrent conversions
329- Can add load balancer + multiple droplets if needed
330- Current traffic: <100 tapes/day = easily handled
331
332## Monitoring & Maintenance
333
334### Health Checks
335- `/health` endpoint for uptime monitoring
336- Cloudflare health checks
337- Alert on 5xx errors or timeouts
338
339### Logs
340- PM2/systemd logs for debugging
341- Rotate logs daily
342- Monitor disk space usage
343
344### Updates
345- Automatic security updates (unattended-upgrades)
346- Manual ffmpeg updates as needed
347- Node.js updates via nvm
348
349## Rollback Plan
350
351If oven service fails:
3521. Revert Netlify functions to inline processing
3532. Use smaller droplet sizes temporarily
3543. Fall back to background function (if <250MB somehow)
355
356## Future Enhancements
357
358### Potential Additions
359- Job queue with Redis (for high traffic)
360- Multiple worker droplets with load balancer
361- Separate thumbnail generation endpoint
362- Progress callbacks (for UI feedback)
363- Video preview generation
364- Format conversion (WebM, different resolutions)
365
366## References
367
368### Similar Services in Repo
369- `/at` - ATProto PDS server (DigitalOcean + Cloudflare DNS)
370- `/grab` - Screenshot worker (Cloudflare Worker + DNS)
371- `/session-server` - WebSocket server example
372
373### External Documentation
374- [DigitalOcean API](https://docs.digitalocean.com/reference/api/)
375- [Cloudflare API](https://developers.cloudflare.com/api/)
376- [ffmpeg Documentation](https://ffmpeg.org/documentation.html)
377- [Express.js](https://expressjs.com/)
378
379## Timeline Estimate
380
381- **Phase 1**: 4-6 hours (service implementation)
382- **Phase 2**: 2-3 hours (deployment automation)
383- **Phase 3**: 2-3 hours (Netlify integration)
384- **Phase 4**: 1-2 hours (testing)
385
386**Total**: ~9-14 hours of development
387
388## Success Criteria
389
390✅ Anonymous tape uploads work in production
391✅ Authenticated tape uploads work in production
392✅ MP4 conversion completes in <30 seconds for 8-second tapes
393✅ ATProto sync happens automatically after conversion
394✅ Netlify functions deploy successfully (<250MB)
395✅ Service has 99% uptime
396✅ Automatic deployment via `deploy.fish`
397✅ DNS automatically configured on deployment