rss email digests over ssh because you're a cool kid herald.dunkirk.sh
go rss rss-reader ssh charm
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

chore: remove md files

dunkirk.sh 1bdf40a5 63aacf78

verified
-810
-376
fix.md
··· 1 - # Code Review: Herald RSS-to-Email Project 2 - 3 - --- 4 - 5 - # Herald - Issue Groups & Priorities 6 - 7 - ## **P0 - Critical (Security & Data Integrity)** ✅ COMPLETE 8 - 9 - - ✅ #14: No Rate Limiting (SSH, SCP, web, email) 10 - - ✅ #15: Token Generation (verify crypto/rand usage) 11 - 12 - ## **P1 - High (Performance & Reliability)** ✅ COMPLETE 13 - 14 - - ✅ #8: N+1 Query Problem (batch operations) 15 - - ✅ #26: No Cleanup of Old seen_items (6 month cleanup job) 16 - - ✅ #23: Missing Graceful Shutdown for Scheduler (panic recovery) 17 - 18 - ## **P2 - Medium (Code Quality & UX)** ✅ COMPLETE 19 - 20 - ### Group A: Input Validation 21 - 22 - - ✅ #16: No SMTP Auth Validation 23 - - ✅ #27: No Feed Validation on Upload 24 - - ✅ #37: No Cron Validation at Upload 25 - - ✅ #36: No Max File Size on SCP Upload 26 - 27 - ### Group B: Performance Tuning 28 - 29 - - ✅ #9: No Prepared Statements 30 - - ✅ #10: Inefficient Sorting in Handlers 31 - - ✅ #11: No HTTP Caching Headers 32 - 33 - ## **P3 - Low (Nice to Have)** ✅ COMPLETE 34 - 35 - ### Group C: Observability 36 - 37 - - ✅ #24: No Metrics/Observability 38 - - ✅ #35: HTTP Server Doesn't Log Requests 39 - - ✅ #22: Inconsistent Logging Levels 40 - 41 - ### Group D: Architecture & Scalability 42 - 43 - - #30: Scheduler Interval is Fixed 44 - - #31: No Pagination on Feed Endpoints 45 - 46 - ### Group E: Code Hygiene 47 - 48 - - ⚠️ #3: Context Timeout Duplication (not an issue) 49 - - ✅ #19: Magic Numbers 50 - - ✅ #18: Error Wrapping Inconsistency (already consistent) 51 - - ✅ #21: Unused Context Parameter 52 - - ✅ #33-34: Minor Code Cleanup 53 - 54 - ### Group F: Documentation 55 - 56 - - ✅ #28: Inconsistent Command Help 57 - - ✅ #29: Config Example Doesn't Match Defaults 58 - 59 - ### Group G: Testing 60 - 61 - - #20: No Tests 62 - 63 - --- 64 - 65 - ## **Critical Issues - COMPLETED** ✅ 66 - 67 - ### 1. **Database Connection Pool Not Configured** ✅ 68 - 69 - **Fixed:** Added WAL mode, busy timeout, and connection pool limits in `store/db.go:16-18` 70 - 71 - ### 2. **Code Duplication in Scheduler** ✅ 72 - 73 - **Fixed:** Refactored into shared `collectNewItems` and `sendDigestAndMarkSeen` helper methods 74 - 75 - ### 3. **No Context Timeout on HTTP Requests** 🔄 76 - 77 - **Location:** `scheduler/fetch.go:41-43` 78 - 79 - While you set a 30s timeout on the context, the HTTP client also has a separate 30s timeout. If the context times out first, the HTTP client won't respect it immediately. 80 - 81 - **Fix:** Use context-aware client without separate timeout. 82 - 83 - ### 4. **Missing Index on Configs Active Status** ✅ 84 - 85 - **Fixed:** Added partial index in `store/db.go:93` 86 - 87 - ### 5. **Race Condition in seen_items** ✅ 88 - 89 - **Fixed:** Using transactions to mark items seen before email send 90 - 91 - ### 6. **Unbounded Memory Growth in Feed Fetching** ✅ 92 - 93 - **Fixed:** Added semaphore limiting to 10 concurrent fetches in `scheduler/fetch.go` 94 - 95 - ### 7. **Silent Failure on Email Send** ✅ 96 - 97 - **Fixed:** Items only marked seen after successful email via transaction commit 98 - 99 - ### 14. **No Rate Limiting** ✅ 100 - 101 - **Fixed:** Added comprehensive rate limiting using `golang.org/x/time/rate`: 102 - - Created reusable `ratelimit.Limiter` with token bucket algorithm (`ratelimit/limiter.go`) 103 - - Web handler middleware: 10 req/sec, burst of 20 per IP (`web/server.go:65-77`) 104 - - SSH authentication: 5 req/sec, burst of 10 per fingerprint (`ssh/server.go:96-101`) 105 - - SCP uploads: 5 req/sec, burst of 10 per user (`ssh/scp.go:107-110`) 106 - - Email sending: 1 per minute per user (`scheduler/scheduler.go:207-210`) 107 - - Added 1MB max file size limit for SCP uploads (`ssh/scp.go:112-115`) 108 - - Rate limiter automatically cleans up inactive entries every 5 minutes 109 - 110 - ### 15. **Token Generation Not Cryptographically Secure** ✅ 111 - 112 - **Already secure:** Confirmed using `crypto/rand` in `store/unsubscribe.go:14` 113 - 114 - --- 115 - 116 - ## **Performance Issues** 117 - 118 - ### 8. **N+1 Query Problem** ✅ 119 - 120 - **Location:** Multiple locations 121 - 122 - - `web/handlers.go:99-103` - Gets feeds for each config in a loop 123 - - `scheduler/scheduler.go` - Checks each item individually for seen status 124 - 125 - **Fixed:** 126 - - Added `GetSeenGUIDs()` batch method in `store/items.go` to check multiple items at once 127 - - Added `GetFeedsByConfigs()` batch method in `store/feeds.go` to fetch feeds for multiple configs 128 - - Updated `scheduler/scheduler.go:collectNewItems()` to use batch GUID checking 129 - - Updated `web/handlers.go` dashboard handler to batch fetch all feeds 130 - 131 - ### 9. **No Prepared Statements** ✅ 132 - 133 - **Location:** All store methods 134 - 135 - **Fixed:** Added prepared statements for 7 most frequent queries: 136 - - `store/db.go:11-73` - Added `preparedStmts` struct and `prepareStatements()` method 137 - - Prepared statements for: markItemSeen, isItemSeen, getSeenItems, getConfig, updateConfigRun, updateFeedMeta, cleanupSeenItems 138 - - Updated store methods in `items.go`, `configs.go`, and `feeds.go` to use prepared statements 139 - - Statements closed on database close for proper cleanup 140 - 141 - ### 10. **Inefficient Sorting in Handlers** ✅ 142 - 143 - **Location:** `web/handlers.go:231-235` and `326-330` 144 - 145 - **Fixed:** 146 - - Added `rssItemWithTime` and `jsonFeedItemWithTime` wrapper structs in `web/handlers.go` 147 - - Parse timestamps once into `time.Time` field when building items 148 - - Sort by parsed `time.Time` directly (single comparison) 149 - - Convert back to original structs for encoding 150 - - Eliminates repeated `time.Parse()` calls during sorting 151 - 152 - ### 11. **No HTTP Caching Headers** ✅ 153 - 154 - **Location:** `web/handlers.go` - all feed handlers 155 - 156 - **Fixed:** 157 - - Added `Cache-Control: public, max-age=300` (5 minute cache) 158 - - Added `ETag` header with format "fingerprint-timestamp" 159 - - Added `Last-Modified` header from cfg.LastRun 160 - - Added conditional request handling for `If-None-Match` (ETag) 161 - - Added conditional request handling for `If-Modified-Since` 162 - - Returns 304 Not Modified when appropriate 163 - - Applied to both RSS and JSON feed handlers in `web/handlers.go` 164 - 165 - ### 12. **Database Migration Runs on Every Connection** 🔄 166 - 167 - **Location:** `store/db.go:26-28` 168 - 169 - Migration runs inside `Open()`, which happens once at startup. However, `Migrate()` is also exposed and called separately in `main.go:160`. The schema execution uses `CREATE TABLE IF NOT EXISTS` which is fine, but it's still unnecessary work. 170 - 171 - --- 172 - 173 - ## **Security Issues** 174 - 175 - ### 13. **Missing Input Validation on Email Addresses** ✅ 176 - 177 - **Already implemented:** Using `net/mail.ParseAddress()` in `config/validate.go:24-26` 178 - 179 - ### 16. **No SMTP Auth Validation** ✅ 180 - 181 - **Location:** `email/send.go:102-105` 182 - 183 - **Fixed:** 184 - - Added `ValidateConfig()` method in `email/send.go:33-95` 185 - - Validates SMTP connectivity and authentication at startup 186 - - Port 465: Uses implicit TLS connection 187 - - Port 587: Connects, then calls STARTTLS before authentication 188 - - Called from `main.go:172-175` after mailer creation 189 - - Returns error on connection, TLS, or auth failure 190 - 191 - ### 17. **SQL Injection Potential in UPSERT** 💉 192 - 193 - **Location:** `store/items.go:29-33` 194 - 195 - While using parameterized queries (good!), the `ON CONFLICT` clause should explicitly name the conflict target for clarity and safety: 196 - 197 - ```sql 198 - ON CONFLICT(feed_id, guid) DO UPDATE SET ... 199 - ``` 200 - 201 - (Actually you already do this correctly, but worth noting for other queries) 202 - 203 - --- 204 - 205 - ## **Code Quality Issues** 206 - 207 - ### 18. **Error Wrapping Inconsistency** ✅ 208 - 209 - **Analysis:** All fmt.Errorf calls already follow consistent "verb: %w" pattern with colon. No changes needed. 210 - 211 - ### 19. **Magic Numbers** 🎩 212 - 213 - **Location:** Multiple 214 - 215 - - `scheduler/scheduler.go:84` - hardcoded 3 months 216 - - `scheduler/scheduler.go:148-150` - hardcoded 5 items threshold 217 - - `web/handlers.go:238` and `332` - hardcoded 100 items limit 218 - - `scheduler/fetch.go:41` - hardcoded 30s timeout 219 - 220 - **Fix:** Extract to constants or config. 221 - 222 - ### 20. **No Tests** 🧪 223 - 224 - **Location:** Entire codebase 225 - 226 - Zero test coverage. Critical business logic (cron parsing, config parsing, email rendering) is untested. 227 - 228 - ### 21. **Unused Context Parameter** ✅ 229 - 230 - **Fixed:** 231 - - Removed unused ctx parameter from `store/db.go:Migrate()` method 232 - - Updated call site in `main.go` to match new signature 233 - 234 - ### 22. **Inconsistent Logging Levels** ✅ 235 - 236 - **Fixed:** 237 - - Established clear criteria: Error for critical failures, Warn for expected/recoverable failures 238 - - Changed 23 log calls from Error to Warn: 239 - - Database read operations in web handlers (14 calls) 240 - - Template rendering failures (6 calls) 241 - - Response encoding failures (2 calls) 242 - - Delete token operation (1 call) 243 - - All logging now follows consistent severity guidelines 244 - 245 - ### 23. **Missing Graceful Shutdown for Scheduler** ✅ 246 - 247 - **Location:** `main.go:194-197` 248 - 249 - **Fixed:** 250 - - Added panic recovery with defer in `scheduler/scheduler.go:tick()` 251 - - Added panic recovery wrapper in `main.go` scheduler goroutine 252 - - Added panic recovery in cleanup job 253 - 254 - --- 255 - 256 - ## **Missing Features** 257 - 258 - ### 24. **No Metrics/Observability** ✅ 259 - 260 - **Fixed:** 261 - - Added `/metrics` endpoint returning JSON with system and application metrics in `web/metrics.go` 262 - - Metrics include: uptime, goroutines, memory stats, request counts, emails sent, feeds fetched, items seen, active configs, errors, rate limit hits 263 - - Added `/health` endpoint for health checks 264 - - Metrics tracked via atomic counters for thread safety 265 - - Integrated into web server with proper HTTP headers 266 - 267 - ### 25. **No Email Validation on Successful Send** ✅ 268 - 269 - You log `"email sent"` but don't verify SMTP actually accepted it (some SMTP servers queue and fail later). 270 - 271 - ### 26. **No Cleanup of Old seen_items** ✅ 272 - 273 - **Fixed:** 274 - - Added `CleanupOldSeenItems()` method in `store/items.go` to delete items older than specified duration 275 - - Added cleanup ticker in `scheduler/scheduler.go:Start()` that runs every 24 hours 276 - - Cleanup runs on startup and then daily, removing items older than 6 months 277 - - Added logging for cleanup operations showing number of items deleted 278 - 279 - ### 27. **No Feed Validation on Upload** ✅ 280 - 281 - **Fixed:** 282 - - Added `ValidateFeedURLs()` function in `config/validate.go:1-48` 283 - - Fetches and parses each feed URL using gofeed parser 284 - - Uses 10 second context timeout, 5 second HTTP client timeout 285 - - Returns error if feed unreachable, returns non-200, or fails to parse 286 - - Called from `ssh/scp.go:136-147` during config upload 287 - - Validation happens after basic config validation, before database operations 288 - 289 - --- 290 - 291 - ## **Documentation Issues** 292 - 293 - ### 28. **Inconsistent Command Help** ✅ 294 - 295 - **Fixed:** 296 - - Updated welcome message in `ssh/server.go:168-177` to include all available commands 297 - - Added missing `activate` and `deactivate` commands to help text 298 - - Welcome message now matches actual available commands in `ssh/commands.go` 299 - 300 - ### 29. **Config Example Doesn't Match Defaults** ✅ 301 - 302 - **Fixed:** 303 - - Updated `README.md:89` to show `inline` default as `false` (matching code in `config/parse.go:27`) 304 - - Documentation now consistent with actual code behavior 305 - 306 - --- 307 - 308 - ## **Architectural Concerns** 309 - 310 - ### 30. **Scheduler Interval is Fixed** ⏲️ 311 - 312 - **Location:** `main.go:172` 313 - 314 - 60-second interval is hardcoded. This doesn't scale well—if you have thousands of users, checking every 60 seconds is wasteful. Consider event-driven scheduling with a priority queue. 315 - 316 - ### 31. **No Pagination on Feed Endpoints** 📄 317 - 318 - **Location:** `web/handlers.go:238` and `332` 319 - 320 - Hardcoded limit of 100 items. Users can't access older items. 321 - 322 - ### 32. **No Transaction for Config Update** ✅ 323 - 324 - **Fixed:** Config upload now uses transactions in `ssh/scp.go:134-161` 325 - 326 - --- 327 - 328 - ## **Minor Issues** 329 - 330 - 33. ✅ **Unused `getCommitHash()` function** - Fixed: Inlined into `main.go:179-195` 331 - 34. ✅ **Inconsistent fingerprint shortening** - Fixed: Standardized to 8 chars using `shortFingerprintLen` constant in `web/handlers.go` 332 - 35. ✅ **HTTP server doesn't log requests** - Fixed: Added logging middleware in `web/server.go:90-116` 333 - 36. ✅ **No max file size on SCP upload** - Fixed: 1MB limit in `ssh/scp.go:112-115` 334 - 37. ✅ **No validation on cron expressions at upload time** - Already implemented via `config.Validate()` and `calculateNextRun()` in config validation 335 - 336 - --- 337 - 338 - ## **Positive Notes** ✅ 339 - 340 - - Good use of Context for cancellation 341 - - Proper use of foreign keys and CASCADE 342 - - Clean separation of concerns (store/scheduler/ssh/web) 343 - - Good use of Charm libraries 344 - - ETag/Last-Modified support for feed fetching 345 - - Unsubscribe functionality implemented 346 - - SQL injection protection with parameterized queries 347 - - Config file validation before accepting uploads 348 - 349 - --- 350 - 351 - ## **Priority Fixes** 352 - 353 - ### **High Priority (Fix ASAP):** 354 - 355 - 1. ✅ Database connection pool configuration (#1) 356 - 2. ✅ Race condition in seen_items (#5) 357 - 3. ✅ Silent failure on email send (#7) 358 - 4. ✅ No rate limiting (#14) 359 - 5. ✅ No transaction for config updates (#32) 360 - 6. ✅ Token generation security (#15) 361 - 7. ✅ Max file size on SCP upload (#36) 362 - 363 - ### **Medium Priority:** 364 - 365 - 6. ✅ Code duplication in scheduler (#2) 366 - 7. N+1 query problems (#8) 367 - 8. ✅ Unbounded feed fetching concurrency (#6) 368 - 9. ✅ Missing input validation (#13) 369 - 10. No cleanup of old data (#26) 370 - 371 - ### **Low Priority (Technical Debt):** 372 - 373 - 11. Add tests (#20) 374 - 12. Add metrics (#24) 375 - 13. Extract magic numbers (#19) 376 - 14. Add HTTP caching (#11)
-434
spec.md
··· 1 - # rss2email-ssh 2 - 3 - A minimal, SSH-powered RSS to email service. Upload a feed config via SCP, get email digests on a schedule. 4 - 5 - ## Goals 6 - 7 - - **Simple**: Single binary, SQLite storage, no external dependencies 8 - - **SSH-native**: Auth via SSH keys, configure via SCP or interactive TUI 9 - - **Pico-compatible**: Same config format as pico.sh/feeds 10 - - **Charm-powered**: Built with the Charm ecosystem 11 - 12 - ## Architecture 13 - 14 - ``` 15 - ┌─────────────────────────────────────────┐ 16 - │ rss2email-ssh │ 17 - ├─────────────────────────────────────────┤ 18 - │ │ 19 - SSH key auth │ ┌─────────────┐ ┌──────────────┐ │ 20 - ─────────────────┼─▶│ wish │───▶│ bubbletea │ │ 21 - │ │ (SSH srv) │ │ (TUI) │ │ 22 - │ └─────────────┘ └──────────────┘ │ 23 - │ │ │ 24 - SCP upload │ ▼ │ 25 - ─────────────────┼─▶┌─────────────┐ ┌──────────────┐ │ 26 - │ │ wish/scp │───▶│ SQLite │ │ 27 - │ │ (files) │ │ (store) │ │ 28 - │ └─────────────┘ └──────┬───────┘ │ 29 - │ │ │ 30 - │ ▼ │ 31 - │ ┌──────────────┐ │ 32 - │ │ Scheduler │ │ 33 - │ │ (cron loop) │ │ 34 - │ └──────┬───────┘ │ 35 - │ │ │ 36 - │ ▼ │ 37 - │ ┌──────────────┐ │ 38 - │ │ SMTP out │──────▶ Email 39 - │ └──────────────┘ │ 40 - └─────────────────────────────────────────┘ 41 - ``` 42 - 43 - ## Charm Libraries 44 - 45 - | Library | Purpose | 46 - |---------|---------| 47 - | [wish](https://github.com/charmbracelet/wish) | SSH server, middleware, SCP handling | 48 - | [lipgloss](https://github.com/charmbracelet/lipgloss) | Styling CLI output | 49 - | [log](https://github.com/charmbracelet/log) | Structured logging | 50 - 51 - ## Config Format 52 - 53 - Pico-compatible plaintext format. Users upload as `feeds.txt` (or any `.txt` file): 54 - 55 - ```text 56 - =: email you@example.com 57 - =: cron 0 8 * * * 58 - =: digest true 59 - =: inline false 60 - => https://blog.example.com/rss 61 - => https://news.ycombinator.com/rss 62 - => https://lobste.rs/rss "Lobsters" 63 - ``` 64 - 65 - ### Directives 66 - 67 - | Directive | Required | Description | 68 - |-----------|----------|-------------| 69 - | `=: email <addr>` | Yes | Recipient email address | 70 - | `=: cron <expr>` | Yes | Standard cron expression (5 fields) | 71 - | `=: digest <bool>` | No | Combine all items into one email (default: true) | 72 - | `=: inline <bool>` | No | Include article content in email (default: false) | 73 - | `=> <url> ["name"]` | Yes (1+) | RSS/Atom feed URL, optional display name | 74 - 75 - Note: Items are filtered to only include those published within the last 3 months. 76 - 77 - ## Data Model 78 - 79 - ### SQLite Schema 80 - 81 - ```sql 82 - -- Users identified by SSH public key fingerprint 83 - CREATE TABLE users ( 84 - id INTEGER PRIMARY KEY, 85 - pubkey_fp TEXT UNIQUE NOT NULL, -- SHA256 fingerprint 86 - pubkey TEXT NOT NULL, -- Full public key 87 - created_at DATETIME DEFAULT CURRENT_TIMESTAMP 88 - ); 89 - 90 - -- Feed configurations (one per uploaded file) 91 - CREATE TABLE configs ( 92 - id INTEGER PRIMARY KEY, 93 - user_id INTEGER NOT NULL REFERENCES users(id), 94 - filename TEXT NOT NULL, 95 - email TEXT NOT NULL, 96 - cron_expr TEXT NOT NULL, 97 - digest BOOLEAN DEFAULT TRUE, 98 - inline_content BOOLEAN DEFAULT FALSE, 99 - raw_text TEXT NOT NULL, -- Original file contents 100 - last_run DATETIME, 101 - next_run DATETIME, 102 - created_at DATETIME DEFAULT CURRENT_TIMESTAMP, 103 - UNIQUE(user_id, filename) 104 - ); 105 - 106 - -- Individual feeds within a config 107 - CREATE TABLE feeds ( 108 - id INTEGER PRIMARY KEY, 109 - config_id INTEGER NOT NULL REFERENCES configs(id) ON DELETE CASCADE, 110 - url TEXT NOT NULL, 111 - name TEXT, -- Optional display name 112 - last_fetched DATETIME, 113 - etag TEXT, -- For conditional requests 114 - last_modified TEXT 115 - ); 116 - 117 - -- Seen items for deduplication 118 - CREATE TABLE seen_items ( 119 - id INTEGER PRIMARY KEY, 120 - feed_id INTEGER NOT NULL REFERENCES feeds(id) ON DELETE CASCADE, 121 - guid TEXT NOT NULL, -- Item GUID or link hash 122 - title TEXT, 123 - link TEXT, 124 - seen_at DATETIME DEFAULT CURRENT_TIMESTAMP, 125 - UNIQUE(feed_id, guid) 126 - ); 127 - ``` 128 - 129 - ## SSH Interface 130 - 131 - ### Authentication 132 - 133 - - Public key auth only (no passwords) 134 - - First connection auto-registers user by key fingerprint 135 - - Config file for allowed keys (optional, default: allow all) 136 - 137 - ```yaml 138 - # config.yaml 139 - allow_all_keys: true # or false to require explicit registration 140 - allowed_keys: 141 - - "ssh-ed25519 AAAA... user@host" 142 - ``` 143 - 144 - ### SCP Upload 145 - 146 - ```bash 147 - # Upload a feed config 148 - scp feeds.txt user@rss.example.com: 149 - 150 - # Upload with custom name 151 - scp feeds.txt user@rss.example.com:work-feeds.txt 152 - 153 - # Download existing config 154 - scp user@rss.example.com:feeds.txt . 155 - ``` 156 - 157 - ### CLI Commands 158 - 159 - Via SSH command execution: 160 - 161 - ```bash 162 - # List configs 163 - ssh rss.example.com ls 164 - 165 - # Show config contents 166 - ssh rss.example.com cat feeds.txt 167 - 168 - # Delete a config 169 - ssh rss.example.com rm feeds.txt 170 - 171 - # Run immediately (don't wait for cron) 172 - ssh rss.example.com run feeds.txt 173 - 174 - # Show recent activity 175 - ssh rss.example.com logs 176 - ``` 177 - 178 - ## Web Interface 179 - 180 - Minimal brutalist web UI (à la pierre.computer). Serves two purposes: 181 - 182 - 1. **Public RSS feed** - Aggregated feed of all items for a user 183 - 2. **Config view** - Shows the raw config file 184 - 185 - ### Routes 186 - 187 - ``` 188 - GET / # Landing page 189 - GET /{fingerprint} # User's public page 190 - GET /{fingerprint}/feed.xml # Aggregated RSS feed 191 - GET /{fingerprint}/feed.json # JSON Feed format 192 - GET /{fingerprint}/{config} # Raw config file 193 - ``` 194 - 195 - ### User Page 196 - 197 - ``` 198 - /{fingerprint} 199 - ``` 200 - 201 - ``` 202 - RSS2EMAIL █ 203 - 204 - ~~~ 205 - 206 - USER: a]D+3xKL... 207 - STATUS: ONLINE 208 - NEXT RUN: 2025-01-09 08:00 UTC 209 - 210 - ~~~ 211 - 212 - CONFIGS: 213 - - [feeds.txt](/abc123/feeds.txt) (3 feeds) 214 - - [work.txt](/abc123/work.txt) (5 feeds) 215 - 216 - FEEDS: 217 - - [RSS](/abc123/feed.xml) 218 - - [JSON](/abc123/feed.json) 219 - ``` 220 - 221 - ### Aggregated Feed 222 - 223 - `/{fingerprint}/feed.xml` returns a standard RSS 2.0 feed containing all items from all the user's subscribed feeds - essentially a "river of news" feed they can subscribe to elsewhere. 224 - 225 - ### Styling 226 - 227 - ```css 228 - * { 229 - font-family: monospace; 230 - background: #000; 231 - color: #fff; 232 - } 233 - 234 - a { 235 - color: #fff; 236 - text-decoration: underline; 237 - } 238 - 239 - pre { 240 - white-space: pre-wrap; 241 - } 242 - ``` 243 - 244 - Single HTML template, no JS, ~20 lines of CSS. 245 - 246 - ## Scheduler 247 - 248 - Background goroutine that: 249 - 250 - 1. Every 60 seconds, queries `configs` where `next_run <= now()` 251 - 2. For each due config: 252 - - Fetch all feeds (parallel, with timeout) 253 - - Filter to unseen items (check `seen_items` table) 254 - - If new items exist, render and send email 255 - - Update `last_run`, calculate and set `next_run` 256 - - Insert new items into `seen_items` 257 - 3. Handle errors gracefully (log, increment retry counter) 258 - 259 - ### Cron Parsing 260 - 261 - Use [adhocore/gronx](https://github.com/adhocore/gronx) for cron expression parsing (same as pico). 262 - 263 - ## Email Rendering 264 - 265 - ### Digest Mode (default) 266 - 267 - One email per config run containing all new items: 268 - 269 - ``` 270 - Subject: RSS Digest: 5 new items 271 - From: rss@example.com 272 - To: you@example.com 273 - Content-Type: multipart/alternative 274 - 275 - ────────────────────────────────────── 276 - Lobsters (2 new) 277 - ────────────────────────────────────── 278 - 279 - ▸ Show HN: I built a thing 280 - https://example.com/article1 281 - 282 - ▸ Why Rust is great 283 - https://example.com/article2 284 - 285 - ────────────────────────────────────── 286 - Example Blog (3 new) 287 - ────────────────────────────────────── 288 - 289 - ▸ My latest post 290 - https://blog.example.com/post1 291 - 292 - ... 293 - ``` 294 - 295 - ### Individual Mode 296 - 297 - One email per item (when `digest: false`). 298 - 299 - ### Templates 300 - 301 - Use Go `html/template` and `text/template` for HTML and plaintext versions. 302 - 303 - ## Project Structure 304 - 305 - ``` 306 - rss2email-ssh/ 307 - ├── main.go # Entry point, CLI flags 308 - ├── ssh/ 309 - │ ├── server.go # wish server setup, middleware 310 - │ ├── scp.go # SCP upload/download handlers 311 - │ └── commands.go # ls, rm, cat, run, logs 312 - ├── web/ 313 - │ ├── server.go # HTTP server 314 - │ ├── handlers.go # Route handlers 315 - │ └── templates/ 316 - │ ├── index.html 317 - │ ├── user.html 318 - │ └── style.css 319 - ├── config/ 320 - │ ├── parse.go # Parse pico-format config files 321 - │ └── validate.go # Validation logic 322 - ├── store/ 323 - │ ├── db.go # SQLite connection, migrations 324 - │ ├── users.go # User CRUD 325 - │ ├── configs.go # Config CRUD 326 - │ └── items.go # Seen items tracking 327 - ├── scheduler/ 328 - │ ├── scheduler.go # Main loop 329 - │ └── fetch.go # RSS fetching with gofeed 330 - ├── email/ 331 - │ ├── render.go # Template rendering 332 - │ ├── send.go # SMTP sending 333 - │ └── templates/ 334 - │ ├── digest.html 335 - │ └── digest.txt 336 - ├── go.mod 337 - ├── go.sum 338 - └── config.example.yaml 339 - ``` 340 - 341 - ## Configuration 342 - 343 - Server configuration via YAML or environment variables: 344 - 345 - ```yaml 346 - # config.yaml 347 - host: 0.0.0.0 348 - port: 2222 349 - 350 - # SSH host keys (generated on first run if missing) 351 - host_key_path: ./host_key 352 - 353 - # Database 354 - db_path: ./rss2email.db 355 - 356 - # SMTP 357 - smtp: 358 - host: smtp.example.com 359 - port: 587 360 - user: sender@example.com 361 - pass: ${SMTP_PASS} # Env var substitution 362 - from: rss@example.com 363 - 364 - # Auth 365 - allow_all_keys: true 366 - ``` 367 - 368 - ## Dependencies 369 - 370 - ```go 371 - require ( 372 - github.com/charmbracelet/wish v1.4.0 373 - github.com/charmbracelet/lipgloss v1.0.0 374 - github.com/charmbracelet/log v0.4.0 375 - github.com/mmcdole/gofeed v1.3.0 376 - github.com/adhocore/gronx v1.19.0 377 - github.com/mattn/go-sqlite3 v1.14.24 378 - gopkg.in/yaml.v3 v3.0.1 379 - ) 380 - ``` 381 - 382 - ## Implementation Phases 383 - 384 - ### Phase 1: Core (MVP) 385 - 386 - - [ ] SSH server with key auth (wish) 387 - - [ ] SCP upload/download 388 - - [ ] Config parsing (pico format) 389 - - [ ] SQLite storage 390 - - [ ] Basic scheduler 391 - - [ ] Plaintext email sending 392 - 393 - ### Phase 2: Polish 394 - 395 - - [ ] Web UI (brutalist style) 396 - - [ ] Aggregated RSS/JSON feeds 397 - - [ ] HTML emails 398 - - [ ] Conditional fetching (ETag/Last-Modified) 399 - - [ ] `logs` command 400 - 401 - ### Phase 3: Nice-to-have 402 - 403 - - [ ] OPML import/export 404 - - [ ] Feed discovery (find RSS from URL) 405 - - [ ] Webhook notifications 406 - - [ ] Metrics endpoint 407 - 408 - ## Example Session 409 - 410 - ```bash 411 - # First time setup - just SSH in 412 - $ ssh rss.example.com 413 - Welcome! Your account has been created. 414 - Upload a config with: scp feeds.txt rss.example.com: 415 - 416 - # Upload a config 417 - $ cat feeds.txt 418 - =: email me@example.com 419 - =: cron 0 8 * * * 420 - => https://lobste.rs/rss 421 - 422 - $ scp feeds.txt rss.example.com: 423 - feeds.txt 100% 89 12.3KB/s 00:00 424 - Config saved! Next run: tomorrow at 8:00 AM 425 - 426 - # Check status 427 - $ ssh rss.example.com ls 428 - feeds.txt 1 feed next: 8:00 AM 429 - 430 - # Run immediately 431 - $ ssh rss.example.com run feeds.txt 432 - Fetched 25 items, 25 new 433 - Email sent to me@example.com 434 - ```