paperbnd.koplugin / CLAUDE.md

# Paperbnd KOReader Plugin - Developer Documentation

> **Document Purpose**: This is technical documentation for developers and maintainers. For user-facing installation and usage instructions, see [README.md](README.md).

A KOReader plugin for syncing reading progress to an AT Protocol PDS using the Popfeed List Item lexicon.

## Overview

Paperbnd allows KOReader users to sync their reading progress to their AT Protocol Personal Data Server (PDS) via the Popfeed social reading platform. The plugin tracks page progress and synchronizes it with existing Popfeed book entries.

## Architecture

The plugin consists of two main components:

### 1. XRPC Client (`xrpc.lua`)

A dedicated module for AT Protocol/XRPC communication that handles:

- **Generic XRPC method invocation** with support for GET and POST requests
- **PDS resolution** via the Slingshot service (Microcosm)
- **Advanced session management** with multi-level token renewal and automatic retry
- **Authentication** using handle and app password
- **Record operations** (list, get, put) for AT Protocol collections

**Key Features:**
- Table-based parameters for flexibility (method, params, body, pds, skip_renewal)
- Multi-level session renewal: refresh token → app password fallback → automatic retry
- Session validation on plugin startup to proactively renew expired tokens
- Authorization headers only on POST requests (GET requests never authenticated)
- Callback support for persisting refreshed tokens to plugin settings
- Handles both 401 and 400 "expired" errors as authentication failures
- Prevents infinite recursion during renewal with `skip_renewal` flag

### 2. Main Plugin (`main.lua`)

The primary plugin file that integrates with KOReader:

- **Settings management** with persistent storage of credentials and book mappings
- **Authentication flow** with two-step credential input (handle, then app password)
- **Book linking** allows users to select from their existing Popfeed list items
- **Progress sync** tracks current page, total pages, and percentage
- **Document hooks** for automatic sync on document close
- **Menu integration** with conditional menu items based on authentication state

## User Workflow

1. **Set Credentials**: User enters their AT Protocol handle and app password
   - Plugin resolves PDS URL via Slingshot service
   - Creates session and stores access/refresh tokens

2. **Link Book**: User selects current document and links it to an existing Popfeed book
   - Fetches user's list items from their PDS
   - Filters for books only
   - Stores mapping between document path and list item rkey

3. **Sync Progress**: Reading progress is synchronized automatically
   - Updates `bookProgress` field with current page, total pages, and percentage
   - Updates `listType` to "currently_reading_books" if needed
   - Can be triggered manually or automatically on document close

4. **Session Management**: Tokens are automatically renewed with multi-level fallback
   - Session validated at plugin startup to ensure fresh tokens
   - Authentication errors (401 or 400 "expired") trigger automatic renewal
   - First attempts refresh with refresh token
   - Falls back to creating new session with app password if refresh token expired
   - Original request automatically retried after successful renewal
   - New tokens saved to settings transparently via callback
   - No user intervention required at any point

## Technical Details

### External Resources

- **KOReader Plugin Development**: https://github.com/koreader/koreader/wiki/Developer-documentation
- **AT Protocol Specifications**: https://atproto.com/specs/atp
- **Popfeed Platform**: https://popfeed.social/
- **Paperbnd Website**: https://paperbnd.club/

### Lexicons Used

- `social.popfeed.feed.listItem` - Individual book entries with progress tracking
- `social.popfeed.feed.list` - Reading lists (e.g., "currently_reading_books")
- `com.atproto.server.createSession` - Initial authentication
- `com.atproto.server.refreshSession` - Token refresh
- `com.atproto.repo.listRecords` - Fetch book collections
- `com.atproto.repo.getRecord` - Fetch individual book records
- `com.atproto.repo.putRecord` - Update book records with progress

### Data Storage

Settings are persisted in `paperbnd.lua` within KOReader's settings directory:
- Handle and app password
- PDS URL
- Access and refresh tokens
- User DID (Decentralized Identifier)
- Document-to-listItem mappings (document path -> {rkey, title, author})

### Book Progress Format

```lua
bookProgress = {
    status = "in_progress",
    percent = 42,           -- Calculated percentage
    currentPage = 150,      -- From KOReader document stats
    totalPages = 357,       -- From KOReader document stats
    updatedAt = "2025-10-20T14:30:00.000Z"  -- ISO 8601 timestamp
}
```

## Design Decisions

### Simplicity First
- Single-file XRPC client for clean separation of concerns
- No future feature speculation - only implemented what's needed
- Direct and straightforward code without unnecessary abstractions

### User Book Selection
Rather than attempting to match books by ISBN or title, users manually link documents to existing Popfeed list items. This approach:
- Avoids complex ISBN DB API integration
- Ensures accuracy (user confirms the correct book)
- Works with any book format supported by KOReader
- Leverages existing Popfeed records as the source of truth

### Advanced Session Management

The plugin implements a sophisticated multi-level session management system that is the core reliability feature.

**Architecture:**

The session management system has three levels of components:

1. **`call()` method in xrpc.lua**: Generic XRPC request handler that detects auth errors (401 or 400 "expired"), calls `renewSession()` on failure, rebuilds requests with new tokens, and retries automatically. This is the single point of entry for all XRPC operations.

2. **`renewSession()` method**: Multi-level fallback coordinator that first attempts `refreshSession()`, then falls back to `createSession()` with app password if refresh token expired. Returns boolean success status.

3. **`validateSession()` method**: Makes lightweight test request at plugin startup to verify token validity and proactively renew expired tokens before user operations.

**Integration with Main Plugin:**

- **Initialization**: Loads credentials from persistent storage, configures XRPC client with tokens and app password, and sets callback for automatic token persistence
- **Token Callback**: Receives new tokens from XRPC client, saves to plugin settings immediately, ensuring credentials never get out of sync
- **Startup Validation**: Runs `validateSession()` silently when plugin loads to ensure tokens are fresh before first use

**Automatic Renewal Flow:**

1. Authentication error detected (401 or 400 "expired")
2. Attempt to refresh using refresh token
3. If refresh token expired, create new session with stored app password
4. If renewal successful, retry the original request automatically
5. Save new tokens to persistent storage via callback

**Example Flow - User syncs progress with expired access token:**

1. `syncProgress()` calls `xrpc:putRecord()`
2. `putRecord()` calls `call()` with POST body
3. Server returns 401 Unauthorized
4. `call()` detects auth error, calls `renewSession()`
5. `renewSession()` tries `refreshSession()`
6. If refresh token valid: new tokens received, callback fired
7. If refresh token expired: `createSession()` with app password
8. `call()` rebuilds request with new access token
9. `call()` retries POST request
10. Request succeeds, progress synced
11. User sees success message, unaware of token renewal

**Benefits:**

- No manual "Refresh Session" button needed
- Tokens can expire for weeks without user intervention
- Original requests automatically retried after renewal
- Seamless user experience without interruptions
- Callback mechanism keeps plugin and XRPC client in sync
- Proactive validation prevents mid-operation failures

### Authentication Strategy
- Uses Slingshot service to resolve PDS from handle (no manual PDS entry)
- App passwords for security (no main account passwords stored)
- Refresh tokens enable long-lived sessions without re-authentication

## Implementation Notes

### Reference Implementation
Built by studying the ReadwiseReader KOReader plugin for:
- Plugin structure and initialization patterns
- UI component creation (menus, dialogs, input forms)
- HTTP request handling with `socket.http` and `ltn12`
- JSON encoding/decoding with `rapidjson`
- Settings persistence with `LuaSettings`
- Document metadata access and event hooks

### AT Protocol Specifics
- GET requests never require authentication headers
- POST requests use Bearer token authentication
- Refresh tokens are used as access tokens for the refresh endpoint
- Both access and refresh tokens are rotated on each refresh
- Session validation uses `com.atproto.server.getSession` endpoint
- Authentication errors can return 401 or 400 with "expired" message

### Error Handling
- User-friendly error messages via `InfoMessage` widgets
- Graceful degradation when operations fail
- Network errors handled with appropriate fallbacks

## Code Quality Assessment

**Current State (December 2024):**

The codebase is well-structured and production-ready with the following characteristics:

**Strengths:**
- Clean separation of concerns (XRPC client vs. plugin logic)
- Comprehensive error handling with user-friendly messages
- Robust session management with multiple fallback mechanisms
- Proper state synchronization between components via callbacks
- No memory leaks or blocking operations
- Defensive programming with validation at critical points
- Clear function naming and logical organization

**Architecture:**
- **XRPC Client (353 lines)**: Self-contained networking layer with 15 public methods
- **Main Plugin (446 lines)**: UI integration with 17 public methods
- **Metadata (6 lines)**: KOReader plugin manifest
- **Total**: 805 lines of well-documented Lua code

**Known Limitations:**
- One TODO in `main.lua` for updating `listUri` when changing reading status (currently commented out at lines 413-416). Implementation would require fetching the user's "currently_reading_books" list URI and updating the `listUri` field when moving a book to that list.
- Session validation at startup runs synchronously (could be async but startup is fast enough)
- No retry logic for network failures (only authentication failures)
- No offline queuing (by design, for simplicity)

**Testing Considerations:**
- Manual testing required (KOReader plugin environment)
- Test scenarios: expired access token, expired refresh token, network errors
- Edge cases: malformed responses, missing credentials, unlinked books

## Future Enhancements (Not Implemented)

Potential features intentionally left out for simplicity:

- **Automatic book detection via ISBN**: Query ISBN databases to automatically match books instead of manual linking
- **Batch syncing multiple books**: Sync progress for all linked books at once
- **Conflict resolution for concurrent edits**: Handle cases where book data is modified from multiple clients
- **Offline queue for syncing**: Queue progress updates when network unavailable and sync when connection restored
- **Automatic list URI updates**: When changing a book's reading status (e.g., to "currently_reading_books"), automatically fetch and update the `listUri` field to point to the correct list. Currently commented out in code - would require additional `listRecords` call to fetch the user's lists and find the matching URI.
- **Retry logic for transient network failures**: Automatically retry failed requests due to temporary network issues
- **Progress indicators for long-running operations**: Show loading spinners or progress bars for network requests

## Files

- `main.lua` - Main plugin with UI and KOReader integration (446 lines)
- `xrpc.lua` - XRPC client for AT Protocol communication (353 lines)
- `_meta.lua` - KOReader plugin manifest (6 lines)
- `README.md` - User-facing documentation with installation and usage instructions
- `CLAUDE.md` - This technical documentation for developers and maintainers

## Troubleshooting

### Common Issues

**"Failed to resolve PDS" error**
- Verify handle is correct (e.g., "user.bsky.social")
- Check internet connection
- Confirm Slingshot service (slingshot.microcosm.blue) is accessible

**"Authentication failed" error**
- Ensure app password is correct (not your main account password)
- Create a new app password if needed from your account settings
- Verify handle format matches your AT Protocol identifier

**"Failed to fetch books: No books found"**
- Books must be added to Popfeed/Paperbnd first before linking
- Books must be in a "currently_reading_books" list
- Use the Popfeed website or app to add books to your account

**"Failed to sync progress" error**
- Check that the book is still linked (may have been unlinked)
- Verify internet connection
- Token renewal should happen automatically, but if persisting, try re-entering credentials

**Book not appearing in link list**
- Ensure book has `creativeWorkType` set to "book"
- Verify book is in "currently_reading_books" list type
- Try unlinking and re-linking the book

### Debug Tips

- Check KOReader logs for detailed error messages
- Verify credentials are saved by checking settings file at `<KOReader settings dir>/paperbnd.lua`
- Test network connectivity with other KOReader plugins
- Ensure document has page count metadata (required for progress percentage)

## Development

The plugin was developed collaboratively with Claude (Anthropic's AI assistant) with these priorities:
- Clean, readable code
- Comprehensive error handling
- User experience focused on simplicity
- No emojis in documentation or messages
- Direct communication style in code comments
- Proactive reliability through startup validation