···4455(Also [on Tangled!](https://tangled.org/@did:plc:ofrbh253gwicbkc5nktqepol/atproto-lastfm-importer))
6677+## Features
88+99+- ✅ **Rate Limiting**: Automatically limits imports to 1K records per day to prevent rate limiting your entire PDS
1010+- ✅ **Multi-Day Imports**: Large imports (>1K records) automatically span multiple days with 24-hour pauses
1111+- ✅ **Resume Support**: Safe to stop (Ctrl+C) and restart - continues from where it left off
1212+- ✅ **Graceful Cancellation**: Press Ctrl+C to stop after the current batch completes
1313+- ✅ **Identity Resolution**: Resolves ATProto handles/DIDs using Slingshot
1414+- ✅ **PDS Auto-Discovery**: Automatically connects to your personal PDS
1515+- ✅ **Dry Run Mode**: Preview records without publishing
1616+- ✅ **Batch Processing**: Configurable batching with rate limit safety
1717+- ✅ **Progress Tracking**: Real-time progress with time estimates
1818+- ✅ **Error Handling**: Continues on errors with detailed reporting
1919+- ✅ **MusicBrainz Support**: Preserves MusicBrainz IDs when available
2020+- ✅ **Chronological Ordering**: Processes oldest first (or newest with `-r` flag)
2121+2222+## Important: Rate Limits
2323+2424+⚠️ **CRITICAL**: Bluesky's AppView has rate limits on PDS instances. Exceeding 10K records per day can rate limit your **ENTIRE PDS**, affecting all users on your instance!
2525+2626+This importer automatically:
2727+- Limits imports to **1,000 records per day** (90% of safe limit)
2828+- Calculates optimal batch sizes and delays
2929+- Pauses 24 hours between days for large imports
3030+- Shows clear progress and time estimates
3131+3232+See: [Bluesky Rate Limits Documentation](https://docs.bsky.app/blog/rate-limits-pds-v3)
3333+734## Setup
835936```bash
1037npm install
3838+npm run build
1139```
12401341## Usage
14421543### Interactive Mode
4444+4545+The simplest way to use the importer - just run it and follow the prompts:
16461747```bash
1818-node importer.js
4848+npm start
1949```
20502121-### With Command Line Arguments
5151+### Command Line Mode
22522323-**Full automation:**
5353+For automation or scripting, provide all parameters via flags:
24542555```bash
2626-node importer.js -f lastfm.csv -i alice.bsky.social -p xxxx-xxxx-xxxx-xxxx -y
2727-```
5656+# Full automation
5757+npm start -- -f lastfm.csv -i alice.bsky.social -p xxxx-xxxx-xxxx-xxxx -y
28582929-**Dry run (preview without publishing):**
5959+# Preview without publishing
6060+npm start -- -f lastfm.csv --dry-run
30613131-```bash
3232-node importer.js -f lastfm.csv --dry-run
6262+# Custom batch settings (advanced users)
6363+npm start -- -f lastfm.csv -i alice.bsky.social -b 20 -d 3000
6464+6565+# Process newest tracks first
6666+npm start -- -f lastfm.csv -i alice.bsky.social -r -y
3367```
34683535-**Custom batch settings:**
6969+## Command Line Options
7070+7171+| Option | Short | Description | Default |
7272+|--------|-------|-------------|---------|
7373+| `--help` | `-h` | Show help message | - |
7474+| `--file <path>` | `-f` | Path to Last.fm CSV export file | (prompted) |
7575+| `--identifier <id>` | `-i` | ATProto handle or DID | (prompted) |
7676+| `--password <pass>` | `-p` | ATProto app password | (prompted) |
7777+| `--batch-size <num>` | `-b` | Records per batch | Auto-calculated |
7878+| `--batch-delay <ms>` | `-d` | Delay between batches in ms | 2000 (min: 1000) |
7979+| `--yes` | `-y` | Skip confirmation prompt | false |
8080+| `--dry-run` | `-n` | Preview without publishing | false |
8181+| `--reverse-chronological` | `-r` | Process newest first | false (oldest first) |
36823737-```bash
3838-node importer.js -f lastfm.csv -i alice.bsky.social -b 20 -d 3000
3939-```
8383+### Batch Settings
8484+8585+The importer automatically calculates optimal batch settings based on your total record count and rate limits. You generally **don't need** to specify batch settings unless you have specific requirements.
8686+8787+**Automatic behavior:**
8888+- For imports < 1K records: Uses default settings (10 records/batch, 2s delay)
8989+- For imports > 1K records: Automatically calculates settings to spread across multiple days
40904141-## Options
9191+**Manual override** (advanced):
9292+- `--batch-size`: Number of records processed per batch (1-50)
9393+- `--batch-delay`: Milliseconds to wait between batches (min: 1000)
42944343-- `-h, --help` - Show help message
4444-- `-f, --file <path>` - Path to Last.fm CSV export file
4545-- `-i, --identifier <id>` - ATProto handle or DID
4646-- `-p, --password <pass>` - ATProto app password
4747-- `-b, --batch-size <num>` - Records per batch (default: 10)
4848-- `-d, --batch-delay <ms>` - Delay between batches in ms (default: 2000)
4949-- `-y, --yes` - Skip confirmation prompt
5050-- `-n, --dry-run` - Preview records without publishing
9595+⚠️ Lower delays increase speed but risk hitting rate limits. The automatic calculation is recommended.
51965297## Getting Your Last.fm Data
539854991. Go to <https://lastfm.ghan.nl/export/>
5555-2. Request your data export in CSV
100100+2. Request your data export in CSV format
561013. Download the CSV file when ready
571024. Use the CSV file path with this script
581035959-## Features
104104+## What Gets Imported
105105+106106+Each Last.fm scrobble becomes an `fm.teal.alpha.feed.play` record with:
601076161-- ✅ Resolves ATProto handles/DIDs using Slingshot
6262-- ✅ Connects to your personal PDS
6363-- ✅ Converts Last.fm scrobbles to `fm.teal.alpha.feed.play` records
6464-- ✅ Follows the official lexicon schema
6565-- ✅ Batch publishing with configurable rate limiting
6666-- ✅ Dry run mode for previewing
6767-- ✅ Progress tracking and error reporting
6868-- ✅ Preserves MusicBrainz IDs when available
108108+### Required Fields
109109+- **trackName**: The name of the track
110110+- **artists**: Array of artist objects (requires `artistName`, optional `artistMbId`)
111111+- **playedTime**: ISO 8601 timestamp of when you listened
112112+- **submissionClientAgent**: Identifies this importer (`lastfm-importer/v0.0.2`)
113113+- **musicServiceBaseDomain**: Always set to `last.fm`
691147070-## Record Format
115115+### Optional Fields (when available)
116116+- **releaseName**: Album/release name
117117+- **releaseMbId**: MusicBrainz release ID
118118+- **recordingMbId**: MusicBrainz recording/track ID
119119+- **originUrl**: Link to the track on Last.fm
711207272-Each scrobble is converted according to the `fm.teal.alpha.feed.play` lexicon:
121121+### Example Record
7312274123```json
75124{
···86135 "recordingMbId": "3a390ad3-fe56-45f2-a073-bebc45d6bde1",
87136 "playedTime": "2025-11-13T23:49:36Z",
88137 "originUrl": "https://www.last.fm/music/Cjbeards/_/Paint+My+Masterpiece",
8989- "submissionClientAgent": "lastfm-importer/v1.0.0",
138138+ "submissionClientAgent": "lastfm-importer/v0.0.2",
90139 "musicServiceBaseDomain": "last.fm"
91140}
92141```
931429494-### Required Fields
143143+## Processing Order
144144+145145+By default, records are processed **oldest first** (chronological order). This means your earliest scrobbles will appear first in your ATProto feed.
146146+147147+Use the `--reverse-chronological` or `-r` flag to process **newest first** instead.
148148+149149+## Multi-Day Imports
150150+151151+For imports exceeding 1,000 records (after applying the 90% safety margin), the importer automatically:
152152+153153+1. **Calculates a schedule**: Splits your import across multiple days
154154+2. **Shows the plan**: Displays which records will be imported each day
155155+3. **Processes Day 1**: Imports the first batch of records
156156+4. **Pauses 24 hours**: Waits a full day before continuing
157157+5. **Repeats**: Continues until all records are imported
158158+159159+**Important notes:**
160160+- You can safely stop (Ctrl+C) and restart the importer
161161+- Progress is preserved - it continues where it left off
162162+- Each day's progress is clearly displayed
163163+- Time estimates account for multi-day duration
164164+165165+Example output for a 5,000 record import:
166166+```
167167+📊 Rate Limiting Information:
168168+ Total records: 5,000
169169+ Daily limit: 900 records/day
170170+ Estimated duration: 6 days
171171+ Batch size: 10 records
172172+ Batch delay: 9600.0s
173173+```
174174+175175+## Dry Run Mode
176176+177177+Preview what will be imported without actually publishing:
178178+179179+```bash
180180+npm start -- -f lastfm.csv --dry-run
181181+```
182182+183183+Dry run shows:
184184+- Total record count
185185+- Rate limiting schedule (if applicable)
186186+- Multi-day import plan (if needed)
187187+- Preview of first 5 records with full details
188188+- MusicBrainz IDs when available
189189+190190+## Error Handling
191191+192192+The importer is designed to be resilient:
193193+194194+- **Network errors**: Records that fail are logged but don't stop the import
195195+- **Invalid data**: Skipped with error messages
196196+- **Authentication issues**: Clear error messages with suggested fixes
197197+- **Rate limit hits**: Automatic adjustment and retry logic
198198+- **Ctrl+C handling**: Gracefully stops after current batch
199199+200200+Failed records are logged but don't prevent the rest of your import from completing.
201201+202202+## Project Structure
203203+204204+```
205205+atproto-lastfm-importer/
206206+├── src/
207207+│ ├── lib/
208208+│ │ ├── auth.ts # Authentication & identity resolution
209209+│ │ ├── cli.ts # Command line argument parsing
210210+│ │ ├── csv.ts # CSV parsing & record conversion
211211+│ │ └── publisher.ts # Batch publishing with rate limiting
212212+│ ├── utils/
213213+│ │ ├── helpers.ts # Utility functions (timing, formatting)
214214+│ │ ├── input.ts # User input handling (prompts, passwords)
215215+│ │ └── rate-limiter.ts # Rate limiting calculations
216216+│ ├── config.ts # Configuration constants
217217+│ └── types.ts # TypeScript type definitions
218218+├── lexicons/ # fm.teal.alpha lexicon definitions
219219+│ └── fm.teal.alpha/
220220+│ └── feed/
221221+│ └── play.json # Play record schema
222222+├── package.json
223223+├── tsconfig.json
224224+└── README.md
225225+```
952269696-- `trackName` - The name of the track
9797-- `artists` - Array of artist objects with `artistName` (required) and optional `artistMbId`
227227+## Development
982289999-### Optional Fields
229229+```bash
230230+# Type checking
231231+npm run type-check
100232101101-- `releaseName` - Album name
102102-- `releaseMbId` - MusicBrainz release ID
103103-- `recordingMbId` - MusicBrainz recording ID
104104-- `playedTime` - ISO 8601 datetime
105105-- `originUrl` - Link to the track
106106-- `submissionClientAgent` - Client identifier
107107-- `musicServiceBaseDomain` - Service domain (e.g., "last.fm")
233233+# Build
234234+npm run build
235235+236236+# Development mode (rebuild + run)
237237+npm run dev
238238+239239+# Clean build artifacts
240240+npm run clean
241241+```
242242+243243+## Technical Details
244244+245245+### Authentication
246246+- Uses Slingshot resolver to discover your PDS from your handle/DID
247247+- Requires an ATProto app password (not your main password)
248248+- Automatically configures the agent for your personal PDS
249249+250250+### Rate Limiting Algorithm
251251+1. Calculates safe daily limit (90% of 1K = 900 records/day)
252252+2. Determines how many days needed for your import
253253+3. Calculates optimal batch size and delay to spread records evenly
254254+4. Enforces minimum 1 second delay between batches
255255+5. Shows clear schedule before starting
256256+257257+### Record Processing
258258+1. Parses CSV using `csv-parse` library
259259+2. Sorts records chronologically (or reverse if `-r` flag)
260260+3. Converts Last.fm format to `fm.teal.alpha.feed.play` schema
261261+4. Validates required fields
262262+5. Publishes in batches with configurable delays
263263+264264+### Data Mapping
265265+- **Track info**: Direct mapping from CSV columns
266266+- **Timestamps**: Converts Unix timestamps to ISO 8601
267267+- **MusicBrainz IDs**: Preserved when present in CSV
268268+- **URLs**: Generated from artist/track names
269269+- **Artists**: Wrapped in array format with optional MBID
108270109271## Lexicon Reference
110272111111-This importer follows the lexicon defined in `/lexicons/fm.teal.alpha/feed/play.json`.
273273+This importer follows the official `fm.teal.alpha` lexicon defined in `/lexicons/fm.teal.alpha/feed/play.json`.
274274+275275+The lexicon defines:
276276+- Required and optional field types
277277+- String length constraints
278278+- Array formats
279279+- Timestamp formatting
280280+- URL validation
281281+282282+## Troubleshooting
283283+284284+### "Handle not found"
285285+- Verify your ATProto handle is correct (e.g., `alice.bsky.social`)
286286+- Make sure you're using a valid DID or handle
287287+288288+### "Invalid credentials"
289289+- Use an **app password**, not your main account password
290290+- Generate app passwords in your account settings
291291+292292+### "Rate limit exceeded"
293293+- The importer should prevent this automatically
294294+- If you see this, wait 24 hours before retrying
295295+- Consider reducing batch size or increasing delay
296296+297297+### "Connection refused"
298298+- Check your internet connection
299299+- Verify your PDS is accessible
300300+- Some PDSs may have firewall rules
301301+302302+### Import seems stuck
303303+- Check progress messages - large imports take time
304304+- Multi-day imports pause for 24 hours between days
305305+- You can safely stop (Ctrl+C) and resume later
306306+307307+## Contributing
308308+309309+Contributions welcome! Please:
310310+1. Fork the repository
311311+2. Create a feature branch
312312+3. Make your changes with tests
313313+4. Submit a pull request
314314+315315+## License
316316+317317+MIT License - See LICENSE file for details
318318+319319+## Credits
320320+321321+- Uses [@atproto/api](https://www.npmjs.com/package/@atproto/api) for ATProto interactions
322322+- CSV parsing via [csv-parse](https://www.npmjs.com/package/csv-parse)
323323+- Identity resolution via [Slingshot](https://slingshot.danner.cloud)
324324+- Follows the `fm.teal.alpha` lexicon standard
325325+326326+---
327327+328328+**Note**: This tool is for personal use. Respect Last.fm's terms of service and rate limits when exporting your data.
-126
STRUCTURE.md
···11-# Last.fm to ATProto Importer - Modular Structure
22-33-## Project Structure
44-55-```plaintext
66-lastfm-importer/
77-├── src/
88-│ ├── index.js # Main entry point
99-│ ├── config.js # Configuration constants
1010-│ ├── lib/ # Core library modules
1111-│ │ ├── auth.js # Authentication & login
1212-│ │ ├── cli.js # CLI argument parsing & help
1313-│ │ ├── csv.js # CSV parsing & conversion
1414-│ │ └── publisher.js # Record publishing logic
1515-│ └── utils/ # Utility functions
1616-│ ├── helpers.js # Helper functions (formatting, batch calculation)
1717-│ ├── input.js # User input & password masking
1818-│ └── killswitch.js # Graceful shutdown handling
1919-├── importer.js # Wrapper for backwards compatibility
2020-└── importer.old.js # Original monolithic version (backup)
2121-```
2222-2323-## Module Responsibilities
2424-2525-### `/src/config.js`
2626-2727-- Configuration constants
2828-- Batch size calculation parameters
2929-- API endpoints and client information
3030-3131-### `/src/lib/auth.js`
3232-3333-- ATProto authentication
3434-- Identity resolution via Slingshot
3535-- Login error handling
3636-3737-### `/src/lib/cli.js`
3838-3939-- Command-line argument parsing
4040-- Help text display
4141-- Input validation
4242-4343-### `/src/lib/csv.js`
4444-4545-- CSV file parsing
4646-- Record conversion to ATProto format
4747-- Chronological sorting
4848-4949-### `/src/lib/publisher.js`
5050-5151-- Batch publishing with rate limiting
5252-- Dry-run preview mode
5353-- Progress tracking and reporting
5454-- Killswitch integration
5555-5656-### `/src/utils/helpers.js`
5757-5858-- Duration formatting
5959-- Optimal batch size calculation (logarithmic algorithm)
6060-- Generic utility functions
6161-6262-### `/src/utils/input.js`
6363-6464-- Interactive prompts
6565-- Password masking with asterisks
6666-- Backspace support
6767-6868-### `/src/utils/killswitch.js`
6969-7070-- SIGINT handler
7171-- Graceful shutdown state management
7272-- Force-quit on second Ctrl+C
7373-7474-## Benefits of Modular Structure
7575-7676-1. **Maintainability**: Each module has a single responsibility
7777-2. **Testability**: Individual modules can be tested in isolation
7878-3. **Reusability**: Modules can be imported and reused
7979-4. **Readability**: Smaller files are easier to understand
8080-5. **Collaboration**: Multiple developers can work on different modules
8181-6. **Debugging**: Easier to locate and fix issues
8282-8383-## Usage
8484-8585-The wrapper file (`importer.js`) maintains backwards compatibility:
8686-8787-```bash
8888-# Still works exactly as before
8989-node importer.js -f lastfm.csv -i handle.bsky.social
9090-9191-# Or use the modular version directly
9292-node src/index.js -f lastfm.csv -i handle.bsky.social
9393-```
9494-9595-## Algorithm Details
9696-9797-### Batch Size Calculation
9898-9999-Located in `/src/utils/helpers.js`:
100100-101101-```javascript
102102-batchSize = BASE + (log2(records/MIN) * SCALING_FACTOR)
103103-```
104104-105105-- **Time Complexity**: O(n) - each record processed once
106106-- **Space Complexity**: O(b) where b is batch size
107107-- **Rate Limit Strategy**: Token bucket approach
108108-- **Adaptive**: Adjusts based on total records and delay settings
109109-110110-### Processing Order
111111-112112-- Default: Chronological (oldest first)
113113-- Option: `--reverse-chronological` for newest first
114114-- Sorted by `playedTime` field
115115-116116-## Future Improvements
117117-118118-With the modular structure, it's now easier to:
119119-120120-- Add unit tests for each module
121121-- Implement different authentication methods
122122-- Support multiple export formats (JSON, XML)
123123-- Add progress persistence (resume interrupted imports)
124124-- Implement retry logic with exponential backoff
125125-- Add statistics and analytics
126126-- Create a web UI that imports these modules
-5
importer.js
···11-#!/usr/bin/env node
22-33-// Wrapper file for backwards compatibility
44-// This imports and runs the modular version
55-import './src/index.js';