+13
-148
README.md
+13
-148
README.md
···
1
-
# Bluesky to Leaflet.pub Reply Synchronization
1
+
## ATProto Experiments
2
2
3
-
A real-time service that monitors Bluesky posts linking to leaflet.pub and synchronizes replies as comments.
3
+
A monorepo of AT Protocol experiments, managed via Yarn workspaces.
4
4
5
-
## Overview
5
+
### Packages
6
6
7
-
This application:
8
-
- Monitors Bluesky Jetstream for real-time post events
9
-
- Tracks Bluesky posts containing leaflet.pub links
10
-
- Detects replies to tracked posts
11
-
- Creates corresponding comments in leaflet.pub via AT Protocol's PDS
12
-
- **Automatically deletes comments when Bluesky replies are deleted** (new!)
7
+
- **[@atproto-experiments/bsky-leaflet-sync](./packages/bsky-leaflet-sync)** — Automatic syncing of Bluesky replies to Leaflet.pub posts as Leaflet.pub comments
13
8
14
-
## Prerequisites
15
-
16
-
- Node.js 18 or higher
17
-
- Yarn package manager
18
-
- Bluesky account with app password
19
-
- Leaflet.pub documents published on AT Protocol
20
-
21
-
## Installation
9
+
### Development
22
10
23
11
```bash
24
-
# Install dependencies
12
+
# Install dependencies for all packages
25
13
yarn install
26
14
27
-
# Build the project
15
+
# Build all packages
28
16
yarn build
29
-
```
30
-
31
-
## Configuration
32
-
33
-
Create a `.env` file based on `.env.example`:
34
-
35
-
```env
36
-
BLUESKY_HANDLE=your.handle.bsky.social
37
-
BLUESKY_APP_PASSWORD=xxxx-xxxx-xxxx-xxxx
38
-
JETSTREAM_URL=wss://jetstream2.us-east.bsky.network
39
-
CURSOR_FILE=data/cursor.json
40
-
LOG_LEVEL=info
41
-
```
42
-
43
-
### Creating a Bluesky App Password
44
-
45
-
1. Log in to Bluesky
46
-
2. Go to Settings → App Passwords
47
-
3. Create a new app password
48
-
4. Copy the password (format: `xxxx-xxxx-xxxx-xxxx`)
49
-
5. Add it to your `.env` file
50
-
51
-
## Usage
52
-
53
-
```bash
54
-
# Development mode (with auto-reload)
55
-
yarn dev
56
-
57
-
# Production mode
58
-
yarn build
59
-
yarn start
60
-
```
61
-
62
-
## How It Works
63
-
64
-
1. **Startup**: Authenticates with Bluesky and loads saved cursor
65
-
2. **Initial Scan**: Queries leaflet.pub documents with `postRef` to build tracking map
66
-
3. **Real-time Monitoring**: Connects to Jetstream and processes events
67
-
4. **Post Tracking**: Adds new posts with leaflet.pub links to tracking
68
-
5. **Reply Detection**: Detects replies to tracked posts
69
-
6. **Comment Creation**: Creates comments in leaflet.pub with reply text and URL
70
-
7. **Deduplication**: Checks existing comments to prevent duplicates
71
-
8. **Reply Deletion**: Detects when Bluesky replies are deleted and removes corresponding comments
72
-
73
-
## Architecture
74
-
75
-
- **Jetstream Client**: WebSocket connection with automatic reconnection
76
-
- **Post Tracker**: In-memory Map for O(1) post lookup
77
-
- **Event Processor**: Filters and routes Jetstream events (creates & deletes)
78
-
- **Comment Checker**: Two-level caching for deduplication
79
-
- **Comment Creator**: Creates PDS records for comments
80
-
- **Comment Deleter**: Removes comments via URL search (no state needed!)
81
-
- **Cursor Manager**: Persists cursor for gapless replay
82
-
83
-
### Deletion Strategy
84
17
85
-
**Completely stateless** using [Constellation API](https://constellation.microcosm.blue/)! When a reply is deleted, we:
86
-
1. Query Constellation for each tracked post: "what replies to this post?"
87
-
2. Check if the deleted reply URI is in the backlinks
88
-
3. If found, delete the comment from that specific Leaflet document
89
-
90
-
**Zero local state** - we query the AT Protocol network graph dynamically via Constellation's real-time index.
91
-
92
-
See [REPLY_DELETION_TRACKING.md](./REPLY_DELETION_TRACKING.md) for the beautiful details.
93
-
94
-
## Logging
95
-
96
-
Logs are written to:
97
-
- Console (colorized for development)
98
-
- `logs/combined.log` (all logs)
99
-
- `logs/error.log` (errors only)
100
-
101
-
## Troubleshooting
102
-
103
-
**Authentication fails:**
104
-
- Verify `BLUESKY_HANDLE` is your full handle (e.g., `user.bsky.social`)
105
-
- Ensure `BLUESKY_APP_PASSWORD` is correct
106
-
- Check that app password has required permissions
107
-
108
-
**No posts tracked:**
109
-
- Verify you have leaflet.pub documents with `postRef` field
110
-
- Check logs for errors during initial scan
111
-
112
-
**Comments not created:**
113
-
- Ensure replies are to tracked posts
114
-
- Check logs for PDS record creation errors
115
-
- Verify leaflet.pub comment schema is correct
116
-
117
-
**Connection issues:**
118
-
- Check `JETSTREAM_URL` is correct
119
-
- Verify internet connection
120
-
- Review logs for reconnection attempts
121
-
122
-
## Development
123
-
124
-
```bash
125
18
# Clean build artifacts
126
19
yarn clean
127
20
128
-
# Build project
129
-
yarn build
130
-
131
-
# Run in development mode
132
-
yarn dev
133
-
```
134
-
135
-
## Project Structure
136
-
137
-
```
138
-
src/
139
-
├── index.ts # Entry point
140
-
├── app.ts # Main application
141
-
├── config.ts # Configuration
142
-
├── auth.ts # Authentication
143
-
├── services/
144
-
│ ├── jetstreamClient.ts # WebSocket client
145
-
│ ├── postTracker.ts # Post tracking
146
-
│ ├── eventProcessor.ts # Event filtering
147
-
│ ├── commentChecker.ts # Deduplication
148
-
│ ├── commentCreator.ts # Comment creation
149
-
│ └── commentDeleter.ts # Comment deletion (stateless!)
150
-
├── utils/
151
-
│ ├── cursorManager.ts # Cursor persistence
152
-
│ ├── urlParser.ts # URL utilities
153
-
│ └── logger.ts # Logging
154
-
└── types/
155
-
├── jetstream.ts # Jetstream types
156
-
└── lexicons.ts # Lexicon types
157
-
```
158
-
159
-
## License
160
-
161
-
MIT
21
+
# Run specific workspace
22
+
yarn start # Runs bsky-leaflet-sync
23
+
yarn dev # Dev mode for bsky-leaflet-sync
162
24
25
+
# Run command in specific workspace
26
+
yarn workspace @atproto-experiments/bsky-leaflet-sync <command>
27
+
```
+8
-13
package.json
+8
-13
package.json
···
1
1
{
2
2
"name": "atproto-experiments",
3
3
"version": "1.0.0",
4
-
"main": "dist/index.js",
4
+
"private": true,
5
5
"author": "Kaushik Chakraborty <git@kaushikc.org>",
6
6
"license": "MIT",
7
+
"workspaces": [
8
+
"packages/*"
9
+
],
7
10
"scripts": {
8
-
"build": "tsc",
9
-
"start": "yarn build && node dist/index.js",
10
-
"dev": "ts-node src/index.ts",
11
-
"clean": "rm -rf dist"
12
-
},
13
-
"dependencies": {
14
-
"@atproto/api": "^0.17.2",
15
-
"dotenv": "^17.2.3",
16
-
"winston": "^3.18.3",
17
-
"ws": "^8.18.3"
11
+
"build": "yarn workspaces run build",
12
+
"clean": "yarn workspaces run clean",
13
+
"start": "yarn workspace @atproto-experiments/bsky-leaflet-sync start",
14
+
"dev": "yarn workspace @atproto-experiments/bsky-leaflet-sync dev"
18
15
},
19
16
"devDependencies": {
20
17
"@types/node": "^24.7.2",
21
-
"@types/ws": "^8.18.1",
22
-
"ts-node": "^10.9.2",
23
18
"typescript": "^5.9.3"
24
19
}
25
20
}
+32
packages/bsky-leaflet-sync/README.md
+32
packages/bsky-leaflet-sync/README.md
···
1
+
# @atproto-experiments/bsky-leaflet-sync
2
+
3
+
Automatic syncing of Bluesky replies to Leaflet.pub posts as Leaflet.pub comments.
4
+
5
+
## Setup
6
+
7
+
1. Copy `.env.example` to `.env` and configure:
8
+
- Bluesky credentials
9
+
- Leaflet.pub credentials
10
+
11
+
2. Install dependencies:
12
+
```bash
13
+
yarn install
14
+
```
15
+
16
+
3. Build:
17
+
```bash
18
+
yarn build
19
+
```
20
+
21
+
4. Run:
22
+
```bash
23
+
yarn start
24
+
```
25
+
26
+
## Development
27
+
28
+
```bash
29
+
# Run in development mode
30
+
yarn dev
31
+
```
32
+
+24
packages/bsky-leaflet-sync/package.json
+24
packages/bsky-leaflet-sync/package.json
···
1
+
{
2
+
"name": "@atproto-experiments/bsky-leaflet-sync",
3
+
"version": "1.0.0",
4
+
"private": true,
5
+
"license": "MIT",
6
+
"main": "dist/index.js",
7
+
"scripts": {
8
+
"build": "tsc",
9
+
"start": "yarn build && node dist/index.js",
10
+
"dev": "ts-node index.ts",
11
+
"clean": "rm -rf dist"
12
+
},
13
+
"dependencies": {
14
+
"@atproto/api": "^0.17.2",
15
+
"dotenv": "^17.2.3",
16
+
"winston": "^3.18.3",
17
+
"ws": "^8.18.3"
18
+
},
19
+
"devDependencies": {
20
+
"@types/ws": "^8.18.1",
21
+
"ts-node": "^10.9.2"
22
+
}
23
+
}
24
+
+10
packages/bsky-leaflet-sync/tsconfig.json
+10
packages/bsky-leaflet-sync/tsconfig.json
src/app.ts
packages/bsky-leaflet-sync/app.ts
src/app.ts
packages/bsky-leaflet-sync/app.ts
src/auth.ts
packages/bsky-leaflet-sync/auth.ts
src/auth.ts
packages/bsky-leaflet-sync/auth.ts
src/config.ts
packages/bsky-leaflet-sync/config.ts
src/config.ts
packages/bsky-leaflet-sync/config.ts
src/index.ts
packages/bsky-leaflet-sync/index.ts
src/index.ts
packages/bsky-leaflet-sync/index.ts
src/services/commentChecker.ts
packages/bsky-leaflet-sync/services/commentChecker.ts
src/services/commentChecker.ts
packages/bsky-leaflet-sync/services/commentChecker.ts
src/services/commentCreator.ts
packages/bsky-leaflet-sync/services/commentCreator.ts
src/services/commentCreator.ts
packages/bsky-leaflet-sync/services/commentCreator.ts
src/services/commentDeleter.ts
packages/bsky-leaflet-sync/services/commentDeleter.ts
src/services/commentDeleter.ts
packages/bsky-leaflet-sync/services/commentDeleter.ts
src/services/eventProcessor.ts
packages/bsky-leaflet-sync/services/eventProcessor.ts
src/services/eventProcessor.ts
packages/bsky-leaflet-sync/services/eventProcessor.ts
src/services/jetstreamClient.ts
packages/bsky-leaflet-sync/services/jetstreamClient.ts
src/services/jetstreamClient.ts
packages/bsky-leaflet-sync/services/jetstreamClient.ts
src/services/postTracker.ts
packages/bsky-leaflet-sync/services/postTracker.ts
src/services/postTracker.ts
packages/bsky-leaflet-sync/services/postTracker.ts
src/types/jetstream.ts
packages/bsky-leaflet-sync/types/jetstream.ts
src/types/jetstream.ts
packages/bsky-leaflet-sync/types/jetstream.ts
src/types/lexicons.ts
packages/bsky-leaflet-sync/types/lexicons.ts
src/types/lexicons.ts
packages/bsky-leaflet-sync/types/lexicons.ts
src/utils/cursorManager.ts
packages/bsky-leaflet-sync/utils/cursorManager.ts
src/utils/cursorManager.ts
packages/bsky-leaflet-sync/utils/cursorManager.ts
src/utils/logger.ts
packages/bsky-leaflet-sync/utils/logger.ts
src/utils/logger.ts
packages/bsky-leaflet-sync/utils/logger.ts
src/utils/urlParser.ts
packages/bsky-leaflet-sync/utils/urlParser.ts
src/utils/urlParser.ts
packages/bsky-leaflet-sync/utils/urlParser.ts
+1
-5
tsconfig.json
+1
-5
tsconfig.json
···
3
3
"target": "ES2022",
4
4
"module": "commonjs",
5
5
"lib": ["ES2022"],
6
-
"outDir": "./dist",
7
-
"rootDir": "./src",
8
6
"strict": true,
9
7
"esModuleInterop": true,
10
8
"skipLibCheck": true,
···
14
12
"declaration": true,
15
13
"declarationMap": true,
16
14
"sourceMap": true
17
-
},
18
-
"include": ["src/**/*"],
19
-
"exclude": ["node_modules", "dist"]
15
+
}
20
16
}
21
17