ewancroft.uk
+22
-2
README.md
+22
-2
README.md
···
10
10
11
11
### Configuration
12
12
13
-
Before running any scripts, you need to configure the project. See `src/.env` (or `src/config.env` if you prefer to keep a template) for the required environment variables:
13
+
Before running any scripts, you need to configure the project. Create a `src/.env` file based on `src/.env.example`:
14
+
15
+
```bash
16
+
cp src/.env.example src/.env
17
+
```
18
+
19
+
Then edit `src/.env` with your actual values:
14
20
15
21
* `BASE_DIR` – the local directory where GitHub repositories will be cloned.
16
22
* `GITHUB_USER` – your GitHub username or organisation.
···
45
51
46
52
---
47
53
54
+
### Testing AT Proto Connection
55
+
56
+
**Before running the full sync**, test your AT Proto connection:
57
+
58
+
```bash
59
+
npm run test-atproto
60
+
```
61
+
62
+
This will:
63
+
- Verify your Bluesky credentials
64
+
- Confirm your DID matches the configuration
65
+
- List any existing `sh.tangled.repo` records
66
+
- Validate the connection to the PDS
67
+
48
68
### Running the Sync Script
49
69
50
-
Once configuration and SSH verification are complete, run:
70
+
Once configuration, SSH verification, and AT Proto testing are complete, run:
51
71
52
72
```bash
53
73
npm run sync
+175
SETUP.md
+175
SETUP.md
···
1
+
# Tangled Sync - Setup & Troubleshooting Guide
2
+
3
+
## Quick Setup Checklist
4
+
5
+
### 1. Install Dependencies
6
+
```bash
7
+
npm install
8
+
```
9
+
10
+
### 2. Configure Environment Variables
11
+
```bash
12
+
# Copy the example env file
13
+
cp src/.env.example src/.env
14
+
15
+
# Edit with your actual values
16
+
nano src/.env # or use your preferred editor
17
+
```
18
+
19
+
**Required values:**
20
+
- `BASE_DIR`: Where to clone repos (e.g., `/Users/you/tangled-repos`)
21
+
- `GITHUB_USER`: Your GitHub username
22
+
- `ATPROTO_DID`: Your AT Proto DID (get from Bluesky settings)
23
+
- `BLUESKY_PDS`: Usually `https://bsky.social`
24
+
- `BLUESKY_USERNAME`: Your Bluesky handle (e.g., `you.bsky.social`)
25
+
- `BLUESKY_PASSWORD`: Use an **app password**, not your main password!
26
+
27
+
### 3. Get Your AT Proto DID
28
+
29
+
Your DID can be found by:
30
+
1. Go to https://bsky.app
31
+
2. Click your profile
32
+
3. Settings → Advanced → Account
33
+
4. Look for "DID" (starts with `did:plc:`)
34
+
35
+
Alternatively, visit: `https://bsky.social/xrpc/com.atproto.identity.resolveHandle?handle=YOUR_HANDLE.bsky.social`
36
+
37
+
### 4. Create an App Password
38
+
39
+
**IMPORTANT:** Do NOT use your main Bluesky password!
40
+
41
+
1. Go to https://bsky.app/settings
42
+
2. Navigate to "App Passwords"
43
+
3. Click "Add App Password"
44
+
4. Give it a name (e.g., "Tangled Sync")
45
+
5. Copy the generated password to your `.env` file
46
+
47
+
### 5. Test AT Proto Connection
48
+
```bash
49
+
npm run test-atproto
50
+
```
51
+
52
+
Expected output:
53
+
```
54
+
✓ Login successful!
55
+
DID: did:plc:...
56
+
Handle: you.bsky.social
57
+
Email: your@email.com
58
+
59
+
✓ Found X existing Tangled repo records
60
+
```
61
+
62
+
### 6. Verify SSH to Tangled
63
+
```bash
64
+
ssh git@tangled.sh
65
+
```
66
+
67
+
You should see a message confirming your SSH key is configured.
68
+
69
+
### 7. Run the Sync
70
+
```bash
71
+
npm run sync
72
+
```
73
+
74
+
---
75
+
76
+
## Common Issues & Solutions
77
+
78
+
### Issue: "Missing Bluesky credentials"
79
+
**Solution:** Check that `src/.env` exists and contains `BLUESKY_USERNAME` and `BLUESKY_PASSWORD`
80
+
81
+
### Issue: "Login failed" or "Invalid credentials"
82
+
**Solution:**
83
+
- Ensure you're using an **app password**, not your main password
84
+
- Check your username includes the full handle (e.g., `you.bsky.social`)
85
+
- Verify credentials are correct
86
+
87
+
### Issue: "DID mismatch"
88
+
**Solution:**
89
+
- Run `npm run test-atproto` to see your actual DID
90
+
- Update `ATPROTO_DID` in `src/.env` to match
91
+
92
+
### Issue: "Could not push to Tangled"
93
+
**Solution:**
94
+
- Verify SSH key is added to Tangled: https://tangled.org/settings/keys
95
+
- Test SSH connection: `ssh git@tangled.sh`
96
+
- Ensure the repository exists on Tangled first
97
+
98
+
### Issue: "Failed to create ATProto record"
99
+
**Solution:**
100
+
- Check that the schema matches (required fields: `name`, `knot`, `createdAt`)
101
+
- Verify your app password has write permissions
102
+
- Check PDS is reachable: `curl https://bsky.social`
103
+
104
+
### Issue: Rate limiting from GitHub API
105
+
**Solution:**
106
+
- GitHub has a rate limit of 60 requests/hour for unauthenticated requests
107
+
- Consider adding GitHub authentication if syncing many repos
108
+
- Wait an hour and try again
109
+
110
+
---
111
+
112
+
## Understanding the Workflow
113
+
114
+
1. **Login to AT Proto**: Authenticates with Bluesky PDS using your credentials
115
+
2. **Fetch GitHub Repos**: Retrieves all public repos from your GitHub account
116
+
3. **Clone Locally**: Downloads repos to `BASE_DIR` if not already present
117
+
4. **Add Tangled Remote**: Adds `tangled` as a git remote
118
+
5. **Push to Tangled**: Pushes the `main` branch to Tangled
119
+
6. **Update README**: Adds a Tangled mirror link to the README
120
+
7. **Create AT Proto Record**: Publishes metadata to the AT Proto network
121
+
122
+
Each repository gets a record in the `sh.tangled.repo` collection with:
123
+
- Repository name and description
124
+
- Source URL (GitHub)
125
+
- Creation timestamp
126
+
- Knot server reference
127
+
- Optional labels and topics
128
+
129
+
---
130
+
131
+
## Verifying Success
132
+
133
+
After running the sync, you can verify:
134
+
135
+
1. **Local repos**: Check `BASE_DIR` for cloned repositories
136
+
2. **Tangled remotes**: Run `git remote -v` in any repo directory
137
+
3. **AT Proto records**: Run `npm run test-atproto` to list records
138
+
4. **Tangled website**: Visit `https://tangled.org/YOUR_DID/REPO_NAME`
139
+
140
+
---
141
+
142
+
## Advanced Configuration
143
+
144
+
### Using a Different PDS
145
+
If you're not using the default Bluesky PDS:
146
+
```bash
147
+
BLUESKY_PDS=https://your-pds.example.com
148
+
```
149
+
150
+
### Syncing Specific Repos Only
151
+
Modify the `getGitHubRepos()` function to filter repos:
152
+
```typescript
153
+
return json
154
+
.filter((r: any) => r.name.startsWith('my-prefix-'))
155
+
.map(...);
156
+
```
157
+
158
+
### Changing the Default Branch
159
+
If your repos use `master` instead of `main`, update:
160
+
```typescript
161
+
run(`git push tangled master`, repoDir);
162
+
```
163
+
164
+
---
165
+
166
+
## Support
167
+
168
+
For issues with:
169
+
- **Tangled**: https://github.com/tangled-dev/tangled
170
+
- **AT Proto**: https://atproto.com/docs
171
+
- **This tool**: Open an issue in the repository
172
+
173
+
---
174
+
175
+
**Happy syncing! 🚀**
+197
USAGE.md
+197
USAGE.md
···
1
+
# 🎉 Tangled Sync - Ready to Use!
2
+
3
+
## Summary of Changes
4
+
5
+
I've improved your Tangled Sync project to ensure proper AT Proto authentication and repository record creation. Here's what was updated:
6
+
7
+
### ✅ What's Fixed
8
+
9
+
1. **Enhanced AT Proto Login**
10
+
- Added better error handling and validation
11
+
- Shows DID and handle on successful login
12
+
- Clearer error messages when authentication fails
13
+
14
+
2. **Corrected Repository Schema**
15
+
- Fixed record structure to match `sh.tangled.repo` lexicon
16
+
- Required fields (`name`, `knot`, `createdAt`) now ordered correctly
17
+
- Optional fields properly marked as optional
18
+
- Added better error handling for record creation
19
+
20
+
3. **Improved Logging**
21
+
- More detailed startup information
22
+
- Better progress tracking during sync
23
+
- Shows AT Proto record URIs when created
24
+
- Success/failure messages are clearer
25
+
26
+
### 📁 New Files Created
27
+
28
+
1. **`src/.env.example`** - Template for your configuration
29
+
2. **`src/test-atproto.ts`** - Test AT Proto connection before syncing
30
+
3. **`src/validate-config.ts`** - Validate your environment setup
31
+
4. **`SETUP.md`** - Comprehensive setup and troubleshooting guide
32
+
33
+
### 🚀 How to Use
34
+
35
+
#### Step 1: Configure Environment
36
+
```bash
37
+
# Copy the example file
38
+
cp src/.env.example src/.env
39
+
40
+
# Edit with your actual values
41
+
nano src/.env
42
+
```
43
+
44
+
You need:
45
+
- Your GitHub username
46
+
- Your AT Proto DID (from Bluesky settings)
47
+
- A Bluesky **app password** (not your main password!)
48
+
- Base directory for repos
49
+
50
+
#### Step 2: Validate Configuration
51
+
```bash
52
+
npm run validate
53
+
```
54
+
55
+
This checks all your environment variables are set correctly.
56
+
57
+
#### Step 3: Test AT Proto Connection
58
+
```bash
59
+
npm run test-atproto
60
+
```
61
+
62
+
This verifies:
63
+
- ✅ Your credentials work
64
+
- ✅ Your DID is correct
65
+
- ✅ You can access the PDS
66
+
- ✅ Shows any existing Tangled repo records
67
+
68
+
#### Step 4: Run the Sync
69
+
```bash
70
+
npm run sync
71
+
```
72
+
73
+
This will:
74
+
1. Login to AT Proto ✅
75
+
2. Fetch your GitHub repos
76
+
3. Clone them locally (if needed)
77
+
4. Add Tangled remotes
78
+
5. Push to Tangled
79
+
6. Update READMEs
80
+
7. Create AT Proto records for each repo ✅
81
+
82
+
### 🔍 What to Check
83
+
84
+
After running the sync, verify:
85
+
86
+
1. **AT Proto Records Created**
87
+
```bash
88
+
npm run test-atproto
89
+
```
90
+
Should show your repos listed
91
+
92
+
2. **Repos on Tangled**
93
+
Visit: `https://tangled.org/YOUR_DID/REPO_NAME`
94
+
95
+
3. **Local Git Remotes**
96
+
```bash
97
+
cd YOUR_BASE_DIR/some-repo
98
+
git remote -v
99
+
```
100
+
Should show both `origin` (GitHub) and `tangled` remotes
101
+
102
+
### 📊 Record Schema
103
+
104
+
Each repository creates a record with this structure:
105
+
106
+
```typescript
107
+
{
108
+
$type: "sh.tangled.repo",
109
+
name: "your-repo-name", // required
110
+
knot: "knot1.tangled.sh", // required
111
+
createdAt: "2024-01-01T00:00:00Z", // required
112
+
description: "Repo description", // optional
113
+
source: "https://github.com/...", // optional
114
+
labels: [], // optional
115
+
}
116
+
```
117
+
118
+
This matches the official `sh.tangled.repo` lexicon schema.
119
+
120
+
### ⚠️ Important Notes
121
+
122
+
1. **Use App Password**: Never use your main Bluesky password. Create an app password in Settings → App Passwords.
123
+
124
+
2. **Check Your DID**: Run `npm run test-atproto` first to ensure your DID in `.env` matches your actual account.
125
+
126
+
3. **SSH Key Required**: Make sure your SSH key is added to Tangled at https://tangled.org/settings/keys
127
+
128
+
4. **Rate Limits**: GitHub API has rate limits (60 req/hour unauthenticated). If you have many repos, consider adding GitHub auth.
129
+
130
+
### 🐛 Troubleshooting
131
+
132
+
**"Missing Bluesky credentials"**
133
+
- Check `src/.env` exists and has `BLUESKY_USERNAME` and `BLUESKY_PASSWORD`
134
+
135
+
**"Login failed"**
136
+
- Verify you're using an app password, not your main password
137
+
- Check username includes full handle (e.g., `you.bsky.social`)
138
+
139
+
**"Could not push to Tangled"**
140
+
- Verify SSH key is configured: `ssh git@tangled.sh`
141
+
- Check repo exists on Tangled
142
+
143
+
**"Failed to create ATProto record"**
144
+
- Run `npm run test-atproto` to check connection
145
+
- Verify your app password has write permissions
146
+
147
+
See `SETUP.md` for more detailed troubleshooting.
148
+
149
+
### 📚 Available Commands
150
+
151
+
```bash
152
+
npm run check # Comprehensive health check (recommended first step!)
153
+
npm run validate # Check environment configuration only
154
+
npm run test-atproto # Test AT Proto connection only
155
+
npm run sync # Run sync (only new repos without AT Proto records)
156
+
npm run sync:force # Force sync all repos (including existing)
157
+
```
158
+
159
+
#### `npm run check` - Comprehensive Health Check
160
+
161
+
This is the **most useful command** for troubleshooting! It runs all checks in one go:
162
+
163
+
- ✅ Configuration validation
164
+
- ✅ AT Proto connection test
165
+
- ✅ SSH connection to Tangled
166
+
- ✅ GitHub API access
167
+
- ✅ Dependencies verification
168
+
169
+
**When to use:**
170
+
- Before your first sync
171
+
- When troubleshooting issues
172
+
- After changing configuration
173
+
- To verify everything is working
174
+
175
+
#### Individual Check Commands
176
+
177
+
**Normal sync** (recommended): Only processes repos that don't have AT Proto records yet. This is efficient and safe for regular use.
178
+
179
+
**Force sync**: Processes all repos regardless of whether they already have records. Use this if you need to:
180
+
- Re-push repos to Tangled
181
+
- Update READMEs for all repos
182
+
- Recover from a partial sync
183
+
184
+
### ✨ Next Steps
185
+
186
+
1. Copy and configure `src/.env`
187
+
2. Run `npm run validate`
188
+
3. Run `npm run test-atproto`
189
+
4. Run `npm run sync`
190
+
191
+
That's it! Your GitHub repos will be synced to Tangled with proper AT Proto records.
192
+
193
+
---
194
+
195
+
**Questions?** Check `SETUP.md` for detailed instructions and troubleshooting.
196
+
197
+
**Happy syncing! 🚀**
+2
-2
package-lock.json
+2
-2
package-lock.json
···
306
306
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
307
307
"dev": true,
308
308
"license": "Apache-2.0",
309
+
"peer": true,
309
310
"bin": {
310
311
"tsc": "bin/tsc",
311
312
"tsserver": "bin/tsserver"
···
328
329
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.14.0.tgz",
329
330
"integrity": "sha512-QQiYxHuyZ9gQUIrmPo3IA+hUl4KYk8uSA7cHrcKd/l3p1OTpZcM0Tbp9x7FAtXdAYhlasd60ncPpgu6ihG6TOA==",
330
331
"dev": true,
331
-
"license": "MIT",
332
-
"peer": true
332
+
"license": "MIT"
333
333
},
334
334
"node_modules/v8-compile-cache-lib": {
335
335
"version": "3.0.1",
+7
-1
package.json
+7
-1
package.json
···
2
2
"name": "tangled-sync",
3
3
"version": "1.0.0",
4
4
"description": "Sync GitHub repos to Tangled with ATProto records",
5
+
"type": "module",
5
6
"main": "src/index.ts",
6
7
"scripts": {
7
-
"sync": "ts-node src/index.ts"
8
+
"check": "ts-node src/check.ts",
9
+
"validate": "ts-node src/validate-config.ts",
10
+
"test-atproto": "ts-node src/test-atproto.ts",
11
+
"sync": "ts-node src/index.ts",
12
+
"sync:force": "ts-node src/index.ts --force"
8
13
},
9
14
"dependencies": {
10
15
"@atproto/api": "^0.17.2",
11
16
"dotenv": "^16.0.0"
12
17
},
13
18
"devDependencies": {
19
+
"@types/node": "^20.0.0",
14
20
"ts-node": "^10.9.2",
15
21
"typescript": "^5.9.3"
16
22
},
+15
src/.env.example
+15
src/.env.example
···
1
+
# Base directory where GitHub repos will be cloned
2
+
BASE_DIR=/path/to/your/repos
3
+
4
+
# Your GitHub username
5
+
GITHUB_USER=your-github-username
6
+
7
+
# Your ATProto DID (e.g., did:plc:abc123...)
8
+
ATPROTO_DID=did:plc:your-did-here
9
+
10
+
# Bluesky PDS URL (usually https://bsky.social)
11
+
BLUESKY_PDS=https://bsky.social
12
+
13
+
# Your Bluesky credentials
14
+
BLUESKY_USERNAME=your-handle.bsky.social
15
+
BLUESKY_PASSWORD=your-app-password
+266
src/check.ts
+266
src/check.ts
···
1
+
import { AtpAgent } from "@atproto/api";
2
+
import dotenv from "dotenv";
3
+
import fs from "fs";
4
+
import path from "path";
5
+
import { fileURLToPath } from "url";
6
+
import { execSync } from "child_process";
7
+
8
+
const __filename = fileURLToPath(import.meta.url);
9
+
const __dirname = path.dirname(__filename);
10
+
11
+
dotenv.config({ path: "./src/.env" });
12
+
13
+
async function runHealthCheck() {
14
+
15
+
console.log("🔍 Running Tangled Sync Health Check...\n");
16
+
17
+
const checks: { category: string; name: string; status: boolean; message: string }[] = [];
18
+
let errors = 0;
19
+
let warnings = 0;
20
+
21
+
// ===== CONFIGURATION CHECKS =====
22
+
console.log("📋 Configuration Checks\n");
23
+
24
+
const envPath = path.join(__dirname, ".env");
25
+
const envExists = fs.existsSync(envPath);
26
+
checks.push({
27
+
category: "config",
28
+
name: ".env file",
29
+
status: envExists,
30
+
message: envExists ? "Found at src/.env" : "Missing! Copy src/.env.example to src/.env"
31
+
});
32
+
if (!envExists) errors++;
33
+
34
+
const requiredVars = [
35
+
{ name: "BASE_DIR", description: "Base directory for repos" },
36
+
{ name: "GITHUB_USER", description: "GitHub username" },
37
+
{ name: "ATPROTO_DID", description: "AT Proto DID" },
38
+
{ name: "BLUESKY_PDS", description: "Bluesky PDS URL" },
39
+
{ name: "BLUESKY_USERNAME", description: "Bluesky username" },
40
+
{ name: "BLUESKY_PASSWORD", description: "Bluesky app password" },
41
+
];
42
+
43
+
requiredVars.forEach(({ name, description }) => {
44
+
const value = process.env[name];
45
+
const exists = !!value && value.trim().length > 0;
46
+
checks.push({
47
+
category: "config",
48
+
name: name,
49
+
status: exists,
50
+
message: exists ? `Set` : `Missing (${description})`
51
+
});
52
+
if (!exists) errors++;
53
+
});
54
+
55
+
// Check BASE_DIR
56
+
const baseDir = process.env.BASE_DIR;
57
+
if (baseDir) {
58
+
const baseDirExists = fs.existsSync(baseDir);
59
+
checks.push({
60
+
category: "config",
61
+
name: "BASE_DIR path",
62
+
status: baseDirExists,
63
+
message: baseDirExists ? `Exists: ${baseDir}` : `Missing (will be created): ${baseDir}`
64
+
});
65
+
if (!baseDirExists) warnings++;
66
+
}
67
+
68
+
// Check DID format
69
+
const did = process.env.ATPROTO_DID;
70
+
if (did) {
71
+
const validDid = did.startsWith("did:plc:") || did.startsWith("did:web:");
72
+
checks.push({
73
+
category: "config",
74
+
name: "DID format",
75
+
status: validDid,
76
+
message: validDid ? "Valid" : "Invalid! Should start with 'did:plc:' or 'did:web:'"
77
+
});
78
+
if (!validDid) errors++;
79
+
}
80
+
81
+
// Check PDS URL
82
+
const pds = process.env.BLUESKY_PDS;
83
+
if (pds) {
84
+
const validPds = pds.startsWith("http://") || pds.startsWith("https://");
85
+
checks.push({
86
+
category: "config",
87
+
name: "PDS URL",
88
+
status: validPds,
89
+
message: validPds ? pds : "Invalid! Should start with 'https://'"
90
+
});
91
+
if (!validPds) errors++;
92
+
}
93
+
94
+
// Print config results
95
+
checks.filter(c => c.category === "config").forEach((check) => {
96
+
const icon = check.status ? "✅" : "❌";
97
+
console.log(`${icon} ${check.name}: ${check.message}`);
98
+
});
99
+
100
+
// ===== AT PROTO CONNECTION CHECK =====
101
+
console.log("\n🔐 AT Proto Connection Check\n");
102
+
103
+
const canTestConnection = process.env.BLUESKY_USERNAME &&
104
+
process.env.BLUESKY_PASSWORD &&
105
+
process.env.BLUESKY_PDS &&
106
+
process.env.ATPROTO_DID;
107
+
108
+
if (canTestConnection) {
109
+
try {
110
+
const agent = new AtpAgent({ service: process.env.BLUESKY_PDS! });
111
+
112
+
const loginResponse = await agent.login({
113
+
identifier: process.env.BLUESKY_USERNAME!,
114
+
password: process.env.BLUESKY_PASSWORD!
115
+
});
116
+
117
+
console.log(`✅ Login successful`);
118
+
console.log(` DID: ${loginResponse.data.did}`);
119
+
console.log(` Handle: ${loginResponse.data.handle}`);
120
+
121
+
if (loginResponse.data.did !== process.env.ATPROTO_DID) {
122
+
console.log(`⚠️ DID mismatch!`);
123
+
console.log(` Expected: ${process.env.ATPROTO_DID}`);
124
+
console.log(` Got: ${loginResponse.data.did}`);
125
+
warnings++;
126
+
}
127
+
128
+
// Test fetching records
129
+
const records = await agent.api.com.atproto.repo.listRecords({
130
+
repo: loginResponse.data.did,
131
+
collection: "sh.tangled.repo",
132
+
limit: 5,
133
+
});
134
+
135
+
console.log(`✅ Can access AT Proto records`);
136
+
console.log(` Found ${records.data.records.length} sample records`);
137
+
138
+
} catch (error: any) {
139
+
console.log(`❌ AT Proto connection failed`);
140
+
console.log(` Error: ${error.message}`);
141
+
errors++;
142
+
}
143
+
} else {
144
+
console.log("⏭️ Skipped (missing credentials)");
145
+
}
146
+
147
+
// ===== SSH CONNECTION CHECK =====
148
+
console.log("\n🔑 SSH Connection Check\n");
149
+
150
+
try {
151
+
const sshTest = execSync("ssh -T git@tangled.sh 2>&1", {
152
+
encoding: "utf-8",
153
+
timeout: 5000
154
+
});
155
+
156
+
if (sshTest.includes("successfully authenticated") || sshTest.includes("Hi")) {
157
+
console.log("✅ SSH connection to Tangled works");
158
+
console.log(` ${sshTest.trim().split('\n')[0]}`);
159
+
} else {
160
+
console.log("⚠️ SSH connection uncertain");
161
+
console.log(` Response: ${sshTest.trim()}`);
162
+
warnings++;
163
+
}
164
+
} catch (error: any) {
165
+
const output = error.stdout?.toString() || error.message;
166
+
167
+
if (output.includes("successfully authenticated") || output.includes("Hi")) {
168
+
console.log("✅ SSH connection to Tangled works");
169
+
} else {
170
+
console.log("❌ SSH connection to Tangled failed");
171
+
console.log(" Make sure your SSH key is added at https://tangled.org/settings/keys");
172
+
errors++;
173
+
}
174
+
}
175
+
176
+
// ===== GITHUB API CHECK =====
177
+
console.log("\n🐙 GitHub API Check\n");
178
+
179
+
if (process.env.GITHUB_USER) {
180
+
try {
181
+
const response = execSync(`curl -s "https://api.github.com/users/${process.env.GITHUB_USER}"`, {
182
+
encoding: "utf-8",
183
+
timeout: 5000
184
+
});
185
+
186
+
const data = JSON.parse(response);
187
+
188
+
if (data.login) {
189
+
console.log(`✅ GitHub user found: ${data.login}`);
190
+
console.log(` Public repos: ${data.public_repos || 0}`);
191
+
} else {
192
+
console.log(`❌ GitHub user not found: ${process.env.GITHUB_USER}`);
193
+
errors++;
194
+
}
195
+
} catch (error: any) {
196
+
console.log(`⚠️ Could not check GitHub API`);
197
+
console.log(` ${error.message}`);
198
+
warnings++;
199
+
}
200
+
} else {
201
+
console.log("⏭️ Skipped (no GITHUB_USER set)");
202
+
}
203
+
204
+
// ===== DEPENDENCIES CHECK =====
205
+
console.log("\n📦 Dependencies Check\n");
206
+
207
+
let hasAtproto = false;
208
+
let hasDotenv = false;
209
+
210
+
try {
211
+
await import("@atproto/api");
212
+
hasAtproto = true;
213
+
console.log("✅ @atproto/api installed");
214
+
} catch {
215
+
console.log("❌ @atproto/api not installed (run: npm install)");
216
+
errors++;
217
+
}
218
+
219
+
try {
220
+
await import("dotenv");
221
+
hasDotenv = true;
222
+
console.log("✅ dotenv installed");
223
+
} catch {
224
+
console.log("❌ dotenv not installed (run: npm install)");
225
+
errors++;
226
+
}
227
+
228
+
// ===== SUMMARY =====
229
+
console.log("\n" + "=".repeat(50));
230
+
231
+
if (errors === 0 && warnings === 0) {
232
+
console.log("✅ All checks passed! Ready to sync.");
233
+
console.log("\nNext steps:");
234
+
console.log(" npm run sync # Sync new repos only");
235
+
console.log(" npm run sync:force # Force sync all repos");
236
+
} else {
237
+
if (errors > 0) {
238
+
console.log(`❌ ${errors} error(s) found - please fix before syncing`);
239
+
}
240
+
if (warnings > 0) {
241
+
console.log(`⚠️ ${warnings} warning(s) - review before syncing`);
242
+
}
243
+
244
+
console.log("\nSee SETUP.md for detailed troubleshooting");
245
+
246
+
if (errors > 0) {
247
+
process.exit(1);
248
+
}
249
+
}
250
+
251
+
console.log("=".repeat(50));
252
+
253
+
// Additional recommendations
254
+
if (process.env.BLUESKY_PASSWORD && !process.env.BLUESKY_PASSWORD.includes("-")) {
255
+
console.log("\n💡 Tip: Your password might be a regular password.");
256
+
console.log(" Consider using an App Password from Bluesky settings for better security.");
257
+
}
258
+
259
+
}
260
+
261
+
// Run the health check
262
+
runHealthCheck().catch((error) => {
263
+
console.error("\n❌ Health check failed with error:");
264
+
console.error(error);
265
+
process.exit(1);
266
+
});
+142
-33
src/index.ts
+142
-33
src/index.ts
···
6
6
7
7
dotenv.config({ path: "./src/.env" });
8
8
9
+
const FORCE_SYNC = process.argv.includes("--force");
10
+
9
11
const BASE_DIR = process.env.BASE_DIR!;
10
12
const GITHUB_USER = process.env.GITHUB_USER!;
11
13
const ATPROTO_DID = process.env.ATPROTO_DID!;
···
17
19
async function login() {
18
20
const username = process.env.BLUESKY_USERNAME;
19
21
const password = process.env.BLUESKY_PASSWORD;
20
-
if (!username || !password) throw new Error("Missing Bluesky credentials");
21
-
await agent.login({ identifier: username, password });
22
-
console.log("[LOGIN] Logged in to Bluesky");
22
+
if (!username || !password) {
23
+
throw new Error("Missing Bluesky credentials. Please set BLUESKY_USERNAME and BLUESKY_PASSWORD in src/.env");
24
+
}
25
+
26
+
try {
27
+
const response = await agent.login({ identifier: username, password });
28
+
console.log(`[LOGIN] Successfully logged in to AT Proto as ${response.data.did}`);
29
+
console.log(`[LOGIN] Session handle: ${response.data.handle}`);
30
+
return response;
31
+
} catch (error: any) {
32
+
console.error("[ERROR] Failed to login to AT Proto:", error.message);
33
+
throw error;
34
+
}
23
35
}
24
36
25
37
async function getGitHubRepos(): Promise<{ clone_url: string; name: string; description?: string }[]> {
···
90
102
return toBase32Sortable(tidBigInt);
91
103
}
92
104
93
-
// Tangled repo schema typing
105
+
// Tangled repo schema typing (matches sh.tangled.repo lexicon)
94
106
interface TangledRepoRecord {
95
107
$type: "sh.tangled.repo";
96
-
knot: string;
97
-
name: string;
98
-
spindle: string;
99
-
description: string;
100
-
source: string;
101
-
labels: string[];
102
-
createdAt: string;
108
+
name: string; // required
109
+
knot: string; // required
110
+
createdAt: string; // required (ISO 8601 datetime)
111
+
spindle?: string; // optional CI runner
112
+
description?: string; // optional, max 140 graphemes
113
+
website?: string; // optional URI
114
+
topics?: string[]; // optional array of topics
115
+
source?: string; // optional source URI
116
+
labels?: string[]; // optional array of at-uri labels
103
117
}
104
118
105
119
// Cache for existing repo records
···
111
125
githubUser: string,
112
126
repoName: string,
113
127
description?: string
114
-
): Promise<string> {
115
-
if (recordCache[repoName]) return recordCache[repoName];
128
+
): Promise<{ tid: string; existed: boolean }> {
129
+
if (recordCache[repoName]) {
130
+
return { tid: recordCache[repoName], existed: true };
131
+
}
116
132
117
133
let cursor: string | undefined = undefined;
118
134
let tid: string | null = null;
···
131
147
tid = record.rkey;
132
148
recordCache[repoName] = tid;
133
149
console.log(`[FOUND] Existing record for ${repoName} (TID: ${tid})`);
134
-
break;
150
+
return { tid, existed: true };
135
151
}
136
152
}
137
153
···
142
158
tid = generateTid();
143
159
const record: TangledRepoRecord = {
144
160
$type: "sh.tangled.repo",
145
-
knot: "knot1.tangled.sh",
146
161
name: repoName,
147
-
spindle: "",
162
+
knot: "knot1.tangled.sh",
163
+
createdAt: new Date().toISOString(),
148
164
description: description ?? repoName,
149
165
source: `https://github.com/${githubUser}/${repoName}`,
150
166
labels: [],
151
-
createdAt: new Date().toISOString(),
152
167
};
153
168
154
-
await agent.api.com.atproto.repo.putRecord({
155
-
repo: atprotoDid,
156
-
collection: "sh.tangled.repo",
157
-
rkey: tid,
158
-
record,
159
-
});
169
+
try {
170
+
const result = await agent.api.com.atproto.repo.putRecord({
171
+
repo: atprotoDid,
172
+
collection: "sh.tangled.repo",
173
+
rkey: tid,
174
+
record,
175
+
});
176
+
console.log(`[CREATED] ATProto record URI: ${result.data.uri}`);
177
+
} catch (error: any) {
178
+
console.error(`[ERROR] Failed to create ATProto record for ${repoName}:`, error.message);
179
+
throw error;
180
+
}
160
181
161
182
recordCache[repoName] = tid;
162
183
console.log(`[CREATED] Tangled record for ${repoName} (TID: ${tid})`);
184
+
return { tid, existed: false };
163
185
}
164
186
165
-
return tid;
187
+
return { tid, existed: false };
166
188
}
167
189
168
190
function updateReadme(baseDir: string, repoName: string, atprotoDid: string) {
···
187
209
}
188
210
189
211
async function main() {
212
+
console.log("[STARTUP] Starting Tangled Sync...");
213
+
if (FORCE_SYNC) {
214
+
console.log("[MODE] Force sync enabled - will process all repos");
215
+
}
216
+
console.log(`[CONFIG] Base directory: ${BASE_DIR}`);
217
+
console.log(`[CONFIG] GitHub user: ${GITHUB_USER}`);
218
+
console.log(`[CONFIG] ATProto DID: ${ATPROTO_DID}`);
219
+
console.log(`[CONFIG] PDS: ${BLUESKY_PDS}`);
220
+
221
+
// Login to AT Proto
190
222
await login();
223
+
224
+
// Ensure base directory exists
191
225
ensureDir(BASE_DIR);
226
+
227
+
// Fetch GitHub repositories
228
+
console.log(`[GITHUB] Fetching repositories for ${GITHUB_USER}...`);
192
229
const repos = await getGitHubRepos();
230
+
console.log(`[GITHUB] Found ${repos.length} repositories`);
231
+
232
+
let reposToProcess = repos;
233
+
let skippedRepos: typeof repos = [];
234
+
235
+
if (!FORCE_SYNC) {
236
+
// Fetch all existing Tangled records upfront
237
+
console.log(`[ATPROTO] Fetching existing Tangled records...`);
238
+
let cursor: string | undefined = undefined;
239
+
const existingRepos = new Set<string>();
240
+
241
+
do {
242
+
const res: any = await agent.api.com.atproto.repo.listRecords({
243
+
repo: ATPROTO_DID,
244
+
collection: "sh.tangled.repo",
245
+
limit: 100,
246
+
cursor,
247
+
});
248
+
249
+
for (const record of res.data.records) {
250
+
const value = record.value as TangledRepoRecord;
251
+
if (value.name) {
252
+
existingRepos.add(value.name);
253
+
recordCache[value.name] = record.rkey;
254
+
}
255
+
}
256
+
257
+
cursor = res.data.cursor;
258
+
} while (cursor);
259
+
260
+
console.log(`[ATPROTO] Found ${existingRepos.size} existing Tangled records`);
261
+
262
+
// Separate repos into new and existing
263
+
reposToProcess = repos.filter(r => !existingRepos.has(r.name));
264
+
skippedRepos = repos.filter(r => existingRepos.has(r.name));
265
+
266
+
console.log(`[INFO] ${reposToProcess.length} new repos to sync`);
267
+
console.log(`[INFO] ${skippedRepos.length} repos already synced (skipping)\n`);
268
+
269
+
if (skippedRepos.length > 0) {
270
+
console.log("[SKIPPED] The following repos already have AT Proto records:");
271
+
skippedRepos.forEach(r => console.log(` - ${r.name}`));
272
+
console.log("");
273
+
}
274
+
} else {
275
+
console.log("[INFO] Processing all ${repos.length} repos (force sync mode)\n");
276
+
}
193
277
194
-
for (const { clone_url, name: repoName, description } of repos) {
195
-
console.log(`[PROGRESS] Processing ${repoName}`);
278
+
let syncedCount = 0;
279
+
let errorCount = 0;
280
+
281
+
for (const { clone_url, name: repoName, description } of reposToProcess) {
282
+
console.log(`\n[PROGRESS] Processing ${repoName} (${syncedCount + 1}/${reposToProcess.length})`);
196
283
const repoDir = path.join(BASE_DIR, repoName);
197
284
198
-
if (!fs.existsSync(repoDir)) {
199
-
run(`git clone ${clone_url} ${repoDir}`);
200
-
console.log(`[CLONE] ${repoName}`);
285
+
try {
286
+
if (!fs.existsSync(repoDir)) {
287
+
run(`git clone ${clone_url} ${repoDir}`);
288
+
console.log(`[CLONE] ${repoName}`);
289
+
} else {
290
+
console.log(`[EXISTS] ${repoName} already cloned`);
291
+
}
292
+
293
+
await ensureTangledRemoteAndPush(repoDir, repoName, clone_url);
294
+
updateReadme(BASE_DIR, repoName, ATPROTO_DID);
295
+
const result = await ensureTangledRecord(agent, ATPROTO_DID, GITHUB_USER, repoName, description);
296
+
297
+
if (!result.existed) {
298
+
syncedCount++;
299
+
}
300
+
} catch (error: any) {
301
+
console.error(`[ERROR] Failed to sync ${repoName}: ${error.message}`);
302
+
errorCount++;
201
303
}
202
-
203
-
await ensureTangledRemoteAndPush(repoDir, repoName, clone_url);
204
-
updateReadme(BASE_DIR, repoName, ATPROTO_DID);
205
-
await ensureTangledRecord(agent, ATPROTO_DID, GITHUB_USER, repoName, description);
304
+
}
305
+
306
+
console.log(`\n${'='.repeat(50)}`);
307
+
console.log(`[COMPLETE] Sync finished!`);
308
+
console.log(` ✅ New repos synced: ${syncedCount}`);
309
+
if (!FORCE_SYNC) {
310
+
console.log(` ⏭️ Repos skipped: ${skippedRepos.length}`);
206
311
}
312
+
if (errorCount > 0) {
313
+
console.log(` ❌ Errors: ${errorCount}`);
314
+
}
315
+
console.log(`${'='.repeat(50)}`);
207
316
}
208
317
209
318
main().catch(console.error);
+71
src/test-atproto.ts
+71
src/test-atproto.ts
···
1
+
import { AtpAgent } from "@atproto/api";
2
+
import dotenv from "dotenv";
3
+
4
+
dotenv.config({ path: "./src/.env" });
5
+
6
+
async function testAtProtoConnection() {
7
+
console.log("Testing AT Proto Connection...\n");
8
+
9
+
const service = process.env.BLUESKY_PDS || "https://bsky.social";
10
+
const username = process.env.BLUESKY_USERNAME;
11
+
const password = process.env.BLUESKY_PASSWORD;
12
+
const atprotoDid = process.env.ATPROTO_DID;
13
+
14
+
console.log(`Service: ${service}`);
15
+
console.log(`Username: ${username}`);
16
+
console.log(`Expected DID: ${atprotoDid}\n`);
17
+
18
+
if (!username || !password) {
19
+
console.error("ERROR: Missing BLUESKY_USERNAME or BLUESKY_PASSWORD");
20
+
process.exit(1);
21
+
}
22
+
23
+
const agent = new AtpAgent({ service });
24
+
25
+
try {
26
+
console.log("Attempting login...");
27
+
const loginResponse = await agent.login({
28
+
identifier: username,
29
+
password
30
+
});
31
+
32
+
console.log("✓ Login successful!");
33
+
console.log(` DID: ${loginResponse.data.did}`);
34
+
console.log(` Handle: ${loginResponse.data.handle}`);
35
+
console.log(` Email: ${loginResponse.data.email || "N/A"}`);
36
+
37
+
if (loginResponse.data.did !== atprotoDid) {
38
+
console.warn(`\n⚠ WARNING: Logged in DID (${loginResponse.data.did}) does not match ATPROTO_DID in .env (${atprotoDid})`);
39
+
console.warn(" Please update your ATPROTO_DID in src/.env");
40
+
}
41
+
42
+
// Test fetching existing records
43
+
console.log("\nFetching existing sh.tangled.repo records...");
44
+
const records = await agent.api.com.atproto.repo.listRecords({
45
+
repo: loginResponse.data.did,
46
+
collection: "sh.tangled.repo",
47
+
limit: 10,
48
+
});
49
+
50
+
console.log(`✓ Found ${records.data.records.length} existing Tangled repo records`);
51
+
52
+
if (records.data.records.length > 0) {
53
+
console.log("\nSample records:");
54
+
records.data.records.slice(0, 3).forEach((record: any) => {
55
+
console.log(` - ${record.value.name} (${record.uri})`);
56
+
});
57
+
}
58
+
59
+
console.log("\n✓ AT Proto connection test completed successfully!");
60
+
61
+
} catch (error: any) {
62
+
console.error("\n✗ AT Proto connection test failed!");
63
+
console.error(`Error: ${error.message}`);
64
+
if (error.status) {
65
+
console.error(`HTTP Status: ${error.status}`);
66
+
}
67
+
process.exit(1);
68
+
}
69
+
}
70
+
71
+
testAtProtoConnection();
+109
src/validate-config.ts
+109
src/validate-config.ts
···
1
+
import dotenv from "dotenv";
2
+
import fs from "fs";
3
+
import path from "path";
4
+
import { fileURLToPath } from "url";
5
+
6
+
const __filename = fileURLToPath(import.meta.url);
7
+
const __dirname = path.dirname(__filename);
8
+
9
+
dotenv.config({ path: "./src/.env" });
10
+
11
+
console.log("🔍 Validating Tangled Sync Configuration...\n");
12
+
13
+
const checks: { name: string; status: boolean; message: string }[] = [];
14
+
15
+
// Check .env file exists
16
+
const envPath = path.join(__dirname, ".env");
17
+
const envExists = fs.existsSync(envPath);
18
+
checks.push({
19
+
name: ".env file",
20
+
status: envExists,
21
+
message: envExists ? "Found at src/.env" : "Missing! Copy src/.env.example to src/.env"
22
+
});
23
+
24
+
// Check required environment variables
25
+
const requiredVars = [
26
+
{ name: "BASE_DIR", description: "Base directory for repos" },
27
+
{ name: "GITHUB_USER", description: "GitHub username" },
28
+
{ name: "ATPROTO_DID", description: "AT Proto DID" },
29
+
{ name: "BLUESKY_PDS", description: "Bluesky PDS URL" },
30
+
{ name: "BLUESKY_USERNAME", description: "Bluesky username" },
31
+
{ name: "BLUESKY_PASSWORD", description: "Bluesky app password" },
32
+
];
33
+
34
+
requiredVars.forEach(({ name, description }) => {
35
+
const value = process.env[name];
36
+
const exists = !!value && value.trim().length > 0;
37
+
checks.push({
38
+
name: `${name}`,
39
+
status: exists,
40
+
message: exists ? `✓ Set (${description})` : `✗ Missing (${description})`
41
+
});
42
+
});
43
+
44
+
// Validate BASE_DIR
45
+
const baseDir = process.env.BASE_DIR;
46
+
if (baseDir) {
47
+
const baseDirExists = fs.existsSync(baseDir);
48
+
checks.push({
49
+
name: "BASE_DIR exists",
50
+
status: baseDirExists,
51
+
message: baseDirExists ? `Directory exists: ${baseDir}` : `Directory missing: ${baseDir} (will be created)`
52
+
});
53
+
}
54
+
55
+
// Validate DID format
56
+
const did = process.env.ATPROTO_DID;
57
+
if (did) {
58
+
const validDid = did.startsWith("did:plc:") || did.startsWith("did:web:");
59
+
checks.push({
60
+
name: "DID format",
61
+
status: validDid,
62
+
message: validDid ? "Valid DID format" : "Invalid! Should start with 'did:plc:' or 'did:web:'"
63
+
});
64
+
}
65
+
66
+
// Validate PDS URL
67
+
const pds = process.env.BLUESKY_PDS;
68
+
if (pds) {
69
+
const validPds = pds.startsWith("http://") || pds.startsWith("https://");
70
+
checks.push({
71
+
name: "PDS URL format",
72
+
status: validPds,
73
+
message: validPds ? `Valid URL: ${pds}` : "Invalid! Should start with 'https://'"
74
+
});
75
+
}
76
+
77
+
// Print results
78
+
console.log("Configuration Check Results:\n");
79
+
let allPassed = true;
80
+
81
+
checks.forEach((check) => {
82
+
const icon = check.status ? "✅" : "❌";
83
+
console.log(`${icon} ${check.name}: ${check.message}`);
84
+
if (!check.status) allPassed = false;
85
+
});
86
+
87
+
console.log("\n" + "=".repeat(50) + "\n");
88
+
89
+
if (allPassed) {
90
+
console.log("✅ All checks passed! You're ready to run:");
91
+
console.log(" npm run test-atproto # Test AT Proto connection");
92
+
console.log(" npm run sync # Run the full sync");
93
+
} else {
94
+
console.log("❌ Some checks failed. Please fix the issues above.");
95
+
console.log(" See SETUP.md for detailed instructions.");
96
+
process.exit(1);
97
+
}
98
+
99
+
// Additional recommendations
100
+
console.log("\n💡 Recommendations:");
101
+
102
+
if (process.env.BLUESKY_PASSWORD && !process.env.BLUESKY_PASSWORD.includes("-")) {
103
+
console.log(" ⚠️ Your password looks like it might be a regular password.");
104
+
console.log(" Consider using an App Password from Bluesky settings.");
105
+
}
106
+
107
+
console.log(" 📚 Read SETUP.md for detailed setup instructions");
108
+
console.log(" 🔐 Never commit your .env file to version control");
109
+
console.log(" 🔑 Make sure your SSH key is added to Tangled");
+7
-3
tsconfig.json
+7
-3
tsconfig.json
···
1
1
{
2
2
"compilerOptions": {
3
3
"target": "ES2022",
4
-
"module": "ESNext",
5
-
"moduleResolution": "bundler",
4
+
"module": "NodeNext",
5
+
"moduleResolution": "NodeNext",
6
6
"rootDir": "./src",
7
7
"outDir": "./dist",
8
8
"strict": true,
···
18
18
"allowJs": false
19
19
},
20
20
"include": ["src/**/*.ts"],
21
-
"exclude": ["node_modules", "dist"]
21
+
"exclude": ["node_modules", "dist"],
22
+
"ts-node": {
23
+
"esm": true,
24
+
"experimentalSpecifierResolution": "node"
25
+
}
22
26
}