···11+# Created by https://www.toptal.com/developers/gitignore/api/node,macos,svelte,vercel
22+# Edit at https://www.toptal.com/developers/gitignore?templates=node,macos,svelte,vercel
33+44+### macOS ###
55+# General
66+.DS_Store
77+.AppleDouble
88+.LSOverride
99+1010+# Icon must end with two \r
1111+Icon
1212+1313+1414+# Thumbnails
1515+._*
1616+1717+# Files that might appear in the root of a volume
1818+.DocumentRevisions-V100
1919+.fseventsd
2020+.Spotlight-V100
2121+.TemporaryItems
2222+.Trashes
2323+.VolumeIcon.icns
2424+.com.apple.timemachine.donotpresent
2525+2626+# Directories potentially created on remote AFP share
2727+.AppleDB
2828+.AppleDesktop
2929+Network Trash Folder
3030+Temporary Items
3131+.apdisk
3232+3333+### macOS Patch ###
3434+# iCloud generated files
3535+*.icloud
3636+3737+### Node ###
3838+# Logs
3939+logs
4040+*.log
4141+npm-debug.log*
4242+yarn-debug.log*
4343+yarn-error.log*
4444+lerna-debug.log*
4545+.pnpm-debug.log*
4646+4747+# Diagnostic reports (https://nodejs.org/api/report.html)
4848+report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
4949+5050+# Runtime data
5151+pids
5252+*.pid
5353+*.seed
5454+*.pid.lock
5555+5656+# Directory for instrumented libs generated by jscoverage/JSCover
5757+lib-cov
5858+5959+# Coverage directory used by tools like istanbul
6060+coverage
6161+*.lcov
6262+6363+# nyc test coverage
6464+.nyc_output
6565+6666+# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files)
6767+.grunt
6868+6969+# Bower dependency directory (https://bower.io/)
7070+bower_components
7171+7272+# node-waf configuration
7373+.lock-wscript
7474+7575+# Compiled binary addons (https://nodejs.org/api/addons.html)
7676+build/Release
7777+7878+# Dependency directories
7979+node_modules/
8080+jspm_packages/
8181+8282+# Snowpack dependency directory (https://snowpack.dev/)
8383+web_modules/
8484+8585+# TypeScript cache
8686+*.tsbuildinfo
8787+8888+# Optional npm cache directory
8989+.npm
9090+9191+# Optional eslint cache
9292+.eslintcache
9393+9494+# Optional stylelint cache
9595+.stylelintcache
9696+9797+# Microbundle cache
9898+.rpt2_cache/
9999+.rts2_cache_cjs/
100100+.rts2_cache_es/
101101+.rts2_cache_umd/
102102+103103+# Optional REPL history
104104+.node_repl_history
105105+106106+# Output of 'npm pack'
107107+*.tgz
108108+109109+# Yarn Integrity file
110110+.yarn-integrity
111111+112112+# dotenv environment variable files
113113+.env
114114+.env.development.local
115115+.env.test.local
116116+.env.production.local
117117+.env.local
118118+119119+# parcel-bundler cache (https://parceljs.org/)
120120+.cache
121121+.parcel-cache
122122+123123+# Next.js build output
124124+.next
125125+out
126126+127127+# Nuxt.js build / generate output
128128+.nuxt
129129+dist
130130+131131+# Gatsby files
132132+.cache/
133133+# Comment in the public line in if your project uses Gatsby and not Next.js
134134+# https://nextjs.org/blog/next-9-1#public-directory-support
135135+# public
136136+137137+# vuepress build output
138138+.vuepress/dist
139139+140140+# vuepress v2.x temp and cache directory
141141+.temp
142142+143143+# Docusaurus cache and generated files
144144+.docusaurus
145145+146146+# Serverless directories
147147+.serverless/
148148+149149+# FuseBox cache
150150+.fusebox/
151151+152152+# DynamoDB Local files
153153+.dynamodb/
154154+155155+# TernJS port file
156156+.tern-port
157157+158158+# Stores VSCode versions used for testing VSCode extensions
159159+.vscode-test
160160+161161+# yarn v2
162162+.yarn/cache
163163+.yarn/unplugged
164164+.yarn/build-state.yml
165165+.yarn/install-state.gz
166166+.pnp.*
167167+168168+### Node Patch ###
169169+# Serverless Webpack directories
170170+.webpack/
171171+172172+# Optional stylelint cache
173173+174174+# SvelteKit build / generate output
175175+.svelte-kit
176176+177177+### Svelte ###
178178+# gitignore template for the SvelteKit, frontend web component framework
179179+# website: https://kit.svelte.dev/
180180+181181+.svelte-kit/
182182+package
183183+184184+### Vercel ###
185185+.vercel
186186+187187+# End of https://www.toptal.com/developers/gitignore/api/node,macos,svelte,vercel
188188+189189+git-diff*.txt
190190+node_modules
+21
packages/tangled-sync/LICENSE
···11+MIT License
22+33+Copyright (c) 2025 Ewan
44+55+Permission is hereby granted, free of charge, to any person obtaining a copy
66+of this software and associated documentation files (the "Software"), to deal
77+in the Software without restriction, including without limitation the rights
88+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
99+copies of the Software, and to permit persons to whom the Software is
1010+furnished to do so, subject to the following conditions:
1111+1212+The above copyright notice and this permission notice shall be included in all
1313+copies or substantial portions of the Software.
1414+1515+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
1616+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
1717+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
1818+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
1919+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
2020+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
2121+SOFTWARE.
+124
packages/tangled-sync/README.md
···11+# Tangled Sync
22+33+**Tangled Sync** is a TypeScript project that automates the process of syncing GitHub repositories to Tangled and publishing ATProto records for each repository. It is designed to streamline your workflow if you want your GitHub projects mirrored on Tangled while also maintaining structured metadata in ATProto.
44+55+This tool is particularly useful for developers and organisations that want a decentralized or alternative hosting layer for their code repositories while keeping them discoverable via ATProto.
66+77+---
88+99+## Getting Started
1010+1111+### Configuration
1212+1313+Before running any scripts, you need to configure the project. Create a `src/.env` file based on `src/.env.example`:
1414+1515+```bash
1616+cp src/.env.example src/.env
1717+```
1818+1919+Then edit `src/.env` with your actual values:
2020+2121+* `BASE_DIR` โ the local directory where GitHub repositories will be cloned.
2222+* `GITHUB_USER` โ your GitHub username or organisation.
2323+* `ATPROTO_DID` โ your ATProto DID (Decentralized Identifier).
2424+* `BLUESKY_PDS` โ the URL of your Bluesky PDS instance.
2525+* `BLUESKY_USERNAME` โ your Bluesky username.
2626+* `BLUESKY_PASSWORD` โ your Bluesky password.
2727+2828+Make sure this file is properly set up before proceeding.
2929+3030+---
3131+3232+### Installation
3333+3434+1. Clone this repository locally.
3535+2. Navigate to the project directory.
3636+3. Run:
3737+3838+```bash
3939+npm install
4040+```
4141+4242+This will install all dependencies required for syncing GitHub repositories and interacting with ATProto.
4343+4444+---
4545+4646+### Verify SSH Connection to Tangled
4747+4848+* If the Tangled remote does not exist for a repository, the script will attempt to create it on first run. This requires a working SSH key associated with your account.
4949+5050+Without proper SSH authentication, repository creation and pushing will fail.
5151+5252+---
5353+5454+### Testing AT Proto Connection
5555+5656+**Before running the full sync**, test your AT Proto connection:
5757+5858+```bash
5959+npm run test-atproto
6060+```
6161+6262+This will:
6363+- Verify your Bluesky credentials
6464+- Confirm your DID matches the configuration
6565+- List any existing `sh.tangled.repo` records
6666+- Validate the connection to the PDS
6767+6868+### Running the Sync Script
6969+7070+Once configuration, SSH verification, and AT Proto testing are complete, run:
7171+7272+```bash
7373+npm run sync
7474+```
7575+7676+What happens during the sync:
7777+7878+1. **Login to Bluesky:** The script authenticates using your credentials to allow publishing ATProto records.
7979+2. **Clone GitHub Repositories:** All repositories under your configured GitHub user are cloned locally (excluding a repository with the same name as your username to avoid recursion).
8080+3. **Ensure Tangled Remotes:** For each repository, a `tangled` remote is added if it doesnโt exist.
8181+4. **Push to Tangled:** The script pushes the `main` branch to Tangled. If your `origin` remoteโs push URL points to Tangled, it will reset it back to GitHub.
8282+5. **Update README:** Each repositoryโs README is updated to include a link to its Tangled mirror, if it isnโt already present.
8383+6. **Create ATProto Records:** Each repository gets a structured record published in ATProto under your DID, including metadata like description, creation date, and source URL.
8484+8585+---
8686+8787+### Notes & Best Practices
8888+8989+* **Directory Management:** The script ensures that your `BASE_DIR` exists and creates it if necessary.
9090+* **Record Uniqueness:** ATProto records use a time-based, sortable ID (TID) to ensure uniqueness. Duplicate IDs are avoided automatically.
9191+* **Error Handling:** If a repository cannot be pushed to Tangled, the script logs a warning but continues processing the remaining repositories.
9292+* **Idempotency:** Running the script multiple times is safe; existing remotes and ATProto records are checked before creation to prevent duplicates.
9393+9494+---
9595+9696+### Example Workflow
9797+9898+```bash
9999+# Run the sync script
100100+npm run sync
101101+```
102102+103103+After execution, youโll see logs detailing which repositories were cloned, which remotes were added, which READMEs were updated, and which ATProto records were created.
104104+105105+This allows you to quickly confirm that all GitHub repositories have been mirrored and documented properly on Tangled.
106106+107107+---
108108+109109+### Contribution & Development
110110+111111+If you plan to contribute:
112112+113113+* Ensure Node.js v18+ and npm v9+ are installed.
114114+* Test the script in a separate directory to avoid accidentally overwriting your production repositories.
115115+* Use `console.log` statements to debug or track progress during development.
116116+* Maintain proper `.env` configuration to avoid leaking credentials.
117117+118118+---
119119+120120+**Tangled Sync** bridges GitHub and Tangled efficiently, providing automatic mirroring, record management, and easy discoverability. Following these steps will ensure a smooth, automated workflow for syncing and publishing your repositories.
121121+122122+## โ Support
123123+124124+If you found this useful, consider [buying me a ko-fi](https://ko-fi.com/ewancroft)!
+175
packages/tangled-sync/SETUP.md
···11+# Tangled Sync - Setup & Troubleshooting Guide
22+33+## Quick Setup Checklist
44+55+### 1. Install Dependencies
66+```bash
77+npm install
88+```
99+1010+### 2. Configure Environment Variables
1111+```bash
1212+# Copy the example env file
1313+cp src/.env.example src/.env
1414+1515+# Edit with your actual values
1616+nano src/.env # or use your preferred editor
1717+```
1818+1919+**Required values:**
2020+- `BASE_DIR`: Where to clone repos (e.g., `/Users/you/tangled-repos`)
2121+- `GITHUB_USER`: Your GitHub username
2222+- `ATPROTO_DID`: Your AT Proto DID (get from Bluesky settings)
2323+- `BLUESKY_PDS`: Usually `https://bsky.social`
2424+- `BLUESKY_USERNAME`: Your Bluesky handle (e.g., `you.bsky.social`)
2525+- `BLUESKY_PASSWORD`: Use an **app password**, not your main password!
2626+2727+### 3. Get Your AT Proto DID
2828+2929+Your DID can be found by:
3030+1. Go to https://bsky.app
3131+2. Click your profile
3232+3. Settings โ Advanced โ Account
3333+4. Look for "DID" (starts with `did:plc:`)
3434+3535+Alternatively, visit: `https://bsky.social/xrpc/com.atproto.identity.resolveHandle?handle=YOUR_HANDLE.bsky.social`
3636+3737+### 4. Create an App Password
3838+3939+**IMPORTANT:** Do NOT use your main Bluesky password!
4040+4141+1. Go to https://bsky.app/settings
4242+2. Navigate to "App Passwords"
4343+3. Click "Add App Password"
4444+4. Give it a name (e.g., "Tangled Sync")
4545+5. Copy the generated password to your `.env` file
4646+4747+### 5. Test AT Proto Connection
4848+```bash
4949+npm run test-atproto
5050+```
5151+5252+Expected output:
5353+```
5454+โ Login successful!
5555+ DID: did:plc:...
5656+ Handle: you.bsky.social
5757+ Email: your@email.com
5858+5959+โ Found X existing Tangled repo records
6060+```
6161+6262+### 6. Verify SSH to Tangled
6363+```bash
6464+ssh git@tangled.sh
6565+```
6666+6767+You should see a message confirming your SSH key is configured.
6868+6969+### 7. Run the Sync
7070+```bash
7171+npm run sync
7272+```
7373+7474+---
7575+7676+## Common Issues & Solutions
7777+7878+### Issue: "Missing Bluesky credentials"
7979+**Solution:** Check that `src/.env` exists and contains `BLUESKY_USERNAME` and `BLUESKY_PASSWORD`
8080+8181+### Issue: "Login failed" or "Invalid credentials"
8282+**Solution:**
8383+- Ensure you're using an **app password**, not your main password
8484+- Check your username includes the full handle (e.g., `you.bsky.social`)
8585+- Verify credentials are correct
8686+8787+### Issue: "DID mismatch"
8888+**Solution:**
8989+- Run `npm run test-atproto` to see your actual DID
9090+- Update `ATPROTO_DID` in `src/.env` to match
9191+9292+### Issue: "Could not push to Tangled"
9393+**Solution:**
9494+- Verify SSH key is added to Tangled: https://tangled.org/settings/keys
9595+- Test SSH connection: `ssh git@tangled.sh`
9696+- Ensure the repository exists on Tangled first
9797+9898+### Issue: "Failed to create ATProto record"
9999+**Solution:**
100100+- Check that the schema matches (required fields: `name`, `knot`, `createdAt`)
101101+- Verify your app password has write permissions
102102+- Check PDS is reachable: `curl https://bsky.social`
103103+104104+### Issue: Rate limiting from GitHub API
105105+**Solution:**
106106+- GitHub has a rate limit of 60 requests/hour for unauthenticated requests
107107+- Consider adding GitHub authentication if syncing many repos
108108+- Wait an hour and try again
109109+110110+---
111111+112112+## Understanding the Workflow
113113+114114+1. **Login to AT Proto**: Authenticates with Bluesky PDS using your credentials
115115+2. **Fetch GitHub Repos**: Retrieves all public repos from your GitHub account
116116+3. **Clone Locally**: Downloads repos to `BASE_DIR` if not already present
117117+4. **Add Tangled Remote**: Adds `tangled` as a git remote
118118+5. **Push to Tangled**: Pushes the `main` branch to Tangled
119119+6. **Update README**: Adds a Tangled mirror link to the README
120120+7. **Create AT Proto Record**: Publishes metadata to the AT Proto network
121121+122122+Each repository gets a record in the `sh.tangled.repo` collection with:
123123+- Repository name and description
124124+- Source URL (GitHub)
125125+- Creation timestamp
126126+- Knot server reference
127127+- Optional labels and topics
128128+129129+---
130130+131131+## Verifying Success
132132+133133+After running the sync, you can verify:
134134+135135+1. **Local repos**: Check `BASE_DIR` for cloned repositories
136136+2. **Tangled remotes**: Run `git remote -v` in any repo directory
137137+3. **AT Proto records**: Run `npm run test-atproto` to list records
138138+4. **Tangled website**: Visit `https://tangled.org/YOUR_DID/REPO_NAME`
139139+140140+---
141141+142142+## Advanced Configuration
143143+144144+### Using a Different PDS
145145+If you're not using the default Bluesky PDS:
146146+```bash
147147+BLUESKY_PDS=https://your-pds.example.com
148148+```
149149+150150+### Syncing Specific Repos Only
151151+Modify the `getGitHubRepos()` function to filter repos:
152152+```typescript
153153+return json
154154+ .filter((r: any) => r.name.startsWith('my-prefix-'))
155155+ .map(...);
156156+```
157157+158158+### Changing the Default Branch
159159+If your repos use `master` instead of `main`, update:
160160+```typescript
161161+run(`git push tangled master`, repoDir);
162162+```
163163+164164+---
165165+166166+## Support
167167+168168+For issues with:
169169+- **Tangled**: https://github.com/tangled-dev/tangled
170170+- **AT Proto**: https://atproto.com/docs
171171+- **This tool**: Open an issue in the repository
172172+173173+---
174174+175175+**Happy syncing! ๐**
+197
packages/tangled-sync/USAGE.md
···11+# ๐ Tangled Sync - Ready to Use!
22+33+## Summary of Changes
44+55+I've improved your Tangled Sync project to ensure proper AT Proto authentication and repository record creation. Here's what was updated:
66+77+### โ What's Fixed
88+99+1. **Enhanced AT Proto Login**
1010+ - Added better error handling and validation
1111+ - Shows DID and handle on successful login
1212+ - Clearer error messages when authentication fails
1313+1414+2. **Corrected Repository Schema**
1515+ - Fixed record structure to match `sh.tangled.repo` lexicon
1616+ - Required fields (`name`, `knot`, `createdAt`) now ordered correctly
1717+ - Optional fields properly marked as optional
1818+ - Added better error handling for record creation
1919+2020+3. **Improved Logging**
2121+ - More detailed startup information
2222+ - Better progress tracking during sync
2323+ - Shows AT Proto record URIs when created
2424+ - Success/failure messages are clearer
2525+2626+### ๐ New Files Created
2727+2828+1. **`src/.env.example`** - Template for your configuration
2929+2. **`src/test-atproto.ts`** - Test AT Proto connection before syncing
3030+3. **`src/validate-config.ts`** - Validate your environment setup
3131+4. **`SETUP.md`** - Comprehensive setup and troubleshooting guide
3232+3333+### ๐ How to Use
3434+3535+#### Step 1: Configure Environment
3636+```bash
3737+# Copy the example file
3838+cp src/.env.example src/.env
3939+4040+# Edit with your actual values
4141+nano src/.env
4242+```
4343+4444+You need:
4545+- Your GitHub username
4646+- Your AT Proto DID (from Bluesky settings)
4747+- A Bluesky **app password** (not your main password!)
4848+- Base directory for repos
4949+5050+#### Step 2: Validate Configuration
5151+```bash
5252+npm run validate
5353+```
5454+5555+This checks all your environment variables are set correctly.
5656+5757+#### Step 3: Test AT Proto Connection
5858+```bash
5959+npm run test-atproto
6060+```
6161+6262+This verifies:
6363+- โ Your credentials work
6464+- โ Your DID is correct
6565+- โ You can access the PDS
6666+- โ Shows any existing Tangled repo records
6767+6868+#### Step 4: Run the Sync
6969+```bash
7070+npm run sync
7171+```
7272+7373+This will:
7474+1. Login to AT Proto โ
7575+2. Fetch your GitHub repos
7676+3. Clone them locally (if needed)
7777+4. Add Tangled remotes
7878+5. Push to Tangled
7979+6. Update READMEs
8080+7. Create AT Proto records for each repo โ
8181+8282+### ๐ What to Check
8383+8484+After running the sync, verify:
8585+8686+1. **AT Proto Records Created**
8787+ ```bash
8888+ npm run test-atproto
8989+ ```
9090+ Should show your repos listed
9191+9292+2. **Repos on Tangled**
9393+ Visit: `https://tangled.org/YOUR_DID/REPO_NAME`
9494+9595+3. **Local Git Remotes**
9696+ ```bash
9797+ cd YOUR_BASE_DIR/some-repo
9898+ git remote -v
9999+ ```
100100+ Should show both `origin` (GitHub) and `tangled` remotes
101101+102102+### ๐ Record Schema
103103+104104+Each repository creates a record with this structure:
105105+106106+```typescript
107107+{
108108+ $type: "sh.tangled.repo",
109109+ name: "your-repo-name", // required
110110+ knot: "knot1.tangled.sh", // required
111111+ createdAt: "2024-01-01T00:00:00Z", // required
112112+ description: "Repo description", // optional
113113+ source: "https://github.com/...", // optional
114114+ labels: [], // optional
115115+}
116116+```
117117+118118+This matches the official `sh.tangled.repo` lexicon schema.
119119+120120+### โ ๏ธ Important Notes
121121+122122+1. **Use App Password**: Never use your main Bluesky password. Create an app password in Settings โ App Passwords.
123123+124124+2. **Check Your DID**: Run `npm run test-atproto` first to ensure your DID in `.env` matches your actual account.
125125+126126+3. **SSH Key Required**: Make sure your SSH key is added to Tangled at https://tangled.org/settings/keys
127127+128128+4. **Rate Limits**: GitHub API has rate limits (60 req/hour unauthenticated). If you have many repos, consider adding GitHub auth.
129129+130130+### ๐ Troubleshooting
131131+132132+**"Missing Bluesky credentials"**
133133+- Check `src/.env` exists and has `BLUESKY_USERNAME` and `BLUESKY_PASSWORD`
134134+135135+**"Login failed"**
136136+- Verify you're using an app password, not your main password
137137+- Check username includes full handle (e.g., `you.bsky.social`)
138138+139139+**"Could not push to Tangled"**
140140+- Verify SSH key is configured: `ssh git@tangled.sh`
141141+- Check repo exists on Tangled
142142+143143+**"Failed to create ATProto record"**
144144+- Run `npm run test-atproto` to check connection
145145+- Verify your app password has write permissions
146146+147147+See `SETUP.md` for more detailed troubleshooting.
148148+149149+### ๐ Available Commands
150150+151151+```bash
152152+npm run check # Comprehensive health check (recommended first step!)
153153+npm run validate # Check environment configuration only
154154+npm run test-atproto # Test AT Proto connection only
155155+npm run sync # Run sync (only new repos without AT Proto records)
156156+npm run sync:force # Force sync all repos (including existing)
157157+```
158158+159159+#### `npm run check` - Comprehensive Health Check
160160+161161+This is the **most useful command** for troubleshooting! It runs all checks in one go:
162162+163163+- โ Configuration validation
164164+- โ AT Proto connection test
165165+- โ SSH connection to Tangled
166166+- โ GitHub API access
167167+- โ Dependencies verification
168168+169169+**When to use:**
170170+- Before your first sync
171171+- When troubleshooting issues
172172+- After changing configuration
173173+- To verify everything is working
174174+175175+#### Individual Check Commands
176176+177177+**Normal sync** (recommended): Only processes repos that don't have AT Proto records yet. This is efficient and safe for regular use.
178178+179179+**Force sync**: Processes all repos regardless of whether they already have records. Use this if you need to:
180180+- Re-push repos to Tangled
181181+- Update READMEs for all repos
182182+- Recover from a partial sync
183183+184184+### โจ Next Steps
185185+186186+1. Copy and configure `src/.env`
187187+2. Run `npm run validate`
188188+3. Run `npm run test-atproto`
189189+4. Run `npm run sync`
190190+191191+That's it! Your GitHub repos will be synced to Tangled with proper AT Proto records.
192192+193193+---
194194+195195+**Questions?** Check `SETUP.md` for detailed instructions and troubleshooting.
196196+197197+**Happy syncing! ๐**
···11+# Base directory where GitHub repos will be cloned
22+BASE_DIR=/path/to/your/repos
33+44+# Your GitHub username
55+GITHUB_USER=your-github-username
66+77+# Your ATProto DID (e.g., did:plc:abc123...)
88+ATPROTO_DID=did:plc:your-did-here
99+1010+# Bluesky PDS URL (usually https://bsky.social)
1111+BLUESKY_PDS=https://bsky.social
1212+1313+# Your Bluesky credentials
1414+BLUESKY_USERNAME=your-handle.bsky.social
1515+BLUESKY_PASSWORD=your-app-password