docs: document webhooks #1074

+272 -111

1 changed file

Diff round #0

expand all

unified split

docs

DOCS.md

+272 -111

docs/DOCS.md

··· 3 3 author: The Tangled Contributors 4 4 date: 21 Sun, Dec 2025 5 5 abstract: | 6 - Tangled is a decentralized code hosting and collaboration 7 - platform. Every component of Tangled is open-source and 8 - self-hostable. [tangled.org](https://tangled.org) also 9 - provides hosting and CI services that are free to use. 10 - 11 - There are several models for decentralized code 12 - collaboration platforms, ranging from ActivityPub’s 13 - (Forgejo) federated model, to Radicle’s entirely P2P model. 14 - Our approach attempts to be the best of both worlds by 15 - adopting the AT Protocol—a protocol for building decentralized 16 - social applications with a central identity 17 - 18 - Our approach to this is the idea of “knots”. Knots are 19 - lightweight, headless servers that enable users to host Git 20 - repositories with ease. Knots are designed for either single 21 - or multi-tenant use which is perfect for self-hosting on a 22 - Raspberry Pi at home, or larger “community” servers. By 23 - default, Tangled provides managed knots where you can host 24 - your repositories for free. 25 - 26 - The appview at tangled.org acts as a consolidated "view" 27 - into the whole network, allowing users to access, clone and 28 - contribute to repositories hosted across different knots 29 - seamlessly. 6 + Tangled is a decentralized code hosting and collaboration 7 + platform. Every component of Tangled is open-source and 8 + self-hostable. [tangled.org](https://tangled.org) also 9 + provides hosting and CI services that are free to use. 10 + 11 + There are several models for decentralized code 12 + collaboration platforms, ranging from ActivityPub’s 13 + (Forgejo) federated model, to Radicle’s entirely P2P model. 14 + Our approach attempts to be the best of both worlds by 15 + adopting the AT Protocol—a protocol for building decentralized 16 + social applications with a central identity 17 + 18 + Our approach to this is the idea of “knots”. Knots are 19 + lightweight, headless servers that enable users to host Git 20 + repositories with ease. Knots are designed for either single 21 + or multi-tenant use which is perfect for self-hosting on a 22 + Raspberry Pi at home, or larger “community” servers. By 23 + default, Tangled provides managed knots where you can host 24 + your repositories for free. 25 + 26 + The appview at tangled.org acts as a consolidated "view" 27 + into the whole network, allowing users to access, clone and 28 + contribute to repositories hosted across different knots 29 + seamlessly. 30 30 --- 31 31 32 32 # Quick start guide ··· 131 131 cd my-project 132 132 133 133 git init 134 - echo "# My Project" > README.md 134 + echo "# My Project" > README.md 135 135 ``` 136 136 137 137 Add some content and push! ··· 278 278 git push tangled main 279 279 ``` 280 280 281 + # Webhooks 282 + 283 + Webhooks allow you to receive HTTP POST notifications when events occur in your repositories. This enables you to integrate Tangled with external services, trigger CI/CD pipelines, send notifications, or automate workflows. 284 + 285 + ## Overview 286 + 287 + Webhooks send HTTP POST requests to URLs you configure whenever specific events happen. Currently, Tangled supports push events, with more event types coming soon. 288 + 289 + ## Configuring webhooks 290 + 291 + To set up a webhook for your repository: 292 + 293 + 1. Navigate to your repository settings 294 + 2. Click the "hooks" tab 295 + 3. Click "add webhook" 296 + 4. Configure your webhook: 297 + - **Payload URL**: The endpoint that will receive the webhook POST requests 298 + - **Secret**: An optional secret key for verifying webhook authenticity (auto-generated if left blank) 299 + - **Events**: Select which events trigger the webhook (currently only push events) 300 + - **Active**: Toggle whether the webhook is enabled 301 + 302 + ## Webhook payload 303 + 304 + ### Push 305 + 306 + When a push event occurs, Tangled sends a POST request with a JSON payload of the format: 307 + 308 + ```json 309 + { 310 + "after": "7b320e5cbee2734071e4310c1d9ae401d8f6cab5", 311 + "before": "c04ddf64eddc90e4e2a9846ba3b43e67a0e2865e", 312 + "pusher": { 313 + "did": "did:plc:hwevmowznbiukdf6uk5dwrrq" 314 + }, 315 + "ref": "refs/heads/main", 316 + "repository": { 317 + "clone_url": "https://tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 318 + "created_at": "2025-09-15T08:57:23Z", 319 + "description": "an example repository", 320 + "fork": false, 321 + "full_name": "did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 322 + "html_url": "https://tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 323 + "name": "some-repo", 324 + "open_issues_count": 5, 325 + "owner": { 326 + "did": "did:plc:hwevmowznbiukdf6uk5dwrrq" 327 + }, 328 + "ssh_url": "ssh://git@tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 329 + "stars_count": 1, 330 + "updated_at": "2025-09-15T08:57:23Z" 331 + } 332 + } 333 + ``` 334 + 335 + ## HTTP headers 336 + 337 + Each webhook request includes the following headers: 338 + 339 + - `Content-Type: application/json` 340 + - `User-Agent: Tangled-Hook/<short-sha>` — User agent with short SHA of the commit 341 + - `X-Tangled-Event: push` — The event type 342 + - `X-Tangled-Hook-ID: <webhook-id>` — The webhook ID 343 + - `X-Tangled-Delivery: <uuid>` — Unique delivery ID 344 + - `X-Tangled-Signature-256: sha256=<hmac>` — HMAC-SHA256 signature (if secret configured) 345 + 346 + ## Verifying webhook signatures 347 + 348 + If you configured a secret, you should verify the webhook signature to ensure requests are authentic. For example, in Go: 349 + 350 + ```go 351 + package main 352 + 353 + import ( 354 + "crypto/hmac" 355 + "crypto/sha256" 356 + "encoding/hex" 357 + "io" 358 + "net/http" 359 + "strings" 360 + ) 361 + 362 + func verifySignature(payload []byte, signatureHeader, secret string) bool { 363 + // Remove 'sha256=' prefix from signature header 364 + signature := strings.TrimPrefix(signatureHeader, "sha256=") 365 + 366 + // Compute expected signature 367 + mac := hmac.New(sha256.New, []byte(secret)) 368 + mac.Write(payload) 369 + expected := hex.EncodeToString(mac.Sum(nil)) 370 + 371 + // Use constant-time comparison to prevent timing attacks 372 + return hmac.Equal([]byte(signature), []byte(expected)) 373 + } 374 + 375 + func webhookHandler(w http.ResponseWriter, r *http.Request) { 376 + // Read the request body 377 + payload, err := io.ReadAll(r.Body) 378 + if err != nil { 379 + http.Error(w, "Bad request", http.StatusBadRequest) 380 + return 381 + } 382 + 383 + // Get signature from header 384 + signatureHeader := r.Header.Get("X-Tangled-Signature-256") 385 + 386 + // Verify signature 387 + if signatureHeader != "" && verifySignature(payload, signatureHeader, yourSecret) { 388 + // Webhook is authentic, process it 389 + processWebhook(payload) 390 + w.WriteHeader(http.StatusOK) 391 + } else { 392 + http.Error(w, "Invalid signature", http.StatusUnauthorized) 393 + } 394 + } 395 + ``` 396 + 397 + ## Delivery retries 398 + 399 + Webhooks are automatically retried on failure: 400 + 401 + - **3 total attempts** (1 initial + 2 retries) 402 + - **Exponential backoff** starting at 1 second, max 10 seconds 403 + - **Retried on**: 404 + - Network errors 405 + - HTTP 5xx server errors 406 + - **Not retried on**: 407 + - HTTP 4xx client errors (bad request, unauthorized, etc.) 408 + 409 + ### Timeouts 410 + 411 + Webhook requests timeout after 30 seconds. If your endpoint needs more time: 412 + 413 + 1. Respond with 200 OK immediately 414 + 2. Process the webhook asynchronously in the background 415 + 416 + ## Example integrations 417 + 418 + ### Discord notifications 419 + 420 + ```javascript 421 + app.post("/webhook", (req, res) => { 422 + const payload = req.body; 423 + 424 + fetch("https://discord.com/api/webhooks/...", { 425 + method: "POST", 426 + headers: { "Content-Type": "application/json" }, 427 + body: JSON.stringify({ 428 + content: `New push to ${payload.repository.full_name}`, 429 + embeds: [ 430 + { 431 + title: `${payload.pusher.login} pushed to ${payload.ref}`, 432 + url: payload.repository.html_url, 433 + color: 0x00ff00, 434 + }, 435 + ], 436 + }), 437 + }); 438 + 439 + res.status(200).send("OK"); 440 + }); 441 + ``` 442 + 281 443 # Knot self-hosting guide 282 444 283 445 So you want to run your own knot server? Great! Here are a few prerequisites: ··· 313 475 and operation tool. For the purpose of this guide, we're 314 476 only concerned with these subcommands: 315 477 316 - * `knot server`: the main knot server process, typically 317 - run as a supervised service 318 - * `knot guard`: handles role-based access control for git 319 - over SSH (you'll never have to run this yourself) 320 - * `knot keys`: fetches SSH keys associated with your knot; 321 - we'll use this to generate the SSH 322 - `AuthorizedKeysCommand` 478 + - `knot server`: the main knot server process, typically 479 + run as a supervised service 480 + - `knot guard`: handles role-based access control for git 481 + over SSH (you'll never have to run this yourself) 482 + - `knot keys`: fetches SSH keys associated with your knot; 483 + we'll use this to generate the SSH 484 + `AuthorizedKeysCommand` 323 485 324 486 ``` 325 487 cd core ··· 432 594 can move these paths if you'd like to store them in another folder. Be careful 433 595 when adjusting these paths: 434 596 435 - * Stop your knot when moving data (e.g. `systemctl stop knotserver`) to prevent 436 - any possible side effects. Remember to restart it once you're done. 437 - * Make backups before moving in case something goes wrong. 438 - * Make sure the `git` user can read and write from the new paths. 597 + - Stop your knot when moving data (e.g. `systemctl stop knotserver`) to prevent 598 + any possible side effects. Remember to restart it once you're done. 599 + - Make backups before moving in case something goes wrong. 600 + - Make sure the `git` user can read and write from the new paths. 439 601 440 602 #### Database 441 603 ··· 519 681 2. Check to see that your knot has synced the key by running 520 682 `knot keys` 521 683 3. Check to see if git is supplying the correct private key 522 - when pushing: `GIT_SSH_COMMAND="ssh -v" git push ...` 684 + when pushing: `GIT_SSH_COMMAND="ssh -v" git push ...` 523 685 4. Check to see if `sshd` on the knot is rejecting the push 524 686 for some reason: `journalctl -xeu ssh` (or `sshd`, 525 687 depending on your machine). These logs are unavailable if ··· 527 689 5. Check to see if the knot itself is rejecting the push, 528 690 depending on your setup, the logs might be in one of the 529 691 following paths: 530 - * `/tmp/knotguard.log` 531 - * `/home/git/log` 532 - * `/home/git/guard.log` 692 + - `/tmp/knotguard.log` 693 + - `/home/git/log` 694 + - `/home/git/guard.log` 533 695 534 696 # Spindles 535 697 ··· 847 1009 848 1010 ### Prerequisites 849 1011 850 - * Go 851 - * Docker (the only supported backend currently) 1012 + - Go 1013 + - Docker (the only supported backend currently) 852 1014 853 1015 ### Configuration 854 1016 855 1017 Spindle is configured using environment variables. The following environment variables are available: 856 1018 857 - * `SPINDLE_SERVER_LISTEN_ADDR`: The address the server listens on (default: `"0.0.0.0:6555"`). 858 - * `SPINDLE_SERVER_DB_PATH`: The path to the SQLite database file (default: `"spindle.db"`). 859 - * `SPINDLE_SERVER_HOSTNAME`: The hostname of the server (required). 860 - * `SPINDLE_SERVER_JETSTREAM_ENDPOINT`: The endpoint of the Jetstream server (default: `"wss://jetstream1.us-west.bsky.network/subscribe"`). 861 - * `SPINDLE_SERVER_DEV`: A boolean indicating whether the server is running in development mode (default: `false`). 862 - * `SPINDLE_SERVER_OWNER`: The DID of the owner (required). 863 - * `SPINDLE_PIPELINES_NIXERY`: The Nixery URL (default: `"nixery.tangled.sh"`). 864 - * `SPINDLE_PIPELINES_WORKFLOW_TIMEOUT`: The default workflow timeout (default: `"5m"`). 865 - * `SPINDLE_PIPELINES_LOG_DIR`: The directory to store workflow logs (default: `"/var/log/spindle"`). 1019 + - `SPINDLE_SERVER_LISTEN_ADDR`: The address the server listens on (default: `"0.0.0.0:6555"`). 1020 + - `SPINDLE_SERVER_DB_PATH`: The path to the SQLite database file (default: `"spindle.db"`). 1021 + - `SPINDLE_SERVER_HOSTNAME`: The hostname of the server (required). 1022 + - `SPINDLE_SERVER_JETSTREAM_ENDPOINT`: The endpoint of the Jetstream server (default: `"wss://jetstream1.us-west.bsky.network/subscribe"`). 1023 + - `SPINDLE_SERVER_DEV`: A boolean indicating whether the server is running in development mode (default: `false`). 1024 + - `SPINDLE_SERVER_OWNER`: The DID of the owner (required). 1025 + - `SPINDLE_PIPELINES_NIXERY`: The Nixery URL (default: `"nixery.tangled.sh"`). 1026 + - `SPINDLE_PIPELINES_WORKFLOW_TIMEOUT`: The default workflow timeout (default: `"5m"`). 1027 + - `SPINDLE_PIPELINES_LOG_DIR`: The directory to store workflow logs (default: `"/var/log/spindle"`). 866 1028 867 1029 ### Running spindle 868 1030 869 - 1. **Set the environment variables.** For example: 1031 + 1. **Set the environment variables.** For example: 870 1032 871 1033 ```shell 872 1034 export SPINDLE_SERVER_HOSTNAME="your-hostname" ··· 900 1062 901 1063 Spindle is a small CI runner service. Here's a high-level overview of how it operates: 902 1064 903 - * Listens for [`sh.tangled.spindle.member`](/lexicons/spindle/member.json) and 904 - [`sh.tangled.repo`](/lexicons/repo.json) records on the Jetstream. 905 - * When a new repo record comes through (typically when you add a spindle to a 906 - repo from the settings), spindle then resolves the underlying knot and 907 - subscribes to repo events (see: 908 - [`sh.tangled.pipeline`](/lexicons/pipeline.json)). 909 - * The spindle engine then handles execution of the pipeline, with results and 910 - logs beamed on the spindle event stream over WebSocket 1065 + - Listens for [`sh.tangled.spindle.member`](/lexicons/spindle/member.json) and 1066 + [`sh.tangled.repo`](/lexicons/repo.json) records on the Jetstream. 1067 + - When a new repo record comes through (typically when you add a spindle to a 1068 + repo from the settings), spindle then resolves the underlying knot and 1069 + subscribes to repo events (see: 1070 + [`sh.tangled.pipeline`](/lexicons/pipeline.json)). 1071 + - The spindle engine then handles execution of the pipeline, with results and 1072 + logs beamed on the spindle event stream over WebSocket 911 1073 912 1074 ### The engine 913 1075 ··· 1356 1518 <details> 1357 1519 <summary><strong>macOS users will have to set up a Nix Builder first</strong></summary> 1358 1520 1359 - In order to build Tangled's dev VM on macOS, you will 1360 - first need to set up a Linux Nix builder. The recommended 1361 - way to do so is to run a [`darwin.linux-builder` 1362 - VM](https://nixos.org/manual/nixpkgs/unstable/#sec-darwin-builder) 1363 - and to register it in `nix.conf` as a builder for Linux 1364 - with the same architecture as your Mac (`linux-aarch64` if 1365 - you are using Apple Silicon). 1366 - 1367 - > IMPORTANT: You must build `darwin.linux-builder` somewhere other than inside 1368 - > the Tangled repo so that it doesn't conflict with the other VM. For example, 1369 - > you can do 1370 - > 1371 - > ```shell 1372 - > cd $(mktemp -d buildervm.XXXXX) && nix run nixpkgs#darwin.linux-builder 1373 - > ``` 1374 - > 1375 - > to store the builder VM in a temporary dir. 1376 - > 1377 - > You should read and follow [all the other intructions][darwin builder vm] to 1378 - > avoid subtle problems. 1379 - 1380 - Alternatively, you can use any other method to set up a 1381 - Linux machine with Nix installed that you can `sudo ssh` 1382 - into (in other words, root user on your Mac has to be able 1383 - to ssh into the Linux machine without entering a password) 1384 - and that has the same architecture as your Mac. See 1385 - [remote builder 1386 - instructions](https://nix.dev/manual/nix/2.28/advanced-topics/distributed-builds.html#requirements) 1387 - for how to register such a builder in `nix.conf`. 1388 - 1389 - > WARNING: If you'd like to use 1390 - > [`nixos-lima`](https://github.com/nixos-lima/nixos-lima) or 1391 - > [Orbstack](https://orbstack.dev/), note that setting them up so that `sudo 1392 - > ssh` works can be tricky. It seems to be [possible with 1393 - > Orbstack](https://github.com/orgs/orbstack/discussions/1669). 1521 + In order to build Tangled's dev VM on macOS, you will 1522 + first need to set up a Linux Nix builder. The recommended 1523 + way to do so is to run a [`darwin.linux-builder` 1524 + VM](https://nixos.org/manual/nixpkgs/unstable/#sec-darwin-builder) 1525 + and to register it in `nix.conf` as a builder for Linux 1526 + with the same architecture as your Mac (`linux-aarch64` if 1527 + you are using Apple Silicon). 1528 + 1529 + > IMPORTANT: You must build `darwin.linux-builder` somewhere other than inside 1530 + > the Tangled repo so that it doesn't conflict with the other VM. For example, 1531 + > you can do 1532 + > 1533 + > ```shell 1534 + > cd $(mktemp -d buildervm.XXXXX) && nix run nixpkgs#darwin.linux-builder 1535 + > ``` 1536 + > 1537 + > to store the builder VM in a temporary dir. 1538 + > 1539 + > You should read and follow [all the other intructions][darwin builder vm] to 1540 + > avoid subtle problems. 1541 + 1542 + Alternatively, you can use any other method to set up a 1543 + Linux machine with Nix installed that you can `sudo ssh` 1544 + into (in other words, root user on your Mac has to be able 1545 + to ssh into the Linux machine without entering a password) 1546 + and that has the same architecture as your Mac. See 1547 + [remote builder 1548 + instructions](https://nix.dev/manual/nix/2.28/advanced-topics/distributed-builds.html#requirements) 1549 + for how to register such a builder in `nix.conf`. 1550 + 1551 + > WARNING: If you'd like to use 1552 + > [`nixos-lima`](https://github.com/nixos-lima/nixos-lima) or 1553 + > [Orbstack](https://orbstack.dev/), note that setting them up so that `sudo 1554 + ssh` works can be tricky. It seems to be [possible with 1555 + > Orbstack](https://github.com/orgs/orbstack/discussions/1669). 1394 1556 1395 1557 </details> 1396 1558 ··· 1463 1625 1464 1626 We follow a commit style similar to the Go project. Please keep commits: 1465 1627 1466 - * **atomic**: each commit should represent one logical change 1467 - * **descriptive**: the commit message should clearly describe what the 1468 - change does and why it's needed 1628 + - **atomic**: each commit should represent one logical change 1629 + - **descriptive**: the commit message should clearly describe what the 1630 + change does and why it's needed 1469 1631 1470 1632 ### Message format 1471 1633 ··· 1491 1653 knotserver/git/service: improve error checking in upload-pack 1492 1654 ``` 1493 1655 1494 - 1495 1656 ### General notes 1496 1657 1497 1658 - PRs get merged "as-is" (fast-forward)—like applying a patch-series 1498 - using `git am`. At present, there is no squashing—so please author 1499 - your commits as they would appear on `master`, following the above 1500 - guidelines. 1659 + using `git am`. At present, there is no squashing—so please author 1660 + your commits as they would appear on `master`, following the above 1661 + guidelines. 1501 1662 - If there is a lot of nesting, for example "appview: 1502 - pages/templates/repo/fragments: ...", these can be truncated down to 1503 - just "appview: repo/fragments: ...". If the change affects a lot of 1504 - subdirectories, you may abbreviate to just the top-level names, e.g. 1505 - "appview: ..." or "knotserver: ...". 1663 + pages/templates/repo/fragments: ...", these can be truncated down to 1664 + just "appview: repo/fragments: ...". If the change affects a lot of 1665 + subdirectories, you may abbreviate to just the top-level names, e.g. 1666 + "appview: ..." or "knotserver: ...". 1506 1667 - Keep commits lowercased with no trailing period. 1507 1668 - Use the imperative mood in the summary line (e.g., "fix bug" not 1508 - "fixed bug" or "fixes bug"). 1669 + "fixed bug" or "fixes bug"). 1509 1670 - Try to keep the summary line under 72 characters, but we aren't too 1510 - fussed about this. 1671 + fussed about this. 1511 1672 - Follow the same formatting for PR titles if filled manually. 1512 1673 - Don't include unrelated changes in the same commit. 1513 1674 - Avoid noisy commit messages like "wip" or "final fix"—rewrite history 1514 - before submitting if necessary. 1675 + before submitting if necessary. 1515 1676 1516 1677 ## Code formatting 1517 1678 ··· 1601 1762 1602 1763 - You may need to ensure that your PDS is timesynced using 1603 1764 NTP: 1604 - * Enable the `ntpd` service 1605 - * Run `ntpd -qg` to synchronize your clock 1765 + - Enable the `ntpd` service 1766 + - Run `ntpd -qg` to synchronize your clock 1606 1767 - You may need to increase the default request timeout: 1607 1768 `NODE_OPTIONS="--network-family-autoselection-attempt-timeout=500"` 1608 1769