tangled.org
oppi.li
+65
-47
pages/blog/ci.md
+65
-47
pages/blog/ci.md
···
19
consistently topped our feature request list. Today, CI is
20
no longer a wishlist item, but a fully-featured reality.
21
22
-
Meet **spindle**: Tangled's new CI runner that brings powerful automation
23
-
directly to your repositories. In typical Tangled fashion we've been dogfooding
24
-
spindle for a while now; this very blog post you're reading was [built and
25
-
published using spindle](link to CI run here).
26
27
28
29
## how spindle works
30
31
-
Spindle is designed around simplicity and the decentralized nature of the AT
32
-
Protocol. Here's the flow: when you push code or open a pull request, the knot
33
-
hosting your repository emits a pipeline event (`sh.tangled.pipeline`). Running
34
-
as a dedicated service, spindle subscribes to these events via websocket
35
-
connections to your knot.
36
37
-
Once triggered, spindle reads your pipeline manifest, spins up the necessary
38
-
execution environment (covered below), and runs your defined workflow steps.
39
-
Throughout execution, it streams real-time logs and status updates
40
-
(`sh.tangled.pipeline.status`) back through websockets, which the Tangled
41
-
appview subscribes to for live updates.
42
43
-
This architecture keeps everything responsive and real-time while maintaining
44
-
the distributed spirit that makes Tangled unique.
45
46
## spindle pipelines
47
48
-
The pipeline manifest is defined in YAML, and should be relatively familiar to
49
-
those that have used other CI solutions. Here's a minimal example:
50
51
```yaml
52
# test.yaml
···
83
84
[^1]: I mean, if it isn't there, it's nowhere.
85
86
-
Finally, define your steps, necessary environment variables and commands.
87
-
Commands can be multi-lined. Let's take a look at how spindle executes workflow
88
-
steps.
89
90
## workflow execution
91
92
-
At present, the spindle "engine" supports just the Docker backend[^2]. Podman is
93
-
known to work with the Docker socket feature enabled. Each step is run in a
94
-
separate container, with the `/tangled/workspace` and `/nix` volumes persisted
95
across steps.
96
97
-
[^2]: Support for additional backends like Firecracker are planned.
98
-
Contributions welcome!
99
100
-
The container image is built using [Nixery](https://nixery.dev). Nixery is a
101
-
nifty little tool that takes a path-separated set of Nix packages and returns an
102
-
OCI image with each package in a separate layer. Try this in your terminal if
103
-
you've got Docker installed:
104
105
```
106
docker run nixery.dev/bash/hello-go hello
···
160
161
## pipeline secrets
162
163
-
Secrets are a bit tricky since atproto has no notion of private data. Secrets
164
-
are instead written directly from the appview to the spindle instance using
165
-
[service
166
-
auth](https://docs.bsky.app/docs/api/com-atproto-server-get-service-auth). In
167
-
essence, the appview makes a signed request using the logged-in user's DID key;
168
-
spindle verifies this signature by fetching the public key from the DID
169
-
document.
170
171
172
···
179
180
## get started now
181
182
-
You can run your own spindle instance pretty easily: the [spindle self-hosting
183
guide](https://tangled.sh/@tangled.sh/core/blob/master/docs/spindle/hosting.md)
184
-
should have you covered. Once done, head to your repository's settings tab and
185
-
set it up! Doesn't work? Feel free to pop into
186
-
[Discord](https://chat.tangled.sh) to get help -- we have a nice little crew
187
-
that's always around to help.
188
189
-
All Tangled users have access to our hosted spindle instance, free of
190
-
charge[^3]. You don't have any more excuses to not migrate to Tangled now --
191
-
[get started](https://tangled.sh/login) with your AT Protocol account today.
192
193
-
[^3]: We can't promise we won't charge for it at some point but there will
194
-
always be a free tier.
···
19
consistently topped our feature request list. Today, CI is
20
no longer a wishlist item, but a fully-featured reality.
21
22
+
Meet **spindle**: Tangled's new CI runner built atop Nix and
23
+
AT Protocol. In typical Tangled fashion we've been
24
+
dogfooding spindle for a while now; this very blog post
25
+
you're reading was [built and published using spindle](link
26
+
to CI run here).
27
28
29
30
## how spindle works
31
32
+
Spindle is designed around simplicity and the decentralized
33
+
nature of the AT Protocol. In ingests "pipeline" records and
34
+
emits job status updates.
35
36
+
When you push code or open a pull request, the knot hosting
37
+
your repository emits a pipeline event
38
+
(`sh.tangled.pipeline`). Running as a dedicated service,
39
+
spindle subscribes to these events via websocket connections
40
+
to your knot.
41
42
+
Once triggered, spindle reads your pipeline manifest, spins
43
+
up the necessary execution environment (covered below), and
44
+
runs your defined workflow steps. Throughout execution, it
45
+
streams real-time logs and status updates
46
+
(`sh.tangled.pipeline.status`) back through websockets,
47
+
which the Tangled appview subscribes to for live updates.
48
+
49
+
Over at the appview, these updates are ingested and stored,
50
+
and logs are streamed live.
51
52
## spindle pipelines
53
54
+
The pipeline manifest is defined in YAML, and should be
55
+
relatively familiar to those that have used other CI
56
+
solutions. Here's a minimal example:
57
58
```yaml
59
# test.yaml
···
90
91
[^1]: I mean, if it isn't there, it's nowhere.
92
93
+
Workflow manifests are intentionally simple. We do not want
94
+
to include a "marketplace" of workflows or complex job
95
+
orchestration. The bulk of the work should be offloaded to a
96
+
build system, and CI should be used simply for finishing
97
+
touches. That being said, this is still the first revision
98
+
for CI, there is a lot more on the roadmap!
99
+
100
+
Let's take a look at how spindle executes workflow steps.
101
102
## workflow execution
103
104
+
At present, the spindle "engine" supports just the Docker
105
+
backend[^2]. Podman is known to work with the Docker socket
106
+
feature enabled. Each step is run in a separate container,
107
+
with the `/tangled/workspace` and `/nix` volumes persisted
108
across steps.
109
110
+
[^2]: Support for additional backends like Firecracker are
111
+
planned. Contributions welcome!
112
113
+
The container image is built using
114
+
[Nixery](https://nixery.dev). Nixery is a nifty little tool
115
+
that takes a path-separated set of Nix packages and returns
116
+
an OCI image with each package in a separate layer. Try this
117
+
in your terminal if you've got Docker installed:
118
119
```
120
docker run nixery.dev/bash/hello-go hello
···
174
175
## pipeline secrets
176
177
+
Secrets are a bit tricky since atproto has no notion of
178
+
private data. Secrets are instead written directly from the
179
+
appview to the spindle instance using [service
180
+
auth](https://docs.bsky.app/docs/api/com-atproto-server-get-service-auth).
181
+
In essence, the appview makes a signed request using the
182
+
logged-in user's DID key; spindle verifies this signature by
183
+
fetching the public key from the DID document.
184
185
186
···
193
194
## get started now
195
196
+
You can run your own spindle instance pretty easily: the
197
+
[spindle self-hosting
198
guide](https://tangled.sh/@tangled.sh/core/blob/master/docs/spindle/hosting.md)
199
+
should have you covered. Once done, head to your
200
+
repository's settings tab and set it up! Doesn't work? Feel
201
+
free to pop into [Discord](https://chat.tangled.sh) to get
202
+
help -- we have a nice little crew that's always around to
203
+
help.
204
205
+
All Tangled users have access to our hosted spindle
206
+
instance, free of charge[^3]. You don't have any more
207
+
excuses to not migrate to Tangled now -- [get
208
+
started](https://tangled.sh/login) with your AT Protocol
209
+
account today.
210
211
+
[^3]: We can't promise we won't charge for it at some point
212
+
but there will always be a free tier.