···2121 AppPassword string `env:"APP_PASSWORD"`
22222323 // uhhhh this is because knot1 is under icy's did
2424- TmpAltAppPassword string `env:"ALT_APP_PASSWORD, required"`
2424+ TmpAltAppPassword string `env:"ALT_APP_PASSWORD"`
2525}
26262727type OAuthConfig struct {
+4-4
appview/db/follow.go
···5656}
57575858type FollowStats struct {
5959- Followers int
6060- Following int
5959+ Followers int64
6060+ Following int64
6161}
62626363func GetFollowerFollowingCount(e Execer, did string) (FollowStats, error) {
6464- followers, following := 0, 0
6464+ var followers, following int64
6565 err := e.QueryRow(
6666 `SELECT
6767 COUNT(CASE WHEN subject_did = ? THEN 1 END) AS followers,
···122122123123 for rows.Next() {
124124 var did string
125125- var followers, following int
125125+ var followers, following int64
126126 if err := rows.Scan(&did, &followers, &following); err != nil {
127127 return nil, err
128128 }
···1111### message format
12121313```
1414-<service/top-level directory>: <affected package/directory>: <short summary of change>
1414+<service/top-level directory>/<affected package/directory>: <short summary of change>
151516161717Optional longer description can go here, if necessary. Explain what the
···2323Here are some examples:
24242525```
2626-appview: state: fix token expiry check in middleware
2626+appview/state: fix token expiry check in middleware
27272828The previous check did not account for clock drift, leading to premature
2929token invalidation.
3030```
31313232```
3333-knotserver: git/service: improve error checking in upload-pack
3333+knotserver/git/service: improve error checking in upload-pack
3434```
35353636
+53-12
docs/hacking.md
···4848redis-server
4949```
50505151-## running a knot
5151+## running knots and spindles
52525353An end-to-end knot setup requires setting up a machine with
5454`sshd`, `AuthorizedKeysCommand`, and git user, which is
5555quite cumbersome. So the nix flake provides a
5656`nixosConfiguration` to do so.
57575858-To begin, grab your DID from http://localhost:3000/settings.
5959-Then, set `TANGLED_VM_KNOT_OWNER` and
6060-`TANGLED_VM_SPINDLE_OWNER` to your DID.
5858+<details>
5959+ <summary><strong>MacOS users will have to setup a Nix Builder first</strong></summary>
6060+6161+ In order to build Tangled's dev VM on macOS, you will
6262+ first need to set up a Linux Nix builder. The recommended
6363+ way to do so is to run a [`darwin.linux-builder`
6464+ VM](https://nixos.org/manual/nixpkgs/unstable/#sec-darwin-builder)
6565+ and to register it in `nix.conf` as a builder for Linux
6666+ with the same architecture as your Mac (`linux-aarch64` if
6767+ you are using Apple Silicon).
6868+6969+ > IMPORTANT: You must build `darwin.linux-builder` somewhere other than inside
7070+ > the tangled repo so that it doesn't conflict with the other VM. For example,
7171+ > you can do
7272+ >
7373+ > ```shell
7474+ > cd $(mktemp -d buildervm.XXXXX) && nix run nixpkgs#darwin.linux-builder
7575+ > ```
7676+ >
7777+ > to store the builder VM in a temporary dir.
7878+ >
7979+ > You should read and follow [all the other intructions][darwin builder vm] to
8080+ > avoid subtle problems.
8181+8282+ Alternatively, you can use any other method to set up a
8383+ Linux machine with `nix` installed that you can `sudo ssh`
8484+ into (in other words, root user on your Mac has to be able
8585+ to ssh into the Linux machine without entering a password)
8686+ and that has the same architecture as your Mac. See
8787+ [remote builder
8888+ instructions](https://nix.dev/manual/nix/2.28/advanced-topics/distributed-builds.html#requirements)
8989+ for how to register such a builder in `nix.conf`.
61906262-If you don't want to [set up a spindle](#running-a-spindle),
6363-you can use any placeholder value.
9191+ > WARNING: If you'd like to use
9292+ > [`nixos-lima`](https://github.com/nixos-lima/nixos-lima) or
9393+ > [Orbstack](https://orbstack.dev/), note that setting them up so that `sudo
9494+ > ssh` works can be tricky. It seems to be [possible with
9595+ > Orbstack](https://github.com/orgs/orbstack/discussions/1669).
64966565-You can now start a lightweight NixOS VM like so:
9797+</details>
9898+9999+To begin, grab your DID from http://localhost:3000/settings.
100100+Then, set `TANGLED_VM_KNOT_OWNER` and
101101+`TANGLED_VM_SPINDLE_OWNER` to your DID. You can now start a
102102+lightweight NixOS VM like so:
6610367104```bash
68105nix run --impure .#vm
···74111with `ssh` exposed on port 2222.
7511276113Once the services are running, head to
7777-http://localhost:3000/knots and hit verify (and similarly,
7878-http://localhost:3000/spindles to verify your spindle). It
7979-should verify the ownership of the services instantly if
8080-everything went smoothly.
114114+http://localhost:3000/knots and hit verify. It should
115115+verify the ownership of the services instantly if everything
116116+went smoothly.
8111782118You can push repositories to this VM with this ssh config
83119block on your main machine:
···97133git push local-dev main
98134```
99135100100-## running a spindle
136136+### running a spindle
101137102138The above VM should already be running a spindle on
103139`localhost:6555`. Head to http://localhost:3000/spindles and
···119155# litecli has a nicer REPL interface:
120156litecli /var/lib/spindle/spindle.db
121157```
158158+159159+If for any reason you wish to disable either one of the
160160+services in the VM, modify [nix/vm.nix](/nix/vm.nix) and set
161161+`services.tangled-spindle.enable` (or
162162+`services.tangled-knot.enable`) to `false`.
+130-54
docs/spindle/pipeline.md
···11-# spindle pipeline manifest
11+# spindle pipelines
22+33+Spindle workflows allow you to write CI/CD pipelines in a simple format. They're located in the `.tangled/workflows` directory at the root of your repository, and are defined using YAML.
44+55+The fields are:
2633-Spindle pipelines are defined under the `.tangled/workflows` directory in a
44-repo. Generally:
77+- [Trigger](#trigger): A **required** field that defines when a workflow should be triggered.
88+- [Engine](#engine): A **required** field that defines which engine a workflow should run on.
99+- [Clone options](#clone-options): An **optional** field that defines how the repository should be cloned.
1010+- [Dependencies](#dependencies): An **optional** field that allows you to list dependencies you may need.
1111+- [Environment](#environment): An **optional** field that allows you to define environment variables.
1212+- [Steps](#steps): An **optional** field that allows you to define what steps should run in the workflow.
51366-* Pipelines are defined in YAML.
77-* Workflows can run using different *engines*.
1414+## Trigger
81599-The most barebones workflow looks like this:
1616+The first thing to add to a workflow is the trigger, which defines when a workflow runs. This is defined using a `when` field, which takes in a list of conditions. Each condition has the following fields:
1717+1818+- `event`: This is a **required** field that defines when your workflow should run. It's a list that can take one or more of the following values:
1919+ - `push`: The workflow should run every time a commit is pushed to the repository.
2020+ - `pull_request`: The workflow should run every time a pull request is made or updated.
2121+ - `manual`: The workflow can be triggered manually.
2222+- `branch`: This is a **required** field that defines which branches the workflow should run for. If used with the `push` event, commits to the branch(es) listed here will trigger the workflow. If used with the `pull_request` event, updates to pull requests targeting the branch(es) listed here will trigger the workflow. This field has no effect with the `manual` event.
2323+2424+For example, if you'd like define a workflow that runs when commits are pushed to the `main` and `develop` branches, or when pull requests that target the `main` branch are updated, or manually, you can do so with:
10251126```yaml
1227when:
1313- - event: ["push"]
2828+ - event: ["push", "manual"]
2929+ branch: ["main", "develop"]
3030+ - event: ["pull_request"]
1431 branch: ["main"]
3232+```
15333434+## Engine
3535+3636+Next is the engine on which the workflow should run, defined using the **required** `engine` field. The currently supported engines are:
3737+3838+- `nixery`: This uses an instance of [Nixery](https://nixery.dev) to run steps, which allows you to add [dependencies](#dependencies) from [Nixpkgs](https://github.com/NixOS/nixpkgs). You can search for packages on https://search.nixos.org, and there's a pretty good chance the package(s) you're looking for will be there.
3939+4040+Example:
4141+4242+```yaml
1643engine: "nixery"
4444+```
4545+4646+## Clone options
17471818-# optional
4848+When a workflow starts, the first step is to clone the repository. You can customize this behavior using the **optional** `clone` field. It has the following fields:
4949+5050+- `skip`: Setting this to `true` will skip cloning the repository. This can be useful if your workflow is doing something that doesn't require anything from the repository itself. This is `false` by default.
5151+- `depth`: This sets the number of commits, or the "clone depth", to fetch from the repository. For example, if you set this to 2, the last 2 commits will be fetched. By default, the depth is set to 1, meaning only the most recent commit will be fetched, which is the commit that triggered the workflow.
5252+- `submodules`: If you use [git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules) in your repository, setting this field to `true` will recursively fetch all submodules. This is `false` by default.
5353+5454+The default settings are:
5555+5656+```yaml
1957clone:
2058 skip: false
2121- depth: 50
2222- submodules: true
5959+ depth: 1
6060+ submodules: false
2361```
24622525-The `when` and `engine` fields are required, while every other aspect
2626-of how the definition is parsed is up to the engine. Currently, a spindle
2727-provides at least one of these built-in engines:
6363+## Dependencies
28642929-## `nixery`
6565+Usually when you're running a workflow, you'll need additional dependencies. The `dependencies` field lets you define which dependencies to get, and from where. It's a key-value map, with the key being the registry to fetch dependencies from, and the value being the list of dependencies to fetch.
30663131-The Nixery engine uses an instance of [Nixery](https://nixery.dev) to run
3232-steps that use dependencies from [Nixpkgs](https://github.com/NixOS/nixpkgs).
3333-3434-Here's an example that uses all fields:
6767+Say you want to fetch Node.js and Go from `nixpkgs`, and a package called `my_pkg` you've made from your own registry at your repository at `https://tangled.sh/@example.com/my_pkg`. You can define those dependencies like so:
35683669```yaml
3737-# build_and_test.yaml
3838-when:
3939- - event: ["push", "pull_request"]
4040- branch: ["main", "develop"]
4141- - event: ["manual"]
4242-4370dependencies:
4444- ## from nixpkgs
7171+ # nixpkgs
4572 nixpkgs:
4673 - nodejs
4747- ## custom registry
4848- git+https://tangled.sh/@oppi.li/statix:
4949- - statix
7474+ - go
7575+ # custom registry
7676+ git+https://tangled.sh/@example.com/my_pkg:
7777+ - my_pkg
7878+```
50795151-steps:
5252- - name: "Install dependencies"
5353- command: "npm install"
5454- environment:
5555- NODE_ENV: "development"
5656- CI: "true"
8080+Now these dependencies are available to use in your workflow!
57815858- - name: "Run linter"
5959- command: "npm run lint"
8282+## Environment
8383+8484+The `environment` field allows you define environment variables that will be available throughout the entire workflow. **Do not put secrets here, these environment variables are visible to anyone viewing the repository. You can add secrets for pipelines in your repository's settings.**
8585+8686+Example:
8787+8888+```yaml
8989+environment:
9090+ GOOS: "linux"
9191+ GOARCH: "arm64"
9292+ NODE_ENV: "production"
9393+ MY_ENV_VAR: "MY_ENV_VALUE"
9494+```
60956161- - name: "Run tests"
6262- command: "npm test"
6363- environment:
6464- NODE_ENV: "test"
6565- JEST_WORKERS: "2"
9696+## Steps
66976767- - name: "Build application"
9898+The `steps` field allows you to define what steps should run in the workflow. It's a list of step objects, each with the following fields:
9999+100100+- `name`: This field allows you to give your step a name. This name is visible in your workflow runs, and is used to describe what the step is doing.
101101+- `command`: This field allows you to define a command to run in that step. The step is run in a Bash shell, and the logs from the command will be visible in the pipelines page on the Tangled website. The [dependencies](#dependencies) you added will be available to use here.
102102+- `environment`: Similar to the global [environment](#environment) config, this **optional** field is a key-value map that allows you to set environment variables for the step. **Do not put secrets here, these environment variables are visible to anyone viewing the repository. You can add secrets for pipelines in your repository's settings.**
103103+104104+Example:
105105+106106+```yaml
107107+steps:
108108+ - name: "Build backend"
109109+ command: "go build"
110110+ environment:
111111+ GOOS: "darwin"
112112+ GOARCH: "arm64"
113113+ - name: "Build frontend"
68114 command: "npm run build"
69115 environment:
70116 NODE_ENV: "production"
117117+```
711187272-environment:
7373- BUILD_NUMBER: "123"
7474- GIT_BRANCH: "main"
119119+## Complete workflow
751207676-## current repository is cloned and checked out at the target ref
7777-## by default.
121121+```yaml
122122+# .tangled/workflows/build.yml
123123+124124+when:
125125+ - event: ["push", "manual"]
126126+ branch: ["main", "develop"]
127127+ - event: ["pull_request"]
128128+ branch: ["main"]
129129+130130+engine: "nixery"
131131+132132+# using the default values
78133clone:
79134 skip: false
8080- depth: 50
8181- submodules: true
8282-```
135135+ depth: 1
136136+ submodules: false
831378484-## git push options
138138+dependencies:
139139+ # nixpkgs
140140+ nixpkgs:
141141+ - nodejs
142142+ - go
143143+ # custom registry
144144+ git+https://tangled.sh/@example.com/my_pkg:
145145+ - my_pkg
851468686-These are push options that can be used with the `--push-option (-o)` flag of git push:
147147+environment:
148148+ GOOS: "linux"
149149+ GOARCH: "arm64"
150150+ NODE_ENV: "production"
151151+ MY_ENV_VAR: "MY_ENV_VALUE"
152152+153153+steps:
154154+ - name: "Build backend"
155155+ command: "go build"
156156+ environment:
157157+ GOOS: "darwin"
158158+ GOARCH: "arm64"
159159+ - name: "Build frontend"
160160+ command: "npm run build"
161161+ environment:
162162+ NODE_ENV: "production"
163163+```
871648888-- `verbose-ci`, `ci-verbose`: enables diagnostics reporting for the CI pipeline, allowing you to see any issues when you push.
8989-- `skip-ci`, `ci-skip`: skips triggering the CI pipeline.
165165+If you want another example of a workflow, you can look at the one [Tangled uses to build the project](https://tangled.sh/@tangled.sh/core/blob/master/.tangled/workflows/build.yml).
···1717 Owner string `env:"OWNER, required"`
1818 Secrets Secrets `env:",prefix=SECRETS_"`
1919 LogDir string `env:"LOG_DIR, default=/var/log/spindle"`
2020+ QueueSize int `env:"QUEUE_SIZE, default=100"`
2121+ MaxJobCount int `env:"MAX_JOB_COUNT, default=2"` // max number of jobs that run at a time
2022}
21232224func (s Server) Did() syntax.DID {