tangled.org / knot-docker
Community maintained Docker config for the knot server
fork atom

Add Spindle container image build #12

open opened by psychedeli.ca targeting main from psychedeli.ca/knot-docker: spindle

This PR adds tngl/spindle as a new image, using the same Dockerfile as the Knot server but in an additional stage of the build. By building the images from the same Dockerfile, we can re-use the clone from a single tag and cache more artifacts between builds. It also introduces a docker bake workflow and a docker-bake.hcl file to describe images built off the master branch and given the :edge tag (which is an emerging convention in container tag nomenclature).

The build succeeded, but I got a bunch of warnings about inconsistent case (my editor was yelling at me when it encountered lowercase Dockerfile instructions):

 24 warnings found (use docker --debug to expand):
 - ConsistentInstructionCasing: Command 'CMD' should match the case of the command majority (lowercase) (line 66)
 - ConsistentInstructionCasing: Command 'FROM' should match the case of the command majority (lowercase) (line 10)
 - ConsistentInstructionCasing: Command 'FROM' should match the case of the command majority (lowercase) (line 13)
 - ConsistentInstructionCasing: Command 'ARG' should match the case of the command majority (lowercase) (line 57)
 - ConsistentInstructionCasing: Command 'COPY' should match the case of the command majority (lowercase) (line 63)
 - FromAsCasing: 'AS' and 'from' keywords' casing do not match (line 16)
 - ConsistentInstructionCasing: Command 'RUN' should match the case of the command majority (lowercase) (line 11)
 - ConsistentInstructionCasing: Command 'LABEL' should match the case of the command majority (lowercase) (line 49)
 - ConsistentInstructionCasing: Command 'LABEL' should match the case of the command majority (lowercase) (line 52)
 - ConsistentInstructionCasing: Command 'LABEL' should match the case of the command majority (lowercase) (line 53)
 - ConsistentInstructionCasing: Command 'RUN' should match the case of the command majority (lowercase) (line 61)
 - ConsistentInstructionCasing: Command 'WORKDIR' should match the case of the command majority (lowercase) (line 65)
 - ConsistentInstructionCasing: Command 'RUN' should match the case of the command majority (lowercase) (line 14)
 - ConsistentInstructionCasing: Command 'FROM' should match the case of the command majority (lowercase) (line 46)
 - ConsistentInstructionCasing: Command 'EXPOSE' should match the case of the command majority (lowercase) (line 47)
 - ConsistentInstructionCasing: Command 'LABEL' should match the case of the command majority (lowercase) (line 50)
 - ConsistentInstructionCasing: Command 'ARG' should match the case of the command majority (lowercase) (line 56)
 - ConsistentInstructionCasing: Command 'RUN' should match the case of the command majority (lowercase) (line 59)
 - ConsistentInstructionCasing: Command 'VOLUME' should match the case of the command majority (lowercase) (line 67)
 - ConsistentInstructionCasing: Command 'HEALTHCHECK' should match the case of the command majority (lowercase) (line 68)
 - ConsistentInstructionCasing: Command 'ENV' should match the case of the command majority (lowercase) (line 17)
 - ConsistentInstructionCasing: Command 'LABEL' should match the case of the command majority (lowercase) (line 51)
 - ConsistentInstructionCasing: Command 'LABEL' should match the case of the command majority (lowercase) (line 54)
 - ConsistentInstructionCasing: Command 'RUN' should match the case of the command majority (lowercase) (line 60)

...so I was wondering if I could commit this other change that formats the Dockerfile properly? But I figured I'd let y'all see the changes first before they're obfuscated by all that reformatting.

Labels

None yet.

Participants 3
AT URI
at://did:plc:cmla2he5nqvubofxm4pzgtyi/sh.tangled.repo.pull/3m6bb2zk52i22
+174 -32
Diff #2
unified split
+34 -5
Dockerfile
··· 1 1 from golang:1.24-alpine as builder 2 - env KNOT_REPO_SCAN_PATH=/home/git/repositories 3 2 env CGO_ENABLED=1 4 3 5 4 arg TAG='v1.11.0-alpha' ··· 7 6 workdir /app 8 7 run apk add git gcc musl-dev 9 8 run git clone -b ${TAG} https://tangled.org/@tangled.org/core . 10 - run go build -o /usr/bin/knot -ldflags '-s -w -extldflags "-static"' ./cmd/knot 11 9 12 - from alpine:edge 10 + FROM builder AS build-knot 11 + RUN go build -o /usr/bin/knot -ldflags '-s -w -extldflags "-static"' ./cmd/knot 12 + 13 + FROM builder AS build-spindle 14 + RUN go build -o /usr/bin/spindle ./cmd/spindle 15 + 16 + from alpine:edge AS knot 17 + ENV KNOT_REPO_SCAN_PATH=/home/git/repositories 13 18 expose 5555 14 19 expose 22 15 20 ··· 31 36 run useradd -u $UID -g $GID -d /home/git git 32 37 run openssl rand -hex 16 | passwd --stdin git 33 38 run mkdir -p /home/git/repositories && chown -R git:git /home/git 34 - copy --from=builder /usr/bin/knot /usr/bin 39 + copy --from=build-knot /usr/bin/knot /usr/bin 35 40 run mkdir /app && chown -R git:git /app 36 41 37 42 healthcheck --interval=60s --timeout=30s --start-period=5s --retries=3 \ 38 43 cmd curl -f http://localhost:5555 || exit 1 44 + entrypoint ["/init"] 39 45 40 - entrypoint ["/init"] 46 + FROM alpine:edge AS spindle 47 + EXPOSE 6555 48 + 49 + LABEL org.opencontainers.image.title="spindle" 50 + LABEL org.opencontainers.image.description="ci server for tangled" 51 + LABEL org.opencontainers.image.source="https://tangled.org/@tangled.org/knot-docker" 52 + LABEL org.opencontainers.image.url="https://tangled.org" 53 + LABEL org.opencontainers.image.vendor="tangled.org" 54 + LABEL org.opencontainers.image.licenses="MIT" 55 + 56 + ARG UID=1000 57 + ARG GID=1000 58 + 59 + RUN adduser --system --uid $UID spindle 60 + RUN addgroup --system --gid $UID spindle 61 + RUN mkdir -p /app && chown -R spindle:spindle /app 62 + 63 + COPY --from=build-spindle /usr/bin/spindle /usr/bin 64 + 65 + WORKDIR /app 66 + CMD ["spindle"] 67 + VOLUME ["/app"] 68 + HEALTHCHECK --interval=60s --timeout=30s --start-period=5s --retries=3 \ 69 + CMD curl -f http://localhost:6555 || exit 1
+46
docker-bake.hcl
··· 1 + variable "UID" { 2 + default = "1000" 3 + description = "User ID for the git and spindle users" 4 + } 5 + 6 + variable "GID" { 7 + default = "1000" 8 + description = "Group ID for the git and spindle users" 9 + } 10 + 11 + # 12 + # Build the latest commit from the master branch 13 + # 14 + target "edge" { 15 + name = "${APP}-edge" 16 + context = "." 17 + target = "${APP}" 18 + matrix = { 19 + APP = ["knot", "spindle"] 20 + } 21 + args = { 22 + TAG = "master" 23 + UID = UID 24 + GID = GID 25 + } 26 + tags = ["tngl/${APP}:edge"] 27 + } 28 + 29 + # 30 + # Build supported legacy releases 31 + # 32 + target "all" { 33 + name = "${APP}-${replace(TAG, ".", "-")}" 34 + context = "." 35 + matrix = { 36 + APP = ["knot", "spindle"] 37 + TAG = ["v1.11.0-alpha", "v1.10.0-alpha", "v1.9.0-alpha", "v1.8.0-alpha"] 38 + } 39 + target = "${APP}" 40 + args = { 41 + TAG = "${TAG}" 42 + UID = UID 43 + GID = GID 44 + } 45 + tags = ["tngl/${APP}:${TAG}"] 46 + }
+38 -10
docker-compose.yml
··· 1 + name: tangled 1 2 services: 2 3 knot: 4 + image: tngl/knot:latest 3 5 build: 4 - context: . 5 - args: 6 - UID: 1000 7 - GID: 1000 6 + target: knot 7 + args: &args 8 + UID: ${UID:-1000} 9 + GID: ${GID:-1000} 10 + TAG: ${TAG:-v1.11.0-alpha} 11 + tags: 12 + - tngl/spindle:latest 13 + - tngl/spindle:${TAG:-v1.11.0-alpha} 8 14 environment: 9 15 KNOT_SERVER_HOSTNAME: ${KNOT_SERVER_HOSTNAME} 10 16 KNOT_SERVER_OWNER: ${KNOT_SERVER_OWNER} ··· 19 25 - "5555:5555" 20 26 - "2222:22" 21 27 restart: always 28 + spindle: 29 + image: tngl/spindle:latest 30 + build: 31 + target: spindle 32 + args: *args 33 + tags: 34 + - tngl/spindle:latest 35 + - tngl/spindle:${TAG:-v1.11.0-alpha} 36 + environment: 37 + SPINDLE_SERVER_HOSTNAME: ${SPINDLE_SERVER_HOSTNAME} 38 + SPINDLE_SERVER_OWNER: ${KNOT_SERVER_OWNER} 39 + volumes: 40 + - ./logs:/var/log/spindle 41 + - ./spindle:/app 42 + ports: 43 + - "6555:6555" 22 44 frontend: 23 45 image: caddy:alpine 24 - command: > 25 - caddy 26 - reverse-proxy 27 - --from ${KNOT_SERVER_HOSTNAME} 28 - --to knot:5555 29 46 depends_on: 30 - - knot 47 + knot: 48 + condition: service_healthy 49 + spindle: 50 + condition: service_healthy 51 + configs: 52 + - source: caddyfile 53 + target: /etc/caddy/Caddyfile 31 54 ports: 32 55 - ${KNOT_SERVER_PORT:-443}:443 33 56 - ${KNOT_SERVER_PORT:-443}:443/udp ··· 35 58 - ./caddy_data:/data 36 59 restart: always 37 60 profiles: ["caddy"] 61 + configs: 62 + caddyfile: 63 + content: | 64 + ${KNOT_SERVER_HOSTNAME} { reverse_proxy knot:5555 } 65 + ${SPINDLE_SERVER_HOSTNAME} { reverse_proxy spindle:6555 }
+56 -17
readme.md
··· 4 4 > This is a community maintained repository, support is not guaranteed. 5 5 6 6 Docker container and compose setup to run a [Tangled](https://tangled.org) knot 7 - and host your own repository data. 7 + and spindle, hosting your own repository data and CI. 8 8 9 9 ## Pre-built Images 10 10 11 - There is a [repository](https://hub.docker.com/r/tngl/knot) of pre-built images 11 + There is a [repository](https://hub.docker.com/r/tngl) of pre-built images 12 12 for tags starting at `v1.8.0-alpha` if you prefer. 13 13 14 - ``` 14 + ```sh 15 15 docker pull tngl/knot:v1.10.0-alpha 16 + docker pull tngl/spindle:v1.10.0-alpha 16 17 ``` 17 18 18 19 Note that these are *not* official images, you use them at your own risk. 19 20 20 - ## Building The Image 21 + ## Building The Images 22 + 23 + Both the knot and spindle images are built using the same `Dockerfile`, since 24 + they're sourced from the same codebase and can therefore share a lot of the 25 + build steps (such as `go mod download`), caching results between them. You 26 + can build the images locally by running `docker bake`: 27 + 28 + ```sh 29 + docker bake 30 + ``` 31 + 32 + Optionally, choose a target image to build: 21 33 22 - By default the `Dockerfile` will build the latest tag, but you can change it 23 - with the `TAG` build argument. 34 + ```sh 35 + docker bake knot 36 + docker bake spindle 37 + ``` 38 + 39 + By default this will build the latest tag, but you can target a specific tag 40 + like so: 24 41 25 42 ```sh 26 - docker build -t knot:latest --build-arg TAG=master . 43 + docker bake knot-v1-10-0-alpha spindle-v1-10-0-alpha 27 44 ``` 28 45 29 - The command above for example will build the latest commit on the `master` 30 - branch. 46 + The command above for example will build the `v1.10.0-alpha` tag for both the 47 + `knot` and `spindle`. They're expressed as individual bake targets, so you can 48 + also optionally specify just one to build at a specific version. 49 + 50 + You can also build all tagged releases: 51 + 52 + ```sh 53 + docker bake all 54 + ``` 31 55 32 - By default it will also create a `git` user with user and group ID 1000:1000, 56 + By default it will also create a `git` / `spindle` user with user and group ID 1000:1000, 33 57 but you can change it with the `UID` and `GID` build arguments. 34 58 35 59 ```sh 36 - docker build -t knot:latest --build-arg UID=$(id -u) GID=$(id -g) 60 + docker bake UID=$(id -u) GID=$(id -g) 37 61 ``` 38 62 39 63 The command above for example will create a user with the host user's UID and GID. 40 64 This is useful if you are bind mounting the repositories and app folder on the host, 41 65 as in the provided `docker-compose.yml` file. 42 66 67 + You can also build the latest commit on `master` for both services by using 68 + the `-edge` targets. These will build to a tag named `:edge` to distinguish 69 + it from the `:latest` release: 70 + 71 + ```sh 72 + docker bake edge 73 + # 74 + # or, with a specific target image... 75 + # 76 + docker bake edge-knot 77 + docker bake edge-spindle 78 + ``` 79 + 43 80 <hr style="margin-bottom: 20px; margin-top: 10px" /> 44 81 45 82 When using compose, these can be specified as build arguments which will be ··· 59 96 60 97 ## Setting Up The Image 61 98 62 - The simplest way to set up your own knot is to use the provided compose file 63 - and run the following: 99 + The simplest way to set up your own knot and spindle is to use the provided 100 + compose file and run the following: 64 101 65 102 ```sh 66 - export KNOT_SERVER_HOSTNAME=example.com 67 - export KNOT_SERVER_OWNER=did:plc:yourdidgoeshere 68 - export KNOT_SERVER_PORT=443 69 - docker compose up -d 103 + export KNOT_SERVER_HOSTNAME="knot.example.com" 104 + export SPINDLE_SERVER_HOSTNAME="spindle.example.com" 105 + export KNOT_SERVER_OWNER="did:plc:yourdidgoeshere" 106 + export KNOT_SERVER_PORT="443" 107 + 108 + docker compose up --detach 70 109 ``` 71 110 72 111 This will setup everything for you including a reverse proxy.

History

3 rounds 4 comments
sign up or login to add to the discussion
psychedeli.ca submitted #2
interdiff
8 commits
expand
5712014d
add build stage for the spindle server
eedeaec2
add spindle to docker-compose and configure builds for each image
dba8ae02
add docker bake config for building edge releases
3cffdc73
update readme to reference the spindle and document how to build images using docker bake
c04d282f
fix adduser/group commands in spindle stage
a4bd85cf
add legacy releases task and info on how to build past releases
01623cf6
rename releases to all
7a3aeec0
add docs around variables
merge conflicts detected
expand
  • Dockerfile:1
expand 0 comments
psychedeli.ca submitted #1
diff interdiff
8 commits
expand
cbda62f0
add build stage for the spindle server
f8140b9f
bump tag to v1.11.0 in Dockerfile
982d224c
add spindle to docker-compose and configure builds for each image
e06b78fb
add docker bake config for building edge releases
3aea6d5a
update readme to reference the spindle and document how to build images using docker bake
864829a3
fix adduser/group commands in spindle stage
6fbf06a9
add legacy releases task and info on how to build past releases
90466519
rename releases to all
expand 4 comments
julien.rbrt.fr

Why not simply using a very simple Dockerfile, like this one for instance: https://tangled.org/strings/julien.rbrt.fr/3m75vdbmdad22

knotbin.com

It's unclear to me why this would be built into the knot docker repo. Not all knot hosters will want to host spindles.

psychedeli.ca

@julien.rbrt.fr all the dependencies and build commands are the same between the two tangled components so sharing their stages between builds makes building both of those programs faster.

@knotbin.com do you feel like the spindle image would be better off built in a different repo? my only reason for PRing here is so both the knot & spindle builds could take advantage of some dependency and artifact caching due to the multi-stage dockerfile. but if it’s better off in a different repo that’s fine too.

knotbin.com

i concur, I think this should be fine to merge once conflicts are fixed.

psychedeli.ca submitted #0
diff
6 commits
expand
cbda62f0
add build stage for the spindle server
f8140b9f
bump tag to v1.11.0 in Dockerfile
982d224c
add spindle to docker-compose and configure builds for each image
e06b78fb
add docker bake config for building edge releases
3aea6d5a
update readme to reference the spindle and document how to build images using docker bake
864829a3
fix adduser/group commands in spindle stage
expand 0 comments