During setup, we register cleanup functions which get executed at the
end of the workflow goroutine (deferred exec of DestroyWorkflow).

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

As it sees use in both spindle and knotserver.

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

Logs are streamed for each running pipeline on a websocket at
/logs/{pipelineID}. engine.TailStep demuxes stdout and stderr from the
container's logs and pipes that out to corresponding stdout and stderr
channels.

These channels are maintained inside engine's container
struct, key'd by the pipeline ID, and protected by a read/write mutex.
engine.LogChannels fetches the stdout/stderr chans as recieve-only if
the pipeline is known to exist.

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

The engine package currently uses the docker client setup the pipeline
and execute steps. The flow is like so:

- setup pipeline network for all steps to join to
- create a volume for the nix store (to persist packages across steps)
- create a volume for the workspace directory
- build a nixery.dev URL with packages we want in the container
- execute each step command in a new container using the same image

It's pretty unfinished still. Things to be done:

- support for other registries; currently only works with nixpkgs
- custom nixery URL
- ... a lot more that I'm forgetting now

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>

Signed-off-by: oppiliappan <me@oppi.li>