- Rust 99%
- Docker <1%
srcball #
Srcball is a Rust service and client library for turning source repositories into cached OCI image-layout artifacts. It is intended as a foundation for language-specific source proxies: Go-module-style proxying, CUE-style OCI modules, private forge mirrors, or any registry that wants source repositories to remain the package authority.
The service is storage-backed. The current implementation ships a filesystem storage backend for normal volumes; the storage trait is intentionally small so an S3 backend can be added without changing resolver, server, or client code.
What works now #
srcball serveruns an HTTP service withnotmadlifecycle management.GET /packages/fetch?package=<repo>&ref=<ref>returns an OCI image-layout tar (application/vnd.oci.image.layout.v1.tar) with the source tree as an OCI layer.GET /packages/metadata?package=<repo>&ref=<ref>returns artifact metadata as JSON.- OCI Distribution-compatible pull endpoints are exposed under
/v2:GET /v2/GET /v2/<package>/manifests/<ref>GET /v2/<package>/blobs/<digest>
- Missing immutable refs are resolved from Git, packaged as OCI artifacts, and stored in the configured volume.
- Filesystem storage also materializes source files under
state/artifacts/modules, similar to Go's module cache. - HEAD or
latestrequests resolve to Go-style pseudo versions:v0.0.0-yyyymmddhhmmss-<12-char-sha>. noretrywraps transient remote Git operations.nodriftruns recurring service maintenance without timer drift.- Filesystem storage tracks access timestamps and can evict least-recently-used OCI artifacts with
srcball serve --max-cache-bytes <bytes>. srcball package --url <repo> [--reference <ref>] [--output artifact.oci.tar]fetches through the same service core without HTTP.
Run locally #
docker compose up --build
By default compose bind-mounts the local directory ./.srcball into the container. Override the host-side location with:
SRCBALL_STATE_DIR=./.srcball docker compose up --build
Compose runs the container as ${SRCBALL_UID:-1000}:${SRCBALL_GID:-1000} so the bind-mounted .srcball directory stays writable by the local user. Override these if your local UID/GID differ:
SRCBALL_UID=$(id -u) SRCBALL_GID=$(id -g) docker compose up --build
When running the binary directly, use the same local state directory with:
SRCBALL_ROOT=.srcball cargo run -p srcball -- serve
The registry listens on http://localhost:3000.
Health check:
curl -fsS http://localhost:3000/healthz
Fetch an OCI layout tar directly:
curl -fL 'http://localhost:3000/packages/fetch?package=github.com/cue-lang/cue&ref=v0.15.0' \
-o cue.oci.tar
Fetch only a subdirectory from a repository with //sub/path:
curl -fL 'http://localhost:3000/packages/fetch?package=github.com/kjuulh/forest//apps/forest&ref=main' \
-o forest-app.oci.tar
Use the OCI registry surface:
curl -fsS http://localhost:3000/v2/
curl -fsS http://localhost:3000/v2/github.com/cue-lang/cue/manifests/v0.15.0
For private repositories, use Git transport credentials already available to the service process: SSH agent/keys, git@..., ssh://..., HTTPS credential helpers, or mounted Git config.
Local state layout #
The default filesystem backend keeps both OCI artifacts and an unpacked source cache. With the local defaults above, this lives in:
.srcball/
artifacts/
blobs/fa/bc/<rest>.tar # cached OCI image-layout tar files
metadata/fa/bc/<rest>.json # srcball metadata by package/version
access/fa/bc/<rest>.atime # LRU access timestamps
refs/fa/bc/<rest>.json # short-TTL HEAD/latest resolution cache
oci/blobs/sha256/fa/bc/<rest> # digest-addressed blobs served by /v2/.../blobs/...
modules/ # unpacked source trees
vcs/fa/bc/<rest>/ # bare Git repositories
Normal module paths are splatted in a Go-module-cache-like shape:
.srcball/artifacts/modules/github.com/acme/project@v1.2.3/
Unsafe or non-registry-like package names, such as absolute local paths, are stored under a hashed _/<hash>@<version> directory. During materialization srcball only writes regular files and directories. Absolute paths, parent-directory traversal, hard links, symlinks, device files, and other special tar entries are skipped.
Eviction removes the cached OCI tar, metadata, access timestamp, materialized source directory, and unreferenced digest blobs. Mutable refs such as HEAD and registry latest are cached briefly before resolving upstream again; configure that with SRCBALL_MUTABLE_REF_TTL_SECS.
Hashed internal keys use two-level fanout directories (fa/bc/<rest>) so large caches do not put every artifact, ref, or Git repository under one flat directory.
Rust library #
In-process service:
use srcball::{FileStorage, GitSourceResolver, PackageSpec, SrcballService};
# async fn example() -> anyhow::Result<()> {
let storage = FileStorage::new(".srcball/artifacts");
let resolver = GitSourceResolver::new(".srcball/vcs");
let service = SrcballService::new(storage, resolver);
let spec = PackageSpec::new("github.com/cue-lang/cue", Some("v0.15.0".into()))?;
let artifact = service.fetch(&spec).await?;
std::fs::write("cue.oci.tar", artifact.body)?;
# Ok(())
# }
HTTP client:
use srcball::{PackageSpec, SrcballClient};
# async fn example() -> Result<(), Box<dyn std::error::Error>> {
let client = SrcballClient::new("http://localhost:3000");
let spec = PackageSpec::new("github.com/cue-lang/cue", Some("v0.15.0".into()))?;
let artifact = client.fetch(&spec).await?;
std::fs::write("cue.oci.tar", artifact.body)?;
# Ok(())
# }
CI #
.github/workflows/ci.yml runs:
cargo fmt --all -- --checkcargo test --workspacedocker build -t srcball:ci .
Milestones #
- Core package model: keep
PackageSpec,ArtifactMetadata,Storage, andSourceResolveras the narrow service contract. - Storage backends: keep filesystem as the default volume backend; add S3 by implementing
Storage. - OCI registry parity: fill in remaining Distribution API behavior needed by stricter clients.
- Resolvers: keep Git as the first resolver; add forge-aware resolvers only when they need authentication or provider-specific URL discovery.
- Language adapters: layer Go proxy, CUE module, and other protocol endpoints on top of
SrcballServicerather than duplicating cache or fetch logic. - Private repositories: rely on Git transport credentials first (
ssh://,git@, credential helpers); add per-request credential injection as a resolver concern. - Operations: expand health checks and observability around resolver latency, cache hit rate, eviction counts, and artifact sizes.