- OCaml 97%
- Raku 3%
bottler #
Homebrew bottle builder and tap manager for OCaml monorepos.
Overview #
Homebrew installs a package from a formula, a Ruby file that says where
the package comes from and how to install it. A bottle is a prebuilt
binary of that package for one platform, a gzipped tar archive that brew
downloads instead of compiling. A tap is a git repository of formulae
that brew tap adds to the local list. Shipping an OCaml binary through
Homebrew takes several steps that each fail in their own way: build a
bottle per platform, upload it somewhere brew can download it, write a
formula that carries the bottle's SHA-256, and push that formula to a
tap. Bottler runs all of them from one
homebrew.yml at the root of a dune monorepo. It builds the bottles,
uploads them to S3-compatible storage (Scaleway), checks that the store
serves exactly the bytes it uploaded, writes the formulae and pushes the
tap.
The formula and tap primitives come from the
ocaml-homebrew
library. Bottler adds the parts specific to a monorepo: the dune release
build, the YAML configuration, the upload and the GitHub Actions workflow
that runs a release in CI.
Bottles are byte-reproducible for a given release version. Bottler writes
the tar archive itself, with fixed metadata (mtime 0, uid and gid 0, empty
user and group names), and compresses it at level 9 through zlib's gzip
wrapper, which writes a header with modification time 0 and no file name;
the one header byte that varies, zlib's operating-system code, is constant
for one platform. No external tar or gzip is involved. The release
version is part of the archived path (NAME/VERSION/bin/NAME) and of the
binary's --version output, so two builds give the same bytes when they
share a version: a fixed PROJECT_VERSION, or the same commit built on
the same day (see Configuration).
Installation #
Install with opam:
$ opam install bottler
If opam cannot find the package, it may not yet be released in the public
opam-repository. Add the overlay repository, then install it:
$ opam repo add samoht https://tangled.org/gazagnaire.org/opam-overlay.git
$ opam update
$ opam install bottler
Usage #
$ bottler config # Show parsed homebrew.yml
$ bottler build # Build bottles for build.platforms
$ bottler upload # Upload bottles to S3
$ bottler formula NAME # Emit a Ruby formula
$ bottler release # Full pipeline: build + upload + tap push
$ bottler workflow # Generate GitHub Actions release workflow
$ bottler doctor # Verify credentials and connectivity
$ bottler login # Set up Scaleway + rclone
bottler release is the command most people run. It builds and uploads
the bottles, reads each one back from the store and compares its digest
before any formula is written, then pushes the tap. When several
homebrew.yml sources share one tap and another release pushes first,
bottler resets its checkout to the new tip, writes its formulae again and
retries.
The other commands look after the tap and the bucket. bottler install
installs or upgrades packages from the tap, and bottler install --local
builds bottles from the working tree and installs them without uploading
anything. bottler list shows every formula in the tap and bottler remove NAME deletes one. bottler gc deletes old bottle versions from
the bucket, bottler usage reports how much space it holds, and bottler keys provisions upload-only Scaleway credentials for CI. bottler init
writes the bin/version helper that makes a binary report the release
version from --version.
Configuration #
Bottler reads homebrew.yml at the project root:
handle: samoht
storage:
bucket: homebrew-bottles
region: fr-par
profile: homebrew-samoht
tap:
clone_url: https://github.com/samoht/homebrew-tap.git
build:
switch: lto-5.4.1 # opam switch the release build runs under
packages: [merlint, prune]
A package is its name when the tree says the rest. Its target is the
dune executable whose public_name is that name, found by reading the
tree's dune files as dune does, and its description is the synopsis
of the opam package that installs the executable: the (package P) of
its stanza, or its project's only package, read from P.opam, or from
the dune-project when there is no such file. The object form sets what
the tree does not say or says otherwise, beside the per-package keys:
packages:
- merlint
- name: mq # a binary no public_name installs as mq
target: ocaml-merge-queue/bin/main.exe
description: "Local merge queue"
companions:
- name: daemon-supervisor
target: ocaml-daemon/bin/supervisor.exe
A name that no executable or several are installed under is refused with
the candidates and a request to set target. tap.push_url is the SSH
form of tap.clone_url on github.com and gitlab.com, and
git@KNOT:HANDLE/REPO for a tangled.org clone URL on the knot tap.knot
names; another forge needs tap.push_url. bottler config prints each
of these values with where it came from.
build:
linux: static # or linuxbrew
switch: lto-5.4.1
checks: [switch-pins] # root aliases the release build also builds
platforms: [arm64_sonoma, x86_64_linux, arm64_linux]
build.platforms lists the bottles a release builds, every one in the
same release so that the formula carries them all; without it, a release
builds the host's platform alone. The host builds its own platform. With
build.linux: static, a Linux platform is built in a container of
build.image (ocaml/opam:alpine-ocaml-5.4 unless given) with Docker, for
that platform's architecture, from any host: the working tree is copied in,
its dependencies installed with opam, and the release build linked with
-ccopt -static, so the binary runs on any Linux distribution and can be
shipped outside Homebrew too. The container's opam root is a named volume
that the next release reuses. The image's current switch builds the
release, so a tree whose release profile needs -use-lto names an image
whose switch has it. A macOS platform builds only on a Mac of its
architecture, and a linuxbrew bottle only on its Linux machine.
build.compiler lists the packages build.switch is created with where
it does not exist, as on a CI runner or in a Linux container: the
compiler, then its options. build.compiler_url pins the first from
elsewhere, such as a fork whose compiler has -use-lto. Both are read
from build.switch on the machine that has it, through opam switch export: its ocaml-base-compiler or ocaml-variants package, its
ocaml-option-* packages and the compiler's pin. That machine hands them
on: a release passes them to the Linux containers it starts, and bottler workflow writes them into the workflow it generates. Where the switch
does not exist and nothing is given, the compiler is
ocaml-base-compiler.5.4.1. Given, they are used as they are:
build:
switch: lto-5.4.1
compiler: [ocaml-variants.5.4.1, ocaml-option-flambda]
compiler_url: git+https://github.com/samoht/ocaml.git#6ca6a45
lto: true
A tree released to opam cannot put -use-lto in its own dune files,
since a stock compiler rejects the flag: build.lto: true links its
release with the flag through a workspace file of bottler's, and needs a
build.compiler that has it. A tree with its own dune-workspace turns
the flag on there instead. bottler workflow generates a GitHub Actions workflow that builds each platform
on a runner of its own with bottler build --platform, uploads the
bottles, and writes the formulas into the tap the workflow runs in.
A macOS bottle is labelled arm64_sonoma or sonoma, so brew pours it on
macOS 14 and later. Bottler builds it with MACOSX_DEPLOYMENT_TARGET=14.0,
and refuses to build under a switch whose archives are built for a later
macOS: a compiler configured on a later macOS uses the functions it found
there, such as pipe2 and dup3 on macOS 27, and the binary would not
start on macOS 14. The refusal gives the command that rebuilds the switch
for macOS 14. Each binary is checked before it is bottled: it must record
no later minimum macOS, and import no function weakly, which is how a
function later than the deployment target is linked.
build.switch is required and has no default. Bottles are built with
--profile=release, which links with -use-lto. LTO is link-time
optimisation: the compiler keeps an intermediate form of every module and
optimises across modules when the program is linked. The stock OCaml
compiler has no -use-lto flag and refuses it; only a compiler built with
that extension accepts it, so this key names a switch whose compiler has it,
usually not the switch used for day-to-day development. Bottler runs opam exec --switch <build.switch> -- dune build --profile=release <targets>,
and bottler doctor checks that the switch exists and that its ocamlopt
accepts -use-lto.
build.checks names root-directory dune aliases the release build builds
beside the first package's binaries, as @@<alias> in the same command. A failing check
fails the build, so no bottle is made, and its output says why. The
monorepo names switch-pins, which refuses a release switch whose pins
drifted from root.opam; no other build depends on that alias.
Each package's primary binary must report the release version from
--version, and bottler checks this with PROJECT_VERSION cleared from the
environment. The packages are built one at a time and each primary is
checked before the next is built, so a binary that reports another version
stops the release at its own build. Companion binaries are built and bottled at the same version,
then installed beside the primary binary by source-built formulae.
The release version is PROJECT_VERSION when it is set, and otherwise
YYYYMMDD-<commit> from git HEAD, with -dirty appended when the tree
has uncommitted changes. The date is the build machine's local date at
build time, not the commit date.
Bottler refuses a configuration it cannot use before building anything. A
missing file, a missing build: section or a missing build.switch each
produce an error that names the file it resolved against the current
directory, the section or key that is absent, and the lines to add. An
unknown key is an error too, so a misspelt field cannot silently drop out
of a formula.
Licence #
ISC License. See LICENSE.md for details.