A collection of +180 reusable WebAssembly modules for building composable CI/CD pipelines with FluentCI fluentci.io/modules
modules scripts README.md
5.4 kB · 90 lines
Markdown
at main

Repository automation on BEAM #

All automation in this directory is implemented in Gleam targeting Erlang/OTP and distributed as the ignored fluentci_modules_ci escript. Erlang FFI provides filesystem, process, archive, cryptographic, socket, and process-isolation primitives. Validation, generation, registry policy, build scheduling, and service lifecycle assertions live in Gleam.

CI installs OTP 29.1.2 and Gleam 1.19.2 through the published setup-erlang and setup-gleam modules, then compiles with warnings treated as errors. A small checksum-verified shell bootstrap installs FluentCI first to break the toolchain bootstrap cycle. No Python interpreter is needed to run these helpers.

When compilation and source generation are authorized:

(cd scripts && gleam build --target erlang --warnings-as-errors && gleam export escript)
escript scripts/fluentci_modules_ci generate-all
escript scripts/fluentci_modules_ci check-all
escript scripts/fluentci_modules_ci validate-modules

generate-all runs these generators in dependency order:

  • generate-cli: CLI source catalog, private schemas, entrypoints, option references.
  • generate-setup: setup catalog, private schemas, and legacy adapters.
  • generate-services: native service profiles.
  • generate-buildx: named Buildx arguments, shallow decoders, private schemas, and references. Explicit --capture refreshes the committed snapshot from Docker Buildx help; it executes the CLI and must be authorized locally.
  • generate-components: typed PureScript public functions, argument descriptions, native return schemas, entrypoints, and component documentation.
  • generate-dependencies: installer imports, manifests, frozen source locks, dependency regression wrappers, and prerequisite inventory.
  • audit-modules: reviewed pipeline purposes and export inventory.
  • generate-readmes: contract-derived examples in PureScript, Gleam, Elixir, ClojureScript, ReScript, Python and TypeScript for every registered module.

Each individual generator supports --check. Templates hold declarative contracts and static source/prose fragments; schemas and executable source remain generated from the reviewed catalogs and those templates. Public list fields remain comma-separated strings where declared; no JSON CLI objects are added.

CI-only commands:

escript scripts/fluentci_modules_ci setup-toolchain --profile full
escript scripts/fluentci_modules_ci setup-toolchain --file setup/erlang/dist/plugin.wasm -- --version 29.1.2
escript scripts/fluentci_modules_ci install-security-test-tools --directory /tmp/security-tools --tools betterleaks,gitleaks,syft
escript scripts/fluentci_modules_ci test-module-batch --modules-env MODULE_BATCH --log-dir /tmp/module-logs
escript scripts/fluentci_modules_ci validate-wasm packaging/dpkg
escript scripts/fluentci_modules_ci test-services
escript scripts/fluentci_modules_ci test

MODULE_BATCH is internal JSON supplied by the GitHub matrix; its environment variable name is passed on the command line. It is not a public module argument. Downloads require HTTPS and verified SHA-256 hashes. Published setup components also require the pinned publisher DID, name, version, and native component header. Fresh setup prefixes preserve compatibility with published 0.2.0 installers.

Module batch builds and tests run with 32 workers by default ( MODULE_BUILD_CONCURRENCY accepts 1–32). Expensive component compilations use a separate limit derived from available host CPU cores and any CPU quota, with at least one compiler and at most 32. COMPONENT_BUILD_CONCURRENCY can override that limit (1–32). This compiler limit is shared across workers. Each module produces a composed dist/ci.wasm; subsequent CLI checks use that exact prebuilt artifact. Service lifecycle operations, including followed logs, use these files too, avoiding source and setup rebuilds on every operation.

Service builds use up to 32 BEAM workers; shared dependency preparation and composition are serialized. Native service lifecycle tests run sequentially in an isolated workspace, check TCP/HTTP/TLS and application protocols, exercise live log interruption, and clean up owned services, volumes, and images. Follow-log checks signal the owned FluentCI viewer PID directly and verify that the background service keeps running. Build and E2E commands stream output immediately. Lifecycle operations print timings, and silent commands emit an idle heartbeat identifying the command, directory, and elapsed time every 30 seconds while preserving their original deadlines. Diagnostic artifacts contain logs/status, never generated AWS credential file contents. SERVICE_BUILD_CONCURRENCY accepts 1–32; SERVICE_LOG_DIR selects diagnostics.

lib/component/build-cli.mjs and inspect-abi.mjs are small adapters to the existing JavaScript component compiler and TypeScript AST APIs. The validator compares their ABI data in Gleam. Module build.mjs entrypoints and JavaScript SDK internals remain the existing compiler adapters. JavaScript Wasm fixture tests call the BEAM validator through lib/component/validate-wasm.mjs.

Helper regression tests are under scripts/test; they do not start Docker, install tools, or execute FluentCI plugins. Real builds and lifecycle verification remain in CI. Follow AGENTS.md before local compilation or test execution.