FluentCI Engine #

If you love FluentCI, please ★ star this repository to show your support 💚. Looking for support? Join our Discord.

FluentCI Engine is a programmable CI/CD engine (used by FluentCI) that is designed to be simple, flexible, and easy to use. It is supposed to run on the host machine without containerization or virtualization, and it is designed to be used with Nix, Pkgx, Devbox, Flox, Devenv, Hermit, EnvHub, Pixi and Mise.

Table of contents #

• Features
• Host execution plans
• Parallel matrix jobs
• Quick Start
• Downloads
• Documentation
• Plugins
  • Rust Plugin Example
  • Available Plugins
  • Builtin functions
• GitHub Actions
• Caching
• OpenTelemetry Tracing
• Logging

✨ Features #

• Simple and easy to use
• Flexible
• No containerization or virtualization
• Built-in support for Nix, Hermit, Pkgx, Devbox, Flox, Devenv, Envhub, Mise and Pixi
• Built-in support for Secrets (backends: Google Secret Manager, AWS Secrets Manager, Azure Key Vault and HashiCorp Vault)
• Built-in support for Services
• Cache support (backends: local, S3, GCS, R2)
• Wasm plugin SDKs for Python, Elixir (CI DSL), Gleam, ReScript, PureScript, and TypeScript; see SDK guides
• GraphQL API, see API Documentation
• OpenTelemetry tracing
• Plugin system in WebAssembly, see examples

Host execution plans #

A shared host executor powers fluentci-engine execute and the legacy GraphQL/Wasm adapters, with bounded parallelism, dependency validation, per-task environments, cancellation, and structured results. See host execution plans for the versioned plan format, functional SDK direction, and current limitations. Elixir host plans provide immutable |> builders, pure JSON compilation, typed results, and managed local execution with cancellation.

Version 0.6.2 adds composable host/transfer/digest primitives, compiled DAG execution, and incremental output publication. The new SDK bindings require matching engine sources. See the executor benchmarks for measured gains and their limits.

Version 0.6.1 provides unified execution, parallel matrices, workflow primitives, and native file and env-file tasks. See the 0.6 compatibility notes for migration guidance, legacy scheduling/service limits, and supported guest build paths.

Parallel matrix jobs #

Pure matrix builders expand named cases and Cartesian products into native tasks, with include/exclude filters, stable IDs, prerequisites and joins. jobs bounds concurrency; each case uses normal execution, caching, logs and cancellation. Matching guest SDKs submit a complete independent plan through execute_plan. See matrix jobs for SDK APIs and current boundaries. This API requires FluentCI Engine 0.6.1 or later and matching guest SDKs.

🚀 Quick Start #

# Clone the repository
git clone https://github.com/fluentci-io/fluentci-engine.git
# Go to the project directory
cd fluentci-engine
# Install dependencies
nix develop
cd ./fixtures
cargo run --release -p fluentci-engine -- serve
# Open the browser and go to http://localhost:6880/graphiql
# See ./fixtures for some GraphQL queries examples

📦 Downloads #

Download FluentCI Engine v0.6.2 for your platform:

• Mac: arm64: fluentci-engine_v0.6.2_aarch64-apple-darwin.tar.gz intel: fluentci-engine_v0.6.2_x86_64-apple-darwin.tar.gz
• Linux: intel: fluentci-engine_v0.6.2_x86_64-unknown-linux-gnu.tar.gz arm64: fluentci-engine_v0.6.2_aarch64-unknown-linux-gnu.tar.gz
• FreeBSD: x86_64: fluentci-engine_v0.6.2_x86_64-unknown-freebsd.tar.gz
• NetBSD: x86_64: fluentci-engine_v0.6.2_x86_64-unknown-netbsd.tar.gz
• OpenBSD: x86_64: fluentci-engine_v0.6.2_x86_64-unknown-openbsd.tar.gz

📚 Documentation #

• API Documentation

🧩 Plugins #

FluentCI Engine supports plugins in WebAssembly. You can write your own plugins in Rust or any other language that can compile to WebAssembly. See examples for more information.

🦀 Rust Plugin Example #

Create a new Rust project:

cargo new nix --lib

Install the extism_pdk, fluentci_types and fluentci_pdk crates:

cargo add extism_pdk fluentci_types fluentci_pdk

Save the following code to src/lib.rs:

use extism_pdk::*;
use fluentci_pdk::dag;
use fluentci_types::nix::NixArgs;

#[plugin_fn]
pub fn exec(command: String) -> FnResult<String> {
    let stdout = dag()
        .nix(NixArgs {
            impure: true
        })?
        .with_exec(command.split_whitespace().collect())?
        .stdout()?;
    Ok(stdout)
}

Set the following in your Cargo.toml:

[lib]
crate-type = ["cdylib"]

Compile the plugin to WebAssembly:

cargo build --release --target wasm32-unknown-unknown

Run the plugin:

fluentci-engine call -m ./target/wasm32-unknown-unknown/release/nix.wasm -- exec nix --version

Source examples include Python projects with uv and native Rust builds with sccache, including compiler-cache statistics and supervised service cleanup.

📑 Available Plugins #

• android
• ansible-lint
• ansible
• apko
• atlas
• base
• bazel
• biome
• black
• buf
• buildx
• bun
• checkmake
• chromatic
• clojure
• cloudflare
• codecov
• cue
• cypress
• dagger
• deno
• depot
• dotnet
• elixir
• fastlane
• firebase
• flakestry
• flipt
• fly
• flutter
• github
• gleam
• go
• gradle
• grype
• heroku
• microcks
• netlify
• nixpacks
• nodejs
• pkl
• playwright
• pulumi
• oxc
• railway
• ruby
• ruff
• rust
• rye
• shuttle
• sonar
• spin
• ssh
• supabase
• syft
• symfony
• terraform
• zig
• wasmer

🌈 Builtin functions #

FluentCI Plugin Development Kit (fluentci_pdk) provides some builtin functions from that you can use in your plugins:

devbox #

Setup a Devbox Environment.

Example:

dag()
  .devbox()?
  .with_exec(vec!["devbox", "version"])?
  .stdout()?;

devenv #

Setup a Devenv Environment.

Example:

dag()
  .devenv()?
  .with_exec(vec!["devenv", "version"])?
  .stdout()?;

directory #

Load a directory at the given path.

Example:

dag()
  .directory(".")?
  .with_exec(vec!["ls", "-la"])?
  .stdout()?;

envhub #

Setup an EnvHub Environment.

Example:

dag()
  .envhub()?
  .with_exec(vec!["envhub", "--version"])?
  .stdout()?;

file #

Load a file at the given path.

Example:

dag()
  .file("Cargo.toml")?
  .path()?;

flox #

Setup a Flox Environment.

Example:

dag()
  .flox()?
  .with_exec(vec!["flox", "--version"])?
  .stdout()?;

git #

Load a Git repository at the given URL.

Example:

dag()
  .git("https://github.com/tsirysndr/me")?
  .branch("main")?
  .tree()?
  .entries()?;

http #

Load a HTTP resource at the given URL.

Example:

dag()
  .http("https://example.com")?
  .path()?;

mise #

Setup a Mise Environment.

Example:

dag()
  .mise()?
  .with_exec(vec!["mise", "--version"])?
  .stdout()?;

nix #

Setup a Nix Environment.

Example:

dag()
  .nix(NixArgs {
    impure: true
  })?
  .with_exec(vec!["nix", "--version"])?
  .stdout()?;

pipeline #

Create a new pipeline.

Example:

dag()
  .pipeline("example")?
  .with_exec(vec!["echo", "Hello, World!"])?
  .stdout()?;

pixi #

Setup a Pixi Environment.

Example:

dag()
  .pixi()?
  .with_exec(vec!["pixi", "--version"])?
  .stdout()?;

pkgx #

Setup a Pkgx Environment.

Example:

dag()
  .pkgx()?
  .with_exec(vec!["pkgx", "--version"])?
  .stdout()?;

🌱 Github Actions #

FluentCI Engine is designed to be used with Github Actions. You can use the fluentci-io/setup-fluentci@v5 action to run your pipelines. See fluentci-io/setup-fluentci for more information.

name: Hello

on:
  push:
    branches:
      - main

jobs:
  setup-fluentci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Setup FluentCI
        uses: fluentci-io/setup-fluentci@v5
        with:
          wasm: true # set to true so WebAssembly plugins can be used
          plugin: base # Name of the Wasm Plugin to use without the .wasm extension,
          # will be downloaded from the registry https://pkg.fluentci.io

          # Arguments to pass to the plugin: function_name args
          args: |
            hello Hello, World!

Native plans also support composable workflow primitives: retries, conditional tasks, bounded finalizers, named outputs and artifacts, resource leases and supervised services (engine 0.6.1 or later and matching SDKs required).

See native file and env-file primitives for filesystem tasks, structured query results, and literal dotenv loading.

⚡ Caching #

FluentCI Engine supports caching. To enable it, set the following environment variables:

• FLUENTCI_CACHE_GCS_BUCKET - GCS bucket name, if you are using Google Cloud Storage
• FLUENTCI_CACHE_S3_ENDPOINT - S3 endpoint, if you are using S3-compatible storage
• FLUENTCI_CACHE_S3_BUCKET - S3 bucket name, if you are using S3-compatible storage
• FLUENTCI_CACHE_CDN_ENDPOINT - CDN endpoint, if you are using CDN (optional) for downloading cache

🔭 OpenTelemetry Tracing #

FluentCI Engine supports OpenTelemetry tracing. To enable it, set the OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_ZIPKIN_ENDPOINT (if you want to use Zipkin) environment variable to the desired endpoint.

📑 Logging #

FluentCI Engine supports sending logs to Baselime. To enable it, set the BASELIME_API_KEY environment variable to the desired API key.

Independent host pipelines can also use the authenticated serve-runs API; see the server run API documentation.

Rust Wasm plugins use Extism and XTP schemas for the host ABI. See the plugin schema and code-generation guide for generating plugins and running release-mode compatibility tests.

Native BSD archive targets, verification, and compiler availability are described in the BSD release guide.

Repository CI modules #

The root fluentci.toml declares PureScript Wasm CI modules under .fluentci. Run fluentci modules to discover functions, fluentci run rust test for native release tests, or fluentci run main check for Rust and provider end-to-end validation. GitHub Actions installs the CLI with fluentci-io/setup-fluentci@next; release modules are compiled once on Ubuntu and invoked as portable Wasm on the native Linux/macOS release runners.