folder structure#

• build/ build system files
• doc/ developer documentation
• platforms/ build system configurations for various target platforms
• lib/ various utility libraries
• sys/ the kernel and its subsystems
• third-party/ integration with 3rd party dependencies (crates.io)
• test/ integration tests, Wasm conformance tests, etc.
• website/ the website (TODO)

naming convention#

• all crates use kebab case both in the folder name and crate name
• OS subsystems and components are prefixed with k e.g. kasync and kmem (these go into sys/) while regular, potentially publish-able crates are not prefixed.

buck2 techniques#

Toolchains#

• Sources rust, clang, lld from the repos Nix flake. The flake uses oxalica's rust overlay to fetch the Rust version declared in rust-toolchain.toml. We then obtain references to these tools using (a tweaked (tweaged??) copy of) buck2.nix.
• We use reindeer for crates.io integration. feature resolution was a bit complicated; we essentially have two partially overlapping dependency graphs: one for target builds and one for execution builds. By enabling the no_std cargo feature for target builds and std/default feature for execution builds we can force dependency resolution to not include incompatible crates for target builds. But this part of the integration remains cumbersome.
• Compiler settings are exposed as Buck2 constraints (opt-level, debuginfo, strip at the moment). This way we can have crate specific overrides later for example. Currently these constraints are just exposed via debug/release aliases that emulate the Rust profiles. I.e. you can run buck2 build -m release to produce an optimized binary.

build-std#

• k23 needs to build the core/alloc/compiler_builtins crates from source. Both because we use custom JSON target specs to control the precise code generation settings and because we have a custom unwinding machine that works on no_std builds. Rebuilding stdlib crates also allows us to apply compiler settings to the whole graph. This is just a good idea in general. It is a bit more involved to set up though.

• We defined a filegroup of our custom JSON targets, added it as the Rust toolchains rust_target_path and then in a select instruct rustc to build with the custom target when os:none and the cpu type matches. Otherwise we always fall back to the hosts target triple (e.g. for tests).

• rust-toolchain.toml conntains the rust-src component which means we can obtain the stdlibs source code from the nix flake too. We then just define rust_library targets for each required dependency and add them as explicit sysroot deps to the toolchain.

• Uncoditionally adding these sysroot deps would lead to a circular dependency problem when building the stdlib deps themselves (since they rely on the rust toolchain themselves of course!) so we define a bootstrap mode transition. When applied, the stdlib crates are compiled without any sysroot deps.

• Similarly not all crates can be compiled using the custom unwind machinery: the loader stage for example cannot unwind because not enough of the machine state is set up yet. We define a second transition that allows kernel and dependencies to be compiled with -Cpanic=unwind while loader and dependencies are compiled using -Cpanic=abort.

Testing#

• we have lots of different tests (regular Rust tests, miri, loom, fuzz tests, test running in emulator, etc.) but not all tests run on all targets. Crates that e.g. include riscv specific assembly cannot be tested on the host and need to be run in QEMU while loom tests for example cannot be run on target.
• This was for the longest time the biggest source of headaches for k23 builds but buck2 constraints make this relatively easy: tests are marked with the platform they are compatible with and buck2 will ignore all incompatible tests by default. This still needs some cleaning up but we're finally on a good path…
• buck2 commands can get quite large, so we have a justfile with quick aliases for common tasks (check builds, running clippy, running tests, running miri tests, etc.)