# OxygenFlow

[![OxygenFlow](https://oxygenhub.io/andymccall/oxygenflow/flow/badge.svg)](https://oxygenhub.io/andymccall/oxygenflow/flow)

Hermetic pipelines for Oxygen repositories:

- a typed pipeline in `.flow/pipeline.kdl`

- jobs run in isolation tiers, Firecracker microVMs by default

- the host writes the receipts

- results go to OxygenHub as checks and appear on its Flow tab

The specification is [docs/spec.md](docs/spec.md). What works today is

[docs/m0.md](docs/m0.md).

## A pipeline

```kdl

schema-version 1

sandbox "alpine" {

    runtime "microvm"

    rootfs "cas://blob/…"      // printed by `flow rootfs import`

    memory-limit "256MiB"

    network "isolated"

}

job "build" {

    sandbox "alpine"

    exec "make"

}

job "test" {

    sandbox "alpine"

    depends-on "build"

    exec "sh" "-eu" "test.sh"   // argv, run as written: no shell unless you name one

}

```

Each job boots its own microVM:

- it has no network device

- the rootfs is read-only

- the workspace is the state being built, read-write at `/src`

- the job runs as a user that is not root

This repository's own pipeline is [.flow/pipeline.kdl](.flow/pipeline.kdl).

The stamp at the top of this page is its result on OxygenHub. It shows the

run of the state you are viewing, not whatever the bookmark holds now, and

it names the state and each job's receipt.

## Deploying the runner on Earth

`make deploy-runner` builds `flow` and the static `flow-init`, installs them

under `/home/claude/flow` (keeping the previous files as `.prev`), re-pins

`flow-init` in `flow.toml` and restarts the `flow-runner` service.

## Commands

    flow check                 # validate .flow/pipeline.kdl, print jobs and the pipeline hash

    flow run [JOB…]            # run the pipeline on this directory, locally, in microVMs

    flow rootfs import T.tgz --sha256 HEX   # make a rootfs image; prints the `rootfs` line to pin

                  [--env NAME=value …]      #   with the image's own environment (a Dockerfile's ENV)

    flow rootfs list

    flow hash FILE             # BLAKE3, for pinning in flow.toml

    flow runner                # claim and run jobs for the hub in flow.toml

## A Flow home

`$FLOW_HOME` (default `~/.local/share/flow`) holds:

- `flow.toml`

- the pinned Firecracker binary, guest kernel and `flow-init`

- rootfs images under `cas/`

- scratch space

Every pinned file is checked against its BLAKE3 before use.

```toml

[microvm]

firecracker = { path = "bin/firecracker", blake3 = "…" }

kernel = { path = "kernel/vmlinux", blake3 = "…" }

init = { path = "bin/flow-init", blake3 = "…" }

[runner]                     # only for `flow runner`

hub = "https://hub.example:7343"

token = "…"                  # listed in the hub's [flow] runner_tokens

ca = "/path/to/hub-ca.pem"   # when the hub's certificate is self-signed

name = "runner-1"

max_jobs = 3                 # jobs at once (default 1)

memory_mib = 24576           # what they may use together; a lone job may use it all

cpus = 8

```

Setting one up needs Linux x86-64 with KVM and e2fsprogs 1.43 or later:

1. Build `flow` with `cargo build --release -p flow`. Build the guest init

   as a static binary:

   `RUSTFLAGS="-C target-feature=+crt-static" cargo build --release -p flow-init --target x86_64-unknown-linux-gnu`.

2. Download Firecracker v1.17.0 from its GitHub release, and check the

   published SHA-256. Download the guest kernel

   `firecracker-ci/v1.15/x86_64/vmlinux-6.1.155` from Firecracker's CI

   bucket.

3. Run `flow rootfs import alpine-minirootfs-3.24.2-x86_64.tar.gz --sha256 …`.

4. Write `flow.toml`, pinning each file by its `flow hash`.

On OxygenHub, `hub.toml` lists the runner token and the repositories whose

pushes run pipelines:

```toml

[flow]

runner_tokens = ["…"]

repos = ["owner/*"]

```

## Tests

`cargo test --workspace` runs every test that needs no VM. The microVM

conformance test also runs when `FLOW_TEST_HOME` names a Flow home and

`FLOW_TEST_ROOTFS` is the hash of an Alpine rootfs in it. The test checks

literal argv, network isolation, the read-only rootfs, and that dependents

are skipped after a failure.

## oxygen as a pinned subrepo

`flow` builds against `deps/oxygen`, a subrepo (`oxy submodule`) pinned to an exact state, so builds are reproducible and OxygenFlow can build this repository without network. `make pins` moves it to oxygen's `main` on Earth (push oxygen first); the next commit records the new pin. Format only this workspace, `cargo fmt -p flow -p flow-core -p flow-init`: `--all` also reaches `deps/`. After a fresh clone, `oxy submodule update --init` fetches it.