OXYGENHUB

Unusual operating systems as VM guests: Haiku first, with flow-init in the guest issue #91 open andymccall opened this · 4 comments

andymccall

Andy, 2026-10-26. The stronger way to run an unusual OS than the host tier: a VM per job on a Linux runner, as the Windows runner does on Earth, with a throwaway snapshot each time.

Sketch, with Haiku (or FreeBSD) as the proof:

  • A guest image: a Haiku x86_64 disk with the toolchain from haikuports (Rust is packaged there, so a native build needs no Haiku sysroot on Linux, unlike the tier-3 cross target with -Zbuild-std), imported and pinned by hash like a rootfs.
  • Booting: QEMU/KVM (Firecracker's virtio-mmio devices are Linux-guest only; Haiku has virtio-pci drivers), with a copy-on-write overlay per job, discarded afterwards.
  • flow-init for the guest: the in-guest agent ported (it speaks to the host over virtio-serial or vsock and runs the job's steps), built for the guest's target, with the portable runner core.
  • Placement: the job asks for os "haiku"; the Linux runner advertises the guests it can boot. The receipt records the guest image hash, so provenance is as strong as for a rootfs.

Depends on the matrix (os as a placement key) and the portable runner core. Haiku first because it is the hardest case (no Linux compatibility layer); FreeBSD would follow the same path.

andymccall set labels feature ·

andymccall

Progress on the Haiku guest (mars, QEMU):

  • R1/beta6. beta5's HaikuPorts branch is retired (its repo URL redirects to an empty listing), so the guest moved to R1/beta6 (hrev59866) with pkgman add-repo …/r1beta6/x86_64/current for both repos and pkgman full-sync. pkgman downloads in the guest fail at random with "Interrupted system call"; rerunning reuses what finished.
  • Rust: HaikuPorts rust_bin 1.94.1.
  • oxy 0.8.0 builds and runs natively (init, commit, diff, log, clone over HTTPS from oxygenhub.io in 3.7 s). One build setting: ZSTD_SYS_USE_PKG_CONFIG=1, so zstd-sys links Haiku's libzstd; the bundled static zstd has hidden symbols that a system DSO also references, and ld refuses the link.
  • oh 0.9.9 builds and runs the same way.
  • flow: wasmtime 49 needs rustc 1.96 and HaikuPorts has 1.94.1, so flow does not build as pinned. Trying wasmtime 47 (rustc 1.94) in a scratch copy to see whether the runner itself ports.
andymccall

flow on Haiku, as far as it goes natively: with wasmtime 47 (rustc 1.94) and the lockfile resolved for 1.94 (kdl 6.5.0), the build stops in cap-primitives 4.0.3, wasmtime-wasi's capability file layer: rustix on Haiku has no DirEntry::file_type, no O_PATH and different st_ino/st_dev types. Porting that is upstream work in cap-std, so the in-guest runner is better done as sketched above: a Linux runner boots the Haiku guest per job and drives it over SSH or virtio-serial, with oxy (which does build there) fetching the source.

Meanwhile oxy 0.8.0 and oh 0.9.9 are released for Haiku (oxygen#154, oxygenhub#344), built in the guest.

andymccall

Decision (2026-10-27): wait for HaikuPorts to ship Rust 1.96 (what wasmtime 49 needs), then look at porting cap-primitives to Haiku so flow builds natively. Until then the VM-per-job route and a flow without wasmtime are parked; oxy and oh are released for Haiku meanwhile.

referenced by 2 changes on 28 Oct 2026

21a3ea4aab88 788f57e4d266

andymccall

2026-10-28 night: flow builds natively on Haiku R1/beta6 with HaikuPorts' Rust 1.94.1 and --no-default-features (oxygenflow 8461ad7a: the WASM tier as the wasm cargo feature; 5b20cf22: the syscall tier, the guest's job code and the load average gated on Linux/Android, noatime where statvfs has it). In the guest: flow --version, flow check and flow tiers run; flow tiers says no tier can run there (microvm and syscall are Linux's, wasm is not in this build).

How: build detached with -j 1 (two rustc at once hung twice in the 2 GB guest's kernel, unkillable; the VM was restarted through its unit), ZSTD_SYS_USE_PKG_CONFIG=1, kdl pinned to 6.5.0 for rustc 1.94.

Next, for a Haiku runner: the host tier of issue #90 (a plain process in a fresh work directory, the tier and an environment fingerprint in the receipt, refusable by policy). It needs runtime "host" in the pipeline format (flow-core), placement on the hub, and the receipt fields, so it is its own piece of work, shared with FreeBSD (issue #99).