OXYGENHUB

Spec §11: runner capabilities and placement fd2d4822f7f3

A runner's profile (architecture, tiers with KVM checked, memory, cpus,
images with their architecture, labels, a measured speed class, load),
image manifests that record architecture, arch on a sandbox and a
rootfs per architecture, placement by hard requirements then ranking,
and declining with a reason. refs #36
Author
Andy McCall @andymccall <[email protected]>
When
· 2026-10-04 16:19 UTC
Signature
✔ signed by Andy McCall
Hash
fd2d4822f7f32ad66c302ca2e144505d65a5a101aa7d889b43749f72fd9ce6e9
Depends on
a083cc802892
Depended on by
85275e1ef3b7 06a4c472b385 0077c7488e5d 89171fca079c 62829d15ef09 630ee66034b2 1abadd4a2fa6
In bookmarks
main newer-pipeline push-uses-login
In tags
v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.5.0 v0.5.1 v0.5.2 v0.5.3
Edits
2 atoms, 0 tree operations, 1 file

Browse the tree at this change

modified docs/spec.md +91 −1
…
402 unchanged lines
403 403
4. Non-Downgrade Attestation: An attestation missing the ML-DSA-65 component
404 404
   MUST be rejected by policy verification.
405 405
5. Hermetic Network Isolation: Jobs configured with network "isolated" MUST
406 -
   have outgoing TCP SYN and UDP packets dropped at the hypervisor boundary.
406 +
   have outgoing TCP SYN and UDP packets dropped at the hypervisor boundary.
407 +
## 11. Runner Capabilities and Placement (proposed 2026-10-04)
408 +
409 +
A runner MUST never take a job it cannot run or cannot finish well: the
410 +
wrong architecture, no hardware virtualisation for the MicroVM tier, too
411 +
little memory, an image it does not hold, or a machine too slow for the
412 +
job's time limit. Today a runner reports its architecture, OS, labels,
413 +
job slots, memory and vCPUs when it registers and on each claim, and a
414 +
claim carries the tiers it can run, the memory and vCPUs free now, and
415 +
the images it holds; the hub matches tier, room, labels, scope and image.
416 +
What is missing: architecture is reported but never matched (an image's
417 +
architecture is implicit in its bytes), a pipeline cannot say which
418 +
architecture it needs or offer a rootfs per architecture, speed and load
419 +
play no part, and a runner cannot decline a job it was handed.
420 +
421 +
### 11.1 Capability Profile
422 +
423 +
At registration and on every claim a runner sends its profile:
424 +
425 +
| Field | Meaning |
426 +
|---|---|
427 +
| `arch`, `os`, `kernel` | `x86_64`, `aarch64`, `riscv64`, …; the kernel release |
428 +
| `tiers` | The sandbox tiers it can honour: `microvm` only with `/dev/kvm` usable (a board without KVM, such as an OrangePi RV2, offers `syscall`, §4.2, and `wasm`) |
429 +
| `memory_mib`, `cpus`, `max_jobs` | What its jobs share |
430 +
| `images` | The rootfs images it holds, each with the architecture its manifest records (§11.2) |
431 +
| `labels` | As §3.1 |
432 +
| `speed` | A class from 1 (slow) to 5 (fast), measured at `flow runner register` and `flow runner bench` by a fixed workload (BLAKE3 over 256 MiB and the compilation of a fixed C file), bucketed so that two machines of the same class finish a job within a factor of two of each other |
433 +
| `load` | The 1-minute load average divided by `cpus`, memory free now, jobs running |
434 +
435 +
### 11.2 Image Manifests Carry the Architecture
436 +
437 +
`flow rootfs import` reads the ELF header of the image's `/bin/sh` and
438 +
records its architecture in the CAS manifest beside the hash. A runner
439 +
refuses to import an image for another architecture (there is no
440 +
emulation tier). An image imported before this section has no recorded
441 +
architecture and is treated as the importing runner's.
442 +
443 +
### 11.3 Job Requirements
444 +
445 +
Hard requirements, all of which a runner MUST satisfy to be offered the
446 +
job: the sandbox's tier, its rootfs (hence the image's architecture),
447 +
`memory-limit`, `cpus`, every `runs-on` label of the job and its sandbox,
448 +
and the runner's scope (OxygenHub spec §13.2). Two additions to the
449 +
language:
450 +
451 +
- `arch "riscv64"` on a sandbox pins the architecture; without it the
452 +
  architecture is the rootfs image's.
453 +
- A rootfs per architecture: `rootfs { x86_64 "cas://blob/…"; riscv64 "cas://blob/…" }`.
454 +
  The job runs on whichever architecture takes it; the receipt and the
455 +
  provenance record the architecture and image used, and a check's
456 +
  description names them.
457 +
458 +
Soft requirements guide placement but never block it: a job's `timeout`
459 +
against the runner's speed class. The hub keeps, per job hash and speed
460 +
class, the median duration of past runs; a runner whose class predicts a
461 +
duration over the limit is offered the job only when no better runner
462 +
is free.
463 +
464 +
### 11.4 Placement
465 +
466 +
The hub filters by the hard requirements, then ranks the runners that
467 +
remain: idle before busy (`load`), a higher speed class first, a runner
468 +
already holding the image before one that must fetch it (§5), fewer jobs
469 +
waiting on it first. A job no runner can satisfy waits and the queue
470 +
says what it waits for: tier, architecture, image, memory, labels or
471 +
scope, so the gap is visible rather than a silent stall.
472 +
473 +
### 11.5 Declining
474 +
475 +
A runner double-checks a claim before it starts: the architecture, that
476 +
`/dev/kvm` opens for the MicroVM tier, that the image's blob is present
477 +
and verifies, that the memory is free. If any check fails it answers the
478 +
hub with `declined` and a reason instead of running and failing; the hub
479 +
re-queues the job, never offers it to that runner again until its
480 +
profile changes, records the decline on the runner's page, and after
481 +
three declines by different runners marks the job `unplaceable` with the
482 +
reasons. A decline is never a failed check on the state.
483 +
484 +
### 11.6 Conformance
485 +
486 +
1. Architecture Safety: a job whose image is `x86_64` MUST never be
487 +
   offered to a `riscv64` runner, with or without the image copied there.
488 +
2. Tier Safety: a runner without `/dev/kvm` MUST NOT be offered a
489 +
   MicroVM job, and MUST decline one if handed it.
490 +
3. Room: a job whose `memory-limit` exceeds the runner's free memory
491 +
   MUST wait, not be killed.
492 +
4. Visibility: a job nobody can take MUST say why on the queue within
493 +
   one scheduling pass.
494 +
5. Preference: with two idle runners that both satisfy a job, the hub
495 +
   MUST offer it to the higher speed class, and to the one holding the
496 +
   image when classes tie.