Spec §11: runner capabilities and placement fd2d4822f7f3
A runner's profile (architecture, tiers with KVM checked, memory, cpus, images with their architecture, labels, a measured speed class, load), image manifests that record architecture, arch on a sandbox and a rootfs per architecture, placement by hard requirements then ranking, and declining with a reason. refs #36
- Author
- Andy McCall @andymccall <[email protected]>
- When
- · 2026-10-04 16:19 UTC
- Signature
- ✔ signed by Andy McCall
- Hash
fd2d4822f7f32ad66c302ca2e144505d65a5a101aa7d889b43749f72fd9ce6e9- Depends on
- a083cc802892
- Depended on by
- 85275e1ef3b7 06a4c472b385 0077c7488e5d 89171fca079c 62829d15ef09 630ee66034b2 1abadd4a2fa6
- In bookmarks
- main newer-pipeline push-uses-login
- In tags
- v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.5.0 v0.5.1 v0.5.2 v0.5.3
- Edits
- 2 atoms, 0 tree operations, 1 file
Browse the tree at this change
modified docs/spec.md +91 −1
| … | 402 unchanged lines |
||
| 403 | 403 | 4. Non-Downgrade Attestation: An attestation missing the ML-DSA-65 component |
|
| 404 | 404 | MUST be rejected by policy verification. |
|
| 405 | 405 | 5. Hermetic Network Isolation: Jobs configured with network "isolated" MUST |
|
| 406 | - | have outgoing TCP SYN and UDP packets dropped at the hypervisor boundary. |
|
| 406 | + | have outgoing TCP SYN and UDP packets dropped at the hypervisor boundary. |
|
| 407 | + | ## 11. Runner Capabilities and Placement (proposed 2026-10-04) |
|
| 408 | + | ||
| 409 | + | A runner MUST never take a job it cannot run or cannot finish well: the |
|
| 410 | + | wrong architecture, no hardware virtualisation for the MicroVM tier, too |
|
| 411 | + | little memory, an image it does not hold, or a machine too slow for the |
|
| 412 | + | job's time limit. Today a runner reports its architecture, OS, labels, |
|
| 413 | + | job slots, memory and vCPUs when it registers and on each claim, and a |
|
| 414 | + | claim carries the tiers it can run, the memory and vCPUs free now, and |
|
| 415 | + | the images it holds; the hub matches tier, room, labels, scope and image. |
|
| 416 | + | What is missing: architecture is reported but never matched (an image's |
|
| 417 | + | architecture is implicit in its bytes), a pipeline cannot say which |
|
| 418 | + | architecture it needs or offer a rootfs per architecture, speed and load |
|
| 419 | + | play no part, and a runner cannot decline a job it was handed. |
|
| 420 | + | ||
| 421 | + | ### 11.1 Capability Profile |
|
| 422 | + | ||
| 423 | + | At registration and on every claim a runner sends its profile: |
|
| 424 | + | ||
| 425 | + | | Field | Meaning | |
|
| 426 | + | |---|---| |
|
| 427 | + | | `arch`, `os`, `kernel` | `x86_64`, `aarch64`, `riscv64`, …; the kernel release | |
|
| 428 | + | | `tiers` | The sandbox tiers it can honour: `microvm` only with `/dev/kvm` usable (a board without KVM, such as an OrangePi RV2, offers `syscall`, §4.2, and `wasm`) | |
|
| 429 | + | | `memory_mib`, `cpus`, `max_jobs` | What its jobs share | |
|
| 430 | + | | `images` | The rootfs images it holds, each with the architecture its manifest records (§11.2) | |
|
| 431 | + | | `labels` | As §3.1 | |
|
| 432 | + | | `speed` | A class from 1 (slow) to 5 (fast), measured at `flow runner register` and `flow runner bench` by a fixed workload (BLAKE3 over 256 MiB and the compilation of a fixed C file), bucketed so that two machines of the same class finish a job within a factor of two of each other | |
|
| 433 | + | | `load` | The 1-minute load average divided by `cpus`, memory free now, jobs running | |
|
| 434 | + | ||
| 435 | + | ### 11.2 Image Manifests Carry the Architecture |
|
| 436 | + | ||
| 437 | + | `flow rootfs import` reads the ELF header of the image's `/bin/sh` and |
|
| 438 | + | records its architecture in the CAS manifest beside the hash. A runner |
|
| 439 | + | refuses to import an image for another architecture (there is no |
|
| 440 | + | emulation tier). An image imported before this section has no recorded |
|
| 441 | + | architecture and is treated as the importing runner's. |
|
| 442 | + | ||
| 443 | + | ### 11.3 Job Requirements |
|
| 444 | + | ||
| 445 | + | Hard requirements, all of which a runner MUST satisfy to be offered the |
|
| 446 | + | job: the sandbox's tier, its rootfs (hence the image's architecture), |
|
| 447 | + | `memory-limit`, `cpus`, every `runs-on` label of the job and its sandbox, |
|
| 448 | + | and the runner's scope (OxygenHub spec §13.2). Two additions to the |
|
| 449 | + | language: |
|
| 450 | + | ||
| 451 | + | - `arch "riscv64"` on a sandbox pins the architecture; without it the |
|
| 452 | + | architecture is the rootfs image's. |
|
| 453 | + | - A rootfs per architecture: `rootfs { x86_64 "cas://blob/…"; riscv64 "cas://blob/…" }`. |
|
| 454 | + | The job runs on whichever architecture takes it; the receipt and the |
|
| 455 | + | provenance record the architecture and image used, and a check's |
|
| 456 | + | description names them. |
|
| 457 | + | ||
| 458 | + | Soft requirements guide placement but never block it: a job's `timeout` |
|
| 459 | + | against the runner's speed class. The hub keeps, per job hash and speed |
|
| 460 | + | class, the median duration of past runs; a runner whose class predicts a |
|
| 461 | + | duration over the limit is offered the job only when no better runner |
|
| 462 | + | is free. |
|
| 463 | + | ||
| 464 | + | ### 11.4 Placement |
|
| 465 | + | ||
| 466 | + | The hub filters by the hard requirements, then ranks the runners that |
|
| 467 | + | remain: idle before busy (`load`), a higher speed class first, a runner |
|
| 468 | + | already holding the image before one that must fetch it (§5), fewer jobs |
|
| 469 | + | waiting on it first. A job no runner can satisfy waits and the queue |
|
| 470 | + | says what it waits for: tier, architecture, image, memory, labels or |
|
| 471 | + | scope, so the gap is visible rather than a silent stall. |
|
| 472 | + | ||
| 473 | + | ### 11.5 Declining |
|
| 474 | + | ||
| 475 | + | A runner double-checks a claim before it starts: the architecture, that |
|
| 476 | + | `/dev/kvm` opens for the MicroVM tier, that the image's blob is present |
|
| 477 | + | and verifies, that the memory is free. If any check fails it answers the |
|
| 478 | + | hub with `declined` and a reason instead of running and failing; the hub |
|
| 479 | + | re-queues the job, never offers it to that runner again until its |
|
| 480 | + | profile changes, records the decline on the runner's page, and after |
|
| 481 | + | three declines by different runners marks the job `unplaceable` with the |
|
| 482 | + | reasons. A decline is never a failed check on the state. |
|
| 483 | + | ||
| 484 | + | ### 11.6 Conformance |
|
| 485 | + | ||
| 486 | + | 1. Architecture Safety: a job whose image is `x86_64` MUST never be |
|
| 487 | + | offered to a `riscv64` runner, with or without the image copied there. |
|
| 488 | + | 2. Tier Safety: a runner without `/dev/kvm` MUST NOT be offered a |
|
| 489 | + | MicroVM job, and MUST decline one if handed it. |
|
| 490 | + | 3. Room: a job whose `memory-limit` exceeds the runner's free memory |
|
| 491 | + | MUST wait, not be killed. |
|
| 492 | + | 4. Visibility: a job nobody can take MUST say why on the queue within |
|
| 493 | + | one scheduling pass. |
|
| 494 | + | 5. Preference: with two idle runners that both satisfy a job, the hub |
|
| 495 | + | MUST offer it to the higher speed class, and to the one holding the |
|
| 496 | + | image when classes tie. |