Architecture awareness: image manifests, arch on a sandbox, archs on the claim 1abadd4a2fa6
flow rootfs import reads the image's architecture from its shell's ELF header, refuses an image for another architecture than this machine's, and records it in cas/<hash>.json; a claim reports each image's architecture so the hub offers a job only to a runner of the right one; a sandbox may say arch "riscv64" (spec §11.2, §11.3). refs #36
- Author
- Andy McCall @andymccall <[email protected]>
- When
- · 2026-10-04 17:35 UTC
- Signature
- ✔ signed by Andy McCall
- Hash
1abadd4a2fa602606b66e9cab64df1d28a4291e796ab1484683ca4c84c786a00- Depends on
- 02b127d2c721 1daca0b6bb8c 720eb8191059 8c0a4e3a8a30 a083cc802892 a72abeeedd78 bf7bb1b80fa3 c3db750ed96e c4285535b03a e0918b887850 e2c125610f43 e576e08617dc fd2d4822f7f3
- Depended on by
- e20897216221 46fe2c815cc3 ea81796bb757 0077c7488e5d 4cf0c99a1486 6d53fb7d00ec 630ee66034b2 f53e6d47b56a
- In bookmarks
- main newer-pipeline push-uses-login
- In tags
- v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.5.0 v0.5.1 v0.5.2 v0.5.3
- Edits
- 19 atoms, 0 tree operations, 11 files
Browse the tree at this change
modified CHANGELOG.md +2 −0
| … | 3 unchanged lines |
||
| 4 | 4 | ||
| 5 | 5 | ## Unreleased |
|
| 6 | 6 | ||
| 7 | + | - **Architecture awareness (#36, spec §11.2).** `flow rootfs import` records an image's architecture from its shell's ELF header and refuses an image for another architecture; a claim reports each image's architecture, and a sandbox may say `arch "riscv64"`. |
|
| 8 | + | ||
| 7 | 9 | - **A job's identity (oxygenhub#103).** `identity "AUDIENCE" { mount-path "/run/secrets/…" }` asks the hub for an OIDC token for that audience, mounted as a file like a secret. |
|
| 8 | 10 | ||
| 9 | 11 | - **Secrets as files (oxygenhub#107).** A job's `secret "name" { mount-path "/run/secrets/…" }` names a secret the hub holds; it comes with the claim and `flow-init` writes it at that path, 0400, on a tmpfs, before the job starts. Never in the environment, the receipt or an artifact (spec §9). `flow run --secret NAME=VALUE` locally. |
|
| … | 14 unchanged lines |
modified crates/flow-core/src/api.rs +5 −0
| … | 27 unchanged lines |
||
| 28 | 28 | /// offered. |
|
| 29 | 29 | #[serde(default, skip_serializing_if = "Option::is_none")] |
|
| 30 | 30 | pub images: Option<Vec<String>>, |
|
| 31 | + | /// The architecture of each image it holds, by hash, as the image's |
|
| 32 | + | /// manifest records it (spec §11.2): the hub offers a job only to a |
|
| 33 | + | /// runner whose architecture its image is for. |
|
| 34 | + | #[serde(default, skip_serializing_if = "Option::is_none")] |
|
| 35 | + | pub image_archs: Option<std::collections::BTreeMap<String, String>>, |
|
| 31 | 36 | /// The jobs this runner holds now: running, or finished with a result |
|
| 32 | 37 | /// not yet reported. The hub ends any other job it has as running on |
|
| 33 | 38 | /// this runner, which the runner lost by restarting or never received |
|
| … | 190 unchanged lines |
modified crates/flow-core/src/canonical.rs +4 −0
| … | 42 unchanged lines |
||
| 43 | 43 | if !s.env.is_empty() { |
|
| 44 | 44 | fields.push((6, Value::Array(s.env.iter().map(|(k, v)| Value::Array(vec![text(k), text(v)])).collect()))); |
|
| 45 | 45 | } |
|
| 46 | + | // The architecture asked for; absent when the image decides. |
|
| 47 | + | if let Some(a) = &s.arch { |
|
| 48 | + | fields.push((7, text(a))); |
|
| 49 | + | } |
|
| 46 | 50 | Value::map(fields) |
|
| 47 | 51 | } |
|
| 48 | 52 | ||
| … | 79 unchanged lines |
modified crates/flow-core/src/lib.rs +3 −1
| … | 10 unchanged lines |
||
| 11 | 11 | pub mod receipt; |
|
| 12 | 12 | ||
| 13 | 13 | pub use canonical::{object_hash, scope_tree_hash, Hash}; |
|
| 14 | - | pub use pipeline::{Identity, Job, Mount, MountSource, Network, Pipeline, Problem, Problems, Runtime, Sandbox, Secret, PIPELINE_PATH}; |
|
| 14 | + | pub use pipeline::{ |
|
| 15 | + | arch_ok, Identity, Job, Mount, MountSource, Network, Pipeline, Problem, Problems, Runtime, Sandbox, Secret, PIPELINE_PATH, |
|
| 16 | + | }; |
|
| 15 | 17 | pub use receipt::{Environment, Receipt}; |
|
| 16 | 18 | ||
| 17 | 19 | #[cfg(test)] |
|
| … | 1 unchanged lines |
modified crates/flow-core/src/pipeline.rs +32 −1
| … | 138 unchanged lines |
||
| 139 | 139 | /// the job does not say; the hub caps it at the site's maximum. |
|
| 140 | 140 | #[serde(default, skip_serializing_if = "Option::is_none")] |
|
| 141 | 141 | pub timeout_s: Option<u64>, |
|
| 142 | + | /// The architecture the jobs need (`arch "riscv64"`, spec §11.3); |
|
| 143 | + | /// without it, the rootfs image's. |
|
| 144 | + | #[serde(default, skip_serializing_if = "Option::is_none")] |
|
| 145 | + | pub arch: Option<String>, |
|
| 146 | + | } |
|
| 147 | + | ||
| 148 | + | /// An architecture name as runners report it: `x86_64`, `aarch64`, `riscv64`. |
|
| 149 | + | pub fn arch_ok(a: &str) -> bool { |
|
| 150 | + | (2..=16).contains(&a.len()) && a.bytes().all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'_') |
|
| 142 | 151 | } |
|
| 143 | 152 | ||
| 144 | 153 | #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] |
|
| … | 493 unchanged lines |
||
| 638 | 647 | let mut env = BTreeMap::new(); |
|
| 639 | 648 | let mut runs_on = Vec::new(); |
|
| 640 | 649 | let mut timeout_s = None; |
|
| 650 | + | let mut arch: Option<String> = None; |
|
| 641 | 651 | for c in block(ctx, node) { |
|
| 642 | 652 | let key = c.name().value(); |
|
| 643 | 653 | if key != "runs-on" && !seen.insert(key.to_string()) { |
|
| … | 39 unchanged lines |
||
| 683 | 693 | None => None, |
|
| 684 | 694 | } |
|
| 685 | 695 | } |
|
| 696 | + | "arch" => { |
|
| 697 | + | arch = match ctx.one_string(c) { |
|
| 698 | + | Some(a) if arch_ok(&a) => Some(a), |
|
| 699 | + | Some(a) => { |
|
| 700 | + | ctx.err(c.span(), format!("arch {a:?} is not an architecture name such as x86_64, aarch64 or riscv64")); |
|
| 701 | + | None |
|
| 702 | + | } |
|
| 703 | + | None => None, |
|
| 704 | + | } |
|
| 705 | + | } |
|
| 686 | 706 | "memory-limit" => { |
|
| 687 | 707 | memory = match ctx.one_string(c) { |
|
| 688 | 708 | Some(m) => match parse_memory(&m) { |
|
| … | 47 unchanged lines |
||
| 736 | 756 | if !named { |
|
| 737 | 757 | return None; |
|
| 738 | 758 | } |
|
| 739 | - | Some(Sandbox { name, runtime: runtime?, rootfs: rootfs?, memory_mib: memory?, cpus: cpus?, network: network?, env, runs_on, timeout_s }) |
|
| 759 | + | Some(Sandbox { |
|
| 760 | + | name, |
|
| 761 | + | runtime: runtime?, |
|
| 762 | + | rootfs: rootfs?, |
|
| 763 | + | memory_mib: memory?, |
|
| 764 | + | cpus: cpus?, |
|
| 765 | + | network: network?, |
|
| 766 | + | env, |
|
| 767 | + | runs_on, |
|
| 768 | + | timeout_s, |
|
| 769 | + | arch, |
|
| 770 | + | }) |
|
| 740 | 771 | } |
|
| 741 | 772 | ||
| 742 | 773 | /// An `on` block: `push` and `tag` with globs (none: any), `manual`, |
|
| … | 453 unchanged lines |
modified crates/flow-core/src/tests.rs +14 −0
| … | 340 unchanged lines |
||
| 341 | 341 | } |
|
| 342 | 342 | ||
| 343 | 343 | #[test] |
|
| 344 | + | fn a_sandbox_may_name_its_architecture() { |
|
| 345 | + | let plain = Pipeline::parse(&doc("job \"b\" {\n sandbox \"alpine\"\n exec \"make\"\n}\n")).unwrap(); |
|
| 346 | + | assert_eq!(plain.sandboxes["alpine"].arch, None); |
|
| 347 | + | let src = |
|
| 348 | + | doc("job \"b\" {\n sandbox \"alpine\"\n exec \"make\"\n}\n").replace("memory-limit", "arch \"riscv64\"\n memory-limit"); |
|
| 349 | + | let p = Pipeline::parse(&src).unwrap(); |
|
| 350 | + | assert_eq!(p.sandboxes["alpine"].arch.as_deref(), Some("riscv64")); |
|
| 351 | + | assert_ne!(p.hash(), plain.hash(), "the architecture is part of the sandbox"); |
|
| 352 | + | let bad = src.replace("riscv64", "RISC V"); |
|
| 353 | + | assert!(Pipeline::parse(&bad).unwrap_err().to_string().contains("not an architecture name")); |
|
| 354 | + | assert!(arch_ok("x86_64") && arch_ok("aarch64") && !arch_ok("x") && !arch_ok("A64")); |
|
| 355 | + | } |
|
| 356 | + | ||
| 357 | + | #[test] |
|
| 344 | 358 | fn jobs_are_told_what_they_build() { |
|
| 345 | 359 | let p = Pipeline::parse(&doc(r#"job "build" { |
|
| 346 | 360 | sandbox "alpine" |
|
| … | 190 unchanged lines |
modified crates/flow/src/home.rs +18 −0
| … | 113 unchanged lines |
||
| 114 | 114 | self.root.join("cas").join(format!("{hash}.ext4")) |
|
| 115 | 115 | } |
|
| 116 | 116 | ||
| 117 | + | /// An image's manifest (`cas/<hash>.json`): what `flow rootfs import` |
|
| 118 | + | /// found out about it. |
|
| 119 | + | pub fn image_manifest(&self, hash: &str) -> PathBuf { |
|
| 120 | + | self.root.join("cas").join(format!("{hash}.json")) |
|
| 121 | + | } |
|
| 122 | + | ||
| 123 | + | /// The architecture an image was imported for, when its manifest says. |
|
| 124 | + | pub fn image_arch(&self, hash: &str) -> Option<String> { |
|
| 125 | + | let text = std::fs::read_to_string(self.image_manifest(hash)).ok()?; |
|
| 126 | + | let v: serde_json::Value = serde_json::from_str(&text).ok()?; |
|
| 127 | + | v["arch"].as_str().map(String::from) |
|
| 128 | + | } |
|
| 129 | + | ||
| 130 | + | /// Every image with a known architecture (spec §11.2). |
|
| 131 | + | pub fn image_archs(&self) -> std::collections::BTreeMap<String, String> { |
|
| 132 | + | self.images().into_iter().filter_map(|h| self.image_arch(&h).map(|a| (h, a))).collect() |
|
| 133 | + | } |
|
| 134 | + | ||
| 117 | 135 | /// The rootfs images this home holds: the hashes of `cas/*.ext4`. |
|
| 118 | 136 | pub fn images(&self) -> Vec<String> { |
|
| 119 | 137 | let Ok(dir) = std::fs::read_dir(self.root.join("cas")) else { |
|
| … | 93 unchanged lines |
modified crates/flow/src/main.rs +45 −0
| … | 234 unchanged lines |
||
| 235 | 235 | Ok(out) |
|
| 236 | 236 | } |
|
| 237 | 237 | ||
| 238 | + | /// The architecture an ELF file was built for, by its header. |
|
| 239 | + | fn elf_arch(path: &Path) -> Option<&'static str> { |
|
| 240 | + | let bytes = std::fs::read(path).ok()?; |
|
| 241 | + | if bytes.len() < 20 || &bytes[..4] != b"\x7fELF" { |
|
| 242 | + | return None; |
|
| 243 | + | } |
|
| 244 | + | let little = bytes[5] == 1; |
|
| 245 | + | let machine = if little { u16::from_le_bytes([bytes[18], bytes[19]]) } else { u16::from_be_bytes([bytes[18], bytes[19]]) }; |
|
| 246 | + | Some(match machine { |
|
| 247 | + | 0x3E => "x86_64", |
|
| 248 | + | 0xB7 => "aarch64", |
|
| 249 | + | 0xF3 => "riscv64", |
|
| 250 | + | 0x03 => "x86", |
|
| 251 | + | 0x28 => "arm", |
|
| 252 | + | 0x15 => "powerpc64", |
|
| 253 | + | 0x16 => "s390x", |
|
| 254 | + | _ => return None, |
|
| 255 | + | }) |
|
| 256 | + | } |
|
| 257 | + | ||
| 258 | + | /// The architecture of an unpacked rootfs: its shell's, or the first |
|
| 259 | + | /// program's found under bin. |
|
| 260 | + | fn tree_arch(tree: &Path) -> Option<&'static str> { |
|
| 261 | + | for rel in ["bin/sh", "bin/busybox", "usr/bin/env", "bin/bash", "usr/bin/sh"] { |
|
| 262 | + | let p = tree.join(rel); |
|
| 263 | + | let p = std::fs::canonicalize(&p).unwrap_or(p); |
|
| 264 | + | if let Some(a) = elf_arch(&p) { |
|
| 265 | + | return Some(a); |
|
| 266 | + | } |
|
| 267 | + | } |
|
| 268 | + | None |
|
| 269 | + | } |
|
| 270 | + | ||
| 238 | 271 | fn rootfs_import(home: &Home, tarball: &Path, sha256: Option<&str>, env: &[String]) -> Result<()> { |
|
| 239 | 272 | let env_file = image_env_file(env)?; |
|
| 240 | 273 | if let Some(want) = sha256 { |
|
| … | 11 unchanged lines |
||
| 252 | 285 | let status = std::process::Command::new("tar").arg("-xf").arg(tarball).arg("-C").arg(&tree).arg("--no-same-owner").status()?; |
|
| 253 | 286 | if !status.success() { |
|
| 254 | 287 | bail!("tar could not unpack {}", tarball.display()); |
|
| 288 | + | } |
|
| 289 | + | // The image's architecture must be this machine's: there is no |
|
| 290 | + | // emulation tier (spec §11.2). |
|
| 291 | + | let arch = tree_arch(&tree).unwrap_or("unknown"); |
|
| 292 | + | if arch != "unknown" && arch != std::env::consts::ARCH { |
|
| 293 | + | bail!( |
|
| 294 | + | "{} is a {arch} image; this runner is {}, and jobs here run no other architecture", |
|
| 295 | + | tarball.display(), |
|
| 296 | + | std::env::consts::ARCH |
|
| 297 | + | ); |
|
| 255 | 298 | } |
|
| 256 | 299 | // Mount points flow-init needs (docs/m0.md §2). |
|
| 257 | 300 | for d in ["src", "tmp", "proc", "dev", "run"] { |
|
| … | 26 unchanged lines |
||
| 284 | 327 | } |
|
| 285 | 328 | let hash = home::hash_file(&img)?; |
|
| 286 | 329 | std::fs::rename(&img, home.rootfs(&hash))?; |
|
| 330 | + | let manifest = serde_json::json!({ "arch": if arch == "unknown" { std::env::consts::ARCH } else { arch }, "imported_ms": std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_millis() as u64).unwrap_or(0) }); |
|
| 331 | + | std::fs::write(home.image_manifest(&hash), manifest.to_string())?; |
|
| 287 | 332 | Ok(hash) |
|
| 288 | 333 | })(); |
|
| 289 | 334 | let _ = std::fs::remove_dir_all(&tmp); |
|
| … | 118 unchanged lines |
modified crates/flow/src/runner.rs +1 −0
| … | 294 unchanged lines |
||
| 295 | 295 | }), |
|
| 296 | 296 | // Read each time: an image imported while the runner runs counts. |
|
| 297 | 297 | images: Some(home.images()), |
|
| 298 | + | image_archs: Some(home.image_archs()), |
|
| 298 | 299 | holding: Some(held.lock().unwrap().iter().copied().collect()), |
|
| 299 | 300 | }; |
|
| 300 | 301 | match hub.claim(&req) { |
|
| … | 414 unchanged lines |
modified docs/m0.md +7 −0
| … | 62 unchanged lines |
||
| 63 | 63 | of its own, before the job starts; it is never in the environment, the |
|
| 64 | 64 | receipt or an artifact (spec §9). Locally, `flow run --secret |
|
| 65 | 65 | NAME=VALUE` supplies it. `vault-ref` is not in M0: the hub is the vault. |
|
| 66 | + | - `arch "riscv64"` on a sandbox asks for that architecture (spec §11.3); |
|
| 67 | + | without it the rootfs image's decides. `flow rootfs import` reads the |
|
| 68 | + | image's architecture from its shell's ELF header, refuses an image for |
|
| 69 | + | another architecture than the runner's, and records it in |
|
| 70 | + | `cas/<hash>.json`; a claim reports each image's architecture |
|
| 71 | + | (`image_archs`) so the hub never offers a job to a runner of the wrong |
|
| 72 | + | one (spec §11.2). |
|
| 66 | 73 | - `identity "sts.amazonaws.com" { mount-path "/run/secrets/aws-token" }` on |
|
| 67 | 74 | a job (up to 4) asks the hub for an OIDC token for that audience, naming |
|
| 68 | 75 | the repository, bookmark, run and job (OxygenHub spec §13.5, |
|
| … | 164 unchanged lines |
modified docs/spec.md +1 −1
| … | 429 unchanged lines |
||
| 430 | 430 | MUST be rejected by policy verification. |
|
| 431 | 431 | 5. Hermetic Network Isolation: Jobs configured with network "isolated" MUST |
|
| 432 | 432 | have outgoing TCP SYN and UDP packets dropped at the hypervisor boundary. |
|
| 433 | - | ## 11. Runner Capabilities and Placement (proposed 2026-10-04) |
|
| 433 | + | ## 11. Runner Capabilities and Placement (proposed 2026-10-04; §11.2 and `arch` of §11.3 done the same day, oxygenflow#36) |
|
| 434 | 434 | ||
| 435 | 435 | A runner MUST never take a job it cannot run or cannot finish well: the |
|
| 436 | 436 | wrong architecture, no hardware virtualisation for the MicroVM tier, too |
|
| … | 86 unchanged lines |