OXYGENHUB

Architecture awareness: image manifests, arch on a sandbox, archs on the claim 1abadd4a2fa6

flow rootfs import reads the image's architecture from its shell's ELF
header, refuses an image for another architecture than this machine's,
and records it in cas/<hash>.json; a claim reports each image's
architecture so the hub offers a job only to a runner of the right
one; a sandbox may say arch "riscv64" (spec §11.2, §11.3). refs #36
Author
Andy McCall @andymccall <[email protected]>
When
· 2026-10-04 17:35 UTC
Signature
✔ signed by Andy McCall
Hash
1abadd4a2fa602606b66e9cab64df1d28a4291e796ab1484683ca4c84c786a00
Depends on
02b127d2c721 1daca0b6bb8c 720eb8191059 8c0a4e3a8a30 a083cc802892 a72abeeedd78 bf7bb1b80fa3 c3db750ed96e c4285535b03a e0918b887850 e2c125610f43 e576e08617dc fd2d4822f7f3
Depended on by
e20897216221 46fe2c815cc3 ea81796bb757 0077c7488e5d 4cf0c99a1486 6d53fb7d00ec 630ee66034b2 f53e6d47b56a
In bookmarks
main newer-pipeline push-uses-login
In tags
v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.5.0 v0.5.1 v0.5.2 v0.5.3
Edits
19 atoms, 0 tree operations, 11 files

Browse the tree at this change

modified CHANGELOG.md +2 −0
…
3 unchanged lines
4 4
5 5
## Unreleased
6 6
7 +
- **Architecture awareness (#36, spec §11.2).** `flow rootfs import` records an image's architecture from its shell's ELF header and refuses an image for another architecture; a claim reports each image's architecture, and a sandbox may say `arch "riscv64"`.
8 +
7 9
- **A job's identity (oxygenhub#103).** `identity "AUDIENCE" { mount-path "/run/secrets/…" }` asks the hub for an OIDC token for that audience, mounted as a file like a secret.
8 10
9 11
- **Secrets as files (oxygenhub#107).** A job's `secret "name" { mount-path "/run/secrets/…" }` names a secret the hub holds; it comes with the claim and `flow-init` writes it at that path, 0400, on a tmpfs, before the job starts. Never in the environment, the receipt or an artifact (spec §9). `flow run --secret NAME=VALUE` locally.
…
14 unchanged lines
modified crates/flow-core/src/api.rs +5 −0
…
27 unchanged lines
28 28
    /// offered.
29 29
    #[serde(default, skip_serializing_if = "Option::is_none")]
30 30
    pub images: Option<Vec<String>>,
31 +
    /// The architecture of each image it holds, by hash, as the image's
32 +
    /// manifest records it (spec §11.2): the hub offers a job only to a
33 +
    /// runner whose architecture its image is for.
34 +
    #[serde(default, skip_serializing_if = "Option::is_none")]
35 +
    pub image_archs: Option<std::collections::BTreeMap<String, String>>,
31 36
    /// The jobs this runner holds now: running, or finished with a result
32 37
    /// not yet reported. The hub ends any other job it has as running on
33 38
    /// this runner, which the runner lost by restarting or never received
…
190 unchanged lines
modified crates/flow-core/src/canonical.rs +4 −0
…
42 unchanged lines
43 43
    if !s.env.is_empty() {
44 44
        fields.push((6, Value::Array(s.env.iter().map(|(k, v)| Value::Array(vec![text(k), text(v)])).collect())));
45 45
    }
46 +
    // The architecture asked for; absent when the image decides.
47 +
    if let Some(a) = &s.arch {
48 +
        fields.push((7, text(a)));
49 +
    }
46 50
    Value::map(fields)
47 51
}
48 52
…
79 unchanged lines
modified crates/flow-core/src/lib.rs +3 −1
…
10 unchanged lines
11 11
pub mod receipt;
12 12
13 13
pub use canonical::{object_hash, scope_tree_hash, Hash};
14 -
pub use pipeline::{Identity, Job, Mount, MountSource, Network, Pipeline, Problem, Problems, Runtime, Sandbox, Secret, PIPELINE_PATH};
14 +
pub use pipeline::{
15 +
    arch_ok, Identity, Job, Mount, MountSource, Network, Pipeline, Problem, Problems, Runtime, Sandbox, Secret, PIPELINE_PATH,
16 +
};
15 17
pub use receipt::{Environment, Receipt};
16 18
17 19
#[cfg(test)]
…
1 unchanged lines
modified crates/flow-core/src/pipeline.rs +32 −1
…
138 unchanged lines
139 139
    /// the job does not say; the hub caps it at the site's maximum.
140 140
    #[serde(default, skip_serializing_if = "Option::is_none")]
141 141
    pub timeout_s: Option<u64>,
142 +
    /// The architecture the jobs need (`arch "riscv64"`, spec §11.3);
143 +
    /// without it, the rootfs image's.
144 +
    #[serde(default, skip_serializing_if = "Option::is_none")]
145 +
    pub arch: Option<String>,
146 +
}
147 +
148 +
/// An architecture name as runners report it: `x86_64`, `aarch64`, `riscv64`.
149 +
pub fn arch_ok(a: &str) -> bool {
150 +
    (2..=16).contains(&a.len()) && a.bytes().all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'_')
142 151
}
143 152
144 153
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
…
493 unchanged lines
638 647
    let mut env = BTreeMap::new();
639 648
    let mut runs_on = Vec::new();
640 649
    let mut timeout_s = None;
650 +
    let mut arch: Option<String> = None;
641 651
    for c in block(ctx, node) {
642 652
        let key = c.name().value();
643 653
        if key != "runs-on" && !seen.insert(key.to_string()) {
…
39 unchanged lines
683 693
                    None => None,
684 694
                }
685 695
            }
696 +
            "arch" => {
697 +
                arch = match ctx.one_string(c) {
698 +
                    Some(a) if arch_ok(&a) => Some(a),
699 +
                    Some(a) => {
700 +
                        ctx.err(c.span(), format!("arch {a:?} is not an architecture name such as x86_64, aarch64 or riscv64"));
701 +
                        None
702 +
                    }
703 +
                    None => None,
704 +
                }
705 +
            }
686 706
            "memory-limit" => {
687 707
                memory = match ctx.one_string(c) {
688 708
                    Some(m) => match parse_memory(&m) {
…
47 unchanged lines
736 756
    if !named {
737 757
        return None;
738 758
    }
739 -
    Some(Sandbox { name, runtime: runtime?, rootfs: rootfs?, memory_mib: memory?, cpus: cpus?, network: network?, env, runs_on, timeout_s })
759 +
    Some(Sandbox {
760 +
        name,
761 +
        runtime: runtime?,
762 +
        rootfs: rootfs?,
763 +
        memory_mib: memory?,
764 +
        cpus: cpus?,
765 +
        network: network?,
766 +
        env,
767 +
        runs_on,
768 +
        timeout_s,
769 +
        arch,
770 +
    })
740 771
}
741 772
742 773
/// An `on` block: `push` and `tag` with globs (none: any), `manual`,
…
453 unchanged lines
modified crates/flow-core/src/tests.rs +14 −0
…
340 unchanged lines
341 341
}
342 342
343 343
#[test]
344 +
fn a_sandbox_may_name_its_architecture() {
345 +
    let plain = Pipeline::parse(&doc("job \"b\" {\n    sandbox \"alpine\"\n    exec \"make\"\n}\n")).unwrap();
346 +
    assert_eq!(plain.sandboxes["alpine"].arch, None);
347 +
    let src =
348 +
        doc("job \"b\" {\n    sandbox \"alpine\"\n    exec \"make\"\n}\n").replace("memory-limit", "arch \"riscv64\"\n    memory-limit");
349 +
    let p = Pipeline::parse(&src).unwrap();
350 +
    assert_eq!(p.sandboxes["alpine"].arch.as_deref(), Some("riscv64"));
351 +
    assert_ne!(p.hash(), plain.hash(), "the architecture is part of the sandbox");
352 +
    let bad = src.replace("riscv64", "RISC V");
353 +
    assert!(Pipeline::parse(&bad).unwrap_err().to_string().contains("not an architecture name"));
354 +
    assert!(arch_ok("x86_64") && arch_ok("aarch64") && !arch_ok("x") && !arch_ok("A64"));
355 +
}
356 +
357 +
#[test]
344 358
fn jobs_are_told_what_they_build() {
345 359
    let p = Pipeline::parse(&doc(r#"job "build" {
346 360
    sandbox "alpine"
…
190 unchanged lines
modified crates/flow/src/home.rs +18 −0
…
113 unchanged lines
114 114
        self.root.join("cas").join(format!("{hash}.ext4"))
115 115
    }
116 116
117 +
    /// An image's manifest (`cas/<hash>.json`): what `flow rootfs import`
118 +
    /// found out about it.
119 +
    pub fn image_manifest(&self, hash: &str) -> PathBuf {
120 +
        self.root.join("cas").join(format!("{hash}.json"))
121 +
    }
122 +
123 +
    /// The architecture an image was imported for, when its manifest says.
124 +
    pub fn image_arch(&self, hash: &str) -> Option<String> {
125 +
        let text = std::fs::read_to_string(self.image_manifest(hash)).ok()?;
126 +
        let v: serde_json::Value = serde_json::from_str(&text).ok()?;
127 +
        v["arch"].as_str().map(String::from)
128 +
    }
129 +
130 +
    /// Every image with a known architecture (spec §11.2).
131 +
    pub fn image_archs(&self) -> std::collections::BTreeMap<String, String> {
132 +
        self.images().into_iter().filter_map(|h| self.image_arch(&h).map(|a| (h, a))).collect()
133 +
    }
134 +
117 135
    /// The rootfs images this home holds: the hashes of `cas/*.ext4`.
118 136
    pub fn images(&self) -> Vec<String> {
119 137
        let Ok(dir) = std::fs::read_dir(self.root.join("cas")) else {
…
93 unchanged lines
modified crates/flow/src/main.rs +45 −0
…
234 unchanged lines
235 235
    Ok(out)
236 236
}
237 237
238 +
/// The architecture an ELF file was built for, by its header.
239 +
fn elf_arch(path: &Path) -> Option<&'static str> {
240 +
    let bytes = std::fs::read(path).ok()?;
241 +
    if bytes.len() < 20 || &bytes[..4] != b"\x7fELF" {
242 +
        return None;
243 +
    }
244 +
    let little = bytes[5] == 1;
245 +
    let machine = if little { u16::from_le_bytes([bytes[18], bytes[19]]) } else { u16::from_be_bytes([bytes[18], bytes[19]]) };
246 +
    Some(match machine {
247 +
        0x3E => "x86_64",
248 +
        0xB7 => "aarch64",
249 +
        0xF3 => "riscv64",
250 +
        0x03 => "x86",
251 +
        0x28 => "arm",
252 +
        0x15 => "powerpc64",
253 +
        0x16 => "s390x",
254 +
        _ => return None,
255 +
    })
256 +
}
257 +
258 +
/// The architecture of an unpacked rootfs: its shell's, or the first
259 +
/// program's found under bin.
260 +
fn tree_arch(tree: &Path) -> Option<&'static str> {
261 +
    for rel in ["bin/sh", "bin/busybox", "usr/bin/env", "bin/bash", "usr/bin/sh"] {
262 +
        let p = tree.join(rel);
263 +
        let p = std::fs::canonicalize(&p).unwrap_or(p);
264 +
        if let Some(a) = elf_arch(&p) {
265 +
            return Some(a);
266 +
        }
267 +
    }
268 +
    None
269 +
}
270 +
238 271
fn rootfs_import(home: &Home, tarball: &Path, sha256: Option<&str>, env: &[String]) -> Result<()> {
239 272
    let env_file = image_env_file(env)?;
240 273
    if let Some(want) = sha256 {
…
11 unchanged lines
252 285
        let status = std::process::Command::new("tar").arg("-xf").arg(tarball).arg("-C").arg(&tree).arg("--no-same-owner").status()?;
253 286
        if !status.success() {
254 287
            bail!("tar could not unpack {}", tarball.display());
288 +
        }
289 +
        // The image's architecture must be this machine's: there is no
290 +
        // emulation tier (spec §11.2).
291 +
        let arch = tree_arch(&tree).unwrap_or("unknown");
292 +
        if arch != "unknown" && arch != std::env::consts::ARCH {
293 +
            bail!(
294 +
                "{} is a {arch} image; this runner is {}, and jobs here run no other architecture",
295 +
                tarball.display(),
296 +
                std::env::consts::ARCH
297 +
            );
255 298
        }
256 299
        // Mount points flow-init needs (docs/m0.md §2).
257 300
        for d in ["src", "tmp", "proc", "dev", "run"] {
…
26 unchanged lines
284 327
        }
285 328
        let hash = home::hash_file(&img)?;
286 329
        std::fs::rename(&img, home.rootfs(&hash))?;
330 +
        let manifest = serde_json::json!({ "arch": if arch == "unknown" { std::env::consts::ARCH } else { arch }, "imported_ms": std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_millis() as u64).unwrap_or(0) });
331 +
        std::fs::write(home.image_manifest(&hash), manifest.to_string())?;
287 332
        Ok(hash)
288 333
    })();
289 334
    let _ = std::fs::remove_dir_all(&tmp);
…
118 unchanged lines
modified crates/flow/src/runner.rs +1 −0
…
294 unchanged lines
295 295
                }),
296 296
                // Read each time: an image imported while the runner runs counts.
297 297
                images: Some(home.images()),
298 +
                image_archs: Some(home.image_archs()),
298 299
                holding: Some(held.lock().unwrap().iter().copied().collect()),
299 300
            };
300 301
            match hub.claim(&req) {
…
414 unchanged lines
modified docs/m0.md +7 −0
…
62 unchanged lines
63 63
  of its own, before the job starts; it is never in the environment, the
64 64
  receipt or an artifact (spec §9). Locally, `flow run --secret
65 65
  NAME=VALUE` supplies it. `vault-ref` is not in M0: the hub is the vault.
66 +
- `arch "riscv64"` on a sandbox asks for that architecture (spec §11.3);
67 +
  without it the rootfs image's decides. `flow rootfs import` reads the
68 +
  image's architecture from its shell's ELF header, refuses an image for
69 +
  another architecture than the runner's, and records it in
70 +
  `cas/<hash>.json`; a claim reports each image's architecture
71 +
  (`image_archs`) so the hub never offers a job to a runner of the wrong
72 +
  one (spec §11.2).
66 73
- `identity "sts.amazonaws.com" { mount-path "/run/secrets/aws-token" }` on
67 74
  a job (up to 4) asks the hub for an OIDC token for that audience, naming
68 75
  the repository, bookmark, run and job (OxygenHub spec §13.5,
…
164 unchanged lines
modified docs/spec.md +1 −1
…
429 unchanged lines
430 430
   MUST be rejected by policy verification.
431 431
5. Hermetic Network Isolation: Jobs configured with network "isolated" MUST
432 432
   have outgoing TCP SYN and UDP packets dropped at the hypervisor boundary.
433 -
## 11. Runner Capabilities and Placement (proposed 2026-10-04)
433 +
## 11. Runner Capabilities and Placement (proposed 2026-10-04; §11.2 and `arch` of §11.3 done the same day, oxygenflow#36)
434 434
435 435
A runner MUST never take a job it cannot run or cannot finish well: the
436 436
wrong architecture, no hardware virtualisation for the MicroVM tier, too
…
86 unchanged lines