OXYGENHUB

A job's identity: identity blocks name an audience and a mount path 720eb8191059

identity "AUDIENCE" { mount-path "/run/secrets/…" } asks the hub for an
OIDC token for that audience, naming the repository, bookmark, run and
job; it comes with the claim and mounts exactly as a secret does. A
local run says the file will be missing. refs oxygenhub#103
Author
Andy McCall @andymccall <[email protected]>
When
· 2026-10-04 16:26 UTC
Signature
✔ signed by Andy McCall
Hash
720eb81910593912de76f3721156c812858801460b73767233b2429c9ec5a4e0
Depends on
a67b97e9a717 c3db750ed96e c4285535b03a c58efc6a74db e0918b887850 e2c125610f43 e576e08617dc fc10d9e80383
Depended on by
f201e1bc5bcd 46fe2c815cc3 1abadd4a2fa6
In bookmarks
main newer-pipeline push-uses-login
In tags
v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.5.0 v0.5.1 v0.5.2 v0.5.3
Edits
20 atoms, 0 tree operations, 8 files

Browse the tree at this change

modified CHANGELOG.md +2 −0
…
3 unchanged lines
4 4
5 5
## Unreleased
6 6
7 +
- **A job's identity (oxygenhub#103).** `identity "AUDIENCE" { mount-path "/run/secrets/…" }` asks the hub for an OIDC token for that audience, mounted as a file like a secret.
8 +
7 9
- **Secrets as files (oxygenhub#107).** A job's `secret "name" { mount-path "/run/secrets/…" }` names a secret the hub holds; it comes with the claim and `flow-init` writes it at that path, 0400, on a tmpfs, before the job starts. Never in the environment, the receipt or an artifact (spec §9). `flow run --secret NAME=VALUE` locally.
8 10
9 11
- **Variables from the hub (oxygenhub#102).** A claim carries `env`, the variables the hub holds for the repository; they join the job's environment under `FLOW_*` and the pipeline's own `env`. `flow run --var NAME=VALUE` does the same locally.
…
12 unchanged lines
modified crates/flow-core/src/canonical.rs +4 −0
…
76 unchanged lines
77 77
    if !j.secrets.is_empty() {
78 78
        fields.push((8, Value::Array(j.secrets.iter().map(|s| Value::Array(vec![text(&s.name), text(&s.mount_path)])).collect())));
79 79
    }
80 +
    // Identities as [audience, mount path] sorted by audience; absent when none.
81 +
    if !j.identities.is_empty() {
82 +
        fields.push((9, Value::Array(j.identities.iter().map(|i| Value::Array(vec![text(&i.audience), text(&i.mount_path)])).collect())));
83 +
    }
80 84
    Value::map(fields)
81 85
}
82 86
…
41 unchanged lines
modified crates/flow-core/src/lib.rs +1 −1
…
9 unchanged lines
10 10
pub mod receipt;
11 11
12 12
pub use canonical::{object_hash, scope_tree_hash, Hash};
13 -
pub use pipeline::{Job, Mount, MountSource, Network, Pipeline, Problem, Problems, Runtime, Sandbox, Secret, PIPELINE_PATH};
13 +
pub use pipeline::{Identity, Job, Mount, MountSource, Network, Pipeline, Problem, Problems, Runtime, Sandbox, Secret, PIPELINE_PATH};
14 14
pub use receipt::{Environment, Receipt};
15 15
16 16
#[cfg(test)]
…
1 unchanged lines
modified crates/flow-core/src/pipeline.rs +72 −3
…
99 unchanged lines
100 100
    /// supplies each by name, the runner mounts it as a file.
101 101
    #[serde(default, skip_serializing_if = "Vec::is_empty")]
102 102
    pub secrets: Vec<Secret>,
103 +
    /// Identities the job asks the hub for (oxygenhub#103): an OIDC token
104 +
    /// for an audience, mounted as a file like a secret.
105 +
    #[serde(default, skip_serializing_if = "Vec::is_empty")]
106 +
    pub identities: Vec<Identity>,
103 107
    /// Labels a runner must have to take this job (`runs-on`), sorted;
104 108
    /// with its sandbox's, what the hub matches against a runner's.
105 109
    #[serde(default, skip_serializing_if = "Vec::is_empty")]
…
78 unchanged lines
184 188
    pub mount_path: String,
185 189
}
186 190
191 +
/// An OIDC token the hub mints for the job (oxygenhub#103), naming the
192 +
/// repository, bookmark, run and job, for `audience`; read as a file.
193 +
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
194 +
pub struct Identity {
195 +
    /// Who the token is for (`sts.amazonaws.com`, a Vault role's audience, …).
196 +
    pub audience: String,
197 +
    /// A clean absolute path under `/run/secrets/`, unique among the
198 +
    /// job's secrets and identities.
199 +
    pub mount_path: String,
200 +
}
201 +
187 202
/// A secret's mount path: under `/run/secrets/`, no `.` or `..`.
188 203
pub fn secret_path_ok(p: &str) -> bool {
189 204
    p.strip_prefix("/run/secrets/").is_some_and(|rest| !rest.is_empty() && rest.split('/').all(|c| !c.is_empty() && c != "." && c != ".."))
…
16 unchanged lines
206 221
pub const MAX_MOUNTS: usize = 4;
207 222
/// The most secrets a job may mount.
208 223
pub const MAX_SECRETS: usize = 8;
224 +
/// The most identities a job may ask for.
225 +
pub const MAX_IDENTITIES: usize = 4;
209 226
/// Sizes a mount may have: 16 MiB to 64 GiB.
210 227
pub const MIN_MOUNT_MIB: u64 = 16;
211 228
pub const MAX_MOUNT_MIB: u64 = 64 * 1024;
…
626 unchanged lines
838 855
    };
839 856
    if job.secrets.iter().any(|s| s.name == name) {
840 857
        ctx.err(node.span(), format!("secret {name:?} is given twice"));
841 -
    } else if job.secrets.iter().any(|s| s.mount_path == mount_path) {
842 -
        ctx.err(node.span(), format!("two secrets mount at {mount_path:?}"));
858 +
    } else if job.secrets.iter().any(|s| s.mount_path == mount_path) || job.identities.iter().any(|i| i.mount_path == mount_path) {
859 +
        ctx.err(node.span(), format!("two files mount at {mount_path:?}"));
843 860
    } else if job.secrets.len() == MAX_SECRETS {
844 861
        ctx.err(node.span(), format!("a job has at most {MAX_SECRETS} secrets"));
845 862
    } else {
846 863
        job.secrets.push(Secret { name, mount_path });
864 +
    }
865 +
}
866 +
867 +
/// `identity "sts.amazonaws.com" { mount-path "/run/secrets/aws-token" }`:
868 +
/// the hub mints an OIDC token for that audience and the runner mounts it
869 +
/// as a file (oxygenhub#103).
870 +
fn parse_identity(ctx: &mut Ctx, node: &KdlNode, job: &mut Job) {
871 +
    let Some(audience) = ctx.one_string(node) else {
872 +
        ctx.err(node.span(), "`identity` takes an audience and a block with mount-path");
873 +
        return;
874 +
    };
875 +
    let aud_ok = !audience.is_empty() && audience.len() <= 255 && !audience.chars().any(|c| c.is_whitespace() || c.is_control());
876 +
    if !aud_ok {
877 +
        ctx.err(node.span(), format!("identity audience {audience:?} must be 1 to 255 characters without spaces"));
878 +
        return;
879 +
    }
880 +
    let mut path = None;
881 +
    for c in block(ctx, node) {
882 +
        ctx.no_children(c);
883 +
        match c.name().value() {
884 +
            "mount-path" => match ctx.one_string(c) {
885 +
                Some(p) if secret_path_ok(&p) => path = Some(p),
886 +
                Some(p) => {
887 +
                    ctx.err(c.span(), format!("identity mount-path {p:?} must be an absolute path under /run/secrets/, without . or .."))
888 +
                }
889 +
                None => {}
890 +
            },
891 +
            other => ctx.err(c.span(), format!("unknown identity setting `{other}`")),
892 +
        }
893 +
    }
894 +
    let Some(mount_path) = path else {
895 +
        ctx.err(node.span(), format!("identity {audience:?} needs a mount-path"));
896 +
        return;
897 +
    };
898 +
    if job.identities.iter().any(|i| i.audience == audience) {
899 +
        ctx.err(node.span(), format!("identity {audience:?} is given twice"));
900 +
    } else if job.identities.iter().any(|i| i.mount_path == mount_path) || job.secrets.iter().any(|s| s.mount_path == mount_path) {
901 +
        ctx.err(node.span(), format!("two files mount at {mount_path:?}"));
902 +
    } else if job.identities.len() == MAX_IDENTITIES {
903 +
        ctx.err(node.span(), format!("a job has at most {MAX_IDENTITIES} identities"));
904 +
    } else {
905 +
        job.identities.push(Identity { audience, mount_path });
847 906
    }
848 907
}
849 908
…
56 unchanged lines
906 965
        exec: vec![],
907 966
        mounts: vec![],
908 967
        secrets: vec![],
968 +
        identities: vec![],
909 969
        outputs: vec![],
910 970
        runs_on: vec![],
911 971
        timeout_s: None,
…
2 unchanged lines
914 974
    let mut outputs_at = None;
915 975
    for c in block(ctx, node) {
916 976
        let key = c.name().value();
917 -
        if key != "scope" && key != "depends-on" && key != "mount" && key != "secret" && key != "runs-on" && !seen.insert(key.to_string()) {
977 +
        if key != "scope"
978 +
            && key != "depends-on"
979 +
            && key != "mount"
980 +
            && key != "secret"
981 +
            && key != "identity"
982 +
            && key != "runs-on"
983 +
            && !seen.insert(key.to_string())
984 +
        {
918 985
            ctx.err(c.span(), format!("`{key}` is given twice"));
919 986
        }
920 987
        match key {
…
55 unchanged lines
976 1043
                parse_output(ctx, c, &mut job);
977 1044
            }
978 1045
            "secret" => parse_secret(ctx, c, &mut job),
1046 +
            "identity" => parse_identity(ctx, c, &mut job),
979 1047
            n if LATER_JOB.contains(&n) => ctx.err(c.span(), format!("`{n}` is not in M0 yet")),
980 1048
            other => ctx.err(c.span(), format!("unknown job setting `{other}`")),
981 1049
        }
…
1 unchanged lines
983 1051
    job.mounts.sort_by(|a, b| a.target.cmp(&b.target));
984 1052
    job.outputs.sort_by(|a, b| a.name.cmp(&b.name));
985 1053
    job.secrets.sort_by(|a, b| a.name.cmp(&b.name));
1054 +
    job.identities.sort_by(|a, b| a.audience.cmp(&b.audience));
986 1055
    // A tmpfs is gone when the job ends; nothing on it can be kept.
987 1056
    for o in &job.outputs {
988 1057
        if let Some(m) = job.mounts.iter().find(|m| m.source == MountSource::Tmpfs && o.path.starts_with(&format!("{}/", m.target))) {
…
21 unchanged lines
modified crates/flow-core/src/tests.rs +40 −1
…
291 unchanged lines
292 292
        ("secret \"t\" {\n    mount-path \"/run/secrets/a\"\n}\nsecret \"t\" {\n    mount-path \"/run/secrets/b\"\n}", "given twice"),
293 293
        (
294 294
            "secret \"t\" {\n    mount-path \"/run/secrets/a\"\n}\nsecret \"u\" {\n    mount-path \"/run/secrets/a\"\n}",
295 -
            "two secrets mount at",
295 +
            "two files mount at",
296 296
        ),
297 297
        ("secret \"t\" {\n    vault-ref \"x\"\n    mount-path \"/run/secrets/a\"\n}", "vault-ref"),
298 298
    ] {
…
3 unchanged lines
302 302
}
303 303
304 304
#[test]
305 +
fn identities_are_asked_for_by_audience_and_mount_like_secrets() {
306 +
    let plain = Pipeline::parse(&doc("job \"d\" {\n    sandbox \"alpine\"\n    exec \"deploy\"\n}\n")).unwrap();
307 +
    let p = Pipeline::parse(&doc(r#"job "d" {
308 +
    sandbox "alpine"
309 +
    identity "sts.amazonaws.com" {
310 +
        mount-path "/run/secrets/aws-token"
311 +
    }
312 +
    identity "https://vault.example" {
313 +
        mount-path "/run/secrets/vault-token"
314 +
    }
315 +
    secret "key" {
316 +
        mount-path "/run/secrets/key"
317 +
    }
318 +
    exec "deploy"
319 +
}"#))
320 +
    .unwrap();
321 +
    let d = &p.jobs["d"];
322 +
    assert_eq!(
323 +
        d.identities.iter().map(|i| i.audience.as_str()).collect::<Vec<_>>(),
324 +
        ["https://vault.example", "sts.amazonaws.com"],
325 +
        "sorted"
326 +
    );
327 +
    assert_ne!(p.hash(), plain.hash(), "identities are part of the job");
328 +
    for (bad, why) in [
329 +
        ("identity \"a b\" {\n    mount-path \"/run/secrets/x\"\n}", "without spaces"),
330 +
        ("identity \"a\" {\n    mount-path \"/tmp/x\"\n}", "under /run/secrets/"),
331 +
        ("identity \"a\" {\n}", "needs a mount-path"),
332 +
        ("identity \"a\" {\n    mount-path \"/run/secrets/x\"\n}\nidentity \"a\" {\n    mount-path \"/run/secrets/y\"\n}", "given twice"),
333 +
        (
334 +
            "identity \"a\" {\n    mount-path \"/run/secrets/x\"\n}\nsecret \"s\" {\n    mount-path \"/run/secrets/x\"\n}",
335 +
            "two files mount at",
336 +
        ),
337 +
    ] {
338 +
        let e = Pipeline::parse(&doc(&format!("job \"d\" {{\n    sandbox \"alpine\"\n    {bad}\n    exec \"deploy\"\n}}\n"))).unwrap_err();
339 +
        assert!(e.to_string().contains(why), "{bad}: {e}");
340 +
    }
341 +
}
342 +
343 +
#[test]
305 344
fn jobs_are_told_what_they_build() {
306 345
    let p = Pipeline::parse(&doc(r#"job "build" {
307 346
    sandbox "alpine"
…
158 unchanged lines
modified crates/flow/src/main.rs +4 −0
…
344 unchanged lines
345 345
            .in_sandbox(sb)
346 346
            .with_flow_env(&[("FLOW_JOB".into(), job.name.clone()), ("FLOW_RUN".into(), "local".into())])
347 347
            .with_flow_env(vars);
348 +
        // Identities come from a hub: a local run has none.
349 +
        for i in &job.identities {
350 +
            println!("── {}: identity {:?} is not available locally; {} will be missing", job.name, i.audience, i.mount_path);
351 +
        }
348 352
        // Each secret the job declares must be given, as the hub would.
349 353
        let inputs = job
350 354
            .secrets
…
53 unchanged lines
modified crates/flow/src/outputs.rs +1 −0
…
69 unchanged lines
70 70
            env: Default::default(),
71 71
            exec: vec!["make".into()],
72 72
            secrets: vec![],
73 +
            identities: vec![],
73 74
            mounts: vec![
74 75
                Mount { source: MountSource::Tmpfs, target: "/src/cache".into(), size_mib: 64 },
75 76
                Mount { source: MountSource::Disk, target: "/src/target".into(), size_mib: 1024 },
…
32 unchanged lines
modified docs/m0.md +6 −0
…
62 unchanged lines
63 63
  of its own, before the job starts; it is never in the environment, the
64 64
  receipt or an artifact (spec §9). Locally, `flow run --secret
65 65
  NAME=VALUE` supplies it. `vault-ref` is not in M0: the hub is the vault.
66 +
- `identity "sts.amazonaws.com" { mount-path "/run/secrets/aws-token" }` on
67 +
  a job (up to 4) asks the hub for an OIDC token for that audience, naming
68 +
  the repository, bookmark, run and job (OxygenHub spec §13.5,
69 +
  oxygenhub#103); it comes with the claim and is mounted exactly as a
70 +
  secret is. A local run has no hub, so `flow run` says the file will be
71 +
  missing.
66 72
- `output { artifact "oxy" { path "/src/target/release/oxy" } }` declares
67 73
  up to 16 **artifacts** (spec §5.2). The path is clean, absolute, under
68 74
  `/src`, and not on a tmpfs mount, which is gone when the job ends. After
…
158 unchanged lines