A job's identity: identity blocks name an audience and a mount path 720eb8191059
identity "AUDIENCE" { mount-path "/run/secrets/…" } asks the hub for an
OIDC token for that audience, naming the repository, bookmark, run and
job; it comes with the claim and mounts exactly as a secret does. A
local run says the file will be missing. refs oxygenhub#103
- Author
- Andy McCall @andymccall <[email protected]>
- When
- · 2026-10-04 16:26 UTC
- Signature
- ✔ signed by Andy McCall
- Hash
720eb81910593912de76f3721156c812858801460b73767233b2429c9ec5a4e0- Depends on
- a67b97e9a717 c3db750ed96e c4285535b03a c58efc6a74db e0918b887850 e2c125610f43 e576e08617dc fc10d9e80383
- Depended on by
- f201e1bc5bcd 46fe2c815cc3 1abadd4a2fa6
- In bookmarks
- main newer-pipeline push-uses-login
- In tags
- v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.5.0 v0.5.1 v0.5.2 v0.5.3
- Edits
- 20 atoms, 0 tree operations, 8 files
Browse the tree at this change
modified CHANGELOG.md +2 −0
| … | 3 unchanged lines |
||
| 4 | 4 | ||
| 5 | 5 | ## Unreleased |
|
| 6 | 6 | ||
| 7 | + | - **A job's identity (oxygenhub#103).** `identity "AUDIENCE" { mount-path "/run/secrets/…" }` asks the hub for an OIDC token for that audience, mounted as a file like a secret. |
|
| 8 | + | ||
| 7 | 9 | - **Secrets as files (oxygenhub#107).** A job's `secret "name" { mount-path "/run/secrets/…" }` names a secret the hub holds; it comes with the claim and `flow-init` writes it at that path, 0400, on a tmpfs, before the job starts. Never in the environment, the receipt or an artifact (spec §9). `flow run --secret NAME=VALUE` locally. |
|
| 8 | 10 | ||
| 9 | 11 | - **Variables from the hub (oxygenhub#102).** A claim carries `env`, the variables the hub holds for the repository; they join the job's environment under `FLOW_*` and the pipeline's own `env`. `flow run --var NAME=VALUE` does the same locally. |
|
| … | 12 unchanged lines |
modified crates/flow-core/src/canonical.rs +4 −0
| … | 76 unchanged lines |
||
| 77 | 77 | if !j.secrets.is_empty() { |
|
| 78 | 78 | fields.push((8, Value::Array(j.secrets.iter().map(|s| Value::Array(vec![text(&s.name), text(&s.mount_path)])).collect()))); |
|
| 79 | 79 | } |
|
| 80 | + | // Identities as [audience, mount path] sorted by audience; absent when none. |
|
| 81 | + | if !j.identities.is_empty() { |
|
| 82 | + | fields.push((9, Value::Array(j.identities.iter().map(|i| Value::Array(vec![text(&i.audience), text(&i.mount_path)])).collect()))); |
|
| 83 | + | } |
|
| 80 | 84 | Value::map(fields) |
|
| 81 | 85 | } |
|
| 82 | 86 | ||
| … | 41 unchanged lines |
modified crates/flow-core/src/lib.rs +1 −1
| … | 9 unchanged lines |
||
| 10 | 10 | pub mod receipt; |
|
| 11 | 11 | ||
| 12 | 12 | pub use canonical::{object_hash, scope_tree_hash, Hash}; |
|
| 13 | - | pub use pipeline::{Job, Mount, MountSource, Network, Pipeline, Problem, Problems, Runtime, Sandbox, Secret, PIPELINE_PATH}; |
|
| 13 | + | pub use pipeline::{Identity, Job, Mount, MountSource, Network, Pipeline, Problem, Problems, Runtime, Sandbox, Secret, PIPELINE_PATH}; |
|
| 14 | 14 | pub use receipt::{Environment, Receipt}; |
|
| 15 | 15 | ||
| 16 | 16 | #[cfg(test)] |
|
| … | 1 unchanged lines |
modified crates/flow-core/src/pipeline.rs +72 −3
| … | 99 unchanged lines |
||
| 100 | 100 | /// supplies each by name, the runner mounts it as a file. |
|
| 101 | 101 | #[serde(default, skip_serializing_if = "Vec::is_empty")] |
|
| 102 | 102 | pub secrets: Vec<Secret>, |
|
| 103 | + | /// Identities the job asks the hub for (oxygenhub#103): an OIDC token |
|
| 104 | + | /// for an audience, mounted as a file like a secret. |
|
| 105 | + | #[serde(default, skip_serializing_if = "Vec::is_empty")] |
|
| 106 | + | pub identities: Vec<Identity>, |
|
| 103 | 107 | /// Labels a runner must have to take this job (`runs-on`), sorted; |
|
| 104 | 108 | /// with its sandbox's, what the hub matches against a runner's. |
|
| 105 | 109 | #[serde(default, skip_serializing_if = "Vec::is_empty")] |
|
| … | 78 unchanged lines |
||
| 184 | 188 | pub mount_path: String, |
|
| 185 | 189 | } |
|
| 186 | 190 | ||
| 191 | + | /// An OIDC token the hub mints for the job (oxygenhub#103), naming the |
|
| 192 | + | /// repository, bookmark, run and job, for `audience`; read as a file. |
|
| 193 | + | #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] |
|
| 194 | + | pub struct Identity { |
|
| 195 | + | /// Who the token is for (`sts.amazonaws.com`, a Vault role's audience, …). |
|
| 196 | + | pub audience: String, |
|
| 197 | + | /// A clean absolute path under `/run/secrets/`, unique among the |
|
| 198 | + | /// job's secrets and identities. |
|
| 199 | + | pub mount_path: String, |
|
| 200 | + | } |
|
| 201 | + | ||
| 187 | 202 | /// A secret's mount path: under `/run/secrets/`, no `.` or `..`. |
|
| 188 | 203 | pub fn secret_path_ok(p: &str) -> bool { |
|
| 189 | 204 | p.strip_prefix("/run/secrets/").is_some_and(|rest| !rest.is_empty() && rest.split('/').all(|c| !c.is_empty() && c != "." && c != "..")) |
|
| … | 16 unchanged lines |
||
| 206 | 221 | pub const MAX_MOUNTS: usize = 4; |
|
| 207 | 222 | /// The most secrets a job may mount. |
|
| 208 | 223 | pub const MAX_SECRETS: usize = 8; |
|
| 224 | + | /// The most identities a job may ask for. |
|
| 225 | + | pub const MAX_IDENTITIES: usize = 4; |
|
| 209 | 226 | /// Sizes a mount may have: 16 MiB to 64 GiB. |
|
| 210 | 227 | pub const MIN_MOUNT_MIB: u64 = 16; |
|
| 211 | 228 | pub const MAX_MOUNT_MIB: u64 = 64 * 1024; |
|
| … | 626 unchanged lines |
||
| 838 | 855 | }; |
|
| 839 | 856 | if job.secrets.iter().any(|s| s.name == name) { |
|
| 840 | 857 | ctx.err(node.span(), format!("secret {name:?} is given twice")); |
|
| 841 | - | } else if job.secrets.iter().any(|s| s.mount_path == mount_path) { |
|
| 842 | - | ctx.err(node.span(), format!("two secrets mount at {mount_path:?}")); |
|
| 858 | + | } else if job.secrets.iter().any(|s| s.mount_path == mount_path) || job.identities.iter().any(|i| i.mount_path == mount_path) { |
|
| 859 | + | ctx.err(node.span(), format!("two files mount at {mount_path:?}")); |
|
| 843 | 860 | } else if job.secrets.len() == MAX_SECRETS { |
|
| 844 | 861 | ctx.err(node.span(), format!("a job has at most {MAX_SECRETS} secrets")); |
|
| 845 | 862 | } else { |
|
| 846 | 863 | job.secrets.push(Secret { name, mount_path }); |
|
| 864 | + | } |
|
| 865 | + | } |
|
| 866 | + | ||
| 867 | + | /// `identity "sts.amazonaws.com" { mount-path "/run/secrets/aws-token" }`: |
|
| 868 | + | /// the hub mints an OIDC token for that audience and the runner mounts it |
|
| 869 | + | /// as a file (oxygenhub#103). |
|
| 870 | + | fn parse_identity(ctx: &mut Ctx, node: &KdlNode, job: &mut Job) { |
|
| 871 | + | let Some(audience) = ctx.one_string(node) else { |
|
| 872 | + | ctx.err(node.span(), "`identity` takes an audience and a block with mount-path"); |
|
| 873 | + | return; |
|
| 874 | + | }; |
|
| 875 | + | let aud_ok = !audience.is_empty() && audience.len() <= 255 && !audience.chars().any(|c| c.is_whitespace() || c.is_control()); |
|
| 876 | + | if !aud_ok { |
|
| 877 | + | ctx.err(node.span(), format!("identity audience {audience:?} must be 1 to 255 characters without spaces")); |
|
| 878 | + | return; |
|
| 879 | + | } |
|
| 880 | + | let mut path = None; |
|
| 881 | + | for c in block(ctx, node) { |
|
| 882 | + | ctx.no_children(c); |
|
| 883 | + | match c.name().value() { |
|
| 884 | + | "mount-path" => match ctx.one_string(c) { |
|
| 885 | + | Some(p) if secret_path_ok(&p) => path = Some(p), |
|
| 886 | + | Some(p) => { |
|
| 887 | + | ctx.err(c.span(), format!("identity mount-path {p:?} must be an absolute path under /run/secrets/, without . or ..")) |
|
| 888 | + | } |
|
| 889 | + | None => {} |
|
| 890 | + | }, |
|
| 891 | + | other => ctx.err(c.span(), format!("unknown identity setting `{other}`")), |
|
| 892 | + | } |
|
| 893 | + | } |
|
| 894 | + | let Some(mount_path) = path else { |
|
| 895 | + | ctx.err(node.span(), format!("identity {audience:?} needs a mount-path")); |
|
| 896 | + | return; |
|
| 897 | + | }; |
|
| 898 | + | if job.identities.iter().any(|i| i.audience == audience) { |
|
| 899 | + | ctx.err(node.span(), format!("identity {audience:?} is given twice")); |
|
| 900 | + | } else if job.identities.iter().any(|i| i.mount_path == mount_path) || job.secrets.iter().any(|s| s.mount_path == mount_path) { |
|
| 901 | + | ctx.err(node.span(), format!("two files mount at {mount_path:?}")); |
|
| 902 | + | } else if job.identities.len() == MAX_IDENTITIES { |
|
| 903 | + | ctx.err(node.span(), format!("a job has at most {MAX_IDENTITIES} identities")); |
|
| 904 | + | } else { |
|
| 905 | + | job.identities.push(Identity { audience, mount_path }); |
|
| 847 | 906 | } |
|
| 848 | 907 | } |
|
| 849 | 908 | ||
| … | 56 unchanged lines |
||
| 906 | 965 | exec: vec![], |
|
| 907 | 966 | mounts: vec![], |
|
| 908 | 967 | secrets: vec![], |
|
| 968 | + | identities: vec![], |
|
| 909 | 969 | outputs: vec![], |
|
| 910 | 970 | runs_on: vec![], |
|
| 911 | 971 | timeout_s: None, |
|
| … | 2 unchanged lines |
||
| 914 | 974 | let mut outputs_at = None; |
|
| 915 | 975 | for c in block(ctx, node) { |
|
| 916 | 976 | let key = c.name().value(); |
|
| 917 | - | if key != "scope" && key != "depends-on" && key != "mount" && key != "secret" && key != "runs-on" && !seen.insert(key.to_string()) { |
|
| 977 | + | if key != "scope" |
|
| 978 | + | && key != "depends-on" |
|
| 979 | + | && key != "mount" |
|
| 980 | + | && key != "secret" |
|
| 981 | + | && key != "identity" |
|
| 982 | + | && key != "runs-on" |
|
| 983 | + | && !seen.insert(key.to_string()) |
|
| 984 | + | { |
|
| 918 | 985 | ctx.err(c.span(), format!("`{key}` is given twice")); |
|
| 919 | 986 | } |
|
| 920 | 987 | match key { |
|
| … | 55 unchanged lines |
||
| 976 | 1043 | parse_output(ctx, c, &mut job); |
|
| 977 | 1044 | } |
|
| 978 | 1045 | "secret" => parse_secret(ctx, c, &mut job), |
|
| 1046 | + | "identity" => parse_identity(ctx, c, &mut job), |
|
| 979 | 1047 | n if LATER_JOB.contains(&n) => ctx.err(c.span(), format!("`{n}` is not in M0 yet")), |
|
| 980 | 1048 | other => ctx.err(c.span(), format!("unknown job setting `{other}`")), |
|
| 981 | 1049 | } |
|
| … | 1 unchanged lines |
||
| 983 | 1051 | job.mounts.sort_by(|a, b| a.target.cmp(&b.target)); |
|
| 984 | 1052 | job.outputs.sort_by(|a, b| a.name.cmp(&b.name)); |
|
| 985 | 1053 | job.secrets.sort_by(|a, b| a.name.cmp(&b.name)); |
|
| 1054 | + | job.identities.sort_by(|a, b| a.audience.cmp(&b.audience)); |
|
| 986 | 1055 | // A tmpfs is gone when the job ends; nothing on it can be kept. |
|
| 987 | 1056 | for o in &job.outputs { |
|
| 988 | 1057 | if let Some(m) = job.mounts.iter().find(|m| m.source == MountSource::Tmpfs && o.path.starts_with(&format!("{}/", m.target))) { |
|
| … | 21 unchanged lines |
modified crates/flow-core/src/tests.rs +40 −1
| … | 291 unchanged lines |
||
| 292 | 292 | ("secret \"t\" {\n mount-path \"/run/secrets/a\"\n}\nsecret \"t\" {\n mount-path \"/run/secrets/b\"\n}", "given twice"), |
|
| 293 | 293 | ( |
|
| 294 | 294 | "secret \"t\" {\n mount-path \"/run/secrets/a\"\n}\nsecret \"u\" {\n mount-path \"/run/secrets/a\"\n}", |
|
| 295 | - | "two secrets mount at", |
|
| 295 | + | "two files mount at", |
|
| 296 | 296 | ), |
|
| 297 | 297 | ("secret \"t\" {\n vault-ref \"x\"\n mount-path \"/run/secrets/a\"\n}", "vault-ref"), |
|
| 298 | 298 | ] { |
|
| … | 3 unchanged lines |
||
| 302 | 302 | } |
|
| 303 | 303 | ||
| 304 | 304 | #[test] |
|
| 305 | + | fn identities_are_asked_for_by_audience_and_mount_like_secrets() { |
|
| 306 | + | let plain = Pipeline::parse(&doc("job \"d\" {\n sandbox \"alpine\"\n exec \"deploy\"\n}\n")).unwrap(); |
|
| 307 | + | let p = Pipeline::parse(&doc(r#"job "d" { |
|
| 308 | + | sandbox "alpine" |
|
| 309 | + | identity "sts.amazonaws.com" { |
|
| 310 | + | mount-path "/run/secrets/aws-token" |
|
| 311 | + | } |
|
| 312 | + | identity "https://vault.example" { |
|
| 313 | + | mount-path "/run/secrets/vault-token" |
|
| 314 | + | } |
|
| 315 | + | secret "key" { |
|
| 316 | + | mount-path "/run/secrets/key" |
|
| 317 | + | } |
|
| 318 | + | exec "deploy" |
|
| 319 | + | }"#)) |
|
| 320 | + | .unwrap(); |
|
| 321 | + | let d = &p.jobs["d"]; |
|
| 322 | + | assert_eq!( |
|
| 323 | + | d.identities.iter().map(|i| i.audience.as_str()).collect::<Vec<_>>(), |
|
| 324 | + | ["https://vault.example", "sts.amazonaws.com"], |
|
| 325 | + | "sorted" |
|
| 326 | + | ); |
|
| 327 | + | assert_ne!(p.hash(), plain.hash(), "identities are part of the job"); |
|
| 328 | + | for (bad, why) in [ |
|
| 329 | + | ("identity \"a b\" {\n mount-path \"/run/secrets/x\"\n}", "without spaces"), |
|
| 330 | + | ("identity \"a\" {\n mount-path \"/tmp/x\"\n}", "under /run/secrets/"), |
|
| 331 | + | ("identity \"a\" {\n}", "needs a mount-path"), |
|
| 332 | + | ("identity \"a\" {\n mount-path \"/run/secrets/x\"\n}\nidentity \"a\" {\n mount-path \"/run/secrets/y\"\n}", "given twice"), |
|
| 333 | + | ( |
|
| 334 | + | "identity \"a\" {\n mount-path \"/run/secrets/x\"\n}\nsecret \"s\" {\n mount-path \"/run/secrets/x\"\n}", |
|
| 335 | + | "two files mount at", |
|
| 336 | + | ), |
|
| 337 | + | ] { |
|
| 338 | + | let e = Pipeline::parse(&doc(&format!("job \"d\" {{\n sandbox \"alpine\"\n {bad}\n exec \"deploy\"\n}}\n"))).unwrap_err(); |
|
| 339 | + | assert!(e.to_string().contains(why), "{bad}: {e}"); |
|
| 340 | + | } |
|
| 341 | + | } |
|
| 342 | + | ||
| 343 | + | #[test] |
|
| 305 | 344 | fn jobs_are_told_what_they_build() { |
|
| 306 | 345 | let p = Pipeline::parse(&doc(r#"job "build" { |
|
| 307 | 346 | sandbox "alpine" |
|
| … | 158 unchanged lines |
modified crates/flow/src/main.rs +4 −0
| … | 344 unchanged lines |
||
| 345 | 345 | .in_sandbox(sb) |
|
| 346 | 346 | .with_flow_env(&[("FLOW_JOB".into(), job.name.clone()), ("FLOW_RUN".into(), "local".into())]) |
|
| 347 | 347 | .with_flow_env(vars); |
|
| 348 | + | // Identities come from a hub: a local run has none. |
|
| 349 | + | for i in &job.identities { |
|
| 350 | + | println!("── {}: identity {:?} is not available locally; {} will be missing", job.name, i.audience, i.mount_path); |
|
| 351 | + | } |
|
| 348 | 352 | // Each secret the job declares must be given, as the hub would. |
|
| 349 | 353 | let inputs = job |
|
| 350 | 354 | .secrets |
|
| … | 53 unchanged lines |
modified crates/flow/src/outputs.rs +1 −0
| … | 69 unchanged lines |
||
| 70 | 70 | env: Default::default(), |
|
| 71 | 71 | exec: vec!["make".into()], |
|
| 72 | 72 | secrets: vec![], |
|
| 73 | + | identities: vec![], |
|
| 73 | 74 | mounts: vec![ |
|
| 74 | 75 | Mount { source: MountSource::Tmpfs, target: "/src/cache".into(), size_mib: 64 }, |
|
| 75 | 76 | Mount { source: MountSource::Disk, target: "/src/target".into(), size_mib: 1024 }, |
|
| … | 32 unchanged lines |
modified docs/m0.md +6 −0
| … | 62 unchanged lines |
||
| 63 | 63 | of its own, before the job starts; it is never in the environment, the |
|
| 64 | 64 | receipt or an artifact (spec §9). Locally, `flow run --secret |
|
| 65 | 65 | NAME=VALUE` supplies it. `vault-ref` is not in M0: the hub is the vault. |
|
| 66 | + | - `identity "sts.amazonaws.com" { mount-path "/run/secrets/aws-token" }` on |
|
| 67 | + | a job (up to 4) asks the hub for an OIDC token for that audience, naming |
|
| 68 | + | the repository, bookmark, run and job (OxygenHub spec §13.5, |
|
| 69 | + | oxygenhub#103); it comes with the claim and is mounted exactly as a |
|
| 70 | + | secret is. A local run has no hub, so `flow run` says the file will be |
|
| 71 | + | missing. |
|
| 66 | 72 | - `output { artifact "oxy" { path "/src/target/release/oxy" } }` declares |
|
| 67 | 73 | up to 16 **artifacts** (spec §5.2). The path is clean, absolute, under |
|
| 68 | 74 | `/src`, and not on a tmpfs mount, which is gone when the job ends. After |
|
| … | 158 unchanged lines |