OXYGENHUB

pipeline.kdl: priority lowers a job's place in the queue, never raises it e87c1c138187

priority "low" (or "normal") on a job, or at the top for every job
that does not say; high and urgent are refused with the reason (they are
the hub's to give, and a person may expedite). Part of the compiled job;
a job without one compiles as before. Spec; parser test.

Refs andymccall/oxygenhub#244
Author
Andy McCall @andymccall <[email protected]>
via
agent Claude Code <[email protected]> · delegate for changes, until 2026-11-13 19:00Z · key ea5f78160088
When
· 2026-10-17 13:09 UTC
Signature
✔ signed by Claude Code
Hash
e87c1c138187a3ab0da6e276809e097c7bbeed04ad7b57fd982e00f35d67c0c6
Depends on
02b127d2c721 a083cc802892 a67b97e9a717 bf7bb1b80fa3 e0918b887850 e576e08617dc fc10d9e80383
Depended on by
f201e1bc5bcd
In bookmarks
main
In tags
v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.5.0 v0.5.1 v0.5.2 v0.5.3
Edits
12 atoms, 0 tree operations, 4 files

Browse the tree at this change

modified crates/flow-core/src/pipeline.rs +47 −2
…
168 unchanged lines
169 169
    /// When it runs, over the pipeline's `on` (oxygenflow#24).
170 170
    #[serde(default, skip_serializing_if = "Option::is_none")]
171 171
    pub on: Option<On>,
172 +
    /// `low` or `normal` (`priority "low"`, or the pipeline's): a pipeline
173 +
    /// may lower its jobs below the repository's level, never raise them
174 +
    /// (oxygenhub#244); higher levels are the hub's to give.
175 +
    #[serde(default, skip_serializing_if = "Option::is_none")]
176 +
    pub priority: Option<String>,
172 177
}
173 178
174 179
impl Job {
…
259 unchanged lines
434 439
        let mut sandboxes = BTreeMap::new();
435 440
        let mut jobs: BTreeMap<String, (Job, SourceSpan)> = BTreeMap::new();
436 441
        let mut on = None;
442 +
        let mut priority = None;
437 443
        for node in doc.nodes() {
438 444
            match node.name().value() {
445 +
                "priority" => {
446 +
                    if priority.is_some() {
447 +
                        ctx.err(node.span(), "`priority` is given twice");
448 +
                    }
449 +
                    priority = parse_priority(&mut ctx, node);
450 +
                }
439 451
                "on" => {
440 452
                    if on.is_some() {
441 453
                        ctx.err(node.span(), "`on` is given twice");
…
27 unchanged lines
469 481
                    }
470 482
                }
471 483
                n if LATER_TOP.contains(&n) => ctx.err(node.span(), format!("`{n}` is not in M0 yet")),
472 -
                n => ctx.err(node.span(), format!("unknown node `{n}`; expected schema-version, on, sandbox or job")),
484 +
                n => ctx.err(node.span(), format!("unknown node `{n}`; expected schema-version, on, priority, sandbox or job")),
473 485
            }
474 486
        }
475 487
        if schema.is_none() && !ctx.problems.iter().any(|p| p.message.contains("schema-version")) {
…
25 unchanged lines
501 513
            ctx.problems.sort_by_key(|p| p.line);
502 514
            return Err(Problems(ctx.problems));
503 515
        }
504 -
        Ok(Pipeline { schema_version: 1, sandboxes, jobs: jobs.into_iter().map(|(k, (j, _))| (k, j)).collect(), on })
516 +
        // The pipeline's priority is each job's that does not say.
517 +
        let jobs = jobs
518 +
            .into_iter()
519 +
            .map(|(k, (mut j, _))| {
520 +
                if j.priority.is_none() {
521 +
                    j.priority.clone_from(&priority);
522 +
                }
523 +
                (k, j)
524 +
            })
525 +
            .collect();
526 +
        Ok(Pipeline { schema_version: 1, sandboxes, jobs, on })
505 527
    }
506 528
507 529
    /// Jobs in an order where each comes after what it depends on (ties by name).
…
266 unchanged lines
774 796
}
775 797
776 798
/// A `timeout` line: one duration, from a minute to a day.
799 +
/// `priority "low"`: a pipeline may lower its jobs, never raise them
800 +
/// (oxygenhub#244).
801 +
fn parse_priority(ctx: &mut Ctx, c: &KdlNode) -> Option<String> {
802 +
    ctx.no_children(c);
803 +
    let p = ctx.one_string(c)?;
804 +
    match p.as_str() {
805 +
        "low" | "normal" => Some(p),
806 +
        "high" | "urgent" => {
807 +
            ctx.err(
808 +
                c.span(),
809 +
                format!("priority {p:?}: a pipeline may lower its jobs (`low`, `normal`), not raise them; higher levels are set on the hub, and a person may expedite a run"),
810 +
            );
811 +
            None
812 +
        }
813 +
        _ => {
814 +
            ctx.err(c.span(), format!("priority {p:?}: expected `low` or `normal`"));
815 +
            None
816 +
        }
817 +
    }
818 +
}
819 +
777 820
fn parse_timeout(ctx: &mut Ctx, c: &KdlNode) -> Option<u64> {
778 821
    let t = ctx.one_string(c)?;
779 822
    match parse_duration(&t) {
…
182 unchanged lines
962 1005
        runs_on: vec![],
963 1006
        timeout_s: None,
964 1007
        on: None,
1008 +
        priority: None,
965 1009
    };
966 1010
    let mut seen = BTreeSet::new();
967 1011
    let mut outputs_at = None;
…
28 unchanged lines
996 1040
                ctx.no_children(c);
997 1041
                job.timeout_s = parse_timeout(ctx, c);
998 1042
            }
1043 +
            "priority" => job.priority = parse_priority(ctx, c),
999 1044
            "on" => job.on = Some(parse_on(ctx, c)),
1000 1045
            "env" => parse_env(ctx, c, &mut job.env),
1001 1046
            "exec" => {
…
60 unchanged lines
modified crates/flow-core/src/tests.rs +21 −0
…
387 unchanged lines
388 388
    assert!(none.iter().any(|(_, m)| m.contains("needs one or more labels")), "{none:?}");
389 389
}
390 390
391 +
/// oxygenhub#244: a pipeline may lower its jobs' priority, for all of them
392 +
/// or one by one, never raise it.
393 +
#[test]
394 +
fn a_pipeline_lowers_its_jobs_priority_and_never_raises_it() {
395 +
    let jobs = "job \"nightly\" {\n    sandbox \"alpine\"\n    exec \"true\"\n}\njob \"check\" {\n    sandbox \"alpine\"\n    priority \"normal\"\n    exec \"true\"\n}\n";
396 +
    let p = Pipeline::parse(&format!("priority \"low\"\n{}", doc(jobs))).unwrap();
397 +
    assert_eq!(p.jobs["nightly"].priority.as_deref(), Some("low"), "the pipeline's");
398 +
    assert_eq!(p.jobs["check"].priority.as_deref(), Some("normal"), "its own");
399 +
    let one = Pipeline::parse(&doc("job \"a\" {\n    sandbox \"alpine\"\n    priority \"low\"\n    exec \"true\"\n}\n")).unwrap();
400 +
    assert_eq!(one.jobs["a"].priority.as_deref(), Some("low"));
401 +
    // None given: serialised as before, so its jobs' hashes stay.
402 +
    let plain = Pipeline::parse(&doc("job \"x\" {\n    sandbox \"alpine\"\n    exec \"true\"\n}\n")).unwrap();
403 +
    assert!(!serde_json::to_string(&plain.jobs["x"]).unwrap().contains("priority"));
404 +
    for high in ["high", "urgent"] {
405 +
        let bad = problems(&doc(&format!("job \"x\" {{\n    sandbox \"alpine\"\n    priority \"{high}\"\n    exec \"true\"\n}}\n")));
406 +
        assert!(bad.iter().any(|(_, m)| m.contains("not raise them")), "{high}: {bad:?}");
407 +
    }
408 +
    let odd = problems(&doc("job \"x\" {\n    sandbox \"alpine\"\n    priority \"soon\"\n    exec \"true\"\n}\n"));
409 +
    assert!(odd.iter().any(|(_, m)| m.contains("expected `low` or `normal`")), "{odd:?}");
410 +
}
411 +
391 412
/// oxygenflow#23: `timeout` on a job, or on its sandbox for every job in
392 413
/// it, as a duration from a minute to a day.
393 414
#[test]
…
54 unchanged lines
modified crates/flow/src/outputs.rs +1 −0
…
77 unchanged lines
78 78
            runs_on: vec![],
79 79
            timeout_s: None,
80 80
            on: None,
81 +
            priority: None,
81 82
        }
82 83
    }
83 84
…
24 unchanged lines
modified docs/spec.md +2 −0
…
178 unchanged lines
179 179
180 180
**Time limits.** A job may say how long it may run with `timeout "90m"` (hours, minutes and seconds, as `"2h"`, `"1h30m"` or `"45s"`, from a minute to a day); a sandbox's `timeout` is the default for its jobs. Without one, the hub's default applies, and the hub caps every job at the site's maximum (both set by an administrator on Admin → Runners). The runner stops a job at its limit, and the hub ends one its runner stopped reporting a few minutes after it. A timeout is part of the compiled job, so a job without one compiles as before.
181 181
182 +
**Priority.** A pipeline may lower its jobs' place in the hub's queue, never raise it (oxygenhub#244): `priority "low"` (or `"normal"`) on a job, or at the top of the file for every job that does not say. A job's level is then the lower of its own and its repository's (the repository's, organisation's or site's, set on the hub), so nightly or bulk jobs can stand back for everyday ones; `high` and `urgent` are refused here, being the hub's to give, and a run a person expedites stays urgent whatever its jobs say. A priority is part of the compiled job; a job without one compiles as before.
183 +
182 184
**Runner labels.** A sandbox or a job may say `runs-on "LABEL" ...`: lower-case letters, digits, `-`, `_` and `.`, at most 64 bytes each, given on one or more lines. A job needs a runner that has every label its sandbox and it name. A runner reports its labels and architecture when it registers and on each claim (`flow runner register --label`). The hub hands a job only to a runner with all of them, within the runner's scope (OxygenHub spec 13.2), and a job no runner can take waits, saying so on the queue. Labels are part of the compiled job, so a job that asks for none compiles as before.
183 185
184 186
### 3.2 Canonical Compilation
…
248 unchanged lines