OXYGENHUB

Diffusion: a bump's run reads the upstream it bumps 180dc43ad0e0

A bump's run is started by the diffusion account, which cannot read a
private upstream, so its subrepo was left out and the build failed (R*
API bumps went red this morning: 'The R* API subrepo is not checked
out'). The run is now given the upstream it bumps, which the consumer
already pins; other subrepos still need read access.

refs #150
Author
Andy McCall @andymccall <[email protected]>
When
· 2026-10-13 10:20 UTC
Signature
✔ signed by Andy McCall
Hash
180dc43ad0e062d9029ba7dc76288a1a1ce7afa2b7407d5ae41df71697f35a82
Depends on
25091e5c3bba 3776b70d9b17 42fa95540468 757195ad7a5d 93be9ce4653e a7e6e8e6094c cc19697f8871
Depended on by
5d90e0e045e2 f7bcee49bf24 4fa0da126b70 73ef9f767132
In bookmarks
api-unknown-fields cancel-landed-runs diffusion-pin-bookmark draft-ready main oh-comment-body-file proposal-list-fast
In tags
v0.5.0 v0.5.1 v0.5.2 v0.6.0 v0.6.1 v0.7.0 v0.8.0 v0.9.0 v0.9.1 v0.9.11 v0.9.15 v0.9.16 v0.9.19 v0.9.2 v0.9.20 v0.9.21 v0.9.22 v0.9.23 v0.9.24 v0.9.3 v0.9.4 v0.9.5 v0.9.6 v0.9.7 v0.9.8 v0.9.9
Edits
11 atoms, 0 tree operations, 3 files

Browse the tree at this change

modified crates/hub/src/flow.rs +13 −1
…
1041 unchanged lines
1042 1042
                }
1043 1043
                hosted = true;
1044 1044
                let role = crate::auth::roles::role_for(&app.db, pusher.as_ref(), other).await.ok().flatten();
1045 -
                if crate::auth::roles::allows(role, crate::auth::Access::Read) {
1045 +
                // A Diffusion bump's run reads the upstream it bumps: the one
1046 +
                // the consumer pins (someone who may write it committed the
1047 +
                // pin), and no other (#150).
1048 +
                let diffused = run.actor == crate::diffusion::ACCOUNT
1049 +
                    && run.bookmark.starts_with("diffusion/")
1050 +
                    && sqlx::query_scalar::<_, i64>("SELECT COUNT(*) FROM diffusion_pins WHERE consumer_id = ? AND upstream_repo_id = ?")
1051 +
                        .bind(run.repo_id)
1052 +
                        .bind(other.id)
1053 +
                        .fetch_one(&app.db)
1054 +
                        .await
1055 +
                        .unwrap_or(0)
1056 +
                        > 0;
1057 +
                if diffused || crate::auth::roles::allows(role, crate::auth::Access::Read) {
1046 1058
                    inputs.push(flow_core::api::SubrepoInput {
1047 1059
                        path: path.clone(),
1048 1060
                        repo: format!("{}/{}", other.owner, other.name),
…
878 unchanged lines
modified crates/hub/tests/flow.rs +26 −5
…
3255 unchanged lines
3256 3256
    db::accounts::create_account(&h.app.db, "andy", "[email protected]", "x", "salt", true).await.unwrap();
3257 3257
    create(&h.app, "andy", "api").await;
3258 3258
    create(&h.app, "andy", "game").await;
3259 -
    // Run every job of the upstream's newest run to a pass.
3260 -
    let pass_all = || async {
3259 +
    // Run every job of the upstream's newest run to a pass, keeping each claim.
3260 +
    let seen: std::sync::Mutex<Vec<serde_json::Value>> = std::sync::Mutex::new(Vec::new());
3261 +
    let seen = &seen;
3262 +
    let hub_app = &h.app;
3263 +
    let pass_all = move || async move {
3264 +
        let h_app = hub_app;
3261 3265
        for _ in 0..20 {
3262 -
            let Some(a) = claim(&h.app).await else {
3266 +
            let Some(a) = claim(h_app).await else {
3263 3267
                tokio::time::sleep(std::time::Duration::from_millis(30)).await;
3264 -
                if !jobs_left(&h.app).await {
3268 +
                if !jobs_left(h_app).await {
3265 3269
                    break;
3266 3270
                }
3267 3271
                continue;
3268 3272
            };
3273 +
            seen.lock().unwrap().push(a.clone());
3269 3274
            let id = a["job_id"].as_i64().unwrap();
3270 3275
            let (name, ph) = (a["job"]["name"].as_str().unwrap().to_string(), a["pipeline_hash"].as_str().unwrap().to_string());
3271 3276
            assert_eq!(
3272 -
                finish(&h.app, id, serde_json::json!({ "exit_code": 0, "duration_ms": 10, "receipt": receipt(&name, &ph, 0) })).await,
3277 +
                finish(h_app, id, serde_json::json!({ "exit_code": 0, "duration_ms": 10, "receipt": receipt(&name, &ph, 0) })).await,
3273 3278
                StatusCode::NO_CONTENT
3274 3279
            );
3275 3280
        }
…
130 unchanged lines
3406 3411
    }
3407 3412
    assert_eq!(behind, 0, "the index followed the landing");
3408 3413
3414 +
    // A private upstream: the bump's run (started by the diffusion account,
3415 +
    // which cannot read it) still gets the subrepo it bumps.
3416 +
    sqlx::query("UPDATE repositories SET visibility = 'private' WHERE name = 'api'").execute(&h.app.db).await.unwrap();
3417 +
    seen.lock().unwrap().clear();
3418 +
    write(api_work.path(), "a6.c", "v6\n");
3419 +
    api.snapshot("t").unwrap();
3420 +
    api.commit("api v6", &[]).unwrap();
3421 +
    assert!(api.push("origin", false).unwrap().iter().all(|r| r.ok));
3422 +
    pass_all().await;
3423 +
    tokio::time::sleep(std::time::Duration::from_millis(300)).await;
3424 +
    pass_all().await;
3425 +
    let claims = seen.lock().unwrap().clone();
3426 +
    let bump_job = claims.iter().find(|c| c["repo"] == "andy/game" && c["bookmark"] == "diffusion/deps/api").expect("the bump's job ran");
3427 +
    let subs: Vec<&str> = bump_job["subrepos"].as_array().unwrap().iter().filter_map(|s| s["path"].as_str()).collect();
3428 +
    assert_eq!(subs, vec!["deps/api"], "{bump_job}");
3429 +
3409 3430
    // A consumer turning Diffusion on is bumped at once to the upstream's
3410 3431
    // newest passing state, without waiting for it to move again.
3411 3432
    create(&h.app, "andy", "game2").await;
…
112 unchanged lines
modified docs/spec.md +1 −1
…
514 unchanged lines
515 515
  - its key is the one the runner's token first signed with. The first signed receipt under a token binds that token to its key in `flow_runner_keys`.
516 516
517 517
  Otherwise the job's result still stands, and the run page says why the receipt was not kept. The run page links each kept receipt, with its signer. The change page lists the receipts for the state whose only head is that change. `/{owner}/{name}/receipts/{attest}` downloads a signed receipt for `oxy receipt verify` or `import`.
518 -
- **Architecture** (OxygenFlow spec §11.2–§11.3, oxygenflow#36): a runner reports each image's architecture on its claims (`image_archs`), kept in `flow_images`; a job is offered only to a runner whose architecture matches its sandbox's `arch`, else its image's as known; unknown matches, as before. The queue says which architecture a waiting job asks for. **Images by hash** (oxygenflow#39, Jetstream): `flow rootfs push HASH` sends a rootfs image as the FastCDC chunks the hub lacks (`POST /flow/images/missing`, `PUT /flow/images/chunks/{hash}`, by a token that may write repositories) and then its manifest (`PUT /flow/images/{hash}`), which the hub accepts only when every chunk is held and they make an image with that hash; chunks are kept zstd-compressed under `<data>/images`, shared by every image, and `jet_images` lists what is held. Runners and accounts fetch with `GET /flow/images/{hash}` and `GET /flow/images/chunks/{hash}`. A runner whose claims say `fetches_images` is offered jobs whose image only the hub holds, and fetches it before its preflight. Each claim also carries the runner's `facts` (OxygenFlow spec §11.1, oxygenflow#40): free disk, load, speed score and class, kernel; the hub keeps the latest on the runner, shows them on Admin › Runners and in `GET /runners`. A job's finish carries the most disk and memory it took (`peak_disk_mib`, `peak_memory_mib`, oxygenflow#41); a new job's `need_disk_mib` is the most its last five runs of the same repository and name took, plus a quarter and 64 MiB, and it is offered only to a runner whose claim reports at least that much free disk; the waiting reason says so. **Ranking** (oxygenflow#42): a job that another free runner which polled in the last 10 s would do better (it holds the job's image while the claimant would fetch it, or has a higher speed class) is held from the claimant for up to 15 s, then any runner that fits takes it; each job records why it ran where it did (`placed`: "holds the image, idle, speed 100 (class 3)", with "after N s held for a better runner" when it was), shown on the run page, in the API and by `oh run view`. A run whose jobs pin an image held by neither the hub nor a runner sets a failing `pipeline/images` check on its state (#137), naming the images and how to send them (`flow rootfs push`), so a proposal pinning a new image shows it before it lands; a later run of the state turns it to passed once every image is held. A placement pass every 15 seconds works out why each ready queued job no runner can take waits, from what each online runner offered at its last claim (scope, tier, architecture, image held, labels, memory and CPUs, a recent decline), and shows it on the job (API `waiting`, `oh run view` and `oh run watch`, its panel open on the run page) and in its check's summary, so a proposal says why its check is stuck (#28). **Diffusion** (#148, `docs/diffusion.md`): a push that moves a repository's default bookmark, or the bookmark its work lands on (the most common target of its proposals landed in the last 90 days), records each Oxygen subrepo pin in the new state (`diffusion_pins`: consumer, bookmark, path, the upstream's Oxygen repository id, the pinned heads); a hub that had no index fills it at start. `GET /repos/{owner}/{name}/consumers` lists the consumers the caller may read, each with its pin and how many changes of this repository's default bookmark it lacks (`behind`, null when the pin names changes this repository does not hold), and the overview shows them as Used by. A repository that pins, or is pinned by, another on the hub has a **Diffusion tab** (#150): Used by (each consumer's pin bookmark, how far behind, its status, its open bump and that bump's newest run, its last landed bump, how it follows) and Follows (each upstream it pins, with the same); its overview shows both in brief, and Admin › Diffusion lists every upstream with its consumers. The status is green when the upstream is checked and the consumer's run on the bump passed, amber when that run passed but the upstream has no checks, red when the run failed, grey while it runs, and up to date when there is nothing to bump. Landing a proposal on the hub refreshes the pin index and propagates as a push does. A consumer follows a subrepo's upstream as its `.oxymodules` entry says (`follow = off | green | any`, `land = propose | auto`, `target` naming the bookmark to bump, `watch` listing paths whose upstream changes are flagged), else as its repository setting says (Settings › Diffusion, or `PUT /repos/{owner}/{name}/diffusion {"follow": ...}` by its owners and admins; migration 0058), else as `[diffusion] follow` in hub.toml says (default `off`). A **bump** moves a consumer's pin at a path to a newer upstream state: one change on the target bookmark's heads that replaces only the pin (oxy-core `build_pin_change`), signed with the hub's key, on the bookmark `diffusion/<path>`, which the hub owns and rebuilds on the target each time (a forced update). When the pin has changes the upstream's state lacks (say, from an upstream proposal not landed yet), the bump moves to both, so a consumer never loses changes it relies on; it is skipped when that combination would conflict, when the pin names changes the upstream does not hold, when the pin already holds the state, or when the bump bookmark already moves it there, and the hub logs why. A bump is **proposed** once per bump bookmark, by the hub's `diffusion` account (made on first use, never able to sign in), into the target, labelled `dependencies`, its body the upstream changes the pin lacks (newest first, linking to the upstream's change pages); a later bump edits the same open proposal. It is triggered when a push moves the upstream's default bookmark (for consumers following `any`) or when a run of that bookmark passes (for `green`). Diffusion never lands a proposal. An upstream with no checks of its own (Flow off, or no run ever of its default bookmark) is **unverified**: it never turns green, so its moves also bump consumers following `green`, and those proposals are marked amber with the `upstream unverified` label and a note in the body; a later bump from a passing run removes the label. A bump already in place refreshes its proposal's text and labels. A consumer's `watch` patterns (`*` within a path segment, `**` across them, `?` one character; a plain path covers everything below it) flag the upstream changes touching a matching file: they are listed first under "Touches what you watch", with the files, and marked ⚑ in the changelog. Turning Diffusion on for a consumer (the setting or the API) catches it up at once: it is bumped to each upstream's newest state whose default-bookmark run passed, or to its newest state when it follows every move. **Images page** (#146): Admin › Runners › Images lists every rootfs image a runner holds, the hub keeps, or a job ran on in the last 90 days (from each job's sandbox), newest use first: its size and architecture, where it is (the hub, each runner holding it, each runner fetching it from the swarm now), the repositories whose jobs used it, and who sent it; an image held nowhere is marked. **The swarm** (#144, oxygenflow docs/jetstream.md §8.1): a runner in an image swarm reports its peer address, its site and the BLAKE3 of its TLS certificate with each claim (`swarm`; without a certificate it is not in the swarm), kept in memory and shown on Admin › Runners. `POST /flow/swarm/announce {image, state}` (by a runner; `fetching`, `done` or `stopped`) returns the online runners in its site that hold the image or announced fetching it in the last 45 seconds, each with its certificate's fingerprint, which the fetcher pins its TLS connection to, and a ticket signed with the hub key (JSON naming the runner, its site and an expiry 15 minutes on, hex, then the signature) that peers check against `/.well-known/oxyhub-key` before serving chunks; a runner that has not reported an address is told to add `[runner.swarm]` (409).
518 +
- **Architecture** (OxygenFlow spec §11.2–§11.3, oxygenflow#36): a runner reports each image's architecture on its claims (`image_archs`), kept in `flow_images`; a job is offered only to a runner whose architecture matches its sandbox's `arch`, else its image's as known; unknown matches, as before. The queue says which architecture a waiting job asks for. **Images by hash** (oxygenflow#39, Jetstream): `flow rootfs push HASH` sends a rootfs image as the FastCDC chunks the hub lacks (`POST /flow/images/missing`, `PUT /flow/images/chunks/{hash}`, by a token that may write repositories) and then its manifest (`PUT /flow/images/{hash}`), which the hub accepts only when every chunk is held and they make an image with that hash; chunks are kept zstd-compressed under `<data>/images`, shared by every image, and `jet_images` lists what is held. Runners and accounts fetch with `GET /flow/images/{hash}` and `GET /flow/images/chunks/{hash}`. A runner whose claims say `fetches_images` is offered jobs whose image only the hub holds, and fetches it before its preflight. Each claim also carries the runner's `facts` (OxygenFlow spec §11.1, oxygenflow#40): free disk, load, speed score and class, kernel; the hub keeps the latest on the runner, shows them on Admin › Runners and in `GET /runners`. A job's finish carries the most disk and memory it took (`peak_disk_mib`, `peak_memory_mib`, oxygenflow#41); a new job's `need_disk_mib` is the most its last five runs of the same repository and name took, plus a quarter and 64 MiB, and it is offered only to a runner whose claim reports at least that much free disk; the waiting reason says so. **Ranking** (oxygenflow#42): a job that another free runner which polled in the last 10 s would do better (it holds the job's image while the claimant would fetch it, or has a higher speed class) is held from the claimant for up to 15 s, then any runner that fits takes it; each job records why it ran where it did (`placed`: "holds the image, idle, speed 100 (class 3)", with "after N s held for a better runner" when it was), shown on the run page, in the API and by `oh run view`. A run whose jobs pin an image held by neither the hub nor a runner sets a failing `pipeline/images` check on its state (#137), naming the images and how to send them (`flow rootfs push`), so a proposal pinning a new image shows it before it lands; a later run of the state turns it to passed once every image is held. A placement pass every 15 seconds works out why each ready queued job no runner can take waits, from what each online runner offered at its last claim (scope, tier, architecture, image held, labels, memory and CPUs, a recent decline), and shows it on the job (API `waiting`, `oh run view` and `oh run watch`, its panel open on the run page) and in its check's summary, so a proposal says why its check is stuck (#28). **Diffusion** (#148, `docs/diffusion.md`): a push that moves a repository's default bookmark, or the bookmark its work lands on (the most common target of its proposals landed in the last 90 days), records each Oxygen subrepo pin in the new state (`diffusion_pins`: consumer, bookmark, path, the upstream's Oxygen repository id, the pinned heads); a hub that had no index fills it at start. `GET /repos/{owner}/{name}/consumers` lists the consumers the caller may read, each with its pin and how many changes of this repository's default bookmark it lacks (`behind`, null when the pin names changes this repository does not hold), and the overview shows them as Used by. A repository that pins, or is pinned by, another on the hub has a **Diffusion tab** (#150): Used by (each consumer's pin bookmark, how far behind, its status, its open bump and that bump's newest run, its last landed bump, how it follows) and Follows (each upstream it pins, with the same); its overview shows both in brief, and Admin › Diffusion lists every upstream with its consumers. The status is green when the upstream is checked and the consumer's run on the bump passed, amber when that run passed but the upstream has no checks, red when the run failed, grey while it runs, and up to date when there is nothing to bump. Landing a proposal on the hub refreshes the pin index and propagates as a push does. A bump's run, started by the `diffusion` account, is given the upstream it bumps as a subrepo whatever that account may read (the consumer's own pin of it was committed by someone who may write the consumer); its other subrepos need read access as usual. A consumer follows a subrepo's upstream as its `.oxymodules` entry says (`follow = off | green | any`, `land = propose | auto`, `target` naming the bookmark to bump, `watch` listing paths whose upstream changes are flagged), else as its repository setting says (Settings › Diffusion, or `PUT /repos/{owner}/{name}/diffusion {"follow": ...}` by its owners and admins; migration 0058), else as `[diffusion] follow` in hub.toml says (default `off`). A **bump** moves a consumer's pin at a path to a newer upstream state: one change on the target bookmark's heads that replaces only the pin (oxy-core `build_pin_change`), signed with the hub's key, on the bookmark `diffusion/<path>`, which the hub owns and rebuilds on the target each time (a forced update). When the pin has changes the upstream's state lacks (say, from an upstream proposal not landed yet), the bump moves to both, so a consumer never loses changes it relies on; it is skipped when that combination would conflict, when the pin names changes the upstream does not hold, when the pin already holds the state, or when the bump bookmark already moves it there, and the hub logs why. A bump is **proposed** once per bump bookmark, by the hub's `diffusion` account (made on first use, never able to sign in), into the target, labelled `dependencies`, its body the upstream changes the pin lacks (newest first, linking to the upstream's change pages); a later bump edits the same open proposal. It is triggered when a push moves the upstream's default bookmark (for consumers following `any`) or when a run of that bookmark passes (for `green`). Diffusion never lands a proposal. An upstream with no checks of its own (Flow off, or no run ever of its default bookmark) is **unverified**: it never turns green, so its moves also bump consumers following `green`, and those proposals are marked amber with the `upstream unverified` label and a note in the body; a later bump from a passing run removes the label. A bump already in place refreshes its proposal's text and labels. A consumer's `watch` patterns (`*` within a path segment, `**` across them, `?` one character; a plain path covers everything below it) flag the upstream changes touching a matching file: they are listed first under "Touches what you watch", with the files, and marked ⚑ in the changelog. Turning Diffusion on for a consumer (the setting or the API) catches it up at once: it is bumped to each upstream's newest state whose default-bookmark run passed, or to its newest state when it follows every move. **Images page** (#146): Admin › Runners › Images lists every rootfs image a runner holds, the hub keeps, or a job ran on in the last 90 days (from each job's sandbox), newest use first: its size and architecture, where it is (the hub, each runner holding it, each runner fetching it from the swarm now), the repositories whose jobs used it, and who sent it; an image held nowhere is marked. **The swarm** (#144, oxygenflow docs/jetstream.md §8.1): a runner in an image swarm reports its peer address, its site and the BLAKE3 of its TLS certificate with each claim (`swarm`; without a certificate it is not in the swarm), kept in memory and shown on Admin › Runners. `POST /flow/swarm/announce {image, state}` (by a runner; `fetching`, `done` or `stopped`) returns the online runners in its site that hold the image or announced fetching it in the last 45 seconds, each with its certificate's fingerprint, which the fetcher pins its TLS connection to, and a ticket signed with the hub key (JSON naming the runner, its site and an expiry 15 minutes on, hex, then the signature) that peers check against `/.well-known/oxyhub-key` before serving chunks; a runner that has not reported an address is told to add `[runner.swarm]` (409).
519 519
- **Variables** (oxygenhub#102): Administration, an organisation's settings and a repository's settings each hold `NAME=value` pairs (a name is a letter or `_` then letters, digits and `_`, up to 64, never `FLOW_*`; a value up to 4096 characters; up to 100 per level) under **Variables**, each page showing what it inherits and which inherited names it overrides. A claim carries the repository's effective set as `env`, inner levels overriding outer; Flow puts them in the job's environment under `FLOW_*` and the pipeline's own `env`, so they are part of the receipt. They are not secrets and are shown in full to whoever sees the page; secrets are mounted by Flow (OxygenFlow spec §9), never put in the environment. Changes are audited (`flow.variable.set`, `flow.variable.deleted`); `flow run --var NAME=VALUE` gives a local run the same.
520 520
- **Secrets** (oxygenhub#107; OxygenFlow spec §9): the same three levels hold **Secrets**, named as a pipeline's `secret "name" { mount-path "/run/secrets/…" }` names them (1 to 64 of `A-Z a-z 0-9 . _ -`, a value up to 64 KiB, up to 100 per level), typed once and sealed under `keys/secrets.key`; pages show names, when each was set and where it comes from, never a value. A claim carries only the secrets its job declares, each with its path and value, from the innermost level that sets the name; one no level sets is noted at the top of the job's log and left out, so the job fails on the missing file rather than running with a stand-in. Flow writes each as a file (0400, the job's user's, on an in-memory mount) before the job starts and never puts it in the environment, the receipt or an artifact. Log intake replaces each value, and its base64, with `***` before a record is stored. Changes are audited (`flow.secret.set`, `flow.secret.deleted`), never with the value.
521 521
- **Artifacts** (OxygenFlow spec §5.2): a job may declare artifacts with `output`.
…
231 unchanged lines