OXYGENHUB

Trusted time on the hub: serve when witnesses first saw an entry 13b27cb2200c

The hub side of #240 (the oxygen side is fd97f4c8). The hub keeps each
cosigned checkpoint by its size (migration 0083, both databases),
lists its witnesses at GET /log/witnesses and in every refs answer
(key 7), and for each logged bookmark serves the first cosigned
checkpoints that hold its entry, with inclusion proofs (ref key 7), from
which oxy takes the integrated time. A test runs it end to end: two
hubs, one witnessing, a client verifying the integrated time.

Pins oxygenflow 013e670e (oxygen fd97f4c8: trusted time, #101 identical
additions unify, #126 marker-like text). Two tests follow: a web edit
leaving oxy's own marker in is refused (the test used a made-up marker,
which is now ordinary text), and the 'no runner can take it' test
settles when the hub's own placement pass races it. docs/transparency.md
says what is built and what is not yet.

Closes #240
Author
Andy McCall @andymccall <[email protected]>
via
agent agent:claude@andymccall <[email protected]> · keyless, under your grant for changes,refs,seal · key 27b43ae806f1
When
· 2026-10-18 21:06 UTC
Signature
◈ sealed by agent:claude@andymccall
Hash
13b27cb2200c246f36d31d83144c047a61ccd07f12b9aa61c2a69c114bb11e2b
Depends on
145306044896 380996db84e5 3b1bc27fe805 42fa95540468 4f40eed0588f 67040b158ae2 6e006eaf1da4 7c06f5b28ade 8a2dbcb56246 965b950f9b96 d23464c0ac60 e5b97ac399e7
Depended on by
b5c199bd6b1a 76fdeedccf44
In bookmarks
api-unknown-fields cancel-landed-runs diffusion-pin-bookmark draft-ready main proposal-list-fast
In tags
v0.6.0 v0.6.1 v0.7.0 v0.8.0 v0.9.0 v0.9.1 v0.9.11 v0.9.15 v0.9.16 v0.9.19 v0.9.2 v0.9.20 v0.9.21 v0.9.22 v0.9.23 v0.9.24 v0.9.3 v0.9.4 v0.9.5 v0.9.6 v0.9.7 v0.9.8 v0.9.9
Edits
16 atoms, 2 tree operations, 9 files

Browse the tree at this change

added crates/hub/migrations-pg/0083_tlog_cosigned_notes.sql +8 −0
1 +
-- Trusted time (oxygenhub#240): each checkpoint the witnesses cosigned,
2 +
-- by its size, with the cosignatures it gathered, so a fetch can show when
3 +
-- a witness first saw an entry (Oxygen spec §14.8, integrated time).
4 +
CREATE TABLE tlog_cosigned_notes (
5 +
    size       BIGINT PRIMARY KEY,
6 +
    note       TEXT NOT NULL,
7 +
    updated_ms BIGINT NOT NULL
8 +
);
added crates/hub/migrations/0083_tlog_cosigned_notes.sql +8 −0
1 +
-- Trusted time (oxygenhub#240): each checkpoint the witnesses cosigned,
2 +
-- by its size, with the cosignatures it gathered, so a fetch can show when
3 +
-- a witness first saw an entry (Oxygen spec §14.8, integrated time).
4 +
CREATE TABLE tlog_cosigned_notes (
5 +
    size       INTEGER PRIMARY KEY,
6 +
    note       TEXT NOT NULL,
7 +
    updated_ms INTEGER NOT NULL
8 +
);
modified crates/hub/src/db/tlog.rs +24 −0
…
94 unchanged lines
95 95
    .await?;
96 96
    Ok(())
97 97
}
98 +
99 +
/// Keep the cosigned checkpoint of `size` entries, with every cosignature
100 +
/// it has gathered so far (#240).
101 +
pub async fn keep_cosigned_note(pool: &Pool, size: i64, note: &str) -> sqlx::Result<()> {
102 +
    sqlx::query(
103 +
        "INSERT INTO tlog_cosigned_notes (size, note, updated_ms) VALUES ($1, $2, $3) \
104 +
         ON CONFLICT (size) DO UPDATE SET note = $2, updated_ms = $3",
105 +
    )
106 +
    .bind(size)
107 +
    .bind(note)
108 +
    .bind(super::now_ms())
109 +
    .execute(pool)
110 +
    .await?;
111 +
    Ok(())
112 +
}
113 +
114 +
/// The first `n` cosigned checkpoints that hold entry `index`: (size, note).
115 +
pub async fn cosigned_notes_after(pool: &Pool, index: i64, n: i64) -> sqlx::Result<Vec<(i64, String)>> {
116 +
    sqlx::query_as("SELECT size, note FROM tlog_cosigned_notes WHERE size > $1 ORDER BY size LIMIT $2")
117 +
        .bind(index)
118 +
        .bind(n)
119 +
        .fetch_all(pool)
120 +
        .await
121 +
}
modified crates/hub/src/tlog.rs +23 −0
…
452 unchanged lines
453 453
        return backfill;
454 454
    };
455 455
    refs.log = Some(note);
456 +
    // The witnesses a client may pin, for integrated times (#240).
457 +
    refs.witnesses = app.tlog.witnesses().iter().map(|p| (p.name.clone(), p.identity.encode())).collect();
456 458
    for r in refs.refs.iter_mut() {
457 459
        let Some(a) = &r.attestation else { continue };
458 460
        let object = a.attest.hash();
…
4 unchanged lines
463 465
            Ok(Some(i)) if (i as u64) < size => {
464 466
                let (Ok(entry), Ok(proof)) = (app.tlog.entry(i as u64), app.tlog.prove_inclusion(size, i as u64)) else { continue };
465 467
                r.logged = Some(oxy_core::protocol::Logged { index: i as u64, entry, proof });
468 +
                // When witnesses first saw it (#240): the first cosigned
469 +
                // checkpoints that hold it, each with the entry's proof.
470 +
                let notes = tokio::task::block_in_place(|| {
471 +
                    tokio::runtime::Handle::current().block_on(db::tlog::cosigned_notes_after(&app.db, i, 2))
472 +
                })
473 +
                .unwrap_or_default();
474 +
                for (at, text) in notes {
475 +
                    if let Ok(proof) = app.tlog.prove_inclusion(at as u64, i as u64) {
476 +
                        r.integration.push(oxy_core::protocol::Integration { note: text, proof });
477 +
                    }
478 +
                }
466 479
            }
467 480
            Ok(Some(_)) => {}
468 481
            Ok(None) => backfill.push(Pending { kind: "ref", repo: a.attest.repo, repo_row: row, object, signers: sig_keys(&a.sigs) }),
…
72 unchanged lines
541 554
        }
542 555
    }
543 556
    if changed {
557 +
        // Kept by size too, for integrated times (#240).
558 +
        db::tlog::keep_cosigned_note(&app.db, cp.size as i64, &note.encode()).await.ok();
544 559
        if let Err(e) = app.tlog.set_note(cp.size, note.encode()) {
545 560
            eprintln!("tlog: keeping the cosigned checkpoint: {e}");
546 561
        }
…
77 unchanged lines
624 639
        Some(note) => ([(header::CONTENT_TYPE, "text/plain; charset=utf-8"), (header::CACHE_CONTROL, "no-cache")], note).into_response(),
625 640
        None => plain(StatusCode::SERVICE_UNAVAILABLE, "no checkpoint yet: the log signs one within seconds"),
626 641
    }
642 +
}
643 +
644 +
/// `GET /log/witnesses` (#240): this log's witnesses, one `NAME BASE64`
645 +
/// line each (the identity object), which `oxy tlog verify` pins.
646 +
pub async fn witnesses(State(app): State<Shared>) -> Response {
647 +
    let body: String =
648 +
        app.tlog.witnesses().iter().map(|p| format!("{} {}\n", p.name, oxy_core::dsse::base64(&p.identity.encode()))).collect();
649 +
    ([(header::CONTENT_TYPE, "text/plain; charset=utf-8"), (header::CACHE_CONTROL, "no-cache")], body).into_response()
627 650
}
628 651
629 652
/// `GET /log/tile/{L}/{N}[.p/{W}]` and `/log/tile/entries/{N}[.p/{W}]`:
…
89 unchanged lines
modified crates/hub/src/web/mod.rs +1 −0
…
85 unchanged lines
86 86
        .route("/metrics", get(crate::metrics::metrics))
87 87
        .route("/log", get(transparency::page))
88 88
        .route("/log/checkpoint", get(crate::tlog::checkpoint))
89 +
        .route("/log/witnesses", get(crate::tlog::witnesses))
89 90
        .route("/log/tile/{*path}", get(crate::tlog::tile))
90 91
        .route("/log/witness/add-checkpoint", axum::routing::post(crate::tlog::add_checkpoint))
91 92
        .route("/.well-known/oxyhub-ca.pem", get(home::hub_ca))
…
765 unchanged lines
modified crates/hub/tests/edits.rs +5 −5
…
282 unchanged lines
283 283
    let blamed = send(app, "GET", "/andy/demo/file/main/a.txt?blame=1", Some(&andy), None).await;
284 284
    assert_eq!(blamed.status, 200, "{}", blamed.body);
285 285
    assert!(blamed.body.contains("oxy conflict") && blamed.body.contains("class=\"blame\"></td>"), "{}", blamed.body);
286 -
    // Leaving markers in is refused; removing them resolves.
286 +
    // Leaving one of oxy's own markers in is refused (marker-like text that
287 +
    // isn't one is ordinary text, oxygen#126); removing them resolves.
288 +
    let raw = send(app, "GET", "/andy/demo/raw/main/a.txt", Some(&andy), None).await.body;
289 +
    let marker = raw.lines().find(|l| l.starts_with("<<<<<<< oxy conflict")).expect("a marker").to_string();
287 290
    let r = send(
288 291
        app,
289 292
        "POST",
290 293
        "/andy/demo/edit/main",
291 294
        Some(&andy),
292 -
        Some(&format!(
293 -
            "_csrf={csrf}&path=a.txt&original=a.txt&content={}&message=half&mode=commit",
294 -
            enc("<<<<<<< oxy conflict 1 of 1, side 1 of 2: change x \"y\"\nLeft\n")
295 -
        )),
295 +
        Some(&format!("_csrf={csrf}&path=a.txt&original=a.txt&content={}&message=half&mode=commit", enc(&format!("{marker}\nLeft\n")))),
296 296
    )
297 297
    .await;
298 298
    assert_ne!(r.status, StatusCode::SEE_OTHER);
…
572 unchanged lines
modified crates/hub/tests/flow.rs +39 −3
…
2606 unchanged lines
2607 2607
    assert!(why.contains("earth-9 does not hold image") && why.contains(&ROOTFS[..12]), "{body}");
2608 2608
    let (_, page) = call(&h.app, "GET", "/andy/demo/flow/1", None, None).await;
2609 2609
    assert!(page.contains("data-waiting") && page.contains("does not hold image"), "the run page says it");
2610 -
    let checks = db::hooks::checks_for_state(&h.app.db, run.repo_id, &run.state).await.unwrap();
2611 -
    let check = checks.iter().find(|c| c.name == "flow/build").unwrap();
2612 -
    assert!(check.summary.contains("does not hold image"), "{check:?}");
2610 +
    // The check says it too. The hub's own placement pass runs beside this
2611 +
    // test's: if it lands a reading from before the runner's offer, the
2612 +
    // next pass puts it right.
2613 +
    let mut summary = String::new();
2614 +
    for _ in 0..40 {
2615 +
        let checks = db::hooks::checks_for_state(&h.app.db, run.repo_id, &run.state).await.unwrap();
2616 +
        summary = checks.iter().find(|c| c.name == "flow/build").unwrap().summary.clone();
2617 +
        if summary.contains("does not hold image") {
2618 +
            break;
2619 +
        }
2620 +
        let build = db::flow::jobs(&h.app.db, run.id).await.unwrap().into_iter().find(|j| j.name == "build").unwrap();
2621 +
        db::flow::set_waiting(&h.app.db, build.id, "").await.unwrap();
2622 +
        oxygenhub::flow::explain_waits(&h.app).await.unwrap();
2623 +
        tokio::time::sleep(std::time::Duration::from_millis(50)).await;
2624 +
    }
2625 +
    assert!(summary.contains("does not hold image"), "{summary}");
2613 2626
}
2614 2627
2615 2628
async fn put_bytes(app: &Shared, path: &str, bearer: &str, bytes: Vec<u8>) -> (StatusCode, String) {
…
2535 unchanged lines
5151 5164
        assert!(Note::parse(&served).unwrap().cosigned_by(&name, &wid).is_ok());
5152 5165
    }
5153 5166
    let cp = Checkpoint::parse(&Note::parse(&log.app.tlog.checkpoint().unwrap()).unwrap().text).unwrap();
5167 +
5168 +
    // Trusted time (#240): the log lists its witnesses, and a fetch's refs
5169 +
    // carry, for a logged bookmark, the cosigned checkpoints that first held
5170 +
    // its entry, from which the client takes the integrated time.
5171 +
    let (_, listed) = call(&log.app, "GET", "/log/witnesses", None, None).await;
5172 +
    assert!(listed.starts_with(&format!("{name} ")), "{listed}");
5173 +
    let url = format!("{}/andy/demo", log.base);
5174 +
    let refs = tokio::task::spawn_blocking(move || {
5175 +
        let opts = RemoteOptions { token: Some("site".into()), ..Default::default() };
5176 +
        open_remote(&url, &opts).unwrap().refs().unwrap()
5177 +
    })
5178 +
    .await
5179 +
    .unwrap();
5180 +
    assert_eq!(refs.witnesses.len(), 1);
5181 +
    assert_eq!(refs.witnesses[0].0, name);
5182 +
    let main = refs.refs.iter().find(|r| r.name == "main").expect("main");
5183 +
    let logged = main.logged.as_ref().expect("logged");
5184 +
    assert!(!main.integration.is_empty(), "a witnessed checkpoint holds it");
5185 +
    let cosigned: Vec<(&str, &[[u8; 32]])> = main.integration.iter().map(|i| (i.note.as_str(), i.proof.as_slice())).collect();
5186 +
    let at = oxy_core::tlog::integrated_time(&logged.entry, logged.index, &cosigned, &origin, &log_id, &[(name.clone(), wid.clone())])
5187 +
        .expect("an integrated time");
5188 +
    assert_eq!(at.witness, name);
5189 +
    assert!(at.size > logged.index);
5154 5190
5155 5191
    let ask = |req: AddCheckpoint| {
5156 5192
        let app = witness.app.clone();
…
592 unchanged lines
replaced deps/oxygenflow +0 −0

No line changes.

modified docs/transparency.md +15 −0
…
116 unchanged lines
117 117
- **Before the log exists for a hub,** or for objects older than it,
118 118
  verification falls back to today's behaviour and says so.
119 119
120 +
**Built (2026-10-18, oxygen fd97f4c8 and the hub after it):** the hub keeps
121 +
each cosigned checkpoint by its size (migration 0083,
122 +
`tlog_cosigned_notes`), lists its witnesses at `GET /log/witnesses` and in
123 +
every refs answer (key 7), and for each logged bookmark serves the first
124 +
cosigned checkpoints that hold its entry, with inclusion proofs (ref key 7).
125 +
oxy pins the witnesses on first use beside the log key, takes the earliest
126 +
trusted cosignature as the integrated time, and checks a delegate's refs
127 +
delegation and a short-lived key's certificate at it (Oxygen spec §14.8).
128 +
Not yet: revocations carry no time today, so they stay absolute rather
129 +
than "revoked at T" (a decision for Andy, with revocation times); a
130 +
failure at the integrated time is flagged, not refused; one trusted
131 +
witness is enough (a k-of-n policy would take the k-th earliest time);
132 +
only bookmark attestations get a time so far (tags, seals and receipts
133 +
next).
134 +
120 135
**RFC 3161 timestamps** from a timestamp authority (TSA) are the other way
121 136
to get independent time. They are worth adding later for objects that never
122 137
reach a hub (a local signature, say), but they need ASN.1 and CMS parsing,
…
58 unchanged lines