OXYGENHUB

Keyless step 2: the hub certifies fresh keys for ten minutes 026a76d251ce

POST /api/v1/keyless/certify: a running job's token, or one of the hub's
OIDC tokens for {base_url}/keyless, proves who asks; the fresh key's
challenge signature over certify_challenge(credential) proves it holds
the key; the hub answers with a certificate (Oxygen spec 14.7) signed
with its key, for ten minutes and never past the credential. Audited.
Spec 13.5.2. Test on SQLite and Postgres: a job's certificate verifies
with the hub's key and names the job; a proof for another credential,
another audience's token and a finished job's token are refused.
Pins oxygen with certificates.

Refs #241
Author
Andy McCall @andymccall <[email protected]>
via
agent Claude Code <[email protected]> · delegate for changes, until 2026-11-13 19:00Z · key ea5f78160088
When
· 2026-10-17 14:27 UTC
Signature
✔ signed by Claude Code
Hash
026a76d251ce302a7a2851dac1e4301c101948317f11876c3eae5f2942dc2844
Depends on
0b203bbf7cc3 107eaba52c9d 380996db84e5 42fa95540468 53a0fb2e4ac6 617c2840446c 6dbd999ad794 90326c9601f9 93be9ce4653e d23464c0ac60
Depended on by
27babfa16580 a66c05a6a881 205d8f863e5c 965b950f9b96 559c39204234 8aa5ef384ef1 2c31915f5c89
In bookmarks
api-unknown-fields cancel-landed-runs diffusion-pin-bookmark draft-ready main oh-comment-body-file proposal-list-fast
In tags
v0.6.0 v0.6.1 v0.7.0 v0.8.0 v0.9.0 v0.9.1 v0.9.11 v0.9.15 v0.9.16 v0.9.19 v0.9.2 v0.9.20 v0.9.21 v0.9.22 v0.9.23 v0.9.24 v0.9.3 v0.9.4 v0.9.5 v0.9.6 v0.9.7 v0.9.8 v0.9.9
Edits
6 atoms, 1 tree operation, 6 files

Browse the tree at this change

added crates/hub/src/keyless.rs +168 −0
1 +
//! Certificates for short-lived keys (oxygenhub#241, docs/keyless.md,
2 +
//! Oxygen spec §14.7): the hub vouches, for ten minutes, for a fresh key
3 +
//! whose holder proves who it is with a credential the hub issued. A Flow
4 +
//! job asks with its job token; anything else with one of the hub's OIDC
5 +
//! tokens for the keyless audience. The key signs, then is thrown away:
6 +
//! nothing long-lived sits on a runner.
7 +
8 +
use crate::api::ApiError;
9 +
use crate::{db, Shared};
10 +
use axum::extract::State;
11 +
use axum::http::{HeaderMap, StatusCode};
12 +
use axum::Json;
13 +
use oxy_core::cert::{certify_challenge, Certificate, SignedCertificate};
14 +
use oxy_core::identity::{verify_challenge, Identity, Scope, Signature};
15 +
use serde::{Deserialize, Serialize};
16 +
17 +
/// How long a certificate lasts.
18 +
pub const LIFETIME_MS: i64 = 10 * 60 * 1000;
19 +
/// How far back a certificate starts, for clocks a little behind.
20 +
const SKEW_MS: i64 = 30 * 1000;
21 +
22 +
/// The OIDC audience a token must have to be exchanged here.
23 +
pub fn audience(base_url: &str) -> String {
24 +
    format!("{}/keyless", base_url.trim_end_matches('/'))
25 +
}
26 +
27 +
#[derive(Deserialize)]
28 +
pub struct CertifyRequest {
29 +
    /// The fresh key's identity object, base64.
30 +
    pub identity: String,
31 +
    /// The key's signature over `certify_challenge(credential)`, base64 CBOR.
32 +
    pub proof: String,
33 +
    /// `changes,refs` and so on.
34 +
    pub scope: String,
35 +
}
36 +
37 +
#[derive(Serialize)]
38 +
pub struct CertifyResponse {
39 +
    /// The signed certificate, base64 (`[certificate, signature]`).
40 +
    pub certificate: String,
41 +
    pub subject: String,
42 +
    pub expires_ms: i64,
43 +
}
44 +
45 +
/// Who a credential says its holder is.
46 +
struct Proven {
47 +
    subject: String,
48 +
    claims: Vec<(String, String)>,
49 +
    /// The credential's own end, which a certificate never outlives.
50 +
    until_ms: i64,
51 +
    target: String,
52 +
}
53 +
54 +
fn bad(status: StatusCode, why: impl Into<String>) -> ApiError {
55 +
    ApiError(status, why.into())
56 +
}
57 +
58 +
/// A running job's token: its repository, bookmark, run and job.
59 +
async fn job_proof(app: &Shared, token: &str) -> Result<Proven, ApiError> {
60 +
    let invalid = || bad(StatusCode::UNAUTHORIZED, "this job token is not valid (the job must be running)");
61 +
    let hash = crate::auth::token_hash(token);
62 +
    let job = db::flow::job_by_token(&app.db, &hash, crate::flow::JOB_TOKEN_MAX_AGE_MS)
63 +
        .await
64 +
        .map_err(|e| bad(StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
65 +
        .ok_or_else(invalid)?;
66 +
    let run = db::flow::run_by_id(&app.db, job.run_id)
67 +
        .await
68 +
        .map_err(|e| bad(StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
69 +
        .ok_or_else(invalid)?;
70 +
    let repo = db::repos::get_by_id(&app.db, run.repo_id)
71 +
        .await
72 +
        .map_err(|e| bad(StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
73 +
        .ok_or_else(invalid)?;
74 +
    let mut claims = vec![
75 +
        ("repository".to_string(), format!("{}/{}", repo.owner, repo.name)),
76 +
        ("bookmark".to_string(), run.bookmark.clone()),
77 +
        ("run".to_string(), run.number.to_string()),
78 +
        ("job".to_string(), job.name.clone()),
79 +
        ("runner".to_string(), job.runner.clone()),
80 +
    ];
81 +
    if !job.environment.is_empty() && !job.approved_by.is_empty() {
82 +
        claims.push(("environment".to_string(), job.environment.clone()));
83 +
        claims.push(("approved_by".to_string(), job.approved_by.clone()));
84 +
    }
85 +
    Ok(Proven {
86 +
        subject: format!("repo:{}/{}:bookmark:{}", repo.owner, repo.name, run.bookmark),
87 +
        claims,
88 +
        until_ms: job.started_ms.unwrap_or(0) + crate::flow::JOB_TOKEN_MAX_AGE_MS,
89 +
        target: format!("{}/{}", repo.owner, repo.name),
90 +
    })
91 +
}
92 +
93 +
/// One of this hub's OIDC tokens for the keyless audience.
94 +
async fn oidc_proof(app: &Shared, token: &str, now: i64) -> Result<Proven, ApiError> {
95 +
    let issuer = app.oidc().await.map_err(|e| bad(StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
96 +
    let aud = audience(app.config.base_url());
97 +
    let claims = issuer
98 +
        .verify(token, &aud, now)
99 +
        .ok_or_else(|| bad(StatusCode::UNAUTHORIZED, format!("the token is not one of this hub's, for audience {aud}, still valid")))?;
100 +
    let subject = claims["sub"].as_str().unwrap_or("").to_string();
101 +
    if subject.is_empty() {
102 +
        return Err(bad(StatusCode::UNAUTHORIZED, "the token names no subject"));
103 +
    }
104 +
    let keep = ["repository", "bookmark", "run_number", "job", "runner", "environment", "approved_by"];
105 +
    let mut out = Vec::new();
106 +
    for k in keep {
107 +
        match &claims[k] {
108 +
            serde_json::Value::String(s) => out.push((k.to_string(), s.clone())),
109 +
            serde_json::Value::Number(n) => out.push((k.to_string(), n.to_string())),
110 +
            _ => {}
111 +
        }
112 +
    }
113 +
    Ok(Proven {
114 +
        target: claims["repository"].as_str().unwrap_or(&subject).to_string(),
115 +
        subject,
116 +
        claims: out,
117 +
        until_ms: claims["exp"].as_i64().unwrap_or(0) * 1000,
118 +
    })
119 +
}
120 +
121 +
/// `POST /api/v1/keyless/certify`: a ten-minute certificate for a fresh
122 +
/// key, for whoever the bearer credential proves.
123 +
pub async fn certify(
124 +
    State(app): State<Shared>,
125 +
    headers: HeaderMap,
126 +
    Json(req): Json<CertifyRequest>,
127 +
) -> Result<Json<CertifyResponse>, ApiError> {
128 +
    let token = oxy_server::bearer_token(&headers)
129 +
        .ok_or_else(|| bad(StatusCode::UNAUTHORIZED, "ask with a job token or one of this hub's OIDC tokens as the bearer"))?
130 +
        .to_string();
131 +
    let now = db::now_ms();
132 +
    let proven = if token.starts_with(crate::flow::JOB_TOKEN_PREFIX) {
133 +
        job_proof(&app, &token).await?
134 +
    } else {
135 +
        oidc_proof(&app, &token, now).await?
136 +
    };
137 +
    let decode = |what: &str, b64: &str| {
138 +
        oxy_core::dsse::base64_decode(b64.trim()).ok_or_else(|| bad(StatusCode::BAD_REQUEST, format!("{what} is not base64")))
139 +
    };
140 +
    let identity =
141 +
        Identity::decode(&decode("identity", &req.identity)?).map_err(|e| bad(StatusCode::BAD_REQUEST, format!("identity: {e}")))?;
142 +
    let proof = oxy_core::cbor::decode(&decode("proof", &req.proof)?)
143 +
        .map_err(|e| e.to_string())
144 +
        .and_then(|v| Signature::from_value(&v).map_err(|e| e.to_string()))
145 +
        .map_err(|e| bad(StatusCode::BAD_REQUEST, format!("proof: {e}")))?;
146 +
    verify_challenge(&identity, &proof, &certify_challenge(&token))
147 +
        .map_err(|e| bad(StatusCode::UNAUTHORIZED, format!("the proof does not show this key's holder asked: {e}")))?;
148 +
    let scope = Scope::parse(&req.scope).map_err(|e| bad(StatusCode::BAD_REQUEST, e))?;
149 +
    let expires = (now + LIFETIME_MS).min(proven.until_ms);
150 +
    if expires <= now {
151 +
        return Err(bad(StatusCode::UNAUTHORIZED, "the credential is about to expire: ask with a fresh one"));
152 +
    }
153 +
    let (key, secret) =
154 +
        crate::keys::load_secret(&app.config.keys_dir()).map_err(|e| bad(StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
155 +
    let cert = Certificate {
156 +
        key: identity,
157 +
        issuer: key,
158 +
        subject: proven.subject.clone(),
159 +
        claims: proven.claims,
160 +
        scope,
161 +
        not_before: (now - SKEW_MS) as u64,
162 +
        expires: expires as u64,
163 +
    };
164 +
    let signed = SignedCertificate::sign(cert, &app.hub_key.identity, &secret);
165 +
    let detail = format!("{} for {} until {}", signed.cert.key_id().short(12), proven.subject, expires);
166 +
    db::accounts::audit(&app.db, None, "keyless.certified", &proven.target, &detail).await.ok();
167 +
    Ok(Json(CertifyResponse { certificate: oxy_core::dsse::base64(&signed.encode()), subject: proven.subject, expires_ms: expires }))
168 +
}
modified crates/hub/src/api/mod.rs +1 −0
…
42 unchanged lines
43 43
pub fn router(asset_limit: usize) -> Router<Shared> {
44 44
    Router::new()
45 45
        .fallback(no_route)
46 +
        .route("/keyless/certify", post(crate::keyless::certify))
46 47
        .route("/flow/claim", post(crate::flow::claim))
47 48
        .route("/flow/runners/register", post(crate::flow::register))
48 49
        .route("/flow/runners/heartbeat", post(crate::flow::heartbeat))
…
2633 unchanged lines
modified crates/hub/src/lib.rs +1 −0
…
17 unchanged lines
18 18
pub mod hooks;
19 19
pub mod images;
20 20
pub mod junit;
21 +
pub mod keyless;
21 22
pub mod keys;
22 23
pub mod levels;
23 24
pub mod live;
…
319 unchanged lines
modified crates/hub/tests/flow.rs +71 −0
…
4701 unchanged lines
4702 4702
    let q = db::flow::queue(&h.app.db).await.unwrap();
4703 4703
    assert!(q.iter().filter(|e| e.name == "later").all(|e| e.priority() == "high"));
4704 4704
}
4705 +
4706 +
/// Keyless (oxygenhub#241): a running job's token gets a ten-minute
4707 +
/// certificate for a fresh key from the hub, signed with the hub's key and
4708 +
/// naming the job's repository and bookmark; a proof made for another
4709 +
/// credential, an OIDC token for another audience, and a finished job's
4710 +
/// token get nothing.
4711 +
#[tokio::test(flavor = "multi_thread")]
4712 +
async fn a_running_job_gets_a_short_certificate_for_a_fresh_key() {
4713 +
    use oxy_core::cert::{certify_challenge, SignedCertificate};
4714 +
    use oxy_core::identity::{Scope, SecretKey};
4715 +
    let h = hub().await;
4716 +
    create(&h.app, "andy", "demo").await;
4717 +
    let work = tempfile::tempdir().unwrap();
4718 +
    let c = client(&format!("{}/andy/demo", h.base), work.path());
4719 +
    write(work.path(), ".flow/pipeline.kdl", &pipeline(""));
4720 +
    write(work.path(), "test.sh", "echo ok\n");
4721 +
    c.snapshot("test").unwrap();
4722 +
    c.commit("Add a pipeline", &[]).unwrap();
4723 +
    assert!(c.push("origin", false).unwrap().iter().all(|r| r.ok));
4724 +
    assert_eq!(runs_of(&h.app, "andy", "demo").await.len(), 1);
4725 +
    let job = claim(&h.app).await.expect("a job");
4726 +
    let token = job["token"].as_str().unwrap().to_string();
4727 +
    let job_id = job["job_id"].as_i64().unwrap();
4728 +
4729 +
    let sk = SecretKey::generate();
4730 +
    let id = sk.identity("job", "[email protected]", 1);
4731 +
    let ask = |credential: &str, signed_for: &str| {
4732 +
        let proof = sk.sign_challenge(&id.hash(), &certify_challenge(signed_for)).unwrap();
4733 +
        let body = serde_json::json!({
4734 +
            "identity": oxy_core::dsse::base64(&id.encode()),
4735 +
            "proof": oxy_core::dsse::base64(&oxy_core::cbor::encode(&proof.to_value())),
4736 +
            "scope": "changes,refs",
4737 +
        });
4738 +
        let (credential, app) = (credential.to_string(), h.app.clone());
4739 +
        async move { call(&app, "POST", "/api/v1/keyless/certify", Some(&credential), Some(body.to_string())).await }
4740 +
    };
4741 +
    let (st, body) = ask(&token, &token).await;
4742 +
    assert_eq!(st, StatusCode::OK, "{body}");
4743 +
    let v: serde_json::Value = serde_json::from_str(&body).unwrap();
4744 +
    let sc = SignedCertificate::decode(&oxy_core::dsse::base64_decode(v["certificate"].as_str().unwrap()).unwrap()).unwrap();
4745 +
    assert!(sc.verify(&h.app.hub_key.identity).is_ok(), "signed with the hub's key");
4746 +
    assert_eq!(sc.cert.key_id(), id.hash());
4747 +
    assert_eq!(sc.cert.subject, "repo:andy/demo:bookmark:main");
4748 +
    assert_eq!(sc.cert.claim("repository"), Some("andy/demo"));
4749 +
    assert!(sc.cert.claim("job").is_some() && sc.cert.claim("run") == Some("1"));
4750 +
    assert_eq!(sc.cert.scope, Scope::CHANGES | Scope::REFS);
4751 +
    assert!(sc.cert.expires - sc.cert.not_before <= 10 * 60_000 + 30_000, "ten minutes");
4752 +
4753 +
    // A proof made for another credential is not this one's.
4754 +
    let (st, body) = ask(&token, "flowjob_someone_else").await;
4755 +
    assert_eq!(st, StatusCode::UNAUTHORIZED, "{body}");
4756 +
    assert!(body.contains("the proof does not show"), "{body}");
4757 +
4758 +
    // The hub's own OIDC token, for the keyless audience, works too; for
4759 +
    // another audience it does not.
4760 +
    let issuer = h.app.oidc().await.unwrap();
4761 +
    let now = oxygenhub::db::now_ms();
4762 +
    let aud = oxygenhub::keyless::audience(h.app.config.base_url());
4763 +
    let good = issuer.mint_for(h.app.config.base_url(), "agent:claude@andy", &aud, now, 600);
4764 +
    let (st, body) = ask(&good, &good).await;
4765 +
    assert_eq!(st, StatusCode::OK, "{body}");
4766 +
    assert!(body.contains("\"subject\":\"agent:claude@andy\""), "{body}");
4767 +
    let other = issuer.mint_for(h.app.config.base_url(), "agent:claude@andy", "sts.amazonaws.com", now, 600);
4768 +
    let (st, _) = ask(&other, &other).await;
4769 +
    assert_eq!(st, StatusCode::UNAUTHORIZED);
4770 +
4771 +
    // A job that has finished asks with a token that no longer counts.
4772 +
    sqlx::query("UPDATE flow_jobs SET status = 'passed' WHERE id = $1").bind(job_id).execute(&h.app.db).await.unwrap();
4773 +
    let (st, body) = ask(&token, &token).await;
4774 +
    assert_eq!(st, StatusCode::UNAUTHORIZED, "{body}");
4775 +
}
replaced deps/oxygenflow +0 −0

No line changes.

modified docs/spec.md +4 −0
…
587 unchanged lines
588 588
589 589
Each user and organisation backs up its repositories hourly to a place it controls: a GCS bucket reached keylessly (the hub mints a short-lived token with `sub = backup:OWNER`, which GCP's workload identity federation exchanges for an access token, so no key exists), or a directory the site allows. A run sends only the content-addressed objects the destination lacks, the refs at that moment and the hub's records (issues, proposals, releases, wiki, settings; never secret values), and is whole once its `DONE` marker is written; `oxygenhub admin restore` brings repositories back from any whole run. Built: Settings › Backups and an organisation's Settings › Backups (how often: off, hourly or daily; a directory from `hub.toml` `[backups] directories` or a GCS bucket with its workload identity provider; a prefix; runs kept 7, 30 or 90 days; "back up now"; the last 50 runs with their outcome), migration 0060 (`backup_settings`, `backup_runs`), the hub's scheduler (one run at a time; a run left running by a restart is marked failed), each repository's and wiki's store copied file by file under the repository's read lock for the final pass, the records as JSON (every table keyed by the repository, and three levels of rows pointing at them; tables and columns holding secrets, tokens, sessions or passwords never), pruning of old runs and unreferenced blobs, an inbox notice to the owner (an organisation's owners) once per spell of failures, and `oxygenhub admin backup run OWNER` and `oxygenhub admin backup restore --from DIR [--run AT]`, which restores the stores and the repositories' rows. A GCS destination is keyless: the hub mints a 10-minute token (`Issuer::mint_for`, `sub = backup:OWNER`, `aud` the provider's name over https), Google's STS exchanges it (`[backups] sts_url`) for an access token scoped `devstorage.read_write`, used with the Cloud Storage JSON API (`[backups] storage_url`) and exchanged again before it expires; TEST writes, reads and removes a probe. The design, the layout at a destination, setup and restore are in `docs/backups.md`.
590 590
591 +
### 13.5.2 Certificates for short-lived keys (oxygenhub#241, docs/keyless.md, step 2 built 2026-10-17)
592 +
593 +
`POST /api/v1/keyless/certify` gives a fresh key a certificate (Oxygen spec §14.7) for ten minutes, signed with the hub's key. The bearer is the credential: a running job's token (subject `repo:OWNER/NAME:bookmark:B`, claims `repository`, `bookmark`, `run`, `job`, `runner`, and `environment` with `approved_by` once an approved job may deploy), or one of this hub's OIDC tokens for the audience `{base_url}/keyless` (its `sub` the subject, and the same claims it carries). The body is `{"identity": base64 identity object, "proof": base64 CBOR Signature, "scope": "changes,refs"}`, the proof the fresh key's challenge signature (§14.3) over `certify_challenge(credential)` (`"oxy1 certify "` and the BLAKE3 of the credential), so it cannot be replayed with another credential. The answer is `{"certificate": base64 [certificate, signature], "subject", "expires_ms"}`; a certificate never outlives its credential, and it starts 30 s back for clocks a little behind. Each one is audited (`keyless.certified`). Which issuers and subjects a repository trusts is its policy's, and signing with these keys is steps 3 to 5 of docs/keyless.md.
594 +
591 595
### 13.6 Coverage reports (oxygenhub#113, done 2026-10-06)
592 596
593 597
Coverage is built in, as Coveralls or Codecov would be, fed by OxygenFlow.
…
167 unchanged lines