| **Scope** | A self-hosted forge for Oxygen repositories: hosting, browsing, review, issues, releases, an HTTP API and webhooks, and the integration point for a runner. It works like GitHub where GitHub is right and differs where the commutative model makes something better possible. |
7
| **Depends on** | `oxy-core` and `oxy-server` from the Oxygen workspace (spec Draft 0.8b); `oxy` clients speaking wire protocol version 1 |
18. [Required changes to Oxygen](#18-required-changes-to-oxygen)
36
19. [Milestones](#19-milestones)
37
20. [Open questions](#20-open-questions)
38
21. [Changelog](#21-changelog)
39
40
---
41
42
## 1. Goals and non-goals
43
44
1. **Work like GitHub.** A developer who knows GitHub must find repositories, history, diffs, review, issues, releases, tokens and webhooks where they expect them, with the same words where the words still apply. The learning curve is Oxygen's model, not the forge.
45
2. **Show what the model makes possible.** Multiple heads, reified conflicts, exact blame, signatures and trust policy, proposals that never go stale, review comments that survive rebases, and approvals that are portable signed objects. Each of these is a first-class page, not a footnote.
46
3. **One binary, one directory.** `oxygenhub serve` links `oxy-core`, embeds the wire protocol server, renders its own pages, and keeps everything under one data directory with SQLite for hub state. A backup is a copy of that directory. There is no separate database server, job queue, cache server or front-end build.
47
4. **The repository is the source of truth.** Everything about code, history, references, signatures and policy lives in the Oxygen object store and is readable with `oxy` alone. Hub tables hold only what Oxygen has no object for: accounts, sessions, issues, review threads, webhooks and check results. Deleting the hub database loses discussion, never history.
48
5. **Off the shelf where it is not the product.** Markdown editing uses EasyMDE, in-browser code editing uses CodeMirror 6, syntax highlighting uses syntect, Markdown rendering uses pulldown-cmark with ammonia sanitising. The hub's own code is the forge logic and the pages.
49
6. **Honest state.** A page never shows something the object store does not say. A signature is shown as verified only when the hub verified it against policy; an approval is shown only when the approving seal exists.
Non-goals for 1.0: hosting Git repositories natively (§14 covers mirrors), a marketplace, discussions, GitHub Pages, and federation between hubs. The runner is specified separately (§13, `docs/runner.md`) and is not required for 1.0.
| Branch | **Bookmark** | An Oxygen bookmark names a set of heads, not one commit |
59
| Commit | **Change** | Changes commute; a change has dependencies, not a parent |
60
| Commit page shows one parent diff | Change page shows the change's own edits, its dependencies and its dependents | The diff of a change is intrinsic; it is the same on every bookmark that contains it |
61
| Merge, merge commit, rebase before merge | **Landing**: add the proposal's changes to the target bookmark | Union of heads; no merge commit; the result is computed before landing, so CI runs on exactly what lands |
62
| Pull request "out of date with base branch" | Never happens; a proposal stays valid as the target moves | Changes commute unless they conflict; a new conflict is shown, not a stale flag |
63
| Merge conflict blocks the PR until the author rebases | Conflict is a reified part of the resulting state; the proposal page shows the exact sides, and any collaborator may push a resolution change | Conflicts are values in the model (spec §9) |
64
| Review comment on a line, goes "outdated" after a force-push | Review comment anchored to an **atom** `(ChangeHash, index)`, follows the line through rebase, squash and split | Atoms are stable identities; rewrites record correspondences |
65
| Approval is a database row | Approval is a **seal** attestation signed by the reviewer's key, stored in the repository | Verifiable offline and after export; survives hub loss |
66
| Required status checks on a commit SHA | Required checks on a **state hash** | A state is content addressed, so a green check on the pre-landing state is exactly the check on the post-landing state |
67
| Branch protection rules | The repository's **trust policy** (spec §14.5), edited through signed proposals | One authority for who may update a reference, enforced identically by every client and server |
68
| Signed commits as a badge | Signature and seal status on every change, with the policy rule that admitted it | Signatures are the protocol, not an add-on |
69
| Tags and releases | Tags are signed attestations; releases attach notes and assets | — |
70
| Actions | Runner (§13), later | Designed from the failures of Actions and GitLab CI |
71
| Fork and PR from a fork | Fork, and proposals from a fork or from a bookmark in the same repository | Same |
| Personal access tokens | Same, scoped | Push also accepts the client's identity signature (§5.4) |
74
| Webhooks with HMAC | Same | — |
75
| REST and GraphQL APIs | JSON REST only | — |
76
77
## 3. Architecture
78
79
### 3.1 One process
80
81
`oxygenhub` is a Rust binary in its own Cargo workspace at `../oxygenhub`, depending on `oxy-core` and `oxy-server` by path (developer edition) or by version (release). It runs one tokio runtime with:
82
83
- an axum router serving pages, static assets, the JSON API, and the wire protocol under every repository's path;
84
- askama templates compiled into the binary; one hand-written stylesheet, also compiled in and served at `/style.css`;
85
- SQLite through `sqlx` (WAL mode, `synchronous=NORMAL`, foreign keys on) for hub state, opened from `<data>/hub.db`;
86
- a small background worker pool (tokio tasks) for webhooks delivery, notification email, search indexing, Git mirror export and cache warming;
- an event bus for live pages and workers (`crates/hub/src/bus.rs`): pushes, checks, Flow job changes, comments, waiting webhook deliveries and the resolver's choices are published on it and heard by the pages' server-sent event streams and the webhook worker. On SQLite it is in process; built for PostgreSQL, a hub run as several processes (`--role web,worker`, docs/roles.md) carries every event to the others over `LISTEN/NOTIFY`, at most once: events say "look now" and name ids, never the data.
There is no separate front-end build. JavaScript is limited to: the theme cookie, EasyMDE (Markdown editing), CodeMirror 6 (file editing), and small progressive enhancements (history graph hover, keyboard shortcuts, live check status). Every page MUST render and every form MUST work with JavaScript disabled, except the two editors, which degrade to a plain `<textarea>`.
`<owner>` and `<name>` MUST match `[A-Za-z0-9][A-Za-z0-9._-]{0,63}` and MUST NOT be `.`, `..`, or end in `.git` (reserved for the mirror URL). The reserved top-level path segments are `api`, `static`, `login`, `logout`, `register`, `settings`, `explore`, `search`, `notifications`, `new`, `orgs`, `admin`, `style.css`, `favicon.ico`, `favicon.svg`, `health`, `avatar`, `help`, `projects`, `assets`, `.well-known`, `live`, `csp-report`, `security.txt`, `security-contact`, `log`, `metrics`, `insights` and `drops`; no owner may take those names. No repository may be named `drops` (in any case): `/{owner}/drops` is the owner's Drops page (§4.5). A drop's repository is `repos/<owner>/drops/<id>/`.
3. Page and API handlers resolve `{owner}/{name}` to a `RepoHandle`, check the viewer's role, and run every `oxy-core` call inside `spawn_blocking` under the repository's locks (§16.2: readers never wait for writers).
templates/ askama; base.html, style.css, one file per page
131
assets/ wordmark, favicons, vendored EasyMDE and CodeMirror bundles
132
migrations/ sqlx migrations
133
docs/ this file, runner.md, deploy.md
134
```
135
136
## 4. Repositories on the server
137
138
### 4.1 Bare repositories
139
140
A hosted repository is an Oxygen repository directory with no working copy checked out and no user workspace. `oxy-core` today always keeps one working-copy change per workspace; the hub never snapshots, commits from a checkout, or touches files outside `.oxy/`. The directory contains only `.oxy/`. §18 lists the small change that lets `Repo::open` skip working-copy bookkeeping for such a directory.
141
142
Creation: `Repo::init` under `<data>/repos/<owner>/<name>` with a fresh repository id, then an initial row in `repositories`. Import: a client `oxy push` into an empty hub repository; or, for Git, §14.
Deletion moves the directory to `<data>/trash/<owner>/<name>-<timestamp>` and a job removes it after `trash_days` (default 30). **Transfer** (oxygenhub#108, Settings › Advanced, the repository's admins) moves it to a user or organisation the admin may create repositories under, keeping its name: a directory move under the repository's write lock (the Git mirror moves with it), the row's owner, and a `repo_redirects` row so the old address redirects on the web (the rest of the path kept) and still resolves for `oxy` remotes and the API. **Archive** marks it read-only with a reason: pushes are refused by the wire protocol, and the web refuses edits, issues, proposals, comments and reviews, while reading, cloning and the admins' settings go on; the header shows an *archived* badge and the admins bring it back from the same place. Both are audited (`repo.transferred`, `repo.archived`, `repo.unarchived`). The **Advanced** section also holds Delete and, ahead of wikis (oxygenhub#104), a disabled Delete wiki data.
### 4.1a Repositories in the owner's bucket (oxygenhub#96)
147
148
A repository can keep its data in its owner's bucket instead of on the hub's disk.
149
150
- **Creating:** the New repository form's Storage section takes a store URL (`s3://BUCKET/PREFIX`, `gs://BUCKET/PREFIX` or `az://ACCOUNT/CONTAINER/PREFIX`, with `?region=` the only setting allowed) and the credentials for it: an S3 access key, a Google service account key, or an Azure service principal. The hub refuses a profile, a key file, an endpoint or `dir://`, which would make it use its own host's credentials, reach its own network or write to its own disk.
151
- **The check:** before anything is made, the bucket or prefix must be empty and pass the store conformance check under a probe prefix. Nothing is created when either fails.
152
- **What the hub keeps:** a bare repository directory whose `storage.url` names the store, with a cache in front (oxygen#68), and a `repo_storage` row holding the URL and the credentials, sealed under `keys/secrets.key`. At start the hub hands every repository's credentials to the cloud stores by URL, so nothing from its host's environment or cloud profiles is used for a repository.
153
- **Pages:** the header shows a *bucket* tag; its title names the URL to the repository's admins only. Settings › Storage shows the store and the last check, checks it again, and replaces the credentials once new ones pass the check. Both are audited as `repo.storage`.
- **Organisation policy:** Settings › Storage on an organisation can require every new repository to live in a bucket, and can list the store URL prefixes allowed (its own buckets, so its data stays where they are). The New repository form refuses anything else for that organisation, saying what is allowed. Repositories that exist are not moved. Changes are audited as `org.storage_policy`.
155
- **Not yet:** forking such a repository is refused (clone and push instead). Deleting one leaves its data in the bucket, which is the owner's. The Sync status and runs on references moved outside the hub are still to come.
A fork copies the object store (hard links where the file system allows, otherwise a copy) and records `forked_from`. Because objects are content addressed and repository ids differ, a proposal from a fork carries the changes across repositories by hash; the hub fetches missing objects from the fork's store directly, never over the network.
160
161
### 4.3 The default bookmark
162
163
Each repository has a default bookmark (`main` unless changed). It is what the repository page, tree, file views and search show, and the target every proposal picks first.
164
165
### 4.4 Visibility
166
167
`public` (readable by anyone including anonymous clients), `internal` (readable by every signed-in account), `private` (readable by collaborators). Wire protocol reads obey the same rule: a private repository answers `404` to an unauthorized client, not `401`, so its existence is not disclosed.
A **drop** is a small snippet of one or more files, as a gist is, and like a gist it is a repository: each drop is its own Oxygen repository, with its own history, visibility and address, cloned and pushed with `oxy`.
172
173
- **Names and addresses:** a drop's repository is named `drops/ID` under its owner (a user), where ID is 24 random lowercase hex digits, unique on the hub (96 bits, so an address cannot be guessed). No repository name can hold a slash and none may be `drops`, so the two never collide. The drop's page is `/{owner}/drops/{id}`, and `oxy clone https://HUB/{owner}/drops/{id}` takes that address: the wire protocol answers at `/{owner}/drops/{id}/oxy/v1/…` with the drop's repository behind it, under the same locks, limits and push path as any repository. The row is in `repositories` with `kind = 'drop'` (migration 0087) and a `title`; its `description` is the drop's.
174
- **Visibility:** *public* drops are listed on their owner's Drops page for everyone; *secret* drops are listed only to their owner (and site administrators) and are readable by anyone with the address, signed in or not, on the web, the API and the wire protocol (a secret drop is stored as `private`, so anything that lists by visibility hides it; `auth::roles` grants the read). The transparency log page names a secret drop's repository to nobody but those it is listed to.
175
- **Never among repositories:** drops are left out of every list, count and search of repositories: profiles, organisation pages, the dashboard and its sky, Explore, `/search`, `GET /api/v1/repos`, forks, backups (for now), Diffusion and the Flow scheduler. `/api/v1/repos/{owner}/drops%2F{id}` is 404; a drop's repository pages under `/{owner}/drops/{id}/…` (history, change, raw) are read-only. Upkeep that covers every stored repository (gc, the history scan, Admin › Storage, which labels them, `oxygenhub admin repo list` and `oxygenhub check`) includes them.
176
- **Pages:** `/drops` goes to your own Drops page, `/{owner}/drops`, which lists drops newest first with their files, and `/drops/new` makes one: a title (optional: the first file's name otherwise), a description, public or secret, and one or more files, each a name and a text area (hub.js adds and removes rows; without it the form has its rows and one empty one). The drop's page shows every file rendered (Markdown as a document, other text highlighted, binary or large files as a download link), the clone command, and for its owner Edit and Delete. Edit is the same form; saving makes one change with only what differs (a file whose name is cleared is removed; files that are not text, pushed with `oxy`, are kept). History (`…/changes/main`), each change with its diff (`…/change/{hash}`), raw files (`…/raw/{ref}/{path}`) and the files at a change (`…/tree/{ref}`) are the repository pages with the drop's own bar (Files, History) in place of the tabs.
177
- **Changes:** what the web and the API make is authored by the owner's linked identity and signed by the hub as its delegate, exactly as web edits are (§6.9), so the owner must have delegated to the hub's key first; the forms say how. The first change makes `main`. At most 50 files of 1 MiB each go in one change from the web or the API (larger ones go with `oxy push`).
178
- **Who may do what:** anyone with the address reads a drop; only its owner writes (the web, the API, and pushes with the owner's token); its owner or a site administrator deletes it. Flow never runs on a drop; its pushes ring no webhooks and close no issues, and are recorded in its push log.
179
- **Push protection** (§15.4) applies to a drop's pushes as to any repository's, with no fingerprint to allow (a drop has no settings), and the hub runs the same rules over what it is asked to commit from the web or the API, refusing before anything is stored.
180
- **Deleting** moves the repository to the trash under its exclusive lock (oxygenhub#249) and removes the row; the address stops answering. Audited as `drop.created`, `drop.edited`, `drop.visibility` and `drop.deleted`.
181
- **API:** `GET /api/v1/drops` (yours, `repo:read`), `POST /api/v1/drops` (`NewDrop`, `repo:write`), `GET /api/v1/users/{name}/drops` (someone's public drops), `GET /api/v1/drops/{id}` (anyone with the id; each text file's content up to 512 KiB), `PATCH /api/v1/drops/{id}` (`DropEdit`: title, description, `secret`, and files to write or, with `content: null`, remove; others are kept) and `DELETE /api/v1/drops/{id}`. `oh drop create|list|view|delete` (docs/oh.md).
182
- **Later:** forks, stars, comments, embedding a drop in another page, and drops in backups.
An account has a username (same grammar as an owner name), a display name, an email (verified before it is used for anything), an Argon2id password hash, an optional TOTP secret, a session salt (rotating it signs out every session), and a set of linked Oxygen identities (§5.3). The first account created becomes the site administrator, and only with the hub's one-time setup code (oxygenhub#245): with no accounts, the hub keeps a random code in `<data>/setup-code` (owner-only, never served) and logs it at start, or uses `[registration] setup_code` (`OXYHUB_REGISTRATION_SETUP_CODE`) for automation; registering with it spends it, and a sign-in provider never makes the first account. `oxygenhub admin user create --admin` needs no code. Registration is open, invitation only, or closed, per configuration.
Sessions are server-side records (oxygenhub#223): the cookie `oxyhub=<id>` holds 32 random bytes in hex, and the hub keeps only the id's BLAKE3 hash, with the account, the account's session salt at sign-in, the times it was made and last used, and the address and browser it came from. A session is good while its salt is still the account's, it has been used in the last 14 days, and it is under 30 days old; the cookie is `HttpOnly`, `Secure` when served over HTTPS, `SameSite=Lax`. Every sign-in (after the code, with two-factor) makes a new session; signing out deletes its record. Password changes, resets and "sign out everywhere" rotate the salt, which ends every session at once; Settings › Security lists the signed-in browsers and signs any one out. Ended sessions are swept nightly. CSRF tokens are bound to the session id. Sign-in is also slowed against guessing (§15.6, oxygenhub#221). Passkeys (WebAuthn) are planned for 1.0 as a second factor or password replacement.
An administrator sets up each provider under Administration › Sign in with: the OAuth client id, the client secret (sealed under `keys/secrets.key`, never shown again) and a switch; the page shows the callback URL to register with the provider (`/login/{provider}/callback`). The login page then offers the usable providers (oxygenhub#106). The flow is OAuth 2.0's authorization code: `/login/{provider}` sets a state cookie (a nonce and the purpose, MACed with the cookie key, ten minutes, path `/login`) and sends the person to the provider; the callback checks the state against the cookie, swaps the code for a token server to server, and asks who it is (GitHub: `/user` and the primary verified address from `/user/emails`; Google: the OpenID `userinfo`, the address only when `email_verified`). Then: a subject already linked signs in; else an account with that verified email is linked and signed in; else, when registration is open, or it is by invitation and an unused invitation came through the round trip (`/login/{provider}?invite=CODE`, carried in the signed state as the purpose `invite:CODE`, used up when the account is made; the registration page shows the provider buttons for a usable invitation, oxygenhub#214), an account is made from the provider's login name (made valid and free), with an unusable password and the email verified when the provider vouched for it; otherwise the person is told to register or sign in and connect. A signed-in person connects or disconnects a provider under Settings › Security; a subject can be linked to one account. Links live in `account_logins`. Audited as `admin.signin`, `signin.connected`, `signin.disconnected`, `login` and `register` with the provider. For tests, `[auth] signin_mock` names a base URL that stands in for every provider.
Both use **account links**: 32 random bytes as hex in a URL, stored only as their BLAKE3 hash, bound to one account and one purpose, expiring, and consumed on first use.
199
200
- **Verification.** Setting or changing an email marks it unverified. When the hub can send mail it sends a link to the new address (valid 48 hours). The link verifies that exact address: one sent to an address the account has since replaced does nothing. The settings page shows whether the email is verified and can send a new link: at most one every two minutes for the same address, and ten an hour per account whatever the addresses, so the form cannot be used to mail other people. Notification email goes only to verified addresses.
201
- **Reset by email.** `/login/reset` takes a username or a verified email. When it matches an active account with a verified email and the hub can send mail, a link valid for one hour goes to that address, at most one every two minutes per account. The page answers the same way whether or not anything matched, so it cannot be used to find accounts.
202
- **Reset by an administrator.** When the hub cannot send mail, or the account has no verified email, an administrator issues a reset link from the admin page (valid 24 hours), shown once to hand over by other means. The audit log records who issued it.
203
- **Using a reset link** sets a new password (the same rules as registration), rotates the session salt so every existing session ends, voids the account's other reset links, and records `password.reset` in the audit log. It does not turn off TOTP: a reset account with two-factor sign-in still needs its code. A banned account cannot be reset.
- **Mail.** `[email] smtp` is an `smtp://` or `smtps://` URL, or `file:///path` for development, which writes each message to that directory as a file instead of sending it. Administration has a Mail panel (#101, #181) whose settings, once saved, override `[email]`: pick a service and give only what it needs (Resend: its API key; Amazon SES: the region and SES SMTP credentials; Mailgun: `us` or `eu` and the SMTP login; another SMTP server: server, port, security, user and password; or an SMTP URL, for anything else and `file://`), and the panel stores one standard SMTP URL, so every service is SMTP underneath and any provider works. Keys and passwords are never shown again; left empty, the saved one stays while the service, server and user do. An `smtp://` URL that says nothing of TLS requires STARTTLS (`?tls=opportunistic` or `?tls=none` say otherwise; a relay on this machine needs none). Off turns mail off, and a button sends a test message to the administrator's own address. Every message goes as plain text and as HTML in the site's look (#182): the Oxygen band with the mark, sent inline so it shows without loading anything, the subject as heading, the body with its links, the message's action as a button with its link beneath, and a footer of the site's links (Explore, Help, Notification settings, and Stop these emails on notifications).
An organisation is an owner with members and teams. Roles on an organisation: `owner`, `member`. Roles on a repository, whether personal or organisational: `read`, `triage` (manage issues and proposals without writing), `write`, `maintain` (settings except deletion and secrets), `admin`. Teams hold members and receive repository roles as a unit. An organisation's default repository role for members is configurable.
An Oxygen identity is a signed object holding an Ed25519 + ML-DSA-65 key pair's public halves (spec §14.2). An account links an identity by uploading the identity object (`oxy identity export`) and proving possession: the hub issues a 32-byte challenge and the client signs it with `oxy identity sign-challenge` (§18); the hub verifies both signatures. A linked key is what makes a pushed change *this account's* change: author attribution in every page comes from the change's signing key, not from who held the token that pushed it. A change signed by an unlinked key shows its identity's name (or KeyId when the identity is unknown) with a "key not linked" tag rather than a user name.
An identity can be linked to at most one account. Rotation (`oxy identity rotate`, `renew`, `delegate add|remove`) produces a successor identity whose object names the old key. When a push brings changes authored by a successor of a linked identity, and every step of the chain kept the key pair, the hub links the successor to the same account itself and records `identity.rotated` in the audit log: delegating to an agent never breaks attribution and needs no re-upload. A rotation to a fresh key pair (`oxy identity rotate`) still needs the new identity linked with a challenge.
1. `Authorization: Bearer <token>` with a personal access token holding `repo:write` on that repository;
221
2. an HTTP signature (spec §15.1): the client signs a server challenge with its identity key and the key is linked to an account with `write`. Reference implementation support is a §18 item; until then tokens are the only method.
222
223
Reads on `public` repositories are anonymous. Reads on `internal` and `private` repositories require the same credentials with `repo:read`.
224
225
### 5.5 Two authorities, one answer
226
227
The hub's roles decide who may reach a repository, open issues, create proposals, edit settings, and *attempt* a push. The repository's Oxygen trust policy (spec §14.5) decides which keys may update which references and whether conflicts may land on them. Both must agree for a push to succeed; the hub MUST NOT bypass or reimplement policy, and it MUST run the same `receive_push` validation `oxy-core` runs everywhere else.
228
229
A repository with no policy accepts any reference update from any account with `write`. The settings page says so plainly and offers to create a policy. Policy editing in the browser produces a **policy proposal**: a signed policy object that roots sign with `oxy policy sign` from their own clients; the hub collects signatures until the threshold is met, then applies the update as an ordinary push. The hub never holds a root key.
The hub has one identity of its own, the **hub delegate key**, generated at `init` and published at `/.well-known/oxyhub-key`. It is used only to sign changes the hub itself creates on behalf of a signed-in user (web edits, §6.9; proposal landings, §7.6). Such a change carries the user's linked key as `author` and the hub key as signer, which the author-or-delegate rule (spec §14.4) accepts only when the user's identity object lists the hub key as a delegate with the `changes` and `refs` scopes (Oxygen spec §14.2; the hub signs the change and attests the reference), still in force: the user opts in once with `oxy identity delegate add <hub key> --scope changes,refs` and re-links the rotated identity (§5.3). Policy is then checked exactly as for the user's own signature, since a rule that names the author covers the author's delegates (spec §14.5). If the user has not delegated, or the policy does not authorise them for the reference, the hub refuses the action and shows the client recipe. Delegation is per user and revocable by the user alone (`oxy identity delegate remove`); the hub never needs a policy entry of its own.
Tokens are 32 random bytes, shown once, stored as BLAKE3 hashes, with a name, an optional expiry, and scopes: `repo:read`, `repo:write`, `repo:admin`, `issues:write`, `proposals:write`, `user:read`, `hooks:admin`, optionally restricted to listed repositories. Tokens authenticate the API and the wire protocol; they never authenticate the pages.
**Device sign-in** (in the shape of RFC 8628) gives a token to an app that cannot take a password, such as `oh` or a phone client. The app posts `{client, scopes}` to `/api/v1/device/code` without a token and gets a device code (kept, hashed, by the hub), a user code `XXXX-XXXX` (consonants), `verification_uri` (`/device`), `verification_uri_complete`, `expires_in` (15 minutes) and `interval` (5 seconds). It shows the user code and link, and polls `/api/v1/device/token` with `{device_code}`, getting `authorization_pending`, `slow_down` (polled within the interval), `access_denied` or `expired_token` as `{error, error_description}` with 400. The user opens `/device`, signed in (with their second factor), sees the app's name and the scopes it asked for, and approves with any non-empty subset of them, or denies. The next poll makes a token named `device: CLIENT` with the approved scopes and returns `{access_token, token_type: "bearer", scope, username}`, once: the code is then gone. No token exists until that poll, so none is stored in the clear. Approvals, denials and device tokens are in the audit log.
The default look (since 2026-10-01, oxygenhub#74) is close to GitHub's, chosen for readability and familiarity: Primer colours, light by default (a `theme` cookie of `dark` or `auto` changes it), the system font stack at 14px, rounded boxes with divided rows, the site band and repository tabs as underline navigation, grey buttons with green primary actions, sentence case throughout, and pill labels. It is `assets/github.css`, loaded over the stylesheet below with every rule scoped to `:root[data-look="github"]`, which the page sets unless the browser chose the classic look with `?look=classic` (a cookie keeps the choice; `?look=github` returns to the default). The Oxygen mark stays the logo, with the wordmark OxygenHub.
The classic look follows the RetroPlay Network's shape (dark by default, `Barlow` for text and `Press Start 2P` for the wordmark and first columns, square corners, thin `--line` rules, a `--panel` header and footer, a full-width `--band` navigation bar with the active tab underlined in `--acc`) with the palette taken from the Oxygen mark shared with the VS Code extension: the mark's cyan `#00F0FF` to blue `#0072FF` gradient on `#070c18`, so `--acc` is cyan, `--acc-2` blue, and the band a deep blue. The mark itself is the logo in the header, the footer and the favicon; the wordmark beside it reads OXYGEN**HUB**. The stylesheet starts from `retroplay-manager/templates/style.css` lines 1–171 (tokens, shell, band, tabs, main, tables, forms, footer) and adds forge components; the store, showcase and game components are not carried over.
The theme is dark, light, or auto (follow the system's `prefers-color-scheme`, and change with it), chosen with the switch in the header (it cycles) or under Settings → Appearance, and kept in a `theme` cookie for a year; dark when unset. A script in the page head applies it before first paint. Syntax highlighting uses the highlighter's scope names as classes prefixed `sy-`, so no scope can collide with the page's own classes.
--ok --bad --warn (new: #ffb347 dark / #b8651b light)
255
--add --del --add-ink --del-ink (new: diff backgrounds and text)
256
--m1 … --m5 (kept for signature status colours: root, signed, sealed, unsigned, invalid)
257
```
258
259
Theme selection: cookie `theme=light|dark|auto`; `auto` follows `prefers-color-scheme`. The `<head>` sets `data-theme` before paint exactly as retroplay-manager does. Fonts are vendored under `/static/fonts/` rather than loaded from Google, so a hub on a private network paints without internet access.
The logo is `assets/icons/oxygenhub.svg`, a copy of the extension's `oxygen.svg` (two linked rings, the molecule), inlined in the header at 22px beside the wordmark.
`header` (wordmark, global search box, "+" menu for new repository/issue/proposal, notifications bell with unread count, avatar menu) · `.band` (Repositories · Explore · Proposals · Issues · Notifications, with admin links pushed right) · repository `.tabs` (Code · Changes · Proposals · Issues · Flow · Releases · Wiki · Artifacts ▾ · More ▾ · Settings, open proposals and issues with a count pill; oxygenhub#254). **Artifacts** holds Releases, Build artifacts, Images, Terraform state and Reports (when there are any); **More** holds Bookmarks, Tags, Diffusion (when used), Trust, Pushes and Conflicts. The menus are `<details>`, so they open without JavaScript; hub.js closes one when another opens, on a click outside and on Escape. Under 760 pixels, where the bar scrolls sideways, a menu opens inline in the bar. Projects belong to the owner and are reached from the site band and the owner's profile; a state with conflicts says so on its overview · `main` at 1180px max (wider than retroplay's 980 because diffs need it; tree and file pages use 100% with 22px gutters) · `footer`.
| `/explore` | Every repository the viewer may see; `q` filters by name, description or topic, `topic` by one topic, `sort` is `activity` (default), `stars` or `name`; each row shows its topics and stars. Topics are set under repository settings (letters, digits and dashes, at most 20); anyone signed in can star a repository, and the overview shows star and watcher counts |
| `/{owner}` | A person: identicon, display name, bio, location, website, accounts elsewhere (a Bluesky handle, a Mastodon address and a GitHub username, each checked and normalised under Settings › Profile and shown with the service's icon as a `rel="me"` link; oxygenhub#105), linked signing keys, organisations, repositories the viewer can see, and their recent activity (never the email). An organisation: identicon, description, members with roles, teams with sizes, repositories, recent activity in them. `/avatar/{name}` serves the account's or organisation's picture: an uploaded photo or logo when there is one, otherwise the identicon (SVG, a mirrored 5×5 pattern and hue from BLAKE3 of the lower-cased name). Pictures are uploaded under Settings › Profile, or an organisation's settings by its owners: PNG, JPEG or WebP recognised by content (not by name), at most 5 MiB and 8 000 pixels a side, decoded, centre-cropped to a square and re-encoded at 400×400 (PNG when it has transparency, JPEG otherwise), so no uploaded bytes are ever served. Removing the picture brings the identicon back. Pictures are cached for five minutes and carry an ETag. |
| `/{owner}/{name}` | Repository overview (§6.4); while empty, a quickstart with the exact commands for this hub (remote, certificate, token, push; start from nothing; import from Git) |
| `/{owner}/{name}/fork`, `/{owner}/{name}/forks` | Fork under an owner the viewer may create repositories for (same visibility, own repository id, objects hard-linked, bookmarks copied, policy not copied); the list of forks the viewer can see. The overview shows "forked from", the fork count, and on a fork a link that opens a proposal upstream with `owner/name:bookmark` as its source |
| `/{owner}/{name}/tree/{ref}/{path}` | Directory listing at a bookmark, tag, change or state (§6.5). A `{ref}` is a bookmark name, a tag name, a change hash or unique prefix, or a state: full change hashes joined with `+` (`a1…+b2…`), the state those heads make, which is how a state with several heads is addressed permanently |
| `/{owner}/{name}/raw/{ref}/{path}` | A file's bytes, as `?plain=1` serves them (images inline with a sniffed type, anything else as an attachment); relative images in rendered Markdown point here |
| `/{owner}/{name}/compare/{a}...{b}` | Changes in `b` not in `a`, and the diff between the two states. Each side is picked from the bookmarks and tags, or typed as a change or state; a swap link reverses them. When both are bookmarks, `b` has changes `a` lacks and the viewer can write, **Propose this** opens a new proposal of `b` into `a` (or links the one already open). `/compare` alone compares the most recently changed other bookmark against the default one; `/compare?base=…&head=…` is the pickers' form |
| `/{owner}/{name}/bookmarks` | Bookmarks with heads, ahead/behind the default, last update, attestation status. Writers delete any bookmark but the default one: what `oxy push --delete` does, through the same checks (refused under a trust policy, which needs a signed attestation only a client can make), logged on the pushes page and sent to webhooks as a push. A pushed deletion of the default bookmark (the hub's setting, not the repository's config) is refused |
| `/{owner}/{name}/proposals`, `/proposal/{n}` | §7. The list filters by state and by `q` (`author:`, `label:`, `target:` and words), sorts by recently updated (default), newest or oldest, and pages 25 at a time |
287
| `/{owner}/{name}/issues`, `/issue/{n}` | §8. The list filters by state and by `q` (`author:`, `label:`, `assignee:`, `milestone:` and words), sorts by newest (pinned first, default), oldest, recently updated or most comments, and pages 25 at a time. The Proposals and Issues tabs show how many are open |
| `/{owner}/{name}/releases`, `/tags` | §9; the two are views of the Releases tab |
289
| `/{owner}/{name}/artifacts` | Build artifacts: the newest 200 artifacts of every run, with the run, job, size, BLAKE3 and when, each linking its download (oxygenhub#254) |
290
| `/{owner}/{name}/images` | The runner images the repository's jobs ran on in the last 90 days: architecture and size when the hub keeps the image, the jobs that used it and when last |
| `/{owner}/{name}/insights`, `/insights/{owner}`, `/admin/insights` | Insights (§13.5.6): OxygenFlow's charts for a repository (anyone who reads it), an owner (the owner, its members, administrators) or the site (administrators) |
| `/{owner}/{name}/terraform` | Terraform state (§13.5.4): the backend block, how a pipeline and a person authenticate, and the states with their serials, last writer, lock and versions kept |
| `/{owner}/{name}/settings/{page}` | A repository's settings (oxygenhub#278), one page per setting, with a sidebar of grouped pages beside it and the current one marked (`aria-current`); on a narrow screen the sidebar is a compact list above the page. **General**: General (`general`; `/settings` shows it), Collaborators (`collaborators`). **Code**: Landing rules (`landing`), Required checks (`checks`), Push protection (`push-protection`). **Flow**: Pipelines (`flow`: OxygenFlow on or off), Runs & priority (`runs`: priority, newest run wins, pushing without a run), Gates & agents (`gates`), Runners (`runners`), Variables (`variables`), Secrets (`secrets`), Coverage (`coverage`). **Integrations**: Webhooks (`webhooks`), AI assistants (`assistants`), Git mirror (`git-mirror`), Diffusion (`diffusion`). **Data**: Storage (`storage`, a repository kept in a bucket only), Backup (`backup`). **Danger zone**: Advanced (`advanced`: transfer, archive, delete wiki data, delete). The repository's maintainers have the pages; Collaborators, Runners, Variables, Secrets, Webhooks, AI assistants, Storage and Advanced are its admins' only, left out of the sidebar and not found for others. An archived repository's settings are its admins' only. Each form posts to its own path, as before, and redirects to the page that holds it with a notice or an error; a runner's registration token is shown once on the Runners page. A kept link to the former one-page settings, `/settings#section`, opens the page that has that section (`hub.js`). Names such as *Settings › Push protection* are these pages |
| `/help` | Getting started with this hub's own address, remote name and certificate filled in (identity, pushing, cloning, proposals, delegating to the hub, pipelines and stamps), the keyboard, and a glossary for people coming from GitHub. Linked from the header band, the footer and the keyboard sheet |
| `/admin/*` | Site administration (site administrators only; others get not found), a sidebar of grouped pages beside the page shown, the current one marked (`aria-current`), as a repository's settings (oxygenhub#288): **Overview** (`/admin`: the site's numbers and a card per group), **People** (Accounts `/admin/accounts`, Registration & invitations `/admin/registration`, Organisations `/admin/organisations`), **Sign-in** (Sign in with `/admin/signin`, Security contact `/admin/security`), **Mail** (`/admin/mail`), **Flow** (Runners, Images, Runner settings, Insights, Variables `/admin/variables`, Secrets `/admin/secrets`), **Integrations** (AI assistants `/admin/assistants`, Diffusion), **Data** (Storage & maintenance, Backups), **Audit** (`/admin/audit`, paged with `?audit_before=`). Each form posts to its own path, as before, and answers with its own page; a kept `/admin#section` link opens that section's page (`hub.js`). Accounts, registration mode, invitations, audit log; `/admin/storage` lists every repository's size on disk, loose objects and packs, and runs maintenance (`oxy gc` with the grace period kept, under the repository's write lock) on one repository or all, audited |
The default bookmark's tree at the root, the rendered `README.md` (or `README`, `README.txt`) below it, and a right column: description and topics, clone URL with copy button (`oxy clone https://…/{owner}/{name}`), heads count with a link when more than one, bookmarks, tags, latest release, collaborators, license detected from `LICENSE`, and the trust badge: **policy**, **no policy**, or **policy expired**.
307
308
A repository whose default bookmark has more than one head shows a `--warn` strip: "main has 3 heads; the tree below is their union" with a link to the bookmarks page. Conflicts in that state show a `--bad` strip linking to the conflicts page.
309
310
### 6.5 Tree
311
312
Directory rows show name, the last change touching the entry (message and relative time, from `log_for_path`, computed lazily per row over the API so a large directory paints immediately), size, and mode glyphs (executable, symlink, subrepository). A subrepository row links to its pin: another hub repository at a state, or a Git URL at a SHA.
313
314
### 6.6 File view and blame
315
316
Text files render with syntect line highlighting, line numbers as anchors (`#L12`, `#L12-L20` selects a range), a "raw", "history", "blame", "edit" toolbar, and the file's last change. Markdown renders by default with a toggle to source. Images and PDFs render inline; other binaries offer download. Files above `render_limit` (default 2 MiB) show the first part and a download link.
317
318
Blame uses `Repo::blame`: every line is exact, because each line is an atom with one authoring change. The gutter shows the change's short hash, author and age, grouped into runs, with the change's signature glyph. Blame is exact through rebase and squash, so there is no "ignore revs" file.
A sidebar beside the file lists the tree down to it: the root and each directory on the file's path expanded (at most 200 entries each, then a link to the directory), the file marked, other directories linking to their tree pages. It is hidden on narrow screens.
321
322
The permalink of a file view is the same path at its state: the target's heads joined with `+` in place of the bookmark or tag. `y` replaces the address with the permalink, keeping the line anchor, as GitHub does. When a line or range is selected, a bar offers **Copy lines** (the text) and **Copy permalink** (the permalink with `#L12-L20`).
323
324
The blame gutter shades each run by the age of its change, from the file's oldest (lightest) to newest (strongest) line, in ten steps, with the age also written out so colour is not the only carrier. Each run links **blame before this change**: the same file blamed at the state of the change's dependencies (the state its author built on), so a line can be traced back through every edit. A change without dependencies has no such link; one with more than 32 has none either, to keep addresses bounded.
**Function history.** For Rust, TypeScript, Python and shell files, tree-sitter finds the functions and methods (qualified by their impl, trait or class, as `Repo::log`) and the file page's sidebar lists them. Each links to `/{owner}/{name}/function/{ref}/{path}?start=…&end=…&name=…`: every change that wrote or deleted one of the function's lines, or deleted a line from between them (Oxygen spec §27, the query `oxy log -L` runs), newest first, with a graph of how those changes depend on one another. It is exact because lines keep their identity across edits; changes elsewhere in the file are not listed. A cross-file semantic graph (callers, callees) is future work.
A dependency graph, not a parent chain. The default view lists the bookmark's changes newest first with lanes drawn between a change and its dependencies (the same layout algorithm as the VS Code extension's webview, ported to Rust and rendered server side as SVG). Heads are marked. Filters: author, path, date, signed/unsigned, message text. Each row: short hash, message first line, author (name, or KeyId when unlinked), relative time, signature glyph, bookmark and tag chips, a conflict chip when the change introduces one.
**Swimlanes.** `/{owner}/{name}/lanes/{ref}` draws a bookmark's newest changes (up to 60) as a board: each lane of the shared lane layout (`oxy_core::lanes`, as `oxy log` and the list view use) is a row, time runs left to right, every change is a card linking to its page, and an edge joins a change to each change of the board it depends on. Independent work stays in separate lanes and there are no merge nodes, since Oxygen has none. The history page links to it.
**Planning on the board.** When the bookmark is the source of an open or draft proposal that the viewer wrote or can write to, the board is editable. Dragging a change onto another asks `POST /{owner}/{name}/lanes/{ref}/plan` (form `change`, `target`) what the drop could mean. The answer is JSON: a **squash** (fold the change into the target) when every dependency of the change is the target or already beneath it, and a **rebase** (make the change depend on the target) unless it already does. Nothing is offered when the target depends on the change, or when either is already on the proposal's target bookmark. The hub does not rewrite the changes itself: rewritten changes must be signed by their author, so each option carries the `oxy` commands to run and a `vscode://oxygenhub.oxygen-vscode/plan?…` link that does the same in the VS Code extension, then force-pushes the bookmark.
Header: full hash, message, author identity (linked account or KeyId), timestamp, signature status with the policy rule that admitted it (or "unsigned", or "invalid"), seals covering it, the bookmarks and tags containing it. Body: dependencies and dependents as chips, the change's edits as a diff (unified or split, per-user preference), atom counts, and the raw change (`oxy show --json`) behind a details block.
341
342
Actions for `write`: open a proposal containing this change, tag it, copy hash. Describing, abandoning, squashing and rebasing are client operations and are not offered here.
Every diff (change, compare, proposal files) shows each file as a collapsible section, open unless it has more than 400 rows. Within a block of changed lines, removals come first and each is paired with the addition in the same position; a pair similar enough (at least 40% of its tokens shared, tokens being identifiers, whitespace runs and single other characters) marks the tokens that changed. `?w=1` hides whitespace-only changes: a pair equal once whitespace is removed is shown as one unchanged line. With scripts, a toolbar switches between unified and split (side by side, pairs opposite each other) views, remembered in a `diff` cookie, and collapses or expands every file.
`CodeMirror 6` with the file's language mode. Saving creates a change with the given message signed by the hub delegate on behalf of the user (§5.5) and adds it to the chosen bookmark, or to a new bookmark plus a proposal, exactly as GitHub's "propose changes" flow. The hub refuses with a clear message when the user has not delegated to the hub key (§5.5) or the policy does not authorise them for the target. Binary files and files over `render_limit` cannot be edited in the browser.
For each bookmark whose state has conflicts: the file (or tree entry), the conflict kind, and each side with the change that produced it and its content, rendered with the same marker structure `oxy checkout` writes. For a text file, **resolve region by region** (`/{owner}/{name}/resolve/{bookmark}/{path}`, writers) shows the lines no one disagrees on and each conflict region with its sides and the change that wrote each; per region the user keeps one side, every side in order, none, or their own text (a side that records a deletion keeps the deletion). Regions are linkable (`#region-N`). The file is put back together and committed through the delegate as a web edit (§6.9), which makes it a `kind = resolution` change depending on every contributor; the page carries the bookmark's state and refuses to commit if the bookmark moved meanwhile. Malformed markers fall back to the editor, which also stays available: "edit the marked file" opens the markers in it. A name conflict (several files want one name, shown as `NAME.oxy-conflict-N`) resolves on the same URL by keeping one candidate: it takes the name as a move, keeping its identity, and the other candidates are removed. A replace conflict (a binary file replaced two ways) resolves the same way by keeping one version, each labelled with the change that wrote it; the commit replaces every version with it (Oxygen spec §9.3). Conflicts between directories or links are resolved from a client. Both resolver pages offer **Resolve in VS Code**, a `vscode://oxygenhub.oxygen-vscode/resolve?url=…&bookmark=…&path=…` link that oxygen-vscode handles: it finds the open checkout of that repository (or offers to clone it), switches to the bookmark after asking, pulls it, and opens the file. Resolving is live: everyone on the same resolver page (a room per repository, bookmark and path, over server-sent events at `/{owner}/{name}/resolve-live/{bookmark}/{path}`) sees who else is there and each other's choices as they are made, region by region, the last choice for a region winning (free text is shared whole, a moment after typing stops); a newcomer gets the choices so far; and when one of them commits, the others are told the resolution landed, with a link to it, and their form is disabled. Rooms are in memory and end when the last viewer leaves; with several web processes, choices and landings reach the room's viewers in every process over the event bus, while presence is each process's own (docs/roles.md). The change page tags resolution changes. Otherwise the page gives the two-line client recipe.
The current policy (serial, expiry, roots, threshold, rules with their patterns, thresholds and `allow_conflicts`), the policy history, the hub delegate key with its fingerprint and which collaborators have delegated to it, every linked account's keys that the policy names, revoked keys, and a verification summary computed from `oxy verify`: counts of signed, sealed, unsigned and invalid changes, with a list of invalid ones. Policy proposals awaiting root signatures are listed with who has signed and a download of the proposal file.
359
360
### 6.12 Pushes page
361
362
The hub keeps its own log of every reference update it accepted or refused: time, account, key, reference, old and new state, force flag, the result and reason. This is the answer to "who moved main and when" and is the hub's audit trail for history; it is not the repository's operation log, which is per workspace.
`/` focuses search, `g c` / `g p` / `g i` jump to Code, Proposals, Issues, `y` on a tree or file page switches to its permalink, `?` shows the sheet. Every interactive element is reachable by keyboard and shows a visible focus ring; a "Skip to content" link comes first; the header, site band, repository and settings tabs are named navigation landmarks, and the current tab carries `aria-current`. Colour is never the only carrier of meaning (glyphs accompany every status colour). Text meets WCAG AA (4.5:1) on every surface in both themes, checked by a test over the theme variables; `--dim` is for rules and decoration, never text. Under 760 pixels the tab bars and wide tables scroll sideways instead of breaking the layout, sectioned pages stack their section list above the content, and the header search hides (the `/` key and the search page remain).
A repository may have a **wiki** (oxygenhub#104): Markdown pages kept as an Oxygen repository of their own under `<data>/wikis/<owner>/<name>`, with no row of its own; it follows the repository's visibility, roles, archiving, transfer and deletion. A page `Getting-Started` is the file `Getting-Started.md` at the root of bookmark `main`; `_Sidebar.md` is shown beside every page, and `Home` is the front page. On the web (`/{owner}/{name}/wiki`, `/wiki/{page}`, `/wiki/{page}/edit`, `/wiki/{page}/history`, `/wiki/_pages` with a search over titles and text, `/wiki/_raw/{path}` for images): readers read, writers create, edit, rename and delete pages, and every edit is a change authored by the user and signed by the hub as their delegate, exactly as a code edit (§6.9), so it needs the same delegation. Links between pages are relative (`[Other](Other)`). By client, the wiki is `oxy clone HUB/{owner}/{name}.wiki`: the wire protocol maps that name onto the wiki store with the repository's permissions, pushes need write access and not archived, start no run and ring no webhook; a repository can therefore not be named `*.wiki`. The Wiki tab shows when a wiki exists or the viewer may start one. Settings › Advanced › Delete wiki data moves the wiki to the trash. Audited as `wiki.edited`, `wiki.deleted`, `wiki.data_deleted`; webhooks get a `wiki` event.
A **proposal** is a request to add a set of changes to a target bookmark of a repository. It is identified by a per-repository number, like a pull request. Its source is either a bookmark of the same repository or of a fork, in which case the proposal's changes are those in the source bookmark's state not in the target's; or an explicit list of change hashes (from the change page or the API).
377
378
A proposal records: number, title, description (Markdown), author, target bookmark, source (bookmark and repository, or explicit hashes), state (`draft`, `open`, `landed`, `closed`), reviewers requested, labels, and the **projected state** (§7.2).
379
380
### 7.2 Projected state
381
382
Whenever the proposal or its target changes, a job computes the projected state: the union of the target's heads and the proposal's changes, loaded through `oxy-core` exactly as a client's `oxy merge` would. From it the page shows:
383
384
- the changes to be added, in dependency order, with signature status;
385
- the diff between the target's state and the projected state;
386
- **conflicts** the landing would reify, each with its sides, in the same rendering as §6.10;
387
- files touched, and check results for the projected state hash (§13).
388
389
Because changes commute, a projected state is stale only when the target has moved; the job then recomputes it. There is no "update branch" button because there is nothing to update. If the recomputation shows a new conflict, the proposal is marked `conflicted` and the author and reviewers are notified.
390
391
### 7.3 Review comments
392
393
A comment is attached to the proposal (general), to a file, or to an **atom**: `(ChangeHash, index)` plus the change that introduced the line. Rendering places it at the atom's current position in the projected state; if the atom is gone (deleted by a later change in the proposal), the comment shows in the file's "resolved lines" section rather than disappearing. When the author squashes or rebases, the correspondence the rewrite recorded maps the old atom to the new one, and the comment follows. Comments are Markdown (EasyMDE), threaded, and resolvable.
When a proposal's target moves (a push, or another proposal landing), its projected state is new and its checks would be for the old one: the hub starts a run of the pipeline on each open proposal's new projected state, labelled `proposal/N` with the trigger `proposal`, unless that state has been run already (#134). Its author, or anyone with triage access, closes and reopens a proposal on its page or with `PATCH /api/v1/repos/{o}/{n}/proposals/{number}` and `{"state": "closed"}` or `{"state": "open"}` (`oh proposal close N [-m MSG]`, `oh proposal reopen N`; #140); asking for the state it already has changes nothing, and landing has its own endpoint. The author or a writer can **ask for a review** from anyone who can read the repository (not the author), under Reviewers on the proposal page or with `POST /api/v1/repos/{o}/{n}/proposals/{number}/reviewers` and `{"reviewers": [...]}`; `DELETE .../reviewers/{username}` withdraws a request. Each person asked gets a notification (`proposal.review_requested`), the timeline records it, the proposal's JSON lists `requested_reviewers`, and a review from that person answers the request. Every review notifies the proposal's author (`proposal.review`). An AI assistant the repository may use (§13.4) can be asked the same way; its answer is a `comment` review recorded via the assistant under the asker's name.
A review is `comment`, `request changes`, or `approve`. An approval MUST be backed by a **seal attestation** (spec §14.4, `kind = seal`) over the proposal's change hashes, signed with a key linked to the reviewer's account. Private keys never enter the browser, so the reviewer signs from a client: `oxy seal --proposal <url>` or `oxy seal <hashes>` (§18, a new command; the seal object already exists in the model) signs and pushes the seal; the hub matches the seal to the proposal and records the approval. The page shows the command ready to copy. An approval not yet backed by a seal is shown as "approval pending signature" and does not count towards requirements.
Landing requirements per repository (settings): N approvals, approvals from code owners (`.oxyowners` in the target state, GitHub `CODEOWNERS` syntax), required checks green on the projected state, no conflicts (or conflicts allowed, matching the policy rule's `allow_conflicts`), and no outstanding "request changes". A repository under an organisation of two or more members starts with one approval required (someone other than the author can give it; nobody may approve their own proposal), any other with none; an admin changes either in Settings. When none is required and nobody has approved, the proposal page says so beside Land, and Land asks for confirmation.
A change may depend on changes in another open proposal. The page shows "depends on #12" and lands only after #12 has landed, or lands both when the user chooses "land with dependencies" and holds the rights for each.
406
407
### 7.6 Landing
408
409
Landing sets the target bookmark to the projected state's heads: a reference update with a fresh ref attestation. It is performed by:
410
411
1. the hub delegate key on behalf of the user landing, when that user has delegated to it (§5.5), the policy authorises them for the target, and they hold `write`; or
412
2. a client: `oxy proposal land <url>` (§18) fetches the proposal's changes, sets the bookmark locally and pushes with the user's own key. The hub notices the push, sees the proposal's changes are now in the target, and marks it `landed`.
413
414
The landed state hash is recorded. Nothing is rewritten: the changes keep their hashes, so links to them stay valid and check results carry over. "Squash on land" is offered as a client-side option only, since it rewrites signed changes.
**Landing over checks** (oxygenhub#261). When required checks are the only thing in the way (pending or failed on the projected state, for instance with the runners down and the fix in this very proposal), a repository maintainer or admin may land now with a reason: *Land now, over the checks* on the proposal (a reason field, and a confirmation), `POST /repos/{owner}/{name}/proposals/{n}/land` with `{"over_checks": "REASON"}`, or `oh proposal land N --over-checks REASON`. Only the checks are passed over: approvals, code owners, changes requested, drafts, dependencies and the trust policy still apply, a writer is refused, and an empty reason is refused. How it landed says so ("web, over checks: the runners are down"), on the proposal, its timeline and the pushes page, and it is audited (`proposal.land_over_checks`, with the reason and the checks that had not passed).
**Land when checks pass** (oxygenhub#261). Anyone who may land a proposal can set it to land by itself: *Land when checks pass* on the proposal, `POST /repos/{owner}/{name}/proposals/{n}/auto-land` (`DELETE` turns it off), or `oh proposal land N --when-ready` (`--off`). Whenever a check is recorded on a state, the open proposals set to land on that projected state are looked at: once every required check has passed and nothing else blocks it, it lands as the person who set it ("when its checks passed (set by NAME)"), if they may still land; a required check that fails or is cancelled turns it off, with a note on the timeline; a push to the source makes a new projected state whose checks run, so it waits again. It shows on the proposal ("lands when its checks pass (set by NAME)") and in the API (`auto_land`); setting and clearing it are on the timeline and audited (`proposal.auto_land`). Migration 0081 (`proposals.auto_land_by`, `auto_land_ms`).
`closed` keeps everything; the source bookmark may be deleted on close if the user asks. A proposal whose changes all appear in the target through another route is marked `landed` automatically with a note.
Per-repository numbered issues sharing the number space with proposals (as GitHub does; so every number is shown with its kind, "issue #1" or "proposal #2", in page headers and in the links `#N` becomes), with title, Markdown body, author, assignees, labels (name, colour, description), a milestone (title, due date, description, open/closed), state (`open`, `closed` with reason `completed` or `not planned`), comments, lock, and pin. Issue templates come from `.oxyhub/ISSUE_TEMPLATE/*.md` in the default state.
**The proposal page.** The conversation is centred at the normal page width; the changes and files tabs are wide for diffs.
429
- **Timeline:** the description, comments, reviews and events are one timeline in time order, each entry beside its author's avatar.
430
- **Editing:** the title, target and description are edited inline (an Edit button beside the title opens the editor below the timeline), not from the sidebar.
431
- **State actions:** ready for review, back to draft, close and reopen sit under the comment box, and only when they apply.
432
- **Sidebar:**
433
- Landing, Closes, Labels and Projects.
434
- Labels: those with triage edit them, or are pointed at the repository's labels page when it has none.
435
- Projects: editors of the owner's projects add the proposal to one (its first column) or take it off. Issues have the same project control.
436
- **Redirects:** every action redirects back to the page, so reloading it (the checks script does, when a check arrives) never repeats the action.
**The chain of work.** A proposal shows the issues it closes (`closes`, `fixes` or `resolves #N` in its description or in any of its changes' messages) in a strip under its title: each issue with its state, then the proposal, then its target bookmark, marked landed once it is. A Closes list sits in its sidebar. An issue shows the proposals that close it, by the same rule, with their states. So the path from a report to the change that landed it can be followed from either end.
Relative links and images in issue, comment and proposal text resolve against the repository's default bookmark, and in release notes against the release's tag. Cross references: `#12` links an issue or proposal; `owner/name#12` across repositories; a 12-hex-or-longer token links a change when it resolves; `closes #12` in a change message closes the issue when the change lands on the default bookmark ("closed by change"), however it arrives: every change in the default bookmark's new state and not in its old one counts, whether a push imported it, a client's `oxy proposal land` moved main to changes the hub already held, or the hub landed a proposal (whose description closes issues too); #151. Any other `#12` in a pushed change's message, on any bookmark, puts "referenced by change <hash> on <bookmark>" on that issue's or proposal's timeline, the hash linking to the change; other repositories' `owner/name#12` are left alone.
A **project** is a board that plans work across the repositories of one owner (a person or an organisation): a number per owner, a title, a description, public or private, open or closed, and ordered **columns** (new projects start with Backlog, Doing and Done). A column holds ordered **cards**; a card is an issue or proposal of one of the owner's repositories, or a note.
447
448
- Who: anyone may see a public project; a private one only its owner (for an organisation, its members) and site administrators. The owner, an organisation's members and administrators may change it. A card for an issue or proposal whose repository the viewer cannot read is not shown at all.
449
- The board works without JavaScript: each card has a column picker and up and down buttons; cards are added by reference (`#12` in a named repository, or `repo#12`) or as a note; an issue or proposal is on a project at most once. Issue and proposal pages list the projects they are on, with their column.
450
- Columns can be added, renamed, moved and removed (removing one moves its cards to the first column). Closing a project keeps it read-only.
Tags come from the repository (spec §10): a name, the state, and a signed attestation when the tagger signed. The tags page lists them with signature status. A release attaches to a tag: notes (Markdown, generated draft from the changes since the previous release), assets (uploaded files, content addressed, size limited by `asset_limit`), a pre-release flag, and a latest marker. Source archives (`.tar.zst`, `.zip`) of the tagged state are generated on demand and cached.
**Trust and audit** (§9.1). `/{owner}/{name}/releases/trust/{tag}` reports on a release; each release on the releases page links it.
457
- **Tag:** the tag's signer and whether its attestation verifies.
458
- **Authors:** everyone whose changes the release adds since the previous release (all of its changes for the first release). For each: how many changes are signed, sealed, unsigned, invalid or signed by revoked keys, and the account their key is linked to.
459
- **Assets:** for each asset, what built it: the run and job, the runner's identity (and whether it is a key one of this hub's runner tokens signs with), and the pinned environment (tier, rootfs hash, environment hash). Also its inputs (the state built, the pipeline hash and the scope hash) and its signed receipt, verified again from the repository.
460
- **Graph:** authors → tagged state → jobs → assets, drawn as SVG. A dashed edge is a build from another state than the tag's.
461
- **Audit:** pass, warn or fail for each claim:
462
- the tag is signed;
463
- the changes are authenticated;
464
- each asset has a verifying signed receipt from a known runner;
465
- each asset was built from the tagged state;
466
- each asset's environment is pinned by hash.
467
468
A hand-uploaded asset, an unsigned tag or an unsigned change is a warning. A signature or receipt that does not verify, or a revoked key, is a failure.
469
- **Export:** `?format=json` downloads the report, and `GET /api/v1/repos/{owner}/{name}/releases/trust/{tag}` returns the same. The JSON carries each signed receipt as hex, so an auditor can check it offline with `oxy receipt verify`.
**Built by.** A release asset links to the build that produced it: the passed OxygenFlow job of this repository with an artifact of the same BLAKE3 (hub spec §13.2). A job with a kept signed receipt comes first, then the newest run. The release page shows the run, the job and the signed receipt with its signer, and the API gives the same as `built_by` on each asset. On a run page, a writer can attach an artifact to a release; it becomes an asset named after the artifact's path, so the link follows by content. A file uploaded by hand links the same way when its bytes match.
Events: proposal opened, review requested, review submitted, comment on a thread you are in, mention (`@name`), landing, check failed on your proposal, issue assigned, policy proposal awaiting your signature, push refused. Each becomes an inbox item (`/notifications`, with unread count in the header) and, when the account chooses, an email through `lettre` with a per-account SMTP-independent unsubscribe link. Watching a repository subscribes to all its events; participating in a thread subscribes to it.
The inbox shows one row per **thread** (the page a notification is about, such as an issue or a proposal), with its newest event, how many updates it had and how many are new. Each notification records why it was sent: **participating** (the account opened, commented on, was assigned, mentioned in or asked to review the thread) or **watching** (it only watches the repository); a thread counts as participating if any of its notifications does. The inbox filters by view (inbox, unread, done), by reason, by kind (`issue`, `proposal`, `release`, …) and by repository, with counts over the whole inbox. A thread can be marked read or unread, and **done**, which sets it aside (read) until something new happens in it. Opening a row marks its thread read and goes to the page.
**Who hears, and of what** (oxygenhub#183, 2026-10-14). A repository's owners (the user, or an organisation's owners) watch it by default and stop by unwatching it, which mutes it (`watches.muted`, migration 0065); others watch by choice. A proposal Diffusion opens is told as one opened on the web is. The actor is never told of their own doing, except through a token they marked as an agent's (Settings › Tokens, "an agent uses this token"): a request with it runs with the token's name in `notify::AGENT`, and its events reach the owner like anyone else's, the actor shown as `andymccall (agent token Claude)`. OxygenFlow tells a run's pusher and the watchers when it fails (`flow.failed`) and when a bookmark is green again (`flow.fixed`: the run before on it failed, or this run had failed and a job of it was run again). Backups tell the owner when a run is not whole (once per spell), when one is whole again (`backup.recovered`), and once a day from 07:00 UTC with the day's runs, outcome, repositories and bytes sent (`backup.summary`). The site's administrators hear of each new account (`account.registered`). Each goes to the inbox, and by email to accounts that turned email on and verified their address.
Global search box with scopes: repositories (name, description, topics), issues and proposals (FTS5 over title, body and comments), changes (message FTS5 over the default bookmark's state, indexed on push), accounts, and code. Code search in 1.0 is a per-repository substring and regular-expression search over the default state's text files, run with a time budget and a result cap, cached per state hash. A trigram index is a 1.x item.
JSON under `/api/v1`, bearer tokens, `application/json`, RFC 7807 problem responses (an unknown route, or a known one with the wrong method, too), cursor pagination (`?after=`), ETags on cacheable resources, and rate limits per token. Resources mirror the pages: `/repos/{owner}/{name}`, `/bookmarks`, `/changes/{hash}`, `/tree/{ref}/{path}`, `/file/{ref}/{path}`, `/blame`, `/history/{ref}` (`?path=`, `?offset=&limit=`, or `?r=REVSET` for the changes a revset selects, evaluated under a two-second budget and answering 400 when it does not parse or goes past it; Oxygen's docs/revset_query_language.md), `/compare`, `/conflicts`, `/proposals` (`?state=open|draft|landed|closed|all`, open ones and drafts by default; a source bookmark may hold `/`, and `owner/name:bookmark` naming the repository itself is its own bookmark), `/proposals/{n}/comments|reviews|checks|files`, `PATCH /proposals/{n}` (the title and description, for its author or a maintainer, and `labels`, the whole set by name, for triage; what is not given stays), `POST /proposals/{n}/land` (repo:write: what the Land button does, `409` with the blockers when it cannot; a landed proposal keeps the changes it landed and reports `landed` with who, how (`web`, or `api (token NAME)`) and when, which its event, the audit log and the push record also say, adding when nobody approved), `/proposals/{n}/content/{base|head}/{path}` (a changed file's bytes in the target's state or the projected state, for editor diff views), `/issues` (creating an issue can set its labels, assignees and milestone; `PATCH /issues/{n}` edits the title and body for the author or triage, and replaces labels, assignees or the milestone for triage, recording each on the timeline; unknown names are refused before anything changes), `/labels` (`POST` creates or changes one, `DELETE /labels/{name}` removes it; triage), `/milestones` (`POST` creates one or changes the description and due date of the one with that title, `PATCH /milestones/{title}` changes title, description, due date or state; triage), `/releases` (`PATCH /releases/{tag}` changes the title, notes or prerelease flag, repo:write), `/tags`, `/trust`, `/pushes`, `/hooks`, `/checks`, `/user`, `/notifications` (the caller's inbox and unread count; `/notifications/{id}/read` and `/notifications/read` mark read; scope `user:read`), `/users/{name}`, `/users/{name}/drops` and `/drops` (§4.5), `/orgs/{name}`, `/checks/{state}` (the checks on a state, named by its hash, by its heads joined with `+`, by the one head of a single-head state, or by a bookmark), `/checks/required` (`GET`, and `PUT` of the whole list by a repository admin: the checks a proposal needs, passed, to land), `/projects/{owner}` (list, create) and `/projects/{owner}/{n}` (the board as the caller may see it, `PATCH` for title, description, privacy and closing, and `owner` to transfer it to an owner the caller may make boards under (#125: it takes the next number there, and its old address redirects), `/cards` to list them by column or add a reference or note to a column named by id or name, `/cards/{id}` to move or remove one; `repo:write` and the page's rights to change), `/coverage` and `/tests` (a bookmark's coverage and JUnit test reports from Flow jobs, newest first, `?bookmark=&limit=`), `POST /coverage` (repo:write, #136: `{"bookmark"}` or `{"change"}` and `content`, an LCOV or Cobertura report measured off the hub, kept for that state as uploaded by the caller, shown "uploaded by NAME" wherever a run number would be, and setting no check; `oh coverage upload FILE`), `/coverage/{id}` (a report with its files) and `/tests/{id}` (a report with its cases, `?status=failed` for the failed and errored ones), `/flow/runs` (`GET`: runs newest first, `?before=N&limit=`; `POST {"bookmark"}` or `{"change"}`, repo:write: run a bookmark's current state, the default bookmark when none is named, or one change's state; with `"jobs": [NAME..]` only those jobs and what they depend on, each one a manual run starts, else `409` naming the ones it does (#130); `oh run start --job NAME`), `/flow/runs/{n}` (a run with its jobs), `/flow/runs/{n}/jobs/{job}/log` (`?after=seq&limit=`) and `/flow/runs/{n}/rerun` (repo:write: a finished run again), both answering `201 {"number", "url"}` or `409` with why nothing started, `/flow/runs/{n}/jobs/{job}/rerun` (repo:write, oxygenflow#31: that job of a finished run again, in the same run, with the jobs that depend on it; what it depends on keeps its result, so a flaky test reruns without its build; their logs, artifacts and reports go, their checks return to queued and the run is running again; `{"queued": N}`, or `409` while the run is live or for a job it lacks; RE-RUN THIS JOB on the run page, `oh run rerun N --job NAME`), `/flow/runs/{n}/cancel` (repo:write: a run still going ends as `cancelled`, its running jobs stopped when their runner next reports; `409` when it had already ended; a push to a bookmark cancels the push runs of it still going), `/search`. A change's author carries the account its key is linked to, or `null` and a `note` saying the key is not linked and where its owner links it; `via` names the delegate (an agent) that signed for the author when one did, with the delegation's scope and expiry and whether it has since been revoked, and the pages show the same as an `agent` badge, a `via` row under the author on the change page, and a count under the author in Contributors (the change page shows a "key not linked" tag, with a link to Settings → Keys for signed-in viewers). Shapes reuse `oxy --json` shapes wherever one exists (change, status, blame, diff) so tools written against the CLI work against the hub. `crates/hub-api` holds the types with `serde` derives and is what the runner and the VS Code extension will link.
Per repository and per organisation: a URL, a secret, an event selection, active flag, and delivery log with redelivery. Each webhook has a **format**: `oxygenhub` (the signed JSON envelope below), `discord` (an embed with a title, link and short detail for the event, sent with mentions disabled so no user text can ping anyone) or `slack` (escaped `text`); a Discord or Slack URL picks its format by default, and every event has a one-line summary for the chat formats. The settings show a webhook's URL with its long path segments and query cut short, since chat webhooks keep a secret token there (the audit log records the same masked form), and each webhook's last delivery result; a webhook can be edited (URL, secret, events, format, active), a blank URL or secret keeping the current one. Repository admins manage a repository's webhooks under its settings; organisation owners manage the organisation's under its settings, and those hear every repository the organisation owns. Payloads are JSON with `X-Oxyhub-Event`, `X-Oxyhub-Delivery`, and `X-Oxyhub-Signature: blake3-keyed=<hex>` computed as keyed BLAKE3 over the body. Events: `push` (reference updated: old and new state, heads, changes), `proposal` (opened, edited, review requested, landed, closed, conflicted), `review`, `comment`, `issue`, `release`, `check`, `policy`. Deliveries retry with backoff up to 24 hours.
A **check** is a result attached to a state hash: name, status (`queued`, `running`, `passed`, `failed`, `cancelled`, `skipped`), a summary, a URL, timing, and an optional signed **seal** from the checker's key over the changes in that state, which makes a passed check a verifiable object. Checks are created through the API (`POST /repos/{o}/{n}/checks`) by any token with `checks:write`, so any CI system can report, and required checks for landing (§7.4) are matched by name against the projected state's hash. The proposal page streams check status updates over server-sent events, which reach it from whichever process recorded the check (the event bus, docs/roles.md).
Because a state is content addressed, the hub deduplicates: a check that passed for a state hash is shown for every proposal and bookmark that reaches that same state.
Pipelines run in **OxygenFlow**, a separate project and specification (`oxygenflow/docs/spec.md`; the first slice is `oxygenflow/docs/m0.md`). A pipeline is `.flow/pipeline.kdl` in the state being built; jobs run in Flow's isolation tiers (MicroVM by default, a syscall sandbox, WASM), never in containers on the hub. The hub's side:
- **Configuration.** `[flow] runner_tokens` (bearer tokens for runners, compared in constant time), `repos` (patterns such as `owner/*` or `owner/name`: the repositories whose pushes start runs by default; empty means none) `job_timeout_minutes` (60) and `drain_after_errors` (5, 0 for never: see Alerts).
508
- **Alerts** (oxygenflow#30). A run that ends failed tells its pusher and the repository's watchers, in their inboxes and by email as each chose (`flow.failed`, from the `oxygenflow` account, which cannot sign in), naming the jobs that did not pass. A runner whose last `drain_after_errors` jobs since its state last changed all ended in an infrastructure error (status `error`: it could not run them, not a failing test) is set to draining and the site administrators are told (`flow.runner_drained`, with the last error); making it active again on Admin › Runners starts the count afresh (migration 0064, `state_ms`).
- **Priority and turns** (oxygenhub#244). Runners take queued jobs by level first (`urgent`, `high`, `normal`, `low`), then by turn: a job's place in its own repository's queue plus that repository's running jobs, oldest first among equals, so repositories take turns and one repository's burst can't hold up the rest. A run's level is its own if it was expedited, else its repository's (Settings › OxygenFlow priority; maintainers choose low, normal or the default, site administrators also high), else its organisation's (its settings page, by its owners), else the site's (Admin › Runners › Settings), else normal. A maintainer expedites a queued or running run to urgent on its page with a reason; it is audited (`flow.expedite`), shown on the run and in the queue, and lapses when the run ends. A pipeline may lower its jobs, never raise them: `priority "low"` (or `"normal"`) on a job or for the whole file (OxygenFlow spec) makes a job's level the lower of its own and its run's, except that an expedited run stays urgent; the queue says `pipeline: low` (migration 0074). Nothing running is stopped for a more urgent job (migration 0073).
- **Per repository.** A site administrator turns OxygenFlow on or off for one repository under Settings → OxygenFlow, which overrides `repos` in either direction; "site default" follows it again. Runs use the site's runners, so repository administrators see the setting but cannot change it. Changes are audited (`repo.flow`) and apply from the next push, with no restart.
- **Triggers.** A push that moves a bookmark of an allowed repository starts a **run** on the bookmark's new state, numbered within its repository from 1 (the run's page is `/{owner}/{name}/flow/{number}`): the hub reads `.flow/pipeline.kdl` from that state and validates it with `flow-core` (the same code `flow check` runs). An invalid pipeline records a run in `error` with the problems and their lines; a state without a pipeline records nothing. Tags and proposals start no runs yet, so only code from people with write access ever runs. A run waiting for a runner says where it stands: on the Flow tab its place in line, on its page also the jobs running now and those queued ahead of it (named when their repository is this one or public, otherwise counted), and in the API `queue_position`. Writers also start runs without moving a bookmark: **Run pipeline** runs a committed state without a push:
512
- On the Flow tab and beside each bookmark on the bookmarks page, it runs a bookmark's current state.
513
- On a change page, it runs the state whose only head is that change. That run has no bookmark and is labelled "change".
514
- The API does the same with `POST /flow/runs` and `{"bookmark"}` or `{"change"}`.
515
516
Viewers who may not start runs get no button. The Flow tab says why: Flow is off for the repository, they are not signed in, or they lack write access. **Re-run** on a finished run's page runs its state again.
517
518
Runs only ever take committed states already in the repository, never uncommitted work. A run writes no history: its receipts are signed statements about a state (Oxygen spec §14.6). Running a state again adds another statement. The environment hash shows whether it ran in the same environment, and a matching result is independent confirmation. The new run's checks replace the old ones for that state, and the run page says what started it (pushed by, started by, re-run of #N by).
- **Runs and jobs.** A run holds the repository, bookmark, state hash and heads, pipeline hash and pusher; one job per pipeline job with its compiled form, sandbox and dependencies. Job states: `queued`, `running`, `passed`, `failed`, `skipped` (a dependency did not pass), `error` (the runner could not run it). A run is `running` while any job is, then `passed` when every job passed, else `failed`.
- **Newest run wins** (oxygenhub#267). A new run cancels the older runs of the same thing still live, so runs do not pile up behind states that have moved on: a push run cancels the older push runs of its bookmark (Diffusion's bumps start push runs), and a proposal's run (bookmark `proposal/N`, started when its target moves) the older runs of that proposal. Each cancelled job says `superseded by run #N`, and the next push run's scope reaches back past what was cancelled. A proposal's run leaves its source bookmark's push runs alone: they test the source head, another state, whose checks the bookmark and other proposals may want. Tag, manual, scheduled and re-run runs were each asked for in their own right, so they supersede nothing and nothing supersedes them; they are cancelled by hand. A repository's maintainers choose under Settings › OxygenFlow › Newest run wins whether superseded runs already running are cancelled too (the default) or finish, with only the queued ones cancelled (`repositories.flow_keep_running`, audited as `repo.flow_keep_running`).
- **Reusing a pass** (oxygenhub#265). A push run does not run a job again when the same job, with the same compiled spec and sandbox (the image pin included), already passed on the very same state in this repository, as main does after a proposal lands its tested projection: the job is created passed, its `error` note `reused from run #N`, carrying that pass's receipt, hash and attestation, and its check reports `passed in run #N on this state, reused`. A run whose jobs are all reused has passed when created, and reports as any passing run (Diffusion's green bumps, a bump's landing). Never reused: a job with an approval gate, an environment, secrets or identity tokens, or one whose artifacts another job of the run takes as inputs; manual runs, re-runs, schedules and tags always run. Maintainers turn it off under Settings › Runs (`repositories.flow_reuse`, audited as `repo.flow_reuse`).
- **Pushing without a run** (oxygenhub#284). A push whose newest change (by its clock) has `[skip flow]`, `[skip ci]` or `[ci skip]` in its message, in any case and outside code (a marker between backticks or in a fenced block only mentions one), starts no jobs: the hub records a run with status `skipped`, finished at once, whose note names the marker, the change and who pushed it, so the run list shows the gap. The marker is in a signed message, so who skipped is verifiable. A skipped run is never a later run's baseline: the next push run's scope reaches back past it, so the changes it brought are still tested. A pushed tag always runs. A repository's maintainers choose under Settings › Runs where skipping is allowed: on any bookmark (the default), on any but the default bookmark, or never (`repositories.flow_skip`: `any`, `others`, `never`; audited as `repo.flow_skip`).
- **Runner API** (runner token): `POST /api/v1/flow/claim` hands out the oldest queued job whose dependencies passed and whose tier the runner offers, marking it `running`, with a **job token**; `204` when there is none. `POST /api/v1/flow/jobs/{id}/log` appends log records (text cut at 16 KiB, at most 50 000 per job); `POST /api/v1/flow/jobs/{id}/finish` records the exit code and receipt (the hub recomputes the receipt hash) and settles dependents. Only the runner holding the job may report on it.
- A runner's page has a Settings form (#98): labels an administrator gives it beside those it reports (a job asking for a label matches either), and a job time limit in minutes below the site's, applied when it claims.
- An administrator creates a one-time registration token, valid for a day and shown once with the command that uses it.
528
- On the runner, `flow runner register https://hub --token …` sends the runner's signing identity, architecture, OS, version and labels (`POST /api/v1/flow/runners/register`) and gets its own credential (`flowrunner_…`, stored hashed).
529
- The runner's key is bound at registration: receipts from that runner signed by another key are not kept.
530
- Runners configured with a `hub.toml` token still work. They are tracked from their first request under the name they claim with, and their key is bound on their first signed receipt.
- **Scopes** (oxygenhub#41): a runner serves the whole site, one owner's repositories (a user or an organisation), or one repository. The registration token carries the scope (typed as empty, `owner` or `owner/repository` when it is made) and the runner gets it; an administrator can change it on the Runners page. A claim returns only queued jobs of repositories in the runner's scope, filtered before the queue is cut short, so a scoped runner never receives another scope's jobs and never waits behind them. A job whose pipeline says `runs-on` (OxygenFlow spec 3.1) goes only to a runner that has all of its job's and sandbox's labels, from those the runner reports on the claim or registered with; until one claims it, the queue says which labels it waits for. `hub.toml` runners serve the whole site. Changes are audited (`flow.runner.scope`). **Settings → Runners** on a repository (its admins) and on an organisation (its owners) lists the runners that may take its jobs, with what each serves; there they make a registration token for a runner of their own scope (the repository, or the organisation's repositories) and remove such runners when they run nothing. Only site administrators manage site runners.
- **The Runners page** lists each runner: online (seen in the last 90 s), offline, draining or disabled; what it serves; signing key; machine; labels; capacity; jobs running now; and when it was last seen.
- **The queue** (on the Runners page): running jobs with the runner holding each, then queued jobs in the order runners claim them.
540
- **API** (site administrators): `GET /api/v1/runners` (each runner, what it serves, its machine, key, last seen and jobs running), `GET /api/v1/runners/{id}/jobs?before=&limit=` (its history, newest first) and `GET /api/v1/runners/queue`.
- **A runner's page** shows its totals (jobs, pass rate, time busy) and its job history, newest first, with links to runs and signed receipts. Jobs record the runner that claimed them (`flow_jobs.runner_id`).
- **Signed receipts** (Oxygen spec §14.6): `finish` may carry `signed_receipt`, the runner's signed receipt as hex. The hub keeps it in the repository when four things hold:
543
- it verifies;
544
- it signs the job's receipt;
545
- it is for the run's state (computed from the run's heads) and this repository;
546
- its key is the one the runner's token first signed with. The first signed receipt under a token binds that token to its key in `flow_runner_keys`.
547
548
Otherwise the job's result still stands, and the run page says why the receipt was not kept. The run page links each kept receipt, with its signer. The change page lists the receipts for the state whose only head is that change. `/{owner}/{name}/receipts/{attest}` downloads a signed receipt for `oxy receipt verify` or `import`.
- **Architecture** (OxygenFlow spec §11.2–§11.3, oxygenflow#36): a runner reports each image's architecture on its claims (`image_archs`), kept in `flow_images`; a job is offered only to a runner whose architecture matches its sandbox's `arch`, else its image's as known. The hub knows an image's architecture from its own files, read through its ext4 to its shell's ELF header (`jet::rootfs`) out of the kept chunks, when it is sent (the manifest's word is taken only when the image cannot be read) and, once at start, for every kept image that had none; an image nobody can read takes the architecture of the runner its first job passed on, and a runner's own report, read from the image it holds, wins over that (oxygenhub#173, Andy's "detect and learn"). Only an image nobody has read or run matches any runner. The queue says which architecture a waiting job asks for. **Images by hash** (oxygenflow#39, Jetstream): `flow rootfs push HASH` sends a rootfs image as the FastCDC chunks the hub lacks (`POST /flow/images/missing`, `PUT /flow/images/chunks/{hash}`, by a token that may write repositories) and then its manifest (`PUT /flow/images/{hash}`), which the hub accepts only when every chunk is held and they make an image with that hash; chunks are kept zstd-compressed under `<data>/images`, shared by every image, and `jet_images` lists what is held. Runners and accounts fetch with `GET /flow/images/{hash}` and `GET /flow/images/chunks/{hash}`. A runner whose claims say `fetches_images` is offered jobs whose image only the hub holds, and fetches it before its preflight. Each claim also carries the runner's `facts` (OxygenFlow spec §11.1, oxygenflow#40): free disk, load, speed score and class, kernel; the hub keeps the latest on the runner, shows them on Admin › Runners and in `GET /runners`. A job's finish carries the most disk and memory it took (`peak_disk_mib`, `peak_memory_mib`, oxygenflow#41); a new job's `need_disk_mib` is the most its last five runs of the same repository and name took, plus a quarter and 64 MiB, and it is offered only to a runner whose claim reports at least that much free disk; the waiting reason says so. **Ranking** (oxygenflow#42): a job that another free runner which polled in the last 10 s would do better (it holds the job's build cache, which counts most: claims list the caches a runner holds, `caches`, each `flow_core::api::cache_slot(repo, job)`, oxygenflow#51, and when each last kept a disk, `cache_times`, so of two runners holding it the one whose copy was kept at least a minute later ranks higher, oxygenflow#54, and `placed` says "holds its build cache (kept 12 min ago)"; or it holds the job's image while the claimant would fetch it; or it has a higher speed class) is held from the claimant for up to 15 s, then any runner that fits takes it; each job records why it ran where it did (`placed`: "holds its build cache, holds the image, idle, speed 100 (class 3)", with "after N s held for a better runner" when it was), shown on the run page, in the API and by `oh run view`. A run whose jobs pin an image held by neither the hub nor a runner sets a failing `pipeline/images` check on its state (#137), naming the images and how to send them (`flow rootfs push`), so a proposal pinning a new image shows it before it lands; a later run of the state turns it to passed once every image is held. A placement pass every 15 seconds works out why each ready queued job no runner can take waits, from what each online runner offered at its last claim (scope, tier, architecture, image held, labels, memory and CPUs, a recent decline), and shows it on the job (API `waiting`, `oh run view` and `oh run watch`, its panel open on the run page) and in its check's summary, so a proposal says why its check is stuck (#28). **Diffusion** (#148, `docs/diffusion.md`): a push that moves a repository's default bookmark, or the bookmark its work lands on (the most common target of its proposals landed in the last 90 days), records each Oxygen subrepo pin in the new state (`diffusion_pins`: consumer, bookmark, path, the upstream's Oxygen repository id, the pinned heads); a hub that had no index fills it at start. `GET /repos/{owner}/{name}/consumers` lists the consumers the caller may read, each with its pin and how many changes of this repository's default bookmark it lacks (`behind`, null when the pin names changes this repository does not hold), and the overview shows them under Diffuses to. `GET /repos/{owner}/{name}/diffusion` (#153) gives both directions, `takes_from` (each upstream the repository pins) and `diffuses_to` (each consumer the caller may read), every entry naming its `upstream` and `consumer`, its `path`, `bookmark`, `behind`, `follow`, open `bump`, the bump's newest `run` and `run_status`, and the pill's halves (`upstream_state`, `consumer_state`: `tone` green, amber, red, grey or off, `word`, `title`); hub-api's `Diffusion`. A repository that pins, or is pinned by, another on the hub has a **Diffusion tab** (#150): Diffuses to (each consumer's pin bookmark, how far behind, its status, its open bump and that bump's newest run, its last landed bump, how it follows) and Takes from (each upstream it pins, with the same); its overview shows both in brief, and Admin › Diffusion lists every upstream with its consumers. Each consumer's status is a two-half pill: the upstream on the left (checked when its newest default-bookmark run passed, untested when it has no checks, failing, or testing) and the consumer on the right (passes or fails for its run on the bump, testing, waiting, or up to date), each half coloured green, amber, red or grey; the API keeps one overall `colour` as well. Landing a proposal on the hub refreshes the pin index and propagates as a push does. A bump's run, started by the `diffusion` account, is given the upstream it bumps as a subrepo whatever that account may read (the consumer's own pin of it was committed by someone who may write the consumer); its other subrepos need read access as usual. A consumer follows a subrepo's upstream as its `.oxymodules` entry says (`follow = off | green | any`, `land = propose | auto`, `target` naming the bookmark to bump, `watch` listing paths whose upstream changes are flagged), else as its repository setting says (Settings › Diffusion, or `PUT /repos/{owner}/{name}/diffusion {"follow": ...}` by its owners and admins; migration 0058), else as `[diffusion] follow` in hub.toml says (default `off`). A **bump** moves a consumer's pin at a path to a newer upstream state: one change on the target bookmark's heads that replaces only the pin (oxy-core `build_pin_change`), signed with the hub's key, on the bookmark `diffusion/<path>`, which the hub owns and rebuilds on the target each time (a forced update). When the pin has changes the upstream's state lacks (say, from an upstream proposal not landed yet), the bump moves to both, so a consumer never loses changes it relies on; it is skipped when that combination would conflict, when the pin names changes the upstream does not hold, when the pin already holds the state, or when the bump bookmark already moves it there, and the hub logs why. A bump is **proposed** once per bump bookmark, by the hub's `diffusion` account (made on first use, never able to sign in), into the target, labelled `dependencies`, its body the upstream changes the pin lacks (newest first, linking to the upstream's change pages); a later bump edits the same open proposal. It is triggered when a push moves the upstream's default bookmark (for consumers following `any`) or when a run of that bookmark passes (for `green`). Diffusion lands a bump only where the consumer chose it (`land = auto` in the entry, else Settings › Diffusion › Landing or `PUT /repos/{owner}/{name}/diffusion {"land": "auto"}`, migration 0061; off by default, #174): once both halves are green (the upstream state it pins passed its own run, and the consumer's run on the bump as it is now passed), as the `diffusion` account through `land_as`, signed by the hub's key, so required checks, approvals and the trust policy decide; never for an unverified upstream; a refusal is noted once on the proposal's timeline. An upstream with no checks of its own (Flow off, or no run ever of its default bookmark) is **unverified**: it never turns green, so its moves also bump consumers following `green`, and those proposals are marked amber with the `upstream unverified` label and a note in the body; a later bump from a passing run removes the label. A bump already in place refreshes its proposal's text and labels. A consumer's `watch` patterns (`*` within a path segment, `**` across them, `?` one character; a plain path covers everything below it) flag the upstream changes touching a matching file: they are listed first under "Touches what you watch", with the files, and marked ⚑ in the changelog. Turning Diffusion on for a consumer (the setting or the API) catches it up at once: it is bumped to each upstream's newest state whose default-bookmark run passed, or to its newest state when it follows every move. **Images page** (#146): Admin › Runners › Images lists every rootfs image a runner holds, the hub keeps, or a job ran on in the last 90 days (from each job's sandbox), newest use first: its size and architecture, where it is (the hub, each runner holding it, each runner fetching it from the swarm now), the repositories whose jobs used it, and who sent it; an image held nowhere is marked. **The swarm** (#144, oxygenflow docs/jetstream.md §8.1): a runner in an image swarm reports its peer address, its site and the BLAKE3 of its TLS certificate with each claim (`swarm`; without a certificate it is not in the swarm), kept in memory and shown on Admin › Runners. `POST /flow/swarm/announce {image, state}` (by a runner; `fetching`, `done` or `stopped`) returns the online runners in its site that hold the image or announced fetching it in the last 45 seconds, each with its certificate's fingerprint, which the fetcher pins its TLS connection to, and a ticket signed with the hub key (JSON naming the runner, its site and an expiry 15 minutes on, hex, then the signature) that peers check against `/.well-known/oxyhub-key` before serving chunks; a runner that has not reported an address is told to add `[runner.swarm]` (409).
- **Variables** (oxygenhub#102): Administration, an organisation's settings and a repository's settings each hold `NAME=value` pairs (a name is a letter or `_` then letters, digits and `_`, up to 64, never `FLOW_*`; a value up to 4096 characters; up to 100 per level) under **Variables**, each page showing what it inherits and which inherited names it overrides. A claim carries the repository's effective set as `env`, inner levels overriding outer; Flow puts them in the job's environment under `FLOW_*` and the pipeline's own `env`, so they are part of the receipt. They are not secrets and are shown in full to whoever sees the page; secrets are mounted by Flow (OxygenFlow spec §9), never put in the environment. Changes are audited (`flow.variable.set`, `flow.variable.deleted`); `flow run --var NAME=VALUE` gives a local run the same.
- **Secrets** (oxygenhub#107; OxygenFlow spec §9): the same three levels hold **Secrets**, named as a pipeline's `secret "name" { mount-path "/run/secrets/…" }` names them (1 to 64 of `A-Z a-z 0-9 . _ -`, a value up to 64 KiB, up to 100 per level), typed once and sealed under `keys/secrets.key`; pages show names, when each was set and where it comes from, never a value. A claim carries only the secrets its job declares, each with its path and value, from the innermost level that sets the name; one no level sets is noted at the top of the job's log and left out, so the job fails on the missing file rather than running with a stand-in. Flow writes each as a file (0400, the job's user's, on an in-memory mount) before the job starts and never puts it in the environment, the receipt or an artifact. Log intake replaces each value, and its base64, with `***` before a record is stored. Changes are audited (`flow.secret.set`, `flow.secret.deleted`), never with the value.
- **Provenance others verify** (oxygenhub#239). For an artifact of a job whose signed receipt the hub kept, `…/flow/{run}/artifacts/{job}/{artifact}/provenance` answers an in-toto Statement v1 with an SLSA provenance v1 predicate, in a DSSE envelope signed with the hub's Ed25519 key. The subject is the artifact by SHA-256 (and BLAKE3); the build definition names the repository, bookmark, job and pipeline hash, and depends on the state built; the run details name the runner as builder, the run as invocation with its start and finish, and the kept receipt's attestation as a byproduct. The runner signed the receipt; the hub, which checked it, states what it says in the shape cosign, policy-controller, Kyverno and slsa-verifier read. `/.well-known/oxyhub-signing.pem` is the key: `cosign verify-blob-attestation --key oxyhub-signing.pem --type slsaprovenance1 --signature tool.intoto.json tool`.
- **Artifacts** (OxygenFlow spec §5.2): a job may declare artifacts with `output`.
554
- The runner holding the job uploads each one with `POST /api/v1/flow/jobs/{id}/artifacts/{name}` (raw bytes, at most 512 MiB).
555
- The hub refuses names the job does not declare. It keeps each artifact as a blob of the run's repository, records its path, size and BLAKE3 in `flow_artifacts`, and answers with the blob's ObjectRef.
556
- `finish` refuses a receipt whose `outputs` are not exactly the uploaded blobs, in name order.
557
- The run page lists each job's artifacts, and `/{owner}/{name}/flow/{n}/artifacts/{job}/{name}` downloads one.
558
- A kept receipt keeps its artifacts through garbage collection (Oxygen spec §14.6).
- **Job tokens.** Random, stored as a hash, valid only for reading the job's repository over the wire protocol while the job is running and for at most two hours. They cannot push, call the API or read other repositories.
560
- **Checks.** Each job is also the check `flow/<job>` on the run's state (§13.1), so proposals, required checks and the editor's status bar see it.
561
- **Pages.** A **Flow** tab lists runs (status, bookmark, state, pusher, when, duration). A run page draws the job graph as SVG, with columns by dependency depth and status by colour and glyph, and each job's log below (folded, the failed ones open); it refreshes while the run is live. Readers of the repository can see both.
`/{owner}/{name}/flow/badge.svg` is a small SVG of the latest run on the default bookmark (`passing`, `failing`, `running`, `no runs`) for a README, for whoever may read the repository. It is a convenience, with the weakness of every badge; the stamps below are the real thing.
**State stamps** replace CI badges, which are mutable images claiming that a branch passes, name no state and cannot be checked. A stamp belongs to a state:
566
567
- `/{owner}/{name}/flow/stamp/{ref}` is an SVG. For a state address (heads joined with `+`) or a change, it shows the newest run of exactly that state: one glyph per job in dependency order, the run's status, the state's short hash and the run's date. It shows "no run" when there is none. The same data is machine-readable in the SVG's `<metadata>`: heads, run number, status, and each job's status and receipt hash. For a bookmark or tag, it redirects to the stamp of the ref's current state, so even the image names what was checked. A finished run's stamp may be cached for an hour; any other stamp for a minute.
568
- In rendered Markdown, the relative paths `.flow/stamp` (as an image) and `.flow/run` (as a link) resolve against the state being viewed. The first is served by `/raw/{ref}/.flow/stamp`, the second by `/file/{ref}/.flow/run` (the run page, or the Flow tab when that state has no run). So `[](.flow/run)` in a README shows how the viewed state did: browsing a tag shows the tag's result, not the bookmark's. A real file at either path takes precedence.
The older runner draft (`docs/runner.md`) is superseded by OxygenFlow; its hub-side ideas (environments, approvals for outsider proposals, caches, artifacts) carry over as later Flow milestones.
**Open in Workspace** gives a signed-in user VS Code in the browser on a state of a repository, with Oxygen VCS inside, without cloning anything to their own machine. It is designed here before it is built (Oxygen backlog P5); parts are marked as they land.
576
577
- **What runs.** openvscode-server (MIT, unmodified) with the Oxygen VCS extension and `oxy`, in a Firecracker microVM per workspace on an OxygenFlow runner (a `workspace` job kind beside pipeline jobs), never on the hub's host. The image is a Flow rootfs like the build images, pinned by hash. A fork of VS Code is considered only if an extension cannot do what is needed.
578
- **Starting.** The button is on a bookmark's row and on a change page, for accounts that may read the repository. The hub records a workspace (account, repository, state, bookmark when opened from one, status) and queues a workspace job that only a runner whose scope covers the repository may claim (§13.2 scopes). One account has at most a few workspaces at once; the hub says which are open.
579
- **Files without a checkout.** The state is mounted read-only in the microVM by **flow-casfs** (OxygenFlow spec §5.1): the tree from the state index, each file's bytes from a cache of renderings keyed by graph key, rendered on first read and shared by every state in which that file has the same history. An overlay on the job's ephemeral disk holds edits. Opening a workspace on the Linux tree therefore costs listing a tree, not writing 96 000 files; the mount is to take under 100 ms.
580
- **History and commits.** The microVM's `oxy` repository fetches the state's history over the wire protocol with a token that reads this repository only, as job tokens do, and its working copy is the mount. Private keys never enter the hub or the microVM, so a commit is pushed to a workspace bookmark (`ws/{account}/{id}`) and signed there by the hub as the account's delegate, exactly as web edits are (refused with instructions until the account has delegated). From the bookmark the user proposes or lands as usual.
581
- **Access.** The hub proxies the editor (HTTPS and WebSocket) at `/{owner}/{name}/workspace/{id}/` to the workspace's owner only, checking the session on every request; the runner reaches the hub over a connection it opens itself, so runners behind NAT work, and accepts editor traffic for a workspace only from the hub with that workspace's secret. Nothing in a workspace is reachable from any other account, or from the network directly.
582
- **Lifecycle.** A workspace stops after 30 minutes without a connection (configurable) and on request. Its overlay is discarded when it stops: anything not pushed to its workspace bookmark is gone, and the editor says so before stopping. A stopped workspace reopens from its bookmark. The audit log records starts, stops and pushes.
583
- **Measured** when built: mount time, microVM boot, time to the editor's first paint, and a full read of the Linux tree through the mount.
**Design notes (2026-10-04), for review before anything is built.**
586
587
- *Data.* A `workspaces` table: id, account, repository, state (ObjectRef), bookmark (nullable), workspace bookmark name (`ws/{account}/{id}`), status (`queued`, `starting`, `open`, `stopping`, `stopped`, `failed`), runner, job id, secret hash, created, last connection, stopped, and why. A workspace is a row plus one Flow job of kind `workspace`; the job's log is the editor's server log. `GET /api/v1/user/workspaces` lists the caller's; `POST /repos/{o}/{n}/workspaces {"state"|"bookmark"}` opens one; `DELETE /workspaces/{id}` stops it. Limits are per account: three open at once (configurable), and a site cap.
588
- *Runner side.* The runner gains a job kind beside pipeline jobs: it boots the pinned workspace rootfs (openvscode-server, `oxy`, the Oxygen VCS extension, a non-root user), mounts the state at `/workspace` through flow-casfs with a writable overlay, runs `oxy` in the microVM against a repository that fetches history lazily over the wire protocol with the job's read-only token, and opens a tunnel back to the hub (the same outbound connection the runner uses for claims) carrying the editor's HTTP and WebSocket traffic. The runner marks the job running when the editor answers on its port, and reports a heartbeat (connections open, last activity) every 30 s in place of log batches; the hub stops the workspace when the heartbeat shows no connection for the idle limit.
589
- *Hub side.* `/{owner}/{name}/workspace/{id}/` is a reverse proxy to that tunnel, for the owner's session only, with the workspace secret in a header the runner checks; CSRF does not apply (the editor is same-origin under the hub, and the proxy forwards only to the workspace the path names). Commits: the extension's `oxy commit` writes to the microVM's repository; `oxy push` goes to the workspace bookmark with the job token, and the hub signs the pushed changes as the account's delegate (web edits' path) if the account has delegated to the hub key with `changes,refs`; otherwise the push is refused with the recipe. The bookmark is ordinary afterwards: propose, land, or delete.
590
- *Security.* The microVM has no credentials but the job token (read this repository, push this workspace bookmark) and the workspace secret; the account's keys never enter it. Network egress from the microVM is off by default (as for pipeline jobs); a site setting may allow it for package installs, with the audit log recording the choice. The overlay is ephemeral and the rootfs is pinned by hash, so a workspace cannot be left altered.
591
- *Costs.* One microVM per open workspace: 2 vCPU and 4 GB by default, from the runner's capacity like any job, so open workspaces compete with pipeline jobs; a runner label `workspace` keeps them on runners meant for it. Idle stop at 30 minutes keeps the fleet small.
592
- *Open questions.* Whether the Oxygen VCS extension should talk to the microVM's `oxy` (simplest; works offline in the VM) or to the hub's API (no `oxy` in the VM, but a second code path for every operation): the first. Whether a stopped workspace's overlay should be kept for a day so an unpushed edit is not lost: probably yes, as a tarball artifact of the job, restored on reopen. Whether to allow terminals: yes, it is the point.
An **assistant** is a chat-completion endpoint the hub may ask to review a proposal or draft its description (oxygenhub#97). Assistants are configured at three levels and combined downwards:
597
598
- **Where.** Administration (site), an organisation's settings, a repository's settings each hold a list of assistants: name (`Claude`, `Antigravity`, `ChatGPT`, `DeepSeek`, …), endpoint URL, API shape (`chat-completions` or `anthropic-messages`), model, extra headers, the token, and an *enabled* switch. Each level also holds a **prompt**. A repository sees the assistants of its organisation and of the site beside its own; an assistant disabled at an outer level is off for everything inside it, whatever an inner level says. The effective prompt for a call is the site's, then the organisation's, then the repository's, each under a heading, so a project adds to what the site says rather than replacing it, and each settings page shows the prompt it inherits above its own.
- **Adding one** (oxygenhub#109). The form offers providers (Anthropic, OpenAI, Google Gemini, DeepSeek, Mistral, Perplexity, Ollama) that fill the name, endpoint and API shape, and *Other* for an endpoint typed by hand. **List models** asks the endpoint's own list (`…/models` beside `…/chat/completions` or `…/messages`, with the token typed or, when editing, the one stored) and offers the chat models in the Model field; on save, a model the endpoint lists none of is refused with some it does offer. An endpoint without a list takes any name.
- **Secrets.** Tokens are sealed at rest with ChaCha20-Poly1305 under `keys/assistants.key` (mode 0600, made at first start beside the cookie key; each token carries its own random nonce), shown only as set or none, and never returned by the API. An empty token suits endpoints that need none. Each settings panel has TEST, a one-line exchange whose answer or error is shown. Each level's changes are in the audit log (`assistant.added`, `.updated`, `.enabled`, `.disabled`, `.deleted`, `.prompt`).
- **Request review.** Under Reviewers on the proposal page, the author or a writer sees an **ask NAME** button per assistant the repository may use. Asking records the event (`asked NAME to review`) and the audit entry (`assistant.asked`), then a background task builds the context (title, description, the list of changes, the diff of each changed file as `-`/`+` lines per hunk, binary files noted, the whole cut to the assistant's context budget less the prompt), prefixes a fixed system prompt (who it is, that it answers for the person who asked, that it cannot approve, the shape of a review) with the levels' prompts, calls the endpoint, and records the answer as a review of verdict `comment` whose author is the person who asked and whose **via** (migration 0036) is the assistant, shown as "bob ↳ agent Claude reviewed" in the timeline, exactly as agent-signed changes are shown (§6). An assistant never approves: approvals count only people, and an assistant's review never replaces the asker's own latest verdict. The proposal's author is told as for any review; the `review` webhook carries `via`. No answer leaves an event (`NAME did not answer: why`) and no review.
- **Draft the description.** The new-proposal form (anyone who may open one) and an open proposal's editor (the author or a maintainer) have a **draft with NAME** button per assistant. It asks the assistant, now, for a description from the title so far, the changes (titles and the first lines of their messages) and the diff, under a fixed system prompt (description field only, what and why, what to look at, how to verify) plus the levels' prompts, and puts the answer in the editor with a note; nothing is saved until the person updates or opens the proposal. The audit log records `assistant.drafted`; the same per-repository hourly limit applies. With JavaScript the button posts to `…/draft/stream` instead, which answers at once with server-sent events (`part` with each piece of text as the assistant writes it, then `done` with the whole draft or `error` with why there is none, each a JSON string, and a keep-alive every 15 s), and the editor fills as the draft comes; without it the form posts and the page returns with the draft. Every assistant call is made as a stream, and its time limit is for silence (90 s for a draft, 300 s for a review) under a 15-minute cap, so a long answer that keeps coming is not cut off (oxygenhub#248).
- **Limits.** One request per assistant per proposal at a time, 30 assistant calls per repository per hour (in memory), a 60-second call timeout, answers kept to 64 KiB, and a context budget (characters) per assistant.
- **Too much for the model** (oxygenhub#248). The context lists every changed file with its +/− counts before the diff, then gives whole files in order while they fit: one that doesn't is cut where the room ends (marked) when a fair amount is left, else left out, and every file cut or left out is named, so the model knows what it did not see. A provider that refuses the prompt as too long for its model (`400`/`413`/`422` saying so) is asked once more with a third of the budget; if that fails too, the event says the proposal was too large at both sizes and to lower the assistant's budget. A timeout says the same about slow or local models. A local Ollama cuts a prompt longer than its context window without saying so: set its context (`OLLAMA_CONTEXT_LENGTH`, or the model's `num_ctx`) to fit the assistant's budget, about four characters a token.
- **API.** `POST /repos/{o}/{n}/proposals/{n}/review-requests {"assistant": NAME}` (the author or a writer; 202 when started, 409 when that assistant is already answering or the repository is over its hourly limit, 422 for a name the repository cannot use); reviews carry `via`. Settings: `GET`/`POST /repos/{o}/{n}/assistants` (repo:admin; `POST` adds or changes by name, with `preset`, an optional write-only `token` that an empty value keeps, and the model checked against the endpoint's list), `DELETE …/assistants/{name}`, and `GET`/`PUT …/assistants/prompt` (the level's text and what it inherits); the same under `/orgs/{org}/assistants` for its owners and `/admin/assistants` for site administrators. `oh assistant list|add|remove|prompt` uses them, with `--org` or `--site`, and reads a token only from an environment variable (`--token-env`). Hub actions by an assistant are attributed "ASSISTANT via WHO" everywhere, so a project can require that a human has approved regardless of what an assistant said.
### 13.5 A job's identity: the hub as an OIDC issuer (done 2026-10-04)
608
609
A job that declares `identity "AUDIENCE" { mount-path "/run/secrets/…" }` (OxygenFlow m0 §3) gets a signed JWT so a cloud, a vault or any service that trusts this hub can let it in without a long-lived secret (oxygenhub#103).
610
611
- **Issuer.** The hub's `base_url`. `GET /.well-known/openid-configuration` publishes the discovery document (`issuer`, `jwks_uri`, RS256, the claims below) and `GET /.well-known/jwks.json` the public key, both cacheable for an hour. The key is RSA 2048 in `keys/oidc.pem` (PKCS#8, mode 0600), made the first time it is needed; its `kid` is the first 16 hex of BLAKE3 over the modulus. Rotation is by replacing the file and restarting; tokens signed by the old key then fail, which a job limit of hours makes harmless.
612
- **Token.** Minted at claim time, one per identity, and delivered through the same channel as secrets: the claim's `secrets` carry it as `identity:AUDIENCE` at the mount path, where Flow writes it 0400 on the in-memory mount. Header `{"alg":"RS256","typ":"JWT","kid"}`; claims `iss`, `sub` = `repo:OWNER/NAME:bookmark:BOOKMARK` (what a trust policy matches), `aud`, `iat`, `nbf` (30 s of skew), `exp` = the job's time limit from now, at most six hours, `jti`, and `repository`, `repository_owner`, `repository_visibility`, `bookmark`, `run_number`, `job`, `state` (the state hash), `runner`, `actor` (who started the run). A local `flow run` has no issuer and says the file will be missing.
613
- **Masking.** Log intake replaces any token this hub signed (recognised by its `kid`) with `***`, as it does secret values.
614
- **Trust, for example.** AWS: an IAM OIDC provider with the hub's URL and audience `sts.amazonaws.com`, a role whose trust policy matches `sub` `repo:andymccall/metalgear:bookmark:main`; the job then calls `aws sts assume-role-with-web-identity --web-identity-token file:///run/secrets/aws-token`. Google: a workload identity pool provider with the same issuer, mapping `assertion.sub`. HashiCorp Vault: the JWT auth method with `bound_audiences` and `bound_claims` on `repository`.
Each user and organisation backs up its repositories hourly to a place it controls: a GCS bucket reached keylessly (the hub mints a short-lived token with `sub = backup:OWNER`, which GCP's workload identity federation exchanges for an access token, so no key exists), or a directory the site allows. A run sends only the content-addressed objects the destination lacks, the refs at that moment and the hub's records (issues, proposals, releases, wiki, settings; never secret values), and is whole once its `DONE` marker is written; `oxygenhub admin restore` brings repositories back from any whole run. Built: Settings › Backups and an organisation's Settings › Backups (how often: off, hourly or daily; a directory from `hub.toml` `[backups] directories` or a GCS bucket with its workload identity provider; a prefix; runs kept 7, 30 or 90 days; "back up now"; the last 50 runs with their outcome), migration 0060 (`backup_settings`, `backup_runs`), the hub's scheduler (one run at a time; a run left running by a restart is marked failed), each repository's and wiki's store copied file by file under the repository's read lock for the final pass, the records as JSON (every table keyed by the repository, and three levels of rows pointing at them; tables and columns holding secrets, tokens, sessions or passwords never), pruning of old runs and unreferenced blobs, an inbox notice to the owner (an organisation's owners) once per spell of failures, and `oxygenhub admin backup run OWNER` and `oxygenhub admin backup restore --from DIR [--run AT]`, which restores the stores and the repositories' rows. A GCS destination is keyless: the hub mints a 10-minute token (`Issuer::mint_for`, `sub = backup:OWNER`, `aud` the provider's name over https), Google's STS exchanges it (`[backups] sts_url`) for an access token scoped `devstorage.read_write`, used with the Cloud Storage JSON API (`[backups] storage_url`) and exchanged again before it expires; TEST writes, reads and removes a probe. The design, the layout at a destination, setup and restore are in `docs/backups.md`.
`POST /api/v1/keyless/certify` gives a fresh key a certificate (Oxygen spec §14.7) for ten minutes, signed with the hub's key. The bearer is the credential: a running job's token (subject `repo:OWNER/NAME:bookmark:B`, claims `repository`, `bookmark`, `run`, `job`, `runner`, and `environment` with `approved_by` once an approved job may deploy), or one of this hub's OIDC tokens for the audience `{base_url}/keyless` (its `sub` the subject, and the same claims it carries). The body is `{"identity": base64 identity object, "proof": base64 CBOR Signature, "scope": "changes,refs"}`, the proof the fresh key's challenge signature (§14.3) over `certify_challenge(credential)` (`"oxy1 certify "` and the BLAKE3 of the credential), so it cannot be replayed with another credential. The answer is `{"certificate": base64 [certificate, signature], "subject", "expires_ms"}`; a certificate never outlives its credential, and it starts 30 s back for clocks a little behind. Each one is audited (`keyless.certified`). A runner may sign a job's receipt with such a key, certified just before it signs (scope `seal`): the receipt carries the certificate (Oxygen spec §14.6), and the hub keeps it when the certificate is this hub's, its subject is the run's repository and bookmark, its `run` and `job` claims are the run's and the job's, and it covers the moment the hub accepts it; no runner key is bound then, and the run names the signer as the runner's name with "(short-lived key)". Which issuers and subjects a repository trusts for changes and references is its policy's (docs/keyless.md, step 4). When a key it certified moves a bookmark, the hub signs an `accepted` attestation of the update with its own clock and serves it with the bookmark (Oxygen spec §14.7, step 4c), so fetches check the certificate at the hub's time. The scope given is what was asked within what the credential allows; asking for none of it is refused.
**Agent grants** (step 5, built 2026-10-17). An agent holds no key: its owner signs an Oxygen grant (§14.7) letting `agent:NAME@ACCOUNT` act for them, with the key their oxy home names (`oh agent grant NAME [--scope changes,seal,refs] [--for 30d]`), and `POST /api/v1/agents/grants {"grant": base64 signed grant}` keeps it and answers with the agent's token (`oxyagent_…`, shown once, kept as its BLAKE3 hash). The hub accepts a grant only for the caller's own `agent:NAME@USERNAME`, naming this hub's key as issuer, signed by a key linked to the caller's account and not yet ended; any personal token of the account may send it, since the signature is what authorises it. The agent exchanges the token at `keyless/certify` for ten-minute certificates carrying the grant (claims `agent` and `grant`, scope at most the grant's, never past its end), so what its fresh keys sign counts in the owner's name; oxy does this itself with `author.signer = keyless` and the token from `oxy auth grant`. The token is also the agent's credential for the protocol (fetch and push): it acts as an agent's token of the owner's account, writing when the grant gives `refs` and reading otherwise, so the agent's home holds no other secret, and oxy offers it when the home has no hub token. `GET /api/v1/agents/grants` lists the caller's grants without tokens; `DELETE /api/v1/agents/grants/{id}` revokes one at once (`oh agent list`, `oh agent revoke ID`). Migration 0075 (`agent_grants`); audited as `agent.granted` and `agent.revoked`.
The hub logs every signed object it accepts, in the order it accepts them, as entries of one Merkle tree (Oxygen spec §14.8):
629
- **What it logs:**
630
- from pushes: reference and tag attestations, seals, the repository's policy, the hub's own acceptances, and the certificates of short-lived signers;
631
- receipts as they are kept, and sealed approvals;
632
- every certificate `keyless/certify` issues.
633
634
An object is logged once (`tlog_entries`, migration 0076, maps objects to entry numbers), and a push is logged before it is answered.
635
- **Storage:** the log is append-only files in `<data>/tlog/`: `entries` (each a u16 length and its bytes), `offsets`, `hashes` (every tree hash in stored order) and `checkpoint`. A torn append is cut back to the last whole entry on start and its hashes rebuilt; nothing is ever rewritten.
636
- **Checkpoints:** every 5 seconds while entries arrive, the hub signs a checkpoint with its key. The origin is the base URL's host and `/log`, and the note carries an Ed25519 line and a composite line.
637
- **Serving:** it serves `GET /log/checkpoint` (no-cache), `GET /log/tile/<L>/<N>[.p/<W>]` and `GET /log/tile/entries/<N>[.p/<W>]` (C2SP tlog-tiles: full tiles cached for a year as immutable, partial ones for 5 s; a tile the log does not hold yet is 404), and `/log`, a page with the checkpoint and the newest entries (a private repository's entries show only their hashes).
- **Refs answers** carry the latest checkpoint and, for each bookmark's attestation, its entry and inclusion proof (Oxygen spec §16; step A3, 2026-10-18). An attestation served but never logged (pushed before the log began) is logged then, and proved on a later fetch. oxy pins the log's key per hub, requires each checkpoint to extend the last it saw (proved from the tiles), and fails a fetch whose hub showed two histories.
- **Witnesses** (step A5, 2026-10-18): a hub can witness other hubs' logs and have its own witnessed (C2SP tlog-witness, Oxygen spec §14.8).
640
- As a witness it answers `POST /log/witness/add-checkpoint`: it cosigns, with its key and clock under its host's name, a checkpoint that verifies with the log's key and extends the last it cosigned (`tlog_witnessed`, migration 0077). It answers 409 with its size when the log guessed it wrong, 422 to a fork, 403 to another key's note and 404 to a log it was not told to witness.
641
- As a log it sends each new checkpoint to its witnesses, with the consistency proof from what each last cosigned (`tlog_cosigned`), and serves their cosignatures with the checkpoint.
642
- Both are set in `hub.toml`; the key is the other hub's `/.well-known/oxyhub-key` in base64:
### 13.5.4 Terraform state (oxygenhub#252, built 2026-10-18)
652
653
The hub keeps Terraform and OpenTofu state per repository, so a pipeline needs no bucket or cloud credential just to hold it. It speaks their HTTP backend at `/api/v1/repos/OWNER/NAME/terraform/state/STATE` (STATE: letters, digits, `-`, `_`, `.`, up to 100):
654
- **GET** the state (204 when there is none);
655
- **POST** a new one: JSON, up to 32 MiB, with `?ID=` the lock's ID while it is locked;
656
- **DELETE** it;
657
- **LOCK** with the lock's JSON: 423 and the holder's lock when someone else holds it, 200 to its holder again;
658
- **UNLOCK** with the lock's JSON or `?ID=`: 409 for another lock's.
659
660
**Who may use it:**
661
- a job, with an identity token of this hub's for the audience `{base_url}/terraform` (its pipeline's `identity` block) or with its runner's job token, for its own repository only;
- a person, with a token of theirs with `repo:write` and the write role on the repository, to read a state as much as to change one: states hold secrets, and every account reads a public or internal repository (oxygenhub#256). The Terraform state page lists the states only to people with the write role.
**Storage:** each state is sealed at rest with the hub's secrets key, and the last 20 versions before it are kept (migration 0079: `terraform_states`, `terraform_state_versions`). Backups carry the sealed bodies, which open only with this hub's key. A repository's Artifacts › Terraform state page (`/{owner}/{name}/terraform`) lists its states, serials, writers, locks and versions, with the backend block to use.
**Versions (oxygenhub#255):** each state's kept versions are listed on that page (serial, writer, when), and the current state and each version download as `.tfstate` files, for people with the write role. A person with the maintain role restores a version: it is written again with a serial above the current one (OpenTofu and Terraform only take a state forward), and the state it replaces is kept as a version, so a restore never loses anything and going forward again is restoring the newer version. A restore is refused while the state is locked, asks for confirmation, and is audited (`terraform.restored`). The API: `GET …/terraform/state/STATE/versions` (newest first: `id`, `serial`, `written_by`, `written_ms`), `GET …/versions/ID` (the state) with the write role, and `POST …/versions/ID/restore` (the new `serial`) with the maintain role, never by a job. Not for the hub's own infrastructure (oxygenhub-gcp), whose state cannot live on the hub it deploys.
### 13.5.5 Metrics (oxygenhub#212, first slice 2026-10-18)
671
672
`GET /metrics` serves the hub's numbers in the Prometheus text format, only to a bearer of `[server] metrics_token` in hub.toml. While the token is unset the route answers 404. It reports:
673
- the build (`oxyhub_build_info`), uptime and resident memory;
674
- the state cache's states, estimated weight, budget and summaries;
675
- requests to bucket stores by kind (with pack listings sent and kept apart), and the time spent in them;
- OxygenFlow jobs queued, waiting for approval and running; runs finished in the last 24 hours by result; runners online and not, and seconds since each last called;
- reference updates pushed in the last 24 hours, accepted and refused, and the bytes the protocol received and sent by endpoint (`push` carries pushes, `fetch` and `objects` fetches), counted before compression;
- HTTP requests by route class (`page:<tab>`, `api:<area>`, `protocol:<endpoint>`, `static` and so on, never a user's or repository's name; at most 300 classes), method and status class, with latency histograms (5 ms to 30 s).
The standard OxygenFlow charts, for people who will not build their own in Grafana: one page at three scopes, so the numbers always agree, the site (`/admin/insights`, administrators, filtered by repository, owner or runner), an owner (`/insights/{owner}`: the owner, the organisation's members, administrators) and a repository (`/{owner}/{name}/insights`, in its bar's More menu and linked from its Flow tab). `?range=` is `1h`, `24h` (the default) or `7d`. It shows:
689
- headline numbers: time to pickup (median, 95th percentile), jobs finished and the share that passed, the median job duration, the jobs queued and running now, runners online and their slots in use;
690
- charts: time to pickup (median and 95th percentile by the job's start), jobs finished per bucket by result, queued and running jobs over time, job duration, and runner utilisation (running jobs over the slots of the runners online now);
691
- tables: each runner (status, busy slots, jobs, failures, median pickup and duration, last heard from) and, for the site and an owner, the busiest repositories.
692
693
**Pickup** is from when a job could run to when a runner took it. A job is *queued* when it is made or run again (`queued_ms`) and *ready* at the latest of that, its dependencies passing and its approval (`ready_ms`, set at its claim; migration 0080, which estimates earlier jobs from their run's start and the earlier stages' ends). A job a runner hands back keeps its times: the person still waits. The charts are inline SVG drawn by the hub (no script or library; coloured by theme classes). The page refreshes its numbers every 30 seconds while visible (`data-refresh`), and is a snapshot without JavaScript. **Wall view** (`?wall=1`, opened in a new window from the page): no menus, larger numbers, for a second monitor. The runners page's queue links its filters to the site's Insights (**Charts ↗**). `[server] grafana_url`, when set, adds an *open in Grafana* link for one's own charts from `/metrics`.
- **Reports.** A job's artifact named `coverage` in LCOV format (`cargo llvm-cov --lcov`, Istanbul's `lcov` reporter, `coverage lcov`, gcov with lcov) or Cobertura XML (`coverage xml`, `cargo llvm-cov --cobertura`, Coverlet, JaCoCo through cover2cover; a report starting with `<`) is read when the job uploads it. In Cobertura a filename is tried against each `<source>`, a `branch` line's `condition-coverage` counts its branches, and a `<method>` is a function, hit when any of its lines ran. The hub writes what it found into the job's log (`coverage: 88.6% of lines (951 of 1073) in 5 files`), or why it could not read it; a bad report never fails the upload. At most 64 MiB is read.
- **Counting.** Paths are taken relative to the job's `/src`; a path outside the repository, or a file the run's state does not hold, is left out and counted. Records for one file are added together. A file with a single record keeps the tool's own `LF`/`LH` totals, since `llvm-cov` declares lines its `DA` lines leave out; the totals then match what the tool printed.
701
- **Stored.** `coverage_reports` keeps each report's totals (lines, functions, branches) with its run, job, bookmark and state; `coverage_files` each file's counts. The per-line counts stay in the artifact's blob and are read again for the file view.
- **Pages.** A Coverage tab, shown once a repository has a report: the bookmark's latest percentage with its change against the report before over a chart of up to 200 runs (each point names its run and opens its report), a strip of facts (lines covered, hits per line, functions, branches, files, the first report, the badge with its Markdown), this run's changes with the files whose coverage moved most, files by coverage band, a chart of covered and missed lines, the recent runs, a bookmark picker, every file with a bar (sorted by lines missed, lowest coverage, or path), and a file view with each line run or missed and how often it ran. `/OWNER/NAME/coverage/badge.svg` is a badge of the default bookmark's latest coverage. The run page shows each job's coverage beside its artifacts.
- **The check.** Each report sets a `coverage` check on its run's state, measured against the default bookmark: its previous report there for a run on it, its latest for a run on any other bookmark (a proposal's). Settings › Coverage can set a minimum percentage and a largest drop in points; broken, the check fails and says why (`50.0%: below the minimum of 60%`), and naming `coverage` among the required checks holds a proposal back. Without rules it only reports. A bookmark's first report shows the files that moved against the default bookmark.
704
- **Oxygen itself:** its `test` job runs under cargo-llvm-cov in the build image (oxygenflow 7705d699) and keeps its report as the `coverage` artifact; the README carries the badge.
- **Reports.** A job's artifact named `junit` in JUnit XML (pytest `--junitxml`, cargo-nextest, Jest's junit reporter, gotestsum) is read when the job uploads it, at most 64 MiB and 50 000 cases. Every `<testcase>` counts, wherever its `<testsuite>` nests; a case is `passed`, `failed` (`<failure>`), `error` (`<error>`) or `skipped`, and a nextest case that failed and then passed on a retry (`flakyFailure`) is a pass. Counts come from the cases, not the suites' attributes. The job's log says what was found (`tests: 630 tests: 2 failed in 3 min 4 s`), or why it could not be read; a bad report never fails the upload.
710
- **Failing jobs.** Declare the artifact with `when "always"` (OxygenFlow spec §5.2) so a failing suite still sends its report; it is then outside the job's receipt, and the run page says so.
711
- **Stored.** `test_reports` keeps the counts and summed test time per report with its run, job, bookmark and state; `test_cases` every case with its status and time, and for a failure or error its message (2 KiB) and its text with the case's output (16 KiB).
712
- **Run page.** Beside each job its counts, linked to a block above its log: passed, failed, errored and skipped, then the first 50 failures, failures first in report order, each collapsible to its message and output.
- **Tests tab.** Shown once a repository has a report: the bookmark's latest pass rate (or how many fail) over a chart of up to 200 runs, the counts and test time against the run before, the tests newly failing and fixed against that job's previous report, tests that both failed and passed in the last 30 reports (flaky), the run's failures, a passed-and-failed chart, the slowest tests and recent runs. `?report=` shows any report, `?bookmark=` another bookmark.
- **A test's history.** `/tests/case?classname=&name=` shows one test across the bookmark's last 200 runs: its result, time (charted) and message in each; the Tests tab links each failure, slow test and flaky one to it.
716
- **The check.** Each report sets a `tests` check on its run's state: `630 passed`, or what fails and which of those passed in the same job's report on the default bookmark (its previous one for a run on it, its latest for a run elsewhere), as `2 failing; 1 new against main: tests.test_io::test_write`. A proposal shows it with its other checks, so it names what the proposal broke.
A repository may enable a **Git mirror**: on every landing on selected bookmarks the hub runs `oxy git export` into `<data>/mirrors/<owner>/<name>.git` and serves it read-only over Git's smart HTTP protocol at `/{owner}/{name}.git` (implemented with the `git` binary's `http-backend` behind the hub's authorization, since reimplementing pack negotiation is not the product). This lets GitHub Actions runners, IDEs and tooling that only speak Git read hosted Oxygen repositories. Pushing to a mirror is refused with a message pointing at `oxy`.
721
722
Import from Git: `oxy git import` at repository creation from an uploaded bundle or a URL, keeping the mapping so later `oxy git fetch` keeps the imported bookmark following the Git branch.
723
724
## 15. Security
725
726
### 15.1 Threat model
727
728
An anonymous internet user, a signed-in user attempting to exceed their role, a collaborator with `write` attempting to move a reference the policy forbids, a malicious repository content author (Markdown, file names, huge objects), and a compromised hub host. The last is bounded by design: the hub holds no root keys, approvals are user-signed seals, history is signed by authors, so a compromised hub can deny service and forge hub-only data (issues, comments) but cannot forge history, approvals or policy without detection.
729
730
### 15.2 Web
731
732
CSRF: every unsafe form carries a token bound to the session (`HMAC(cookie_key, "csrf:" || session mac)`), and the `Origin` header is checked when present. Cookies are `HttpOnly`, `SameSite=Lax`, `Secure` over HTTPS. Content Security Policy: `default-src 'self'`, no inline scripts except the theme snippet by hash, `img-src 'self' data:`; rendered Markdown is sanitised by ammonia with an allow-list; rendered HTML files are never served inline (raw file responses use `Content-Disposition: attachment` and `X-Content-Type-Options: nosniff`) except images with sniffed, allow-listed types. Path parameters are validated before touching the file system; paths inside repositories are resolved against the object store, never the disk.
733
734
### 15.3 Passwords, sessions, tokens
735
736
Argon2id with the defaults of the `argon2` crate, per-password salt; TOTP with a 30-second step and one-step drift; recovery codes; sessions as in §5.1; tokens stored as BLAKE3 hashes and compared in constant time; login rate limited per account and per address; all session-changing actions logged in the audit log.
737
738
### 15.4 Repository content
739
740
Object sizes are bounded by the wire protocol server's body limits (per push and per object); rendering is bounded by `render_limit` and by time budgets; archives are streamed; symlinks in trees are shown, never followed on the server; subrepository pins are displayed, never fetched by the hub.
**Push protection** (oxygenhub#268). A push that adds what looks like a credential is refused before any of it is stored, so the secret never enters the repository's history. The hub implements the protocol server's `vet_push` hook (Oxygen spec §16.4): after the push is decoded and before anything is stored, bound or moved, it scans what the push *adds*, the lines its changes insert and the new blobs that replace a file's content (blobs the hub already holds are not new), never history it holds.
743
744
- **Rules** are data, `crates/hub/src/push_protection.toml`, compiled in and versioned (`version`, shown in every refusal as `rules vN`). Each has an id, a description, a regex matched on bytes (its `secret` group, else the whole match, is the secret), optional keywords that must occur (case-insensitive) before the regex runs, and optionally a regex the same file or inserted text must also match, for secrets that are only one when paired. The first set: AWS access key IDs, AWS secret access keys next to a key ID, Google Cloud service account keys (`"type": "service_account"` with a `private_key`), Azure storage connection strings, GitHub tokens (`ghp_`, `gho_`, `ghu_`, `ghs_`, `ghr_`, `github_pat_`), GitLab personal access tokens (`glpat-`), this hub's own credentials (personal access tokens `oxyhub_`, agent grant tokens `oxyagent_`, OxygenFlow job tokens `flowjob_`, runner credentials `flowrunner_` and registration tokens `flowreg_`), Slack tokens (`xoxb-` and kin), Stripe live keys (`sk_live_`, `rk_live_`), OpenSSH and PEM private keys (a `BEGIN … PRIVATE KEY` line, a body and its `END` line; the header alone, as in documentation, is not one). Rules are tried in order and a match overlapping an earlier rule's is dropped, so one secret is reported once, by its most specific rule.
745
- **Findings.** Each names its rule, the file and line (the line as the change that adds it leaves the file, worked out from the repository's state with the push's changes applied in memory, only when there is a finding), a fingerprint (the first 16 hex digits of BLAKE3 of the rule id, a NUL and the secret) and the line with the secret masked (its first four characters kept). The secret itself is never shown, logged or stored.
- **Budget**, so pushes stay fast: at most `scan_limit` bytes scanned per push request (16 MiB), each file or inserted text at most `scan_file_limit` (1 MiB; larger ones, and binary blobs, with a NUL in the first 8000 bytes, are skipped), and `scan_time_ms` (2000) in all, all under `[limits]` in `hub.toml`. Past the size or time budget the rest goes unscanned and the push is **accepted**: refusing would block every large import (a whole history pushed in parts, each part with its own budget) for what is most likely nothing. The accepted references' push log entries say how much went unscanned (`push protection scanned 16.0 MiB and stopped at its size budget: … more were not scanned`). History the hub already holds is the history scan's (below).
- **Refusal.** The push fails with HTTP `422`, nothing of it stored or moved, and a plain text the client prints as it is: one line per finding, `path:line: description (fingerprint F)` with its masked line under it (20 at most, then a count), then what to do: take the value out of the change that adds it (`oxy amend`, or `oxy abandon` a change that only adds it) and push again, revoke the credential if it was real, or have a maintainer allow the fingerprint at `{base_url}/{owner}/{name}/settings/push-protection`. A push through an agent's token (an agent-marked token or a keyless agent's grant) adds a paragraph for the agent naming it: remove the value and push again, do not work around the refusal, and tell its person if it was asked to commit the value, since only a maintainer can allow one. The refusal is recorded in the push log (`/{owner}/{name}/pushes`, each named reference refused with `push protection: N possible secrets (path:line, …)`), and its findings (rule, path, line, fingerprint, masked line, pusher and agent) in `secret_findings` (the newest 200 per repository) for the maintainers.
- **Bypass.** Under Settings › Push protection (maintain role) a maintainer sees the recent findings not yet allowed and allows one by its fingerprint with a reason (required, at most 500 characters), from the finding's row or by typing a fingerprint from a refusal: a row in `secret_bypasses` (repository, fingerprint, the rule it was found under, reason, who, when), audited as `secret.bypass`. A finding whose fingerprint is allowed passes; the allowed fingerprints are listed with their reasons. A writer cannot allow one. Each allowed fingerprint has a **remove** button (maintain role, `POST …/settings/secret-bypass/remove`), audited as `secret.bypass_removed` with the reason it had: pushes adding that value are refused again and its history findings are open again (oxygenhub#277).
- **Setting.** Per repository, `repositories.push_protection`, on by default; Settings › Push protection (maintain role) turns it off or on, audited as `repo.push_protection`. A wiki follows its repository's setting and bypasses (migration 0085, both databases).
- **History scan** (oxygenhub#277). The same rules run over what each repository already holds, so a secret committed before push protection existed, or while it was off, is found too. What is scanned is every change reachable from the repository's bookmarks, read as push protection reads a push: the lines each change inserts and the text blobs it puts in place of a file's content (within `scan_file_limit`; binary and lazy blobs are skipped). It is the worker role's `secrets.history_scan` task (docs/roles.md), under its leader lock: ten seconds after the process starts it passes over every repository (so the first deploy scans every repository's whole history), then over a repository again when a push to it is announced on the bus (`repo:N`, `push`), when a maintainer asks or turns push protection back on, and every 15 minutes over all of them for what the bus missed (a landing, a lost event). Repositories with push protection off are skipped; wikis are not scanned.
751
- **Progress and budget.** Each repository's scan is a row in `secret_scans`: the rules version it ran with, the heads whose history is scanned (`done_heads`), and while a scan is under way the heads it works toward (`target_heads`, the bookmarks' heads when it started) and how far it got (`position` of `pending` changes). The changes to scan are those reachable from the target and not from the done heads, ordered by clock then hash, so every pass sees the same list and resumes at `position`. A pass over one repository stops at `history_scan_limit` bytes (64 MiB; a change counts at least one byte) or `history_scan_time_ms` (5000) wall-clock, under `[limits]`, after at least one change; the worker then moves to the next repository and comes back, so every repository progresses in turn. When the list is done the target becomes the done heads: after a push only the new changes are scanned. A push during a scan waits for it to finish. A target that no longer loads (a bookmark rewritten and collected) is replaced by the bookmarks as they are. A new rules `version` scans every history again. `history_scan_limit = 0` turns the scan off.
752
- **Low priority.** One repository at a time, one pass each in turn, on one blocking thread that rests as long as it worked (at most half a core), 200 ms between repositories, holding only the repository's shared lock, as pages do: it never holds up a push, a page or a gc.
753
- **Findings** go into `secret_findings` with `source` `history` (refused pushes' are `push`), the change that adds the value (`change_hash`), the first bookmark that reaches it (the default bookmark first) and the file and line as that change leaves the file, worked out from the state of its dependencies with it applied. One row per value and file however many changes carry it (a unique index on repository, fingerprint and path for `history` rows), so a rescan adds nothing known and keeps what was resolved; they are not trimmed with the 200 push findings.
754
- **Found in history.** Settings › Push protection (maintain role) shows the scan's state (scanning N of M changes, or scanned up to the bookmarks and when, with totals and the last error) and the open findings, oldest first: each with its change (linked), bookmark, file and line, masked line and fingerprint, and two actions. **Allow** is the push bypass above (a value that is not a secret is then allowed for pushes too). **Mark resolved** (`POST …/settings/secret-resolve`, a reason required, at most 500 characters) says the secret was revoked or rotated, since removing it from the files now leaves it in every earlier change; it records who, when and how, audited as `secret.resolved`, and moves the finding to a Resolved list. **Scan history now** (`POST …/settings/secret-scan`, maintain role, audited as `secret.history_scan`) forgets what was scanned (findings and marks stay) and puts the repository first. A writer can do none of these (migration 0086, both databases).
The hub delegate key lives in `<data>/keys/` with mode `0600`. Its identity object is published at `/.well-known/oxyhub-key` so policy holders can admit it. TLS: the hub serves HTTPS itself with the configured certificate, or plain HTTP behind a terminating proxy, or on the developer edition a self-signed certificate it generates at `init` (with `rcgen`) whose CA is downloadable from the trust page and usable with `OXY_CA`.
759
760
### 15.6 Rate limits and abuse
761
762
Per address for anonymous requests, per account and per token otherwise, with separate budgets for pages, API, search and the wire protocol; registration and login have their own. Administrators can ban accounts and block addresses.
763
764
## 16. Performance and caching
765
766
### 16.1 Targets
767
768
Repository overview and tree pages in under 100 ms on a warm hub for a repository of 100 000 changes; a change page in under 200 ms; a projected state for a 50-change proposal in under 2 s; wire protocol throughput limited by disk and network, not the hub.
Every `oxy-core` call runs on the blocking pool. **Readers never wait** (oxygenhub#249): each repository has a writer lock that pushes, gc, landings, web edits, Diffusion bumps, Flow's blobs, mirror exports, forks and a backup's last pass take in turn, and a lifecycle lock that everyone holds shared and only deleting or moving the repository holds alone. Page and API reads and fetches take only the shared lifecycle lock, so a push or a gc never holds them up: the store gives each handle one head for its life and finds what a repack moved (a reload and a retry on a miss; repacks retire what they replace for an hour before it is deleted; Oxygen spec §13.4). `oxy-core` itself is single-threaded per `Repo`, so a `Repo` is opened per request and dropped after; store handles are cheap to open because packs are memory mapped.
Keyed by state hash, so they never go stale: loaded `State` objects (LRU by memory, up to 256 and `OXYHUB_STATE_CACHE_MB`, default 3 GiB), a state's brief (its change count, latest change and whether it is conflicted, #145: kept in memory and on disk under `<data>/cache/summaries/`, worked out four at a time when a page needs many and as soon as a push moves a bookmark, so a restart never makes a bookmark list or an overview load every bookmark's state again), tree listings, rendered file views, blame, history graph layouts per bookmark head set, projected states per proposal, archives, code search results. Keyed by content: rendered Markdown (BLAKE3 of the text and its two link bases, 32 MiB) and syntax highlighted files (BLAKE3 of the file name, which picks the syntax, and the text, 64 MiB), each an LRU bounded by the bytes it holds, rendering outside the lock; an entry larger than a quarter of its budget is not kept. Keyed by repository: bookmark and tag lists, invalidated by the push path. `ETag` and `Cache-Control: private, max-age=0, must-revalidate` on pages; `immutable` on content-addressed responses (raw files by hash, archives by state). A request that takes two seconds or more is logged at warn with its method, path (never the query) and status.
History pages paginate by cursor; the graph renders a window of 100 changes with lazy extension; `log_for_path` for tree rows is fetched per row after the page paints and cached per (state, path). The projected state job uses `load_state` on the target and applies only the proposal's changes on top.
781
782
## 17. Operations: configuration, storage, backup
783
784
### 17.1 Configuration
785
786
`hub.toml` in the data directory; every key overridable by `OXYHUB_<SECTION>_<KEY>`:
787
788
```toml
789
[server]
790
bind = "0.0.0.0:7300" # plain HTTP
791
tls_bind = "0.0.0.0:7343" # HTTPS, when tls.cert and tls.key (or self_signed) are set
792
base_url = "https://earth.andymccall.xyz:7343"
793
behind_proxy = false # trust X-Forwarded-* when true
SQLite in WAL mode; migrations embedded and applied at start; `PRAGMA user_version` guards downgrades. Repositories are ordinary Oxygen repositories; `oxy gc` runs per repository as a nightly job, one repository at a time under its write lock, at `retention.gc_hour` UTC (default 3; 24 or more turns it off), and `oxygenhub gc` runs the same pass by hand. Each repository's run is logged with what it pruned and packed.
oxygenhub admin --data DIR user|repo|token … offline administration
838
oxygenhub backup --data DIR --to FILE consistent snapshot: sqlite backup API plus repos and assets as tar.zst
839
oxygenhub check --data DIR verify every repository's objects and signatures, report
840
```
841
842
### 17.4 Backup and restore
843
844
`backup` uses SQLite's online backup API for the database and copies `repos/`, `assets/`, `keys/` and `hub.toml`; `cache/` is excluded. Restore is extraction plus `serve`. Because objects are content addressed, a repository directory copied while a push is in flight is still valid up to its last committed reference update.
845
846
### 17.5 Developer edition on Earth
847
848
Earth (`earth.andymccall.xyz`, Ubuntu 26.04, x86_64, 4 cores, 31 GiB, Docker, ports 22, 6400, 6670, 7842, 8080 in use) runs the developer edition as a systemd service under a dedicated `oxygenhub` system user with `<data> = /home/oxygenhub/data`, HTTP on 7300 and HTTPS on 7343 with a self-signed certificate. Source is synchronised from the development machine with `rsync` (the Oxygen workspace and the hub) and built on Earth with a rustup toolchain installed for the `claude` account; `docs/deploy.md` gives the exact steps. Once the hub runs, `oxygen`, `oxygen-vscode` and `oxygenhub` are pushed to it as its first repositories and become their own remotes; that fulfils the self-hosting rule without any Git involvement.
849
850
## 18. Required changes to Oxygen
851
852
Each is small and lives in the Oxygen workspace, tracked in its `docs/plan.md`:
853
854
1. **Bare repositories.** `Repo::open_bare(dir)`: open a `.oxy/` with no workspace; refuse working-copy operations with a clear error. `oxy init --bare`.
855
2. **Server embedding.** `oxy-server` exposes `router_with(resolver, authorizer)` where the resolver maps a path (possibly `owner/name`) to a repository directory and the authorizer decides read and push per request, so the hub embeds it instead of copying handlers. Repository names with one `/` allowed when the resolver permits.
856
3. **Challenge signing.** `oxy identity sign-challenge <bytes>` and `oxy_core::identity::verify_challenge`, for linking keys (§5.3) and HTTP signature authentication (§5.4).
857
4. **Seal command.** `oxy seal <hash>..` signs a seal attestation over the given changes with the identity key and pushes it on the next push (the seal object and its verification exist, spec §14.4; today only the sync path creates seals). `oxy seal --proposal <url>` fetches the hash list from the hub API first.
858
5. **Landing from a client.** `oxy proposal land <url>` and `oxy proposal show <url>` as thin API clients.
859
6. **Delegate signing of hub-created changes.** A library entry point to build and sign a change from an edit set with `author = user key, signer = hub key`, without a workspace (used by web edits, browser conflict resolution and landings). `oxy identity delegate add|remove` already exists.
860
7. **History for the hub.** `Repo::log_window(heads, cursor, limit)` for paginated history, and a `graph::layout` module shared with the CLI so both draw the same lanes.
861
8. **Push hook.** `receive_push` reports, per accepted reference, the old and new state and the list of newly stored changes, for the pushes log and webhooks.
862
9. **Machine interface.** `oxy verify --json` already exists; add `oxy policy propose` output as a downloadable proposal file (it exists as a file format, spec §14.5) and `oxy tag list --json` is already present.
863
864
## 19. Milestones
865
866
| | Delivers | Done when |
867
|---|---|---|
868
| **0.1 Browse** | Data directory, config, SQLite, embedded wire server, repository creation by `oxy push`, theme and shell, overview, tree, file, raw, Markdown, history, change page, compare, bookmarks, tags, blame, conflicts page, trust page (read-only) | `oxygen`, `oxygen-vscode` and `oxygenhub` are hosted on Earth and browsable; `oxy clone` and `oxy push` work against it |
869
| **0.2 Accounts and access** | Registration, sessions, TOTP, profile, organisations and teams, roles, visibility, tokens, linked identities with challenge, pushes log, settings pages, audit log | Pushes are attributed to linked keys; a private repository is invisible to outsiders |
870
| **0.3 Proposals** | Proposals from bookmarks and forks, projected state, diff, conflicts, atom-anchored comments, reviews with seal-backed approvals, requirements, landing by delegate or client, stacking | A change proposed, reviewed with a comment that survives a rebase, sealed, and landed, all from the pages plus `oxy seal` |
871
| **0.4 Issues and notifications** | Issues, labels, milestones, cross references, templates, inbox, email, watching, mentions, search over issues and proposals | — |
872
| **0.5 API, webhooks, checks** | `/api/v1` complete for every page, `hub-api` crate, webhooks with delivery log, checks with seals, required checks, SSE status | The VS Code extension can list proposals and checks through the API |
873
| **0.6 Web edits, releases, mirror** | CodeMirror editing and new file with delegate signing, browser conflict resolution, releases with assets and archives, Git mirror over smart HTTP, code search | A GitHub Actions job can clone a hosted repository over the mirror |
874
| **0.7 Runner** | `docs/runner.md`, `oxyrun`, job-claim API, seals from the runner | A pipeline in `oxygen` builds and tests on Earth and its result appears as a required check |
875
| **1.0** | Passkeys, backup and restore verified, load test at the §16.1 targets, security review of §15, documentation, packaged release (binary, container image, systemd unit) | — |
876
877
## 20. Open questions
878
879
1. **Seal signing UX.** Requiring `oxy seal` from a client for every approval is honest but slower than a button. A browser-held signing key (WebAuthn `prf` extension deriving an Ed25519 key) could sign approvals in the browser with hardware backing; ML-DSA-65 in the browser is heavier. Decide after 0.3 ships with the client flow.
880
2. **Delegate key custody.** One hub key delegated to by many users means one key whose compromise lets an attacker sign as any of them (still subject to policy and to each user's revocation). Per-user delegate keys held by the hub would narrow that blast radius at the cost of a key per user. Decide before 0.6.
881
3. **Postgres.** SQLite is enough for one host; an organisation-scale hub may want Postgres. `sqlx` keeps the option open, but every query would need to be dialect-clean. Decide when someone needs it.
882
4. **Comment anchoring for lines that are not atoms.** Comments on deleted lines and on binary files fall back to file anchors; is that good enough?
5. **Pipeline language.** KDL is the current choice for `.oxyhub/pipelines`; Starlark would give loops and functions at the cost of a real evaluator. Settle in `docs/runner.md`.