OxygenHub — Specification
| Status | Draft 0.1 |
| Scope | A self-hosted forge for Oxygen repositories: hosting, browsing, review, issues, releases, an HTTP API and webhooks, and the integration point for a runner. It works like GitHub where GitHub is right and differs where the commutative model makes something better possible. |
| Depends on | oxy-core and oxy-server from the Oxygen workspace (spec Draft 0.8b); oxy clients speaking wire protocol version 1 |
| Companion documents | docs/runner.md (Oxygen Runner, later), docs/deploy.md (developer edition on Earth) |
| Date | 2026-09-27 |
| Audience | Implementers, security reviewers, and the person who runs it |
The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be interpreted as described in RFC 2119.
Table of contents
- Goals and non-goals
- How it compares with GitHub
- Architecture
- Repositories on the server
- Accounts, identities and authorization
- Web interface
- Proposals and review
- Issues
- Releases and tags
- Notifications
- Search
- HTTP API and webhooks
- Checks and OxygenFlow
- Git interoperability
- Security
- Performance and caching
- Operations: configuration, storage, backup
- Required changes to Oxygen
- Milestones
- Open questions
- Changelog
1. Goals and non-goals
- Work like GitHub. A developer who knows GitHub must find repositories, history, diffs, review, issues, releases, tokens and webhooks where they expect them, with the same words where the words still apply. The learning curve is Oxygen's model, not the forge.
- Show what the model makes possible. Multiple heads, reified conflicts, exact blame, signatures and trust policy, proposals that never go stale, review comments that survive rebases, and approvals that are portable signed objects. Each of these is a first-class page, not a footnote.
- One binary, one directory.
oxygenhub servelinksoxy-core, embeds the wire protocol server, renders its own pages, and keeps everything under one data directory with SQLite for hub state. A backup is a copy of that directory. There is no separate database server, job queue, cache server or front-end build. - The repository is the source of truth. Everything about code, history, references, signatures and policy lives in the Oxygen object store and is readable with
oxyalone. Hub tables hold only what Oxygen has no object for: accounts, sessions, issues, review threads, webhooks and check results. Deleting the hub database loses discussion, never history. - Off the shelf where it is not the product. Markdown editing uses EasyMDE, in-browser code editing uses CodeMirror 6, syntax highlighting uses syntect, Markdown rendering uses pulldown-cmark with ammonia sanitising. The hub's own code is the forge logic and the pages.
- Honest state. A page never shows something the object store does not say. A signature is shown as verified only when the hub verified it against policy; an approval is shown only when the approving seal exists.
Non-goals for 1.0: hosting Git repositories natively (§14 covers mirrors), a marketplace, wikis, project boards, discussions, GitHub Pages, and federation between hubs. The runner is specified separately (§13, docs/runner.md) and is not required for 1.0.
2. How it compares with GitHub
| GitHub | OxygenHub | Why it differs |
|---|---|---|
| Repository, owner, organisation | Same | — |
| Branch | Bookmark | An Oxygen bookmark names a set of heads, not one commit |
| Commit | Change | Changes commute; a change has dependencies, not a parent |
| Commit page shows one parent diff | Change page shows the change's own edits, its dependencies and its dependents | The diff of a change is intrinsic; it is the same on every bookmark that contains it |
| Merge, merge commit, rebase before merge | Landing: add the proposal's changes to the target bookmark | Union of heads; no merge commit; the result is computed before landing, so CI runs on exactly what lands |
| Pull request "out of date with base branch" | Never happens; a proposal stays valid as the target moves | Changes commute unless they conflict; a new conflict is shown, not a stale flag |
| Merge conflict blocks the PR until the author rebases | Conflict is a reified part of the resulting state; the proposal page shows the exact sides, and any collaborator may push a resolution change | Conflicts are values in the model (spec §9) |
| Review comment on a line, goes "outdated" after a force-push | Review comment anchored to an atom (ChangeHash, index), follows the line through rebase, squash and split | Atoms are stable identities; rewrites record correspondences |
| Approval is a database row | Approval is a seal attestation signed by the reviewer's key, stored in the repository | Verifiable offline and after export; survives hub loss |
| Required status checks on a commit SHA | Required checks on a state hash | A state is content addressed, so a green check on the pre-landing state is exactly the check on the post-landing state |
| Branch protection rules | The repository's trust policy (spec §14.5), edited through signed proposals | One authority for who may update a reference, enforced identically by every client and server |
| Signed commits as a badge | Signature and seal status on every change, with the policy rule that admitted it | Signatures are the protocol, not an add-on |
| Tags and releases | Tags are signed attestations; releases attach notes and assets | — |
| Actions | Runner (§13), later | Designed from the failures of Actions and GitLab CI |
| Fork and PR from a fork | Fork, and proposals from a fork or from a bookmark in the same repository | Same |
| Issues, labels, milestones | Same, smaller | — |
| Personal access tokens | Same, scoped | Push also accepts the client's identity signature (§5.4) |
| Webhooks with HMAC | Same | — |
| REST and GraphQL APIs | JSON REST only | — |
3. Architecture
3.1 One process
oxygenhub is a Rust binary in its own Cargo workspace at ../oxygenhub, depending on oxy-core and oxy-server by path (developer edition) or by version (release). It runs one tokio runtime with:
- an axum router serving pages, static assets, the JSON API, and the wire protocol under every repository's path;
- askama templates compiled into the binary; one hand-written stylesheet, also compiled in and served at
/style.css; - SQLite through
sqlx(WAL mode,synchronous=NORMAL, foreign keys on) for hub state, opened from<data>/hub.db; - a small background worker pool (tokio tasks) for webhooks delivery, notification email, search indexing, Git mirror export and cache warming;
- an in-process cache (§16).
There is no separate front-end build. JavaScript is limited to: the theme cookie, EasyMDE (Markdown editing), CodeMirror 6 (file editing), and small progressive enhancements (history graph hover, keyboard shortcuts, live check status). Every page MUST render and every form MUST work with JavaScript disabled, except the two editors, which degrade to a plain <textarea>.
3.2 Layout on disk
<data>/
hub.db SQLite database (and -wal, -shm)
hub.toml configuration (§17.1), optional; environment overrides
keys/ the hub's own identity (delegate key, §5.5) and TLS material
repos/<owner>/<name>/ one Oxygen repository per directory, bare (§4.1)
mirrors/<owner>/<name>.git optional Git mirror (§14)
assets/<owner>/<name>/ release assets, content addressed by BLAKE3
cache/ rebuildable: rendered pages, graph layouts, search index
<owner> and <name> MUST match [A-Za-z0-9][A-Za-z0-9._-]{0,63} and MUST NOT be ., .., or end in .git (reserved for the mirror URL). The reserved top-level path segments are api, static, login, logout, register, settings, explore, search, notifications, new, orgs, admin, style.css, favicon.ico, favicon.svg, health; no owner may take those names.
3.3 Request path
- Middleware: request id, tracing span, body limits, session cookie decode, CSRF check on unsafe methods (§15.2), rate limits (§15.6).
- Wire protocol routes
/{owner}/{name}/oxy/v1/*go to the embeddedoxy-serverhandlers with the hub's repository resolver and authorizer (§18). - Page and API handlers resolve
{owner}/{name}to aRepoHandle, check the viewer's role, and run everyoxy-corecall insidespawn_blockingunder the repository's read-write lock (§16.2). - Templates receive plain structs; no template touches
oxy-coredirectly.
3.4 Crate layout
oxygenhub/
Cargo.toml workspace: crates/hub (binary), crates/hub-api (types shared with clients)
crates/hub/src/
main.rs CLI: serve, init, admin (§17.3)
config.rs hub.toml and environment
db/ sqlx migrations and queries, one module per table group
repos/ RepoHandle, locks, cache, bare repository operations
auth/ sessions, passwords, TOTP, tokens, identity linking
web/ page handlers, one module per area (repo, change, proposal, issue, …)
api/ JSON handlers
protocol.rs embedding oxy-server
render/ markdown, syntax highlighting, diff rendering, graph layout
jobs/ background workers
templates/ askama; base.html, style.css, one file per page
assets/ wordmark, favicons, vendored EasyMDE and CodeMirror bundles
migrations/ sqlx migrations
docs/ this file, runner.md, deploy.md
4. Repositories on the server
4.1 Bare repositories
A hosted repository is an Oxygen repository directory with no working copy checked out and no user workspace. oxy-core today always keeps one working-copy change per workspace; the hub never snapshots, commits from a checkout, or touches files outside .oxy/. The directory contains only .oxy/. §18 lists the small change that lets Repo::open skip working-copy bookkeeping for such a directory.
Creation: Repo::init under <data>/repos/<owner>/<name> with a fresh repository id, then an initial row in repositories. Import: a client oxy push into an empty hub repository; or, for Git, §14.
Deletion moves the directory to <data>/trash/<owner>/<name>-<timestamp> and a job removes it after trash_days (default 30). Renames are a directory move under the repository's write lock plus a redirects row so old URLs resolve for 90 days.
4.2 Forks
A fork copies the object store (hard links where the file system allows, otherwise a copy) and records forked_from. Because objects are content addressed and repository ids differ, a proposal from a fork carries the changes across repositories by hash; the hub fetches missing objects from the fork's store directly, never over the network.
4.3 The default bookmark
Each repository has a default bookmark (main unless changed). It is what the repository page, tree, file views and search show, and the target every proposal picks first.
4.4 Visibility
public (readable by anyone including anonymous clients), internal (readable by every signed-in account), private (readable by collaborators). Wire protocol reads obey the same rule: a private repository answers 404 to an unauthorized client, not 401, so its existence is not disclosed.
5. Accounts, identities and authorization
5.1 Accounts
An account has a username (same grammar as an owner name), a display name, an email (verified before it is used for anything), an Argon2id password hash, an optional TOTP secret, a session salt (rotating it signs out every session), and a set of linked Oxygen identities (§5.3). The first account created becomes the site administrator. Registration is open, invitation only, or closed, per configuration.
Sessions are a signed cookie oxyhub=<username>.<mac> where mac = HMAC-SHA256(cookie_key, "session:" || username || ":" || session_salt), HttpOnly, Secure when served over HTTPS, SameSite=Lax. Password changes and "sign out everywhere" rotate the salt. Passkeys (WebAuthn) are planned for 1.0 as a second factor or password replacement.
5.1.1 Email verification and password reset
Both use account links: 32 random bytes as hex in a URL, stored only as their BLAKE3 hash, bound to one account and one purpose, expiring, and consumed on first use.
- Verification. Setting or changing an email marks it unverified. When the hub can send mail it sends a link to the new address (valid 48 hours). The link verifies that exact address: one sent to an address the account has since replaced does nothing. The settings page shows whether the email is verified and can send a new link: at most one every two minutes for the same address, and ten an hour per account whatever the addresses, so the form cannot be used to mail other people. Notification email goes only to verified addresses.
- Reset by email.
/login/resettakes a username or a verified email. When it matches an active account with a verified email and the hub can send mail, a link valid for one hour goes to that address, at most one every two minutes per account. The page answers the same way whether or not anything matched, so it cannot be used to find accounts. - Reset by an administrator. When the hub cannot send mail, or the account has no verified email, an administrator issues a reset link from the admin page (valid 24 hours), shown once to hand over by other means. The audit log records who issued it.
- Using a reset link sets a new password (the same rules as registration), rotates the session salt so every existing session ends, voids the account's other reset links, and records
password.resetin the audit log. It does not turn off TOTP: a reset account with two-factor sign-in still needs its code. A banned account cannot be reset. - Mail.
[email] smtpis ansmtp://orsmtps://URL, orfile:///pathfor development, which writes each message to that directory as a file instead of sending it.
5.2 Organisations and teams
An organisation is an owner with members and teams. Roles on an organisation: owner, member. Roles on a repository, whether personal or organisational: read, triage (manage issues and proposals without writing), write, maintain (settings except deletion and secrets), admin. Teams hold members and receive repository roles as a unit. An organisation's default repository role for members is configurable.
5.3 Linked identities
An Oxygen identity is a signed object holding an Ed25519 + ML-DSA-65 key pair's public halves (spec §14.2). An account links an identity by uploading the identity object (oxy identity export) and proving possession: the hub issues a 32-byte challenge and the client signs it with oxy identity sign-challenge (§18); the hub verifies both signatures. A linked key is what makes a pushed change this account's change: author attribution in every page comes from the change's signing key, not from who held the token that pushed it. A change signed by an unlinked key shows its identity's name (or KeyId when the identity is unknown) with a "key not linked" tag rather than a user name.
An identity can be linked to at most one account. Rotation (oxy identity rotate, renew, delegate add|remove) produces a successor identity whose object names the old key. When a push brings changes authored by a successor of a linked identity, and every step of the chain kept the key pair, the hub links the successor to the same account itself and records identity.rotated in the audit log: delegating to an agent never breaks attribution and needs no re-upload. A rotation to a fresh key pair (oxy identity rotate) still needs the new identity linked with a challenge.
5.4 Authentication for the wire protocol
push requires one of:
Authorization: Bearer <token>with a personal access token holdingrepo:writeon that repository;- an HTTP signature (spec §15.1): the client signs a server challenge with its identity key and the key is linked to an account with
write. Reference implementation support is a §18 item; until then tokens are the only method.
Reads on public repositories are anonymous. Reads on internal and private repositories require the same credentials with repo:read.
5.5 Two authorities, one answer
The hub's roles decide who may reach a repository, open issues, create proposals, edit settings, and attempt a push. The repository's Oxygen trust policy (spec §14.5) decides which keys may update which references and whether conflicts may land on them. Both must agree for a push to succeed; the hub MUST NOT bypass or reimplement policy, and it MUST run the same receive_push validation oxy-core runs everywhere else.
A repository with no policy accepts any reference update from any account with write. The settings page says so plainly and offers to create a policy. Policy editing in the browser produces a policy proposal: a signed policy object that roots sign with oxy policy sign from their own clients; the hub collects signatures until the threshold is met, then applies the update as an ordinary push. The hub never holds a root key.
The hub has one identity of its own, the hub delegate key, generated at init and published at /.well-known/oxyhub-key. It is used only to sign changes the hub itself creates on behalf of a signed-in user (web edits, §6.9; proposal landings, §7.6). Such a change carries the user's linked key as author and the hub key as signer, which the author-or-delegate rule (spec §14.4) accepts only when the user's identity object lists the hub key as a delegate with the changes and refs scopes (Oxygen spec §14.2; the hub signs the change and attests the reference), still in force: the user opts in once with oxy identity delegate add <hub key> --scope changes,refs and re-links the rotated identity (§5.3). Policy is then checked exactly as for the user's own signature, since a rule that names the author covers the author's delegates (spec §14.5). If the user has not delegated, or the policy does not authorise them for the reference, the hub refuses the action and shows the client recipe. Delegation is per user and revocable by the user alone (oxy identity delegate remove); the hub never needs a policy entry of its own.
5.6 Personal access tokens
Tokens are 32 random bytes, shown once, stored as BLAKE3 hashes, with a name, an optional expiry, and scopes: repo:read, repo:write, repo:admin, issues:write, proposals:write, user:read, hooks:admin, optionally restricted to listed repositories. Tokens authenticate the API and the wire protocol; they never authenticate the pages.
Device sign-in (in the shape of RFC 8628) gives a token to an app that cannot take a password, such as oh or a phone client. The app posts {client, scopes} to /api/v1/device/code without a token and gets a device code (kept, hashed, by the hub), a user code XXXX-XXXX (consonants), verification_uri (/device), verification_uri_complete, expires_in (15 minutes) and interval (5 seconds). It shows the user code and link, and polls /api/v1/device/token with {device_code}, getting authorization_pending, slow_down (polled within the interval), access_denied or expired_token as {error, error_description} with 400. The user opens /device, signed in (with their second factor), sees the app's name and the scopes it asked for, and approves with any non-empty subset of them, or denies. The next poll makes a token named device: CLIENT with the approved scopes and returns {access_token, token_type: "bearer", scope, username}, once: the code is then gone. No token exists until that poll, so none is stored in the clear. Approvals, denials and device tokens are in the audit log.
6. Web interface
6.1 Theme
The default look (since 2026-10-01, oxygenhub#74) is close to GitHub's, chosen for readability and familiarity: Primer colours, light by default (a theme cookie of dark or auto changes it), the system font stack at 14px, rounded boxes with divided rows, the site band and repository tabs as underline navigation, grey buttons with green primary actions, sentence case throughout, and pill labels. It is assets/github.css, loaded over the stylesheet below with every rule scoped to :root[data-look="github"], which the page sets unless the browser chose the classic look with ?look=classic (a cookie keeps the choice; ?look=github returns to the default). The Oxygen mark stays the logo, with the wordmark OxygenHub.
The classic look follows the RetroPlay Network's shape (dark by default, Barlow for text and Press Start 2P for the wordmark and first columns, square corners, thin --line rules, a --panel header and footer, a full-width --band navigation bar with the active tab underlined in --acc) with the palette taken from the Oxygen mark shared with the VS Code extension: the mark's cyan #00F0FF to blue #0072FF gradient on #070c18, so --acc is cyan, --acc-2 blue, and the band a deep blue. The mark itself is the logo in the header, the footer and the favicon; the wordmark beside it reads OXYGENHUB. The stylesheet starts from retroplay-manager/templates/style.css lines 1–171 (tokens, shell, band, tabs, main, tables, forms, footer) and adds forge components; the store, showcase and game components are not carried over.
The theme is dark, light, or auto (follow the system's prefers-color-scheme, and change with it), chosen with the switch in the header (it cycles) or under Settings → Appearance, and kept in a theme cookie for a year; dark when unset. A script in the page head applies it before first paint. Syntax highlighting uses the highlighter's scope names as classes prefixed sy-, so no scope can collide with the page's own classes.
Tokens (dark default, light under :root[data-theme="light"]):
--bg --panel --soft --ink --mute --line --dim
--acc --band --band-ink --band-line --now --now-ink
--ok --bad --warn (new: #ffb347 dark / #b8651b light)
--add --del --add-ink --del-ink (new: diff backgrounds and text)
--m1 … --m5 (kept for signature status colours: root, signed, sealed, unsigned, invalid)
Theme selection: cookie theme=light|dark|auto; auto follows prefers-color-scheme. The <head> sets data-theme before paint exactly as retroplay-manager does. Fonts are vendored under /static/fonts/ rather than loaded from Google, so a hub on a private network paints without internet access.
The logo is assets/icons/oxygenhub.svg, a copy of the extension's oxygen.svg (two linked rings, the molecule), inlined in the header at 22px beside the wordmark.
6.2 Page shell
header (wordmark, global search box, "+" menu for new repository/issue/proposal, notifications bell with unread count, avatar menu) · .band (Repositories · Explore · Proposals · Issues · Notifications, with admin links pushed right) · repository .tabs (Code · Changes · Proposals · Issues · Releases · Trust · Settings, each with a count pill) · main at 1180px max (wider than retroplay's 980 because diffs need it; tree and file pages use 100% with 22px gutters) · footer.
Every page has a <title> of the form <thing> · <owner>/<name> · OxygenHub.
6.3 Pages
| Path | Page |
|---|---|
/ | Signed in: dashboard (your repositories, proposals awaiting your review, assigned issues, recent activity). Anonymous: explore. |
/explore | Every repository the viewer may see; q filters by name, description or topic, topic by one topic, sort is activity (default), stars or name; each row shows its topics and stars. Topics are set under repository settings (letters, digits and dashes, at most 20); anyone signed in can star a repository, and the overview shows star and watcher counts |
/{owner} | A person: identicon, display name, bio, location, website, linked signing keys, organisations, repositories the viewer can see, and their recent activity (never the email). An organisation: identicon, description, members with roles, teams with sizes, repositories, recent activity in them. /avatar/{name} serves the account's or organisation's picture: an uploaded photo or logo when there is one, otherwise the identicon (SVG, a mirrored 5×5 pattern and hue from BLAKE3 of the lower-cased name). Pictures are uploaded under Settings › Profile, or an organisation's settings by its owners: PNG, JPEG or WebP recognised by content (not by name), at most 5 MiB and 8 000 pixels a side, decoded, centre-cropped to a square and re-encoded at 400×400 (PNG when it has transparency, JPEG otherwise), so no uploaded bytes are ever served. Removing the picture brings the identicon back. Pictures are cached for five minutes and carry an ETag. |
/{owner}/{name} | Repository overview (§6.4); while empty, a quickstart with the exact commands for this hub (remote, certificate, token, push; start from nothing; import from Git) |
/{owner}/{name}/fork, /{owner}/{name}/forks | Fork under an owner the viewer may create repositories for (same visibility, own repository id, objects hard-linked, bookmarks copied, policy not copied); the list of forks the viewer can see. The overview shows "forked from", the fork count, and on a fork a link that opens a proposal upstream with owner/name:bookmark as its source |
/{owner}/{name}/tree/{ref}/{path} | Directory listing at a bookmark, tag, change or state (§6.5). A {ref} is a bookmark name, a tag name, a change hash or unique prefix, or a state: full change hashes joined with + (a1…+b2…), the state those heads make, which is how a state with several heads is addressed permanently |
/{owner}/{name}/file/{ref}/{path} | File view, ?plain=1 raw, ?blame=1 blame (§6.6) |
/{owner}/{name}/raw/{ref}/{path} | A file's bytes, as ?plain=1 serves them (images inline with a sniffed type, anything else as an attachment); relative images in rendered Markdown point here |
/{owner}/{name}/changes/{ref} | History (§6.7) |
/{owner}/{name}/change/{hash} | Change page (§6.8) |
/{owner}/{name}/compare/{a}...{b} | Changes in b not in a, and the diff between the two states. Each side is picked from the bookmarks and tags, or typed as a change or state; a swap link reverses them. When both are bookmarks, b has changes a lacks and the viewer can write, Propose this opens a new proposal of b into a (or links the one already open). /compare alone compares the most recently changed other bookmark against the default one; /compare?base=…&head=… is the pickers' form |
/{owner}/{name}/bookmarks | Bookmarks with heads, ahead/behind the default, last update, attestation status. Writers delete any bookmark but the default one: what oxy push --delete does, through the same checks (refused under a trust policy, which needs a signed attestation only a client can make), logged on the pushes page and sent to webhooks as a push. A pushed deletion of the default bookmark (the hub's setting, not the repository's config) is refused |
/{owner}/{name}/conflicts | Every bookmark whose state carries conflicts, each with its regions (§6.10) |
/{owner}/{name}/proposals, /proposal/{n} | §7. The list filters by state and by q (author:, label:, target: and words), sorts by recently updated (default), newest or oldest, and pages 25 at a time |
/{owner}/{name}/issues, /issue/{n} | §8. The list filters by state and by q (author:, label:, assignee:, milestone: and words), sorts by newest (pinned first, default), oldest, recently updated or most comments, and pages 25 at a time. The Proposals and Issues tabs show how many are open |
/{owner}/{name}/releases, /tags | §9 |
/{owner}/{name}/trust | Policy, roots, rules, hub delegate status, verification summary (§6.11) |
/{owner}/{name}/pushes | Reference update log: who pushed what, when, with which key, result (§6.12) |
/{owner}/{name}/settings/* | General, collaborators, bookmarks (default, delete), tokens, webhooks, mirror, danger zone |
/{owner}/{name}/edit/{ref}/{path}, /new/{ref}/{path} | Web edit (§6.9) |
/settings/* | Profile, security (password, TOTP, sessions), keys (linked identities), tokens, notifications, theme |
/notifications | Inbox (§10) |
/projects/{owner}, /projects/{owner}/{n} | Projects of a person or organisation (§8.2) |
/search | §11 |
/help | Getting started with this hub's own address, remote name and certificate filled in (identity, pushing, cloning, proposals, delegating to the hub, pipelines and stamps), the keyboard, and a glossary for people coming from GitHub. Linked from the header band, the footer and the keyboard sheet |
/admin/* | Site: accounts, registration mode, invitations, audit log; /admin/storage lists every repository's size on disk, loose objects and packs, and runs maintenance (oxy gc with the grace period kept, under the repository's write lock) on one repository or all, audited |
6.4 Repository overview
The default bookmark's tree at the root, the rendered README.md (or README, README.txt) below it, and a right column: description and topics, clone URL with copy button (oxy clone https://…/{owner}/{name}), heads count with a link when more than one, bookmarks, tags, latest release, collaborators, license detected from LICENSE, and the trust badge: policy, no policy, or policy expired.
A repository whose default bookmark has more than one head shows a --warn strip: "main has 3 heads; the tree below is their union" with a link to the bookmarks page. Conflicts in that state show a --bad strip linking to the conflicts page.
6.5 Tree
Directory rows show name, the last change touching the entry (message and relative time, from log_for_path, computed lazily per row over the API so a large directory paints immediately), size, and mode glyphs (executable, symlink, subrepository). A subrepository row links to its pin: another hub repository at a state, or a Git URL at a SHA.
6.6 File view and blame
Text files render with syntect line highlighting, line numbers as anchors (#L12, #L12-L20 selects a range), a "raw", "history", "blame", "edit" toolbar, and the file's last change. Markdown renders by default with a toggle to source. Images and PDFs render inline; other binaries offer download. Files above render_limit (default 2 MiB) show the first part and a download link.
Blame uses Repo::blame: every line is exact, because each line is an atom with one authoring change. The gutter shows the change's short hash, author and age, grouped into runs, with the change's signature glyph. Blame is exact through rebase and squash, so there is no "ignore revs" file.
A sidebar beside the file lists the tree down to it: the root and each directory on the file's path expanded (at most 200 entries each, then a link to the directory), the file marked, other directories linking to their tree pages. It is hidden on narrow screens.
The permalink of a file view is the same path at its state: the target's heads joined with + in place of the bookmark or tag. y replaces the address with the permalink, keeping the line anchor, as GitHub does. When a line or range is selected, a bar offers Copy lines (the text) and Copy permalink (the permalink with #L12-L20).
The blame gutter shades each run by the age of its change, from the file's oldest (lightest) to newest (strongest) line, in ten steps, with the age also written out so colour is not the only carrier. Each run links blame before this change: the same file blamed at the state of the change's dependencies (the state its author built on), so a line can be traced back through every edit. A change without dependencies has no such link; one with more than 32 has none either, to keep addresses bounded.
Function history. For Rust, TypeScript, Python and shell files, tree-sitter finds the functions and methods (qualified by their impl, trait or class, as Repo::log) and the file page's sidebar lists them. Each links to /{owner}/{name}/function/{ref}/{path}?start=…&end=…&name=…: every change that wrote or deleted one of the function's lines, or deleted a line from between them (Oxygen spec §27, the query oxy log -L runs), newest first, with a graph of how those changes depend on one another. It is exact because lines keep their identity across edits; changes elsewhere in the file are not listed. A cross-file semantic graph (callers, callees) is future work.
6.7 History
A dependency graph, not a parent chain. The default view lists the bookmark's changes newest first with lanes drawn between a change and its dependencies (the same layout algorithm as the VS Code extension's webview, ported to Rust and rendered server side as SVG). Heads are marked. Filters: author, path, date, signed/unsigned, message text. Each row: short hash, message first line, author (name, or KeyId when unlinked), relative time, signature glyph, bookmark and tag chips, a conflict chip when the change introduces one.
?path= gives the history of one path.
Swimlanes. /{owner}/{name}/lanes/{ref} draws a bookmark's newest changes (up to 60) as a board: each lane of the shared lane layout (oxy_core::lanes, as oxy log and the list view use) is a row, time runs left to right, every change is a card linking to its page, and an edge joins a change to each change of the board it depends on. Independent work stays in separate lanes and there are no merge nodes, since Oxygen has none. The history page links to it.
Planning on the board. When the bookmark is the source of an open or draft proposal that the viewer wrote or can write to, the board is editable. Dragging a change onto another asks POST /{owner}/{name}/lanes/{ref}/plan (form change, target) what the drop could mean. The answer is JSON: a squash (fold the change into the target) when every dependency of the change is the target or already beneath it, and a rebase (make the change depend on the target) unless it already does. Nothing is offered when the target depends on the change, or when either is already on the proposal's target bookmark. The hub does not rewrite the changes itself: rewritten changes must be signed by their author, so each option carries the oxy commands to run and a vscode://oxygenhub.oxygen-vscode/plan?… link that does the same in the VS Code extension, then force-pushes the bookmark.
6.8 Change page
Header: full hash, message, author identity (linked account or KeyId), timestamp, signature status with the policy rule that admitted it (or "unsigned", or "invalid"), seals covering it, the bookmarks and tags containing it. Body: dependencies and dependents as chips, the change's edits as a diff (unified or split, per-user preference), atom counts, and the raw change (oxy show --json) behind a details block.
Actions for write: open a proposal containing this change, tag it, copy hash. Describing, abandoning, squashing and rebasing are client operations and are not offered here.
6.8.1 Diffs
Every diff (change, compare, proposal files) shows each file as a collapsible section, open unless it has more than 400 rows. Within a block of changed lines, removals come first and each is paired with the addition in the same position; a pair similar enough (at least 40% of its tokens shared, tokens being identifiers, whitespace runs and single other characters) marks the tokens that changed. ?w=1 hides whitespace-only changes: a pair equal once whitespace is removed is shown as one unchanged line. With scripts, a toolbar switches between unified and split (side by side, pairs opposite each other) views, remembered in a diff cookie, and collapses or expands every file.
6.9 Web edit
CodeMirror 6 with the file's language mode. Saving creates a change with the given message signed by the hub delegate on behalf of the user (§5.5) and adds it to the chosen bookmark, or to a new bookmark plus a proposal, exactly as GitHub's "propose changes" flow. The hub refuses with a clear message when the user has not delegated to the hub key (§5.5) or the policy does not authorise them for the target. Binary files and files over render_limit cannot be edited in the browser.
6.10 Conflicts page
For each bookmark whose state has conflicts: the file (or tree entry), the conflict kind, and each side with the change that produced it and its content, rendered with the same marker structure oxy checkout writes. For a text file, resolve region by region (/{owner}/{name}/resolve/{bookmark}/{path}, writers) shows the lines no one disagrees on and each conflict region with its sides and the change that wrote each; per region the user keeps one side, every side in order, none, or their own text (a side that records a deletion keeps the deletion). Regions are linkable (#region-N). The file is put back together and committed through the delegate as a web edit (§6.9), which makes it a kind = resolution change depending on every contributor; the page carries the bookmark's state and refuses to commit if the bookmark moved meanwhile. Malformed markers fall back to the editor, which also stays available: "edit the marked file" opens the markers in it. A name conflict (several files want one name, shown as NAME.oxy-conflict-N) resolves on the same URL by keeping one candidate: it takes the name as a move, keeping its identity, and the other candidates are removed. A replace conflict (a binary file replaced two ways) resolves the same way by keeping one version, each labelled with the change that wrote it; the commit replaces every version with it (Oxygen spec §9.3). Conflicts between directories or links are resolved from a client. Both resolver pages offer Resolve in VS Code, a vscode://oxygenhub.oxygen-vscode/resolve?url=…&bookmark=…&path=… link that oxygen-vscode handles: it finds the open checkout of that repository (or offers to clone it), switches to the bookmark after asking, pulls it, and opens the file. Resolving is live: everyone on the same resolver page (a room per repository, bookmark and path, over server-sent events at /{owner}/{name}/resolve-live/{bookmark}/{path}) sees who else is there and each other's choices as they are made, region by region, the last choice for a region winning (free text is shared whole, a moment after typing stops); a newcomer gets the choices so far; and when one of them commits, the others are told the resolution landed, with a link to it, and their form is disabled. Rooms are in memory and end when the last viewer leaves. The change page tags resolution changes. Otherwise the page gives the two-line client recipe.
6.11 Trust page
The current policy (serial, expiry, roots, threshold, rules with their patterns, thresholds and allow_conflicts), the policy history, the hub delegate key with its fingerprint and which collaborators have delegated to it, every linked account's keys that the policy names, revoked keys, and a verification summary computed from oxy verify: counts of signed, sealed, unsigned and invalid changes, with a list of invalid ones. Policy proposals awaiting root signatures are listed with who has signed and a download of the proposal file.
6.12 Pushes page
The hub keeps its own log of every reference update it accepted or refused: time, account, key, reference, old and new state, force flag, the result and reason. This is the answer to "who moved main and when" and is the hub's audit trail for history; it is not the repository's operation log, which is per workspace.
6.13 Keyboard and accessibility
/ focuses search, g c / g p / g i jump to Code, Proposals, Issues, y on a tree or file page switches to its permalink, ? shows the sheet. Every interactive element is reachable by keyboard and shows a visible focus ring; a "Skip to content" link comes first; the header, site band, repository and settings tabs are named navigation landmarks, and the current tab carries aria-current. Colour is never the only carrier of meaning (glyphs accompany every status colour). Text meets WCAG AA (4.5:1) on every surface in both themes, checked by a test over the theme variables; --dim is for rules and decoration, never text. Under 760 pixels the tab bars and wide tables scroll sideways instead of breaking the layout, sectioned pages stack their section list above the content, and the header search hides (the / key and the search page remain).
7. Proposals and review
7.1 Definition
A proposal is a request to add a set of changes to a target bookmark of a repository. It is identified by a per-repository number, like a pull request. Its source is either a bookmark of the same repository or of a fork, in which case the proposal's changes are those in the source bookmark's state not in the target's; or an explicit list of change hashes (from the change page or the API).
A proposal records: number, title, description (Markdown), author, target bookmark, source (bookmark and repository, or explicit hashes), state (draft, open, landed, closed), reviewers requested, labels, and the projected state (§7.2).
7.2 Projected state
Whenever the proposal or its target changes, a job computes the projected state: the union of the target's heads and the proposal's changes, loaded through oxy-core exactly as a client's oxy merge would. From it the page shows:
- the changes to be added, in dependency order, with signature status;
- the diff between the target's state and the projected state;
- conflicts the landing would reify, each with its sides, in the same rendering as §6.10;
- files touched, and check results for the projected state hash (§13).
Because changes commute, a projected state is stale only when the target has moved; the job then recomputes it. There is no "update branch" button because there is nothing to update. If the recomputation shows a new conflict, the proposal is marked conflicted and the author and reviewers are notified.
7.3 Review comments
A comment is attached to the proposal (general), to a file, or to an atom: (ChangeHash, index) plus the change that introduced the line. Rendering places it at the atom's current position in the projected state; if the atom is gone (deleted by a later change in the proposal), the comment shows in the file's "resolved lines" section rather than disappearing. When the author squashes or rebases, the correspondence the rewrite recorded maps the old atom to the new one, and the comment follows. Comments are Markdown (EasyMDE), threaded, and resolvable.
7.4 Reviews and approvals
The author or a writer can ask for a review from anyone who can read the repository (not the author), under Reviewers on the proposal page or with POST /api/v1/repos/{o}/{n}/proposals/{number}/reviewers and {"reviewers": [...]}; DELETE .../reviewers/{username} withdraws a request. Each person asked gets a notification (proposal.review_requested), the timeline records it, the proposal's JSON lists requested_reviewers, and a review from that person answers the request. Every review notifies the proposal's author (proposal.review).
A review is comment, request changes, or approve. An approval MUST be backed by a seal attestation (spec §14.4, kind = seal) over the proposal's change hashes, signed with a key linked to the reviewer's account. Private keys never enter the browser, so the reviewer signs from a client: oxy seal --proposal <url> or oxy seal <hashes> (§18, a new command; the seal object already exists in the model) signs and pushes the seal; the hub matches the seal to the proposal and records the approval. The page shows the command ready to copy. An approval not yet backed by a seal is shown as "approval pending signature" and does not count towards requirements.
Landing requirements per repository (settings): N approvals, approvals from code owners (.oxyowners in the target state, GitHub CODEOWNERS syntax), required checks green on the projected state, no conflicts (or conflicts allowed, matching the policy rule's allow_conflicts), and no outstanding "request changes". A repository under an organisation of two or more members starts with one approval required (someone other than the author can give it; nobody may approve their own proposal), any other with none; an admin changes either in Settings. When none is required and nobody has approved, the proposal page says so beside Land, and Land asks for confirmation.
7.5 Stacked proposals
A change may depend on changes in another open proposal. The page shows "depends on #12" and lands only after #12 has landed, or lands both when the user chooses "land with dependencies" and holds the rights for each.
7.6 Landing
Landing sets the target bookmark to the projected state's heads: a reference update with a fresh ref attestation. It is performed by:
- the hub delegate key on behalf of the user landing, when that user has delegated to it (§5.5), the policy authorises them for the target, and they hold
write; or - a client:
oxy proposal land <url>(§18) fetches the proposal's changes, sets the bookmark locally and pushes with the user's own key. The hub notices the push, sees the proposal's changes are now in the target, and marks itlanded.
The landed state hash is recorded. Nothing is rewritten: the changes keep their hashes, so links to them stay valid and check results carry over. "Squash on land" is offered as a client-side option only, since it rewrites signed changes.
7.7 Closing and reopening
closed keeps everything; the source bookmark may be deleted on close if the user asks. A proposal whose changes all appear in the target through another route is marked landed automatically with a note.
8. Issues
Per-repository numbered issues sharing the number space with proposals (as GitHub does; so every number is shown with its kind, "issue #1" or "proposal #2", in page headers and in the links #N becomes), with title, Markdown body, author, assignees, labels (name, colour, description), a milestone (title, due date, description, open/closed), state (open, closed with reason completed or not planned), comments, lock, and pin. Issue templates come from .oxyhub/ISSUE_TEMPLATE/*.md in the default state.
The proposal page. The conversation is centred at the normal page width; the changes and files tabs are wide for diffs.
- Timeline: the description, comments, reviews and events are one timeline in time order, each entry beside its author's avatar.
- Editing: the title, target and description are edited inline (an Edit button beside the title opens the editor below the timeline), not from the sidebar.
- State actions: ready for review, back to draft, close and reopen sit under the comment box, and only when they apply.
- Sidebar:
- Landing, Closes, Labels and Projects.
- Labels: those with triage edit them, or are pointed at the repository's labels page when it has none.
- Projects: editors of the owner's projects add the proposal to one (its first column) or take it off. Issues have the same project control.
- Redirects: every action redirects back to the page, so reloading it (the checks script does, when a check arrives) never repeats the action.
The chain of work. A proposal shows the issues it closes (closes, fixes or resolves #N in its description or in any of its changes' messages) in a strip under its title: each issue with its state, then the proposal, then its target bookmark, marked landed once it is. A Closes list sits in its sidebar. An issue shows the proposals that close it, by the same rule, with their states. So the path from a report to the change that landed it can be followed from either end.
Relative links and images in issue, comment and proposal text resolve against the repository's default bookmark, and in release notes against the release's tag. Cross references: #12 links an issue or proposal; owner/name#12 across repositories; a 12-hex-or-longer token links a change when it resolves; closes #12 in a change message closes the issue when the change lands on the default bookmark ("closed by change"). Any other #12 in a pushed change's message, on any bookmark, puts "referenced by change on " on that issue's or proposal's timeline, the hash linking to the change; other repositories' owner/name#12 are left alone.
Import from GitHub or GitLab is a 1.x item.
8.2 Projects
A project is a board that plans work across the repositories of one owner (a person or an organisation): a number per owner, a title, a description, public or private, open or closed, and ordered columns (new projects start with Backlog, Doing and Done). A column holds ordered cards; a card is an issue or proposal of one of the owner's repositories, or a note.
- Who: anyone may see a public project; a private one only its owner (for an organisation, its members) and site administrators. The owner, an organisation's members and administrators may change it. A card for an issue or proposal whose repository the viewer cannot read is not shown at all.
- The board works without JavaScript: each card has a column picker and up and down buttons; cards are added by reference (
#12in a named repository, orrepo#12) or as a note; an issue or proposal is on a project at most once. Issue and proposal pages list the projects they are on, with their column. - Columns can be added, renamed, moved and removed (removing one moves its cards to the first column). Closing a project keeps it read-only.
9. Releases and tags
Tags come from the repository (spec §10): a name, the state, and a signed attestation when the tagger signed. The tags page lists them with signature status. A release attaches to a tag: notes (Markdown, generated draft from the changes since the previous release), assets (uploaded files, content addressed, size limited by asset_limit), a pre-release flag, and a latest marker. Source archives (.tar.zst, .zip) of the tagged state are generated on demand and cached.
Trust and audit (§9.1). /{owner}/{name}/releases/trust/{tag} reports on a release; each release on the releases page links it.
-
Tag: the tag's signer and whether its attestation verifies.
-
Authors: everyone whose changes the release adds since the previous release (all of its changes for the first release). For each: how many changes are signed, sealed, unsigned, invalid or signed by revoked keys, and the account their key is linked to.
-
Assets: for each asset, what built it: the run and job, the runner's identity (and whether it is a key one of this hub's runner tokens signs with), and the pinned environment (tier, rootfs hash, environment hash). Also its inputs (the state built, the pipeline hash and the scope hash) and its signed receipt, verified again from the repository.
-
Graph: authors → tagged state → jobs → assets, drawn as SVG. A dashed edge is a build from another state than the tag's.
-
Audit: pass, warn or fail for each claim:
- the tag is signed;
- the changes are authenticated;
- each asset has a verifying signed receipt from a known runner;
- each asset was built from the tagged state;
- each asset's environment is pinned by hash.
A hand-uploaded asset, an unsigned tag or an unsigned change is a warning. A signature or receipt that does not verify, or a revoked key, is a failure.
-
Export:
?format=jsondownloads the report, andGET /api/v1/repos/{owner}/{name}/releases/trust/{tag}returns the same. The JSON carries each signed receipt as hex, so an auditor can check it offline withoxy receipt verify.
Built by. A release asset links to the build that produced it: the passed OxygenFlow job of this repository with an artifact of the same BLAKE3 (hub spec §13.2). A job with a kept signed receipt comes first, then the newest run. The release page shows the run, the job and the signed receipt with its signer, and the API gives the same as built_by on each asset. On a run page, a writer can attach an artifact to a release; it becomes an asset named after the artifact's path, so the link follows by content. A file uploaded by hand links the same way when its bytes match.
10. Notifications
Events: proposal opened, review requested, review submitted, comment on a thread you are in, mention (@name), landing, check failed on your proposal, issue assigned, policy proposal awaiting your signature, push refused. Each becomes an inbox item (/notifications, with unread count in the header) and, when the account chooses, an email through lettre with a per-account SMTP-independent unsubscribe link. Watching a repository subscribes to all its events; participating in a thread subscribes to it.
The inbox shows one row per thread (the page a notification is about, such as an issue or a proposal), with its newest event, how many updates it had and how many are new. Each notification records why it was sent: participating (the account opened, commented on, was assigned, mentioned in or asked to review the thread) or watching (it only watches the repository); a thread counts as participating if any of its notifications does. The inbox filters by view (inbox, unread, done), by reason, by kind (issue, proposal, release, …) and by repository, with counts over the whole inbox. A thread can be marked read or unread, and done, which sets it aside (read) until something new happens in it. Opening a row marks its thread read and goes to the page.
11. Search
Global search box with scopes: repositories (name, description, topics), issues and proposals (FTS5 over title, body and comments), changes (message FTS5 over the default bookmark's state, indexed on push), accounts, and code. Code search in 1.0 is a per-repository substring and regular-expression search over the default state's text files, run with a time budget and a result cap, cached per state hash. A trigram index is a 1.x item.
12. HTTP API and webhooks
12.1 API
JSON under /api/v1, bearer tokens, application/json, RFC 7807 problem responses (an unknown route too), cursor pagination (?after=), ETags on cacheable resources, and rate limits per token. Resources mirror the pages: /repos/{owner}/{name}, /bookmarks, /changes/{hash}, /tree/{ref}/{path}, /file/{ref}/{path}, /blame, /history/{ref} (?path=, ?offset=&limit=, or ?r=REVSET for the changes a revset selects, evaluated under a two-second budget and answering 400 when it does not parse or goes past it; Oxygen's docs/revset_query_language.md), /compare, /conflicts, /proposals (?state=open|draft|landed|closed|all, open ones and drafts by default; a source bookmark may hold /, and owner/name:bookmark naming the repository itself is its own bookmark), /proposals/{n}/comments|reviews|checks|files, PATCH /proposals/{n} (the title and description, for its author or a maintainer, and labels, the whole set by name, for triage; what is not given stays), POST /proposals/{n}/land (repo:write: what the Land button does, 409 with the blockers when it cannot; a landed proposal keeps the changes it landed and reports landed with who, how (web, or api (token NAME)) and when, which its event, the audit log and the push record also say, adding when nobody approved), /proposals/{n}/content/{base|head}/{path} (a changed file's bytes in the target's state or the projected state, for editor diff views), /issues (creating an issue can set its labels, assignees and milestone; PATCH /issues/{n} edits the title and body for the author or triage, and replaces labels, assignees or the milestone for triage, recording each on the timeline; unknown names are refused before anything changes), /labels (POST creates or changes one, DELETE /labels/{name} removes it; triage), /milestones (POST creates one or changes the description and due date of the one with that title, PATCH /milestones/{title} changes title, description, due date or state; triage), /releases (PATCH /releases/{tag} changes the title, notes or prerelease flag, repo:write), /tags, /trust, /pushes, /hooks, /checks, /user, /notifications (the caller's inbox and unread count; /notifications/{id}/read and /notifications/read mark read; scope user:read), /users/{name}, /orgs/{name}, /checks/{state} (the checks on a state, named by its hash, by its heads joined with +, by the one head of a single-head state, or by a bookmark), /checks/required (GET, and PUT of the whole list by a repository admin: the checks a proposal needs, passed, to land), /projects/{owner} (list, create) and /projects/{owner}/{n} (the board as the caller may see it, PATCH for title, description, privacy and closing, /cards to add a reference or note to a column named by id or name, /cards/{id} to move or remove one; repo:write and the page's rights to change), /flow/runs (GET: runs newest first, ?before=N&limit=; POST {"bookmark"} or {"change"}, repo:write: run a bookmark's current state, the default bookmark when none is named, or one change's state), /flow/runs/{n} (a run with its jobs), /flow/runs/{n}/jobs/{job}/log (?after=seq&limit=) and /flow/runs/{n}/rerun (repo:write: a finished run again), both answering 201 {"number", "url"} or 409 with why nothing started, /flow/runs/{n}/cancel (repo:write: a run still going ends as cancelled, its running jobs stopped when their runner next reports; 409 when it had already ended; a push to a bookmark cancels the push runs of it still going), /search. A change's author carries the account its key is linked to, or null and a note saying the key is not linked and where its owner links it; via names the delegate (an agent) that signed for the author when one did, with the delegation's scope and expiry and whether it has since been revoked, and the pages show the same as an agent badge, a via row under the author on the change page, and a count under the author in Contributors (the change page shows a "key not linked" tag, with a link to Settings → Keys for signed-in viewers). Shapes reuse oxy --json shapes wherever one exists (change, status, blame, diff) so tools written against the CLI work against the hub. crates/hub-api holds the types with serde derives and is what the runner and the VS Code extension will link.
12.2 Webhooks
Per repository and per organisation: a URL, a secret, an event selection, active flag, and delivery log with redelivery. Each webhook has a format: oxygenhub (the signed JSON envelope below), discord (an embed with a title, link and short detail for the event, sent with mentions disabled so no user text can ping anyone) or slack (escaped text); a Discord or Slack URL picks its format by default, and every event has a one-line summary for the chat formats. The settings show a webhook's URL with its long path segments and query cut short, since chat webhooks keep a secret token there (the audit log records the same masked form), and each webhook's last delivery result; a webhook can be edited (URL, secret, events, format, active), a blank URL or secret keeping the current one. Repository admins manage a repository's webhooks under its settings; organisation owners manage the organisation's under its settings, and those hear every repository the organisation owns. Payloads are JSON with X-Oxyhub-Event, X-Oxyhub-Delivery, and X-Oxyhub-Signature: blake3-keyed=<hex> computed as keyed BLAKE3 over the body. Events: push (reference updated: old and new state, heads, changes), proposal (opened, edited, review requested, landed, closed, conflicted), review, comment, issue, release, check, policy. Deliveries retry with backoff up to 24 hours.
13. Checks and OxygenFlow
13.1 Checks
A check is a result attached to a state hash: name, status (queued, running, passed, failed, cancelled, skipped), a summary, a URL, timing, and an optional signed seal from the checker's key over the changes in that state, which makes a passed check a verifiable object. Checks are created through the API (POST /repos/{o}/{n}/checks) by any token with checks:write, so any CI system can report, and required checks for landing (§7.4) are matched by name against the projected state's hash. The proposal page streams check status updates over server-sent events.
Because a state is content addressed, the hub deduplicates: a check that passed for a state hash is shown for every proposal and bookmark that reaches that same state.
13.2 OxygenFlow runs
Pipelines run in OxygenFlow, a separate project and specification (oxygenflow/docs/spec.md; the first slice is oxygenflow/docs/m0.md). A pipeline is .flow/pipeline.kdl in the state being built; jobs run in Flow's isolation tiers (MicroVM by default, a syscall sandbox, WASM), never in containers on the hub. The hub's side:
-
Configuration.
[flow] runner_tokens(bearer tokens for runners, compared in constant time),repos(patterns such asowner/*orowner/name: the repositories whose pushes start runs by default; empty means none) andjob_timeout_minutes(60). -
Per repository. A site administrator turns OxygenFlow on or off for one repository under Settings → OxygenFlow, which overrides
reposin either direction; "site default" follows it again. Runs use the site's runners, so repository administrators see the setting but cannot change it. Changes are audited (repo.flow) and apply from the next push, with no restart. -
Triggers. A push that moves a bookmark of an allowed repository starts a run on the bookmark's new state, numbered within its repository from 1 (the run's page is
/{owner}/{name}/flow/{number}): the hub reads.flow/pipeline.kdlfrom that state and validates it withflow-core(the same codeflow checkruns). An invalid pipeline records a run inerrorwith the problems and their lines; a state without a pipeline records nothing. Tags and proposals start no runs yet, so only code from people with write access ever runs. A run waiting for a runner says where it stands: on the Flow tab its place in line, on its page also the jobs running now and those queued ahead of it (named when their repository is this one or public, otherwise counted), and in the APIqueue_position. Writers also start runs without moving a bookmark: Run pipeline runs a committed state without a push:- On the Flow tab and beside each bookmark on the bookmarks page, it runs a bookmark's current state.
- On a change page, it runs the state whose only head is that change. That run has no bookmark and is labelled "change".
- The API does the same with
POST /flow/runsand{"bookmark"}or{"change"}.
Viewers who may not start runs get no button. The Flow tab says why: Flow is off for the repository, they are not signed in, or they lack write access. Re-run on a finished run's page runs its state again.
Runs only ever take committed states already in the repository, never uncommitted work. A run writes no history: its receipts are signed statements about a state (Oxygen spec §14.6). Running a state again adds another statement. The environment hash shows whether it ran in the same environment, and a matching result is independent confirmation. The new run's checks replace the old ones for that state, and the run page says what started it (pushed by, started by, re-run of #N by).
-
Runs and jobs. A run holds the repository, bookmark, state hash and heads, pipeline hash and pusher; one job per pipeline job with its compiled form, sandbox and dependencies. Job states:
queued,running,passed,failed,skipped(a dependency did not pass),error(the runner could not run it). A run isrunningwhile any job is, thenpassedwhen every job passed, elsefailed. -
Runner API (runner token):
POST /api/v1/flow/claimhands out the oldest queued job whose dependencies passed and whose tier the runner offers, marking itrunning, with a job token;204when there is none.POST /api/v1/flow/jobs/{id}/logappends log records (text cut at 16 KiB, at most 50 000 per job);POST /api/v1/flow/jobs/{id}/finishrecords the exit code and receipt (the hub recomputes the receipt hash) and settles dependents. Only the runner holding the job may report on it. -
Runners (Admin → Runners, site administrators).
- Registering:
- An administrator creates a one-time registration token, valid for a day and shown once with the command that uses it.
- On the runner,
flow runner register https://hub --token …sends the runner's signing identity, architecture, OS, version and labels (POST /api/v1/flow/runners/register) and gets its own credential (flowrunner_…, stored hashed). - The runner's key is bound at registration: receipts from that runner signed by another key are not kept.
- Runners configured with a
hub.tomltoken still work. They are tracked from their first request under the name they claim with, and their key is bound on their first signed receipt.
- Scopes (oxygenhub#41): a runner serves the whole site, one owner's repositories (a user or an organisation), or one repository. The registration token carries the scope (typed as empty,
ownerorowner/repositorywhen it is made) and the runner gets it; an administrator can change it on the Runners page. A claim returns only queued jobs of repositories in the runner's scope, filtered before the queue is cut short, so a scoped runner never receives another scope's jobs and never waits behind them. A job whose pipeline saysruns-on(OxygenFlow spec 3.1) goes only to a runner that has all of its job's and sandbox's labels, from those the runner reports on the claim or registered with; until one claims it, the queue says which labels it waits for.hub.tomlrunners serve the whole site. Changes are audited (flow.runner.scope). Settings → Runners on a repository (its admins) and on an organisation (its owners) lists the runners that may take its jobs, with what each serves; there they make a registration token for a runner of their own scope (the repository, or the organisation's repositories) and remove such runners when they run nothing. Only site administrators manage site runners. - Each claim carries the runner's description and marks it seen.
- The Runners page lists each runner: online (seen in the last 90 s), offline, draining or disabled; what it serves; signing key; machine; labels; capacity; jobs running now; and when it was last seen.
- Actions:
- Drain: the runner finishes its jobs and claims return nothing.
- Disable: its requests are refused.
- Resume.
- Remove: refused while the runner has running jobs. Its credential stops working, and its jobs keep its name.
- The queue (on the Runners page): running jobs with the runner holding each, then queued jobs in the order runners claim them.
- API (site administrators):
GET /api/v1/runners(each runner, what it serves, its machine, key, last seen and jobs running),GET /api/v1/runners/{id}/jobs?before=&limit=(its history, newest first) andGET /api/v1/runners/queue. - A runner's page shows its totals (jobs, pass rate, time busy) and its job history, newest first, with links to runs and signed receipts. Jobs record the runner that claimed them (
flow_jobs.runner_id).
- Registering:
-
Signed receipts (Oxygen spec §14.6):
finishmay carrysigned_receipt, the runner's signed receipt as hex. The hub keeps it in the repository when four things hold:- it verifies;
- it signs the job's receipt;
- it is for the run's state (computed from the run's heads) and this repository;
- its key is the one the runner's token first signed with. The first signed receipt under a token binds that token to its key in
flow_runner_keys.
Otherwise the job's result still stands, and the run page says why the receipt was not kept. The run page links each kept receipt, with its signer. The change page lists the receipts for the state whose only head is that change.
/{owner}/{name}/receipts/{attest}downloads a signed receipt foroxy receipt verifyorimport. -
Artifacts (OxygenFlow spec §5.2): a job may declare artifacts with
output.- The runner holding the job uploads each one with
POST /api/v1/flow/jobs/{id}/artifacts/{name}(raw bytes, at most 512 MiB). - The hub refuses names the job does not declare. It keeps each artifact as a blob of the run's repository, records its path, size and BLAKE3 in
flow_artifacts, and answers with the blob's ObjectRef. finishrefuses a receipt whoseoutputsare not exactly the uploaded blobs, in name order.- The run page lists each job's artifacts, and
/{owner}/{name}/flow/{n}/artifacts/{job}/{name}downloads one. - A kept receipt keeps its artifacts through garbage collection (Oxygen spec §14.6).
- The runner holding the job uploads each one with
-
Job tokens. Random, stored as a hash, valid only for reading the job's repository over the wire protocol while the job is running and for at most two hours. They cannot push, call the API or read other repositories.
-
Checks. Each job is also the check
flow/<job>on the run's state (§13.1), so proposals, required checks and the editor's status bar see it. -
Pages. A Flow tab lists runs (status, bookmark, state, pusher, when, duration). A run page draws the job graph as SVG, with columns by dependency depth and status by colour and glyph, and each job's log below (folded, the failed ones open); it refreshes while the run is live. Readers of the repository can see both.
/{owner}/{name}/flow/badge.svg is a small SVG of the latest run on the default bookmark (passing, failing, running, no runs) for a README, for whoever may read the repository. It is a convenience, with the weakness of every badge; the stamps below are the real thing.
State stamps replace CI badges, which are mutable images claiming that a branch passes, name no state and cannot be checked. A stamp belongs to a state:
/{owner}/{name}/flow/stamp/{ref}is an SVG. For a state address (heads joined with+) or a change, it shows the newest run of exactly that state: one glyph per job in dependency order, the run's status, the state's short hash and the run's date. It shows "no run" when there is none. The same data is machine-readable in the SVG's<metadata>: heads, run number, status, and each job's status and receipt hash. For a bookmark or tag, it redirects to the stamp of the ref's current state, so even the image names what was checked. A finished run's stamp may be cached for an hour; any other stamp for a minute.- In rendered Markdown, the relative paths
.flow/stamp(as an image) and.flow/run(as a link) resolve against the state being viewed. The first is served by/raw/{ref}/.flow/stamp, the second by/file/{ref}/.flow/run(the run page, or the Flow tab when that state has no run). So[](.flow/run)in a README shows how the viewed state did: browsing a tag shows the tag's result, not the bookmark's. A real file at either path takes precedence. - Later, stamps also say which runner key signed each receipt, and which jobs reused an earlier receipt.
The older runner draft (docs/runner.md) is superseded by OxygenFlow; its hub-side ideas (environments, approvals for outsider proposals, caches, artifacts) carry over as later Flow milestones.
13.3 Web workspaces (planned)
Open in Workspace gives a signed-in user VS Code in the browser on a state of a repository, with Oxygen VCS inside, without cloning anything to their own machine. It is designed here before it is built (Oxygen backlog P5); parts are marked as they land.
- What runs. openvscode-server (MIT, unmodified) with the Oxygen VCS extension and
oxy, in a Firecracker microVM per workspace on an OxygenFlow runner (aworkspacejob kind beside pipeline jobs), never on the hub's host. The image is a Flow rootfs like the build images, pinned by hash. A fork of VS Code is considered only if an extension cannot do what is needed. - Starting. The button is on a bookmark's row and on a change page, for accounts that may read the repository. The hub records a workspace (account, repository, state, bookmark when opened from one, status) and queues a workspace job that only a runner whose scope covers the repository may claim (§13.2 scopes). One account has at most a few workspaces at once; the hub says which are open.
- Files without a checkout. The state is mounted read-only in the microVM by flow-casfs (OxygenFlow spec §5.1): the tree from the state index, each file's bytes from a cache of renderings keyed by graph key, rendered on first read and shared by every state in which that file has the same history. An overlay on the job's ephemeral disk holds edits. Opening a workspace on the Linux tree therefore costs listing a tree, not writing 96 000 files; the mount is to take under 100 ms.
- History and commits. The microVM's
oxyrepository fetches the state's history over the wire protocol with a token that reads this repository only, as job tokens do, and its working copy is the mount. Private keys never enter the hub or the microVM, so a commit is pushed to a workspace bookmark (ws/{account}/{id}) and signed there by the hub as the account's delegate, exactly as web edits are (refused with instructions until the account has delegated). From the bookmark the user proposes or lands as usual. - Access. The hub proxies the editor (HTTPS and WebSocket) at
/{owner}/{name}/workspace/{id}/to the workspace's owner only, checking the session on every request; the runner reaches the hub over a connection it opens itself, so runners behind NAT work, and accepts editor traffic for a workspace only from the hub with that workspace's secret. Nothing in a workspace is reachable from any other account, or from the network directly. - Lifecycle. A workspace stops after 30 minutes without a connection (configurable) and on request. Its overlay is discarded when it stops: anything not pushed to its workspace bookmark is gone, and the editor says so before stopping. A stopped workspace reopens from its bookmark. The audit log records starts, stops and pushes.
- Measured when built: mount time, microVM boot, time to the editor's first paint, and a full read of the Linux tree through the mount.
Design notes (2026-10-04), for review before anything is built.
- Data. A
workspacestable: id, account, repository, state (ObjectRef), bookmark (nullable), workspace bookmark name (ws/{account}/{id}), status (queued,starting,open,stopping,stopped,failed), runner, job id, secret hash, created, last connection, stopped, and why. A workspace is a row plus one Flow job of kindworkspace; the job's log is the editor's server log.GET /api/v1/user/workspaceslists the caller's;POST /repos/{o}/{n}/workspaces {"state"|"bookmark"}opens one;DELETE /workspaces/{id}stops it. Limits are per account: three open at once (configurable), and a site cap. - Runner side. The runner gains a job kind beside pipeline jobs: it boots the pinned workspace rootfs (openvscode-server,
oxy, the Oxygen VCS extension, a non-root user), mounts the state at/workspacethrough flow-casfs with a writable overlay, runsoxyin the microVM against a repository that fetches history lazily over the wire protocol with the job's read-only token, and opens a tunnel back to the hub (the same outbound connection the runner uses for claims) carrying the editor's HTTP and WebSocket traffic. The runner marks the job running when the editor answers on its port, and reports a heartbeat (connections open, last activity) every 30 s in place of log batches; the hub stops the workspace when the heartbeat shows no connection for the idle limit. - Hub side.
/{owner}/{name}/workspace/{id}/is a reverse proxy to that tunnel, for the owner's session only, with the workspace secret in a header the runner checks; CSRF does not apply (the editor is same-origin under the hub, and the proxy forwards only to the workspace the path names). Commits: the extension'soxy commitwrites to the microVM's repository;oxy pushgoes to the workspace bookmark with the job token, and the hub signs the pushed changes as the account's delegate (web edits' path) if the account has delegated to the hub key withchanges,refs; otherwise the push is refused with the recipe. The bookmark is ordinary afterwards: propose, land, or delete. - Security. The microVM has no credentials but the job token (read this repository, push this workspace bookmark) and the workspace secret; the account's keys never enter it. Network egress from the microVM is off by default (as for pipeline jobs); a site setting may allow it for package installs, with the audit log recording the choice. The overlay is ephemeral and the rootfs is pinned by hash, so a workspace cannot be left altered.
- Costs. One microVM per open workspace: 2 vCPU and 4 GB by default, from the runner's capacity like any job, so open workspaces compete with pipeline jobs; a runner label
workspacekeeps them on runners meant for it. Idle stop at 30 minutes keeps the fleet small. - Open questions. Whether the Oxygen VCS extension should talk to the microVM's
oxy(simplest; works offline in the VM) or to the hub's API (nooxyin the VM, but a second code path for every operation): the first. Whether a stopped workspace's overlay should be kept for a day so an unpushed edit is not lost: probably yes, as a tarball artifact of the job, restored on reopen. Whether to allow terminals: yes, it is the point.
14. Git interoperability
A repository may enable a Git mirror: on every landing on selected bookmarks the hub runs oxy git export into <data>/mirrors/<owner>/<name>.git and serves it read-only over Git's smart HTTP protocol at /{owner}/{name}.git (implemented with the git binary's http-backend behind the hub's authorization, since reimplementing pack negotiation is not the product). This lets GitHub Actions runners, IDEs and tooling that only speak Git read hosted Oxygen repositories. Pushing to a mirror is refused with a message pointing at oxy.
Import from Git: oxy git import at repository creation from an uploaded bundle or a URL, keeping the mapping so later oxy git fetch keeps the imported bookmark following the Git branch.
15. Security
15.1 Threat model
An anonymous internet user, a signed-in user attempting to exceed their role, a collaborator with write attempting to move a reference the policy forbids, a malicious repository content author (Markdown, file names, huge objects), and a compromised hub host. The last is bounded by design: the hub holds no root keys, approvals are user-signed seals, history is signed by authors, so a compromised hub can deny service and forge hub-only data (issues, comments) but cannot forge history, approvals or policy without detection.
15.2 Web
CSRF: every unsafe form carries a token bound to the session (HMAC(cookie_key, "csrf:" || session mac)), and the Origin header is checked when present. Cookies are HttpOnly, SameSite=Lax, Secure over HTTPS. Content Security Policy: default-src 'self', no inline scripts except the theme snippet by hash, img-src 'self' data:; rendered Markdown is sanitised by ammonia with an allow-list; rendered HTML files are never served inline (raw file responses use Content-Disposition: attachment and X-Content-Type-Options: nosniff) except images with sniffed, allow-listed types. Path parameters are validated before touching the file system; paths inside repositories are resolved against the object store, never the disk.
15.3 Passwords, sessions, tokens
Argon2id with the defaults of the argon2 crate, per-password salt; TOTP with a 30-second step and one-step drift; recovery codes; sessions as in §5.1; tokens stored as BLAKE3 hashes and compared in constant time; login rate limited per account and per address; all session-changing actions logged in the audit log.
15.4 Repository content
Object sizes are bounded by the wire protocol server's body limits (per push and per object); rendering is bounded by render_limit and by time budgets; archives are streamed; symlinks in trees are shown, never followed on the server; subrepository pins are displayed, never fetched by the hub.
15.5 Keys and TLS
The hub delegate key lives in <data>/keys/ with mode 0600. Its identity object is published at /.well-known/oxyhub-key so policy holders can admit it. TLS: the hub serves HTTPS itself with the configured certificate, or plain HTTP behind a terminating proxy, or on the developer edition a self-signed certificate it generates at init (with rcgen) whose CA is downloadable from the trust page and usable with OXY_CA.
15.6 Rate limits and abuse
Per address for anonymous requests, per account and per token otherwise, with separate budgets for pages, API, search and the wire protocol; registration and login have their own. Administrators can ban accounts and block addresses.
16. Performance and caching
16.1 Targets
Repository overview and tree pages in under 100 ms on a warm hub for a repository of 100 000 changes; a change page in under 200 ms; a projected state for a 50-change proposal in under 2 s; wire protocol throughput limited by disk and network, not the hub.
16.2 Locks and blocking
Every oxy-core call runs on the blocking pool. Each repository has a tokio::sync::RwLock: pushes, landings, web edits and mirror exports take the write lock; page reads take the read lock. oxy-core itself is single-threaded per Repo, so a Repo is opened per request and dropped after; store handles are cheap to open because packs are memory mapped.
16.3 Caches
Keyed by state hash, so they never go stale: loaded State objects (LRU by memory), tree listings, rendered file views, blame, history graph layouts per bookmark head set, projected states per proposal, archives, code search results. Keyed by content: rendered Markdown (BLAKE3 of the text and its two link bases, 32 MiB) and syntax highlighted files (BLAKE3 of the file name, which picks the syntax, and the text, 64 MiB), each an LRU bounded by the bytes it holds, rendering outside the lock; an entry larger than a quarter of its budget is not kept. Keyed by repository: bookmark and tag lists, invalidated by the push path. ETag and Cache-Control: private, max-age=0, must-revalidate on pages; immutable on content-addressed responses (raw files by hash, archives by state).
16.4 Large repositories
History pages paginate by cursor; the graph renders a window of 100 changes with lazy extension; log_for_path for tree rows is fetched per row after the page paints and cached per (state, path). The projected state job uses load_state on the target and applies only the proposal's changes on top.
17. Operations: configuration, storage, backup
17.1 Configuration
hub.toml in the data directory; every key overridable by OXYHUB_<SECTION>_<KEY>:
[server]
bind = "0.0.0.0:7300" # plain HTTP
tls_bind = "0.0.0.0:7343" # HTTPS, when tls.cert and tls.key (or self_signed) are set
base_url = "https://oxygenhub.io"
behind_proxy = false # trust X-Forwarded-* when true
[tls]
self_signed = true # developer edition; otherwise cert = "...", key = "..."
[registration]
mode = "invite" # open | invite | closed
[limits]
render_limit = "2MiB"
asset_limit = "512MiB"
push_body_limit = "4GiB"
[email]
smtp = "smtp://…" # optional
from = "OxygenHub <hub@example>"
[mirror]
git = "/usr/bin/git"
[retention]
trash_days = 30
webhook_log_days = 30
17.2 Storage
SQLite in WAL mode; migrations embedded and applied at start; PRAGMA user_version guards downgrades. Repositories are ordinary Oxygen repositories; oxy gc runs per repository as a nightly job, one repository at a time under its write lock, at retention.gc_hour UTC (default 3; 24 or more turns it off), and oxygenhub gc runs the same pass by hand. Each repository's run is logged with what it pruned and packed.
17.3 Command line
oxygenhub init --data DIR [--admin NAME] create the data directory, keys, database, first admin
oxygenhub serve --data DIR run
oxygenhub admin --data DIR user|repo|token … offline administration
oxygenhub backup --data DIR --to FILE consistent snapshot: sqlite backup API plus repos and assets as tar.zst
oxygenhub check --data DIR verify every repository's objects and signatures, report
17.4 Backup and restore
backup uses SQLite's online backup API for the database and copies repos/, assets/, keys/ and hub.toml; cache/ is excluded. Restore is extraction plus serve. Because objects are content addressed, a repository directory copied while a push is in flight is still valid up to its last committed reference update.
17.5 Developer edition on Earth
Earth (earth.andymccall.xyz, Ubuntu 26.04, x86_64, 4 cores, 31 GiB, Docker, ports 22, 6400, 6670, 7842, 8080 in use) runs the developer edition as a systemd service under a dedicated oxygenhub system user with <data> = /home/oxygenhub/data, HTTP on 7300 and HTTPS on 7343 with a self-signed certificate. Source is synchronised from the development machine with rsync (the Oxygen workspace and the hub) and built on Earth with a rustup toolchain installed for the claude account; docs/deploy.md gives the exact steps. Once the hub runs, oxygen, oxygen-vscode and oxygenhub are pushed to it as its first repositories and become their own remotes; that fulfils the self-hosting rule without any Git involvement.
18. Required changes to Oxygen
Each is small and lives in the Oxygen workspace, tracked in its docs/plan.md:
- Bare repositories.
Repo::open_bare(dir): open a.oxy/with no workspace; refuse working-copy operations with a clear error.oxy init --bare. - Server embedding.
oxy-serverexposesrouter_with(resolver, authorizer)where the resolver maps a path (possiblyowner/name) to a repository directory and the authorizer decides read and push per request, so the hub embeds it instead of copying handlers. Repository names with one/allowed when the resolver permits. - Challenge signing.
oxy identity sign-challenge <bytes>andoxy_core::identity::verify_challenge, for linking keys (§5.3) and HTTP signature authentication (§5.4). - Seal command.
oxy seal <hash>..signs a seal attestation over the given changes with the identity key and pushes it on the next push (the seal object and its verification exist, spec §14.4; today only the sync path creates seals).oxy seal --proposal <url>fetches the hash list from the hub API first. - Landing from a client.
oxy proposal land <url>andoxy proposal show <url>as thin API clients. - Delegate signing of hub-created changes. A library entry point to build and sign a change from an edit set with
author = user key, signer = hub key, without a workspace (used by web edits, browser conflict resolution and landings).oxy identity delegate add|removealready exists. - History for the hub.
Repo::log_window(heads, cursor, limit)for paginated history, and agraph::layoutmodule shared with the CLI so both draw the same lanes. - Push hook.
receive_pushreports, per accepted reference, the old and new state and the list of newly stored changes, for the pushes log and webhooks. - Machine interface.
oxy verify --jsonalready exists; addoxy policy proposeoutput as a downloadable proposal file (it exists as a file format, spec §14.5) andoxy tag list --jsonis already present.
19. Milestones
| Delivers | Done when | |
|---|---|---|
| 0.1 Browse | Data directory, config, SQLite, embedded wire server, repository creation by oxy push, theme and shell, overview, tree, file, raw, Markdown, history, change page, compare, bookmarks, tags, blame, conflicts page, trust page (read-only) | oxygen, oxygen-vscode and oxygenhub are hosted on Earth and browsable; oxy clone and oxy push work against it |
| 0.2 Accounts and access | Registration, sessions, TOTP, profile, organisations and teams, roles, visibility, tokens, linked identities with challenge, pushes log, settings pages, audit log | Pushes are attributed to linked keys; a private repository is invisible to outsiders |
| 0.3 Proposals | Proposals from bookmarks and forks, projected state, diff, conflicts, atom-anchored comments, reviews with seal-backed approvals, requirements, landing by delegate or client, stacking | A change proposed, reviewed with a comment that survives a rebase, sealed, and landed, all from the pages plus oxy seal |
| 0.4 Issues and notifications | Issues, labels, milestones, cross references, templates, inbox, email, watching, mentions, search over issues and proposals | — |
| 0.5 API, webhooks, checks | /api/v1 complete for every page, hub-api crate, webhooks with delivery log, checks with seals, required checks, SSE status | The VS Code extension can list proposals and checks through the API |
| 0.6 Web edits, releases, mirror | CodeMirror editing and new file with delegate signing, browser conflict resolution, releases with assets and archives, Git mirror over smart HTTP, code search | A GitHub Actions job can clone a hosted repository over the mirror |
| 0.7 Runner | docs/runner.md, oxyrun, job-claim API, seals from the runner | A pipeline in oxygen builds and tests on Earth and its result appears as a required check |
| 1.0 | Passkeys, backup and restore verified, load test at the §16.1 targets, security review of §15, documentation, packaged release (binary, container image, systemd unit) | — |
20. Open questions
- Seal signing UX. Requiring
oxy sealfrom a client for every approval is honest but slower than a button. A browser-held signing key (WebAuthnprfextension deriving an Ed25519 key) could sign approvals in the browser with hardware backing; ML-DSA-65 in the browser is heavier. Decide after 0.3 ships with the client flow. - Delegate key custody. One hub key delegated to by many users means one key whose compromise lets an attacker sign as any of them (still subject to policy and to each user's revocation). Per-user delegate keys held by the hub would narrow that blast radius at the cost of a key per user. Decide before 0.6.
- Postgres. SQLite is enough for one host; an organisation-scale hub may want Postgres.
sqlxkeeps the option open, but every query would need to be dialect-clean. Decide when someone needs it. - Comment anchoring for lines that are not atoms. Comments on deleted lines and on binary files fall back to file anchors; is that good enough?
- Pipeline language. KDL is the current choice for
.oxyhub/pipelines; Starlark would give loops and functions at the cost of a real evaluator. Settle indocs/runner.md. - Federation. Proposals across hubs (a fork on another hub) are natural in a content-addressed model and out of scope for 1.0.
21. Changelog
- Draft 0.1 (2026-09-27): initial specification.