OXYGENHUB
◆ show-stat oxygen / docs / spec.md
138.4 KiB · Markdown · last changed oxy show --stat: the files a change touched, with lines added and removed 17 days ago renderedsource blame history raw
ff0af1ee unknown 1
# Oxygen (`oxy`) — Version Control System Specification
2
3
| | |
4
|---|---|
6caacfff Andy McCall ↶ 5
| **Status** | Draft 0.8r |
6f8aeb99 Andy McCall ↶ 6
| **Format version** | 1 (changes: 2) |
7
| **Date** | 2026-09-29 |
ff0af1ee unknown 8
| **Audience** | Implementers, security reviewers, and contributors |
9
10
The key words MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY are to be interpreted as described in RFC 2119.
11
12
---
13
14
## Table of contents
15
16
1. [Overview]#1-overview
17
2. [Goals, non-goals, and lineage]#2-goals-non-goals-and-lineage
18
3. [Threat model]#3-threat-model
19
4. [Terminology]#4-terminology
20
5. [Object model and hashing]#5-object-model-and-hashing
21
6. [The change graph]#6-the-change-graph
22
7. [Files, trees, and atoms]#7-files-trees-and-atoms
23
8. [Dependencies and commutation]#8-dependencies-and-commutation
24
9. [Conflicts]#9-conflicts
25
10. [States, bookmarks, and tags]#10-states-bookmarks-and-tags
26
11. [Checkpoints]#11-checkpoints
27
12. [Operation log, workspaces, and the working copy]#12-operation-log-workspaces-and-the-working-copy
28
13. [Storage engine]#13-storage-engine
29
14. [Identity, signatures, and trust policy]#14-identity-signatures-and-trust-policy
30
15. [Transport]#15-transport
31
16. [Wire protocol]#16-wire-protocol
32
17. [Partial, sparse, and shallow repositories]#17-partial-sparse-and-shallow-repositories
33
18. [Locks for unmergeable files]#18-locks-for-unmergeable-files
34
19. [Repository configuration files]#19-repository-configuration-files
35
20. [Hooks and server policy]#20-hooks-and-server-policy
36
21. [Garbage collection]#21-garbage-collection
37
22. [Client profiles and retro hardware]#22-client-profiles-and-retro-hardware
38
23. [Git interoperability]#23-git-interoperability
39
24. [Command-line interface]#24-command-line-interface
40
25. [Performance targets]#25-performance-targets
41
26. [Conformance and test vectors]#26-conformance-and-test-vectors
42
27. [Open questions]#27-open-questions
43
44
---
45
46
## 1. Overview
47
48
Oxygen (`oxy`) is an open-source, distributed version control system written in Rust. It is built for multi-gigabyte repositories, monorepos, and high-frequency development loops, and it defines a reduced client profile so that older and low-specification machines, which current version control systems exclude, can still take part.
49
50
Its distinguishing design choices are:
51
52
- **History is a set of changes, not a sequence of snapshots.** A change records what it did to identified lines and tree nodes, and which earlier changes it depends on. Changes that do not depend on each other commute: applying them in any order yields the same result.
53
- **Every change has two identifiers.** A random, stable *change ID* that survives rewriting, and a content-addressed *change hash* that makes history tamper-evident. Neither is optional.
54
- **Conflicts are data, not errors.** A conflicted state can be committed, pushed, fetched, and built on. Resolution is itself a change.
55
- **Every mutation of repository state is logged and undoable** through an operation log.
56
- **Content is stored in a content-defined-chunked, BLAKE3-addressed store**, so large files and large histories deduplicate naturally and can be fetched lazily.
57
- **Every change and every reference update is authenticated** with composite Ed25519 + ML-DSA-65 signatures, either on the change itself or on a push-time seal covering it, and verification is driven by an in-repository trust policy.
58
- **Transport is pluggable.** HTTPS is mandatory for every server, Noise is available for peer-to-peer use, and a plaintext transport exists for constrained clients on trusted networks, with integrity provided end to end by the object model rather than by the channel.
59
60
## 2. Goals, non-goals, and lineage
61
62
### 2.1 Goals
63
64
- Correct, deterministic merges of concurrent work without merge commits for independent changes.
65
- Stable identifiers for human and tooling use across rebase, amend, and reorder.
66
- Tamper-evident history and verifiable authorship, robust against future large quantum computers.
67
- Repositories of tens of gigabytes with millions of files, with `status` and `commit` latency independent of repository size.
68
- Partial clone by path, depth, and blob size as a first-class mode, not an afterthought.
69
- Interoperability with Git repositories in both directions.
70
- A documented Thin client profile for machines with a 68000-class CPU and 1 MiB of RAM.
71
72
### 2.2 Non-goals
73
74
- Running on machines with less than 512 KiB of RAM.
75
- Replacing code review, CI, or issue tracking.
76
- Semantic (language-aware) merging. Merging is line-based for text and replace-based for binary.
77
- Confidentiality of repository contents at rest in version 1 (see §27).
78
79
### 2.3 Lineage
80
81
The change model is the patch theory of Darcs and Pijul. The stable-ID-plus-hash split, the first-class working-copy revision, and the operation log come from Jujutsu. The chunked content store follows bup, restic, and Xet. The signed reference model follows The Update Framework (TUF) and gittuf. Oxygen's contribution is combining these with a partial-fetch-first protocol, composite post-quantum signatures, and a constrained-client profile in a single coherent design.
82
83
## 3. Threat model
84
85
### 3.1 Adversaries
86
87
| Adversary | Capabilities |
88
|---|---|
89
| **Network attacker** | Observe, modify, drop, delay, and replay traffic between client and server. |
90
| **Malicious or compromised server** | Serve arbitrary objects and references, withhold objects, and serve stale references. |
91
| **Malicious collaborator** | Hold a valid signing key authorized for some references and attempt to exceed that authorization, forge others' authorship, or introduce colliding identifiers. |
92
| **Compromised client host** | Full control of one contributor's machine, including their private keys. |
93
94
### 3.2 Security goals
95
96
- **G1 Integrity.** A client that has verified a state's heads can detect any modification to any object reachable from them.
97
- **G2 Authenticity.** Every change and every reference update carries a signature verifiable against an identity, and policy decides whether that identity is authorized.
98
- **G3 Freshness.** A client can detect when a server presents a reference older than one it has already seen.
99
- **G4 Non-downgrade.** An attacker cannot strip the post-quantum half of a composite signature and have it verify.
100
- **G5 Availability under adversarial input.** Pathological file contents cannot cause super-linear chunking, hashing, or graph work.
101
102
### 3.3 Explicitly out of scope
103
104
- Confidentiality against a malicious server (repository contents are visible to whoever hosts them).
105
- Recovery from a compromised signing key beyond revocation and policy update.
106
- Protection against a compromised client host altering its own working copy.
107
108
Every mechanism in §§5, 14, 15, and 16 is annotated with the goal it serves.
109
110
## 4. Terminology
111
112
| Term | Meaning |
113
|---|---|
114
| **Object** | An immutable, content-addressed byte string with a type. |
115
| **ObjectRef** | The 32-byte BLAKE3 hash of an object, including its domain-separation header. |
116
| **Change** | The unit of history: tree operations, edits, dependencies, and metadata. |
117
| **ChangeId** | A 128-bit random identifier that is stable across rewrites of a change. |
118
| **ChangeHash** | The ObjectRef of a change. Changes with different content have different hashes. |
119
| **Atom** | An identified unit of file content: one line of a text file. |
120
| **AtomId** | `(ChangeHash of introducing change, u32 index)`. |
121
| **Node** | A file, directory, or symlink, identified by a 128-bit random **NodeId** stable across renames. |
122
| **Placement** | The `(parent NodeId, name)` pair that positions a node in the tree. |
86e682c2 Andy McCall ↶ 123
| **State** | A dependency-closed set of changes. Fully determined by its maximal elements (**heads**), and named by a `state` object that lists them. |
ff0af1ee unknown 124
| **Bookmark** | A named, mutable pointer to a state. Equivalent to a Git branch. |
125
| **Tag** | A named, immutable, signed pointer to a state. |
126
| **Checkpoint** | A materialized snapshot of a state, used as a cache. |
127
| **Operation** | A recorded mutation of local repository state (bookmarks, workspaces). |
128
| **Workspace** | A checkout directory associated with a repository. |
129
| **Attestation** | A signed statement about a reference, tag, batch of changes, or key. |
130
| **Identity** | An object holding a contributor's public keys and metadata. |
131
132
## 5. Object model and hashing
133
134
### 5.1 Encoding
135
136
All objects are encoded as deterministic CBOR (RFC 8949 §4.2.1): definite lengths, shortest integer encodings, map keys sorted bytewise, no floating-point values, no indefinite-length items. Map keys are small unsigned integers; the tables in this document give the key number in brackets, e.g. `deps [2]`.
137
138
Every object begins with the field `version [0]`, an unsigned integer. This document defines version 1. Readers MUST reject objects whose version they do not understand.
139
140
### 5.2 Hashing (G1)
141
142
```
143
ObjectRef = BLAKE3-256( header || body )
144
header    = "oxy1 " || type-name || " " || decimal(len(body)) || 0x00
145
```
146
86e682c2 Andy McCall ↶ 147
`type-name` is one of `chunk`, `blob`, `change`, `checkpoint`, `op`, `attest`, `identity`, `policy`, `state`. The header prevents an object of one type from being presented as another. BLAKE3 with a 256-bit output retains 128-bit collision resistance against quantum attackers.
ff0af1ee unknown 148
149
The all-zero ObjectRef is reserved and means "self" inside a change (§7.4).
150
151
### 5.3 Object types
152
153
| Type | Purpose | Section |
154
|---|---|---|
155
| `chunk` | Raw bytes, at most 1 MiB. | §13.2 |
156
| `blob` | Ordered list of chunk refs plus total length; represents file content. | §13.2 |
157
| `change` | A history entry. | §6 |
158
| `checkpoint` | Materialized state. | §11 |
159
| `op` | Operation log entry. | §12 |
160
| `attest` | A signed statement. | §14.4 |
161
| `identity` | A contributor's keys. | §14.2 |
162
| `policy` | The repository's trust policy. | §14.5 |
86e682c2 Andy McCall ↶ 163
| `state` | A sorted list of heads naming a state. | §10.1 |
fc1a466a Andy McCall ↶ 164
| `receipt` | What a runner ran and its result. | §14.6 |
ff0af1ee unknown 165
166
### 5.4 Signatures are outside the hash
167
168
Signatures never appear inside a hashed object. A signed object is transported and stored as:
169
170
```
171
Signed = { object [0]: bytes, sigs [1]: [Signature] }
172
```
173
174
Adding a signature, re-signing, or attaching a seal (§14.4) therefore never changes an ObjectRef.
175
5b1114a0 Andy McCall ↶ 176
### 5.5 Encoding conventions
177
178
The field tables in this document give each map key in brackets. Where a structure is a choice between variants, or a compact reference, the following encodings apply:
179
180
| Item | Encoding |
181
|---|---|
182
| `TreeOp` | Map with key 0 = variant tag: 0 `AddNode`, 1 `RemoveNode`, 2 `Move`, 3 `SetMode`; key 1 = node id; `AddNode` adds 2 kind, 3 mode, 4 parent, 5 name; `RemoveNode` adds 2 placement ref; `Move` adds 2 old placement ref, 4 parent, 5 name; `SetMode` adds 2 old mode ref, 3 mode. |
6f8aeb99 Andy McCall ↶ 183
| `Edit` | Map with key 0 = variant tag: 0 `Insert`, 1 `Delete`, 2 `Replace`; key 1 = node id; `Insert` adds 2 after, 3 before, and either 4 lines (array of byte strings) or, in a version 2 change, 5 text (the text blob's ObjectRef) and 6 count (the number of lines); `Delete` adds 2 atom ranges; `Replace` adds 2 old (bytes or null), 3 new (bytes or null). |
86fe29fd Andy McCall ↶ 184
| `NodeKind` | Unsigned integer: 0 file, 1 dir, 2 symlink, 3 subrepo (§7.8). |
185
| Subrepo pin | The opaque content of a `subrepo` node: a CBOR map `{0: 1, 1: kind, 2: locator, 3: pin, 4: bookmark}` where `kind` is 0 for an Oxygen repository (`locator` = repository id bytes(16), `pin` = a state object's heads as `[ChangeHash]`) or 1 for a Git repository (`locator` = null, `pin` = commit sha as text); `bookmark` is the branch or bookmark name the pin was taken from. |
5b1114a0 Andy McCall ↶ 186
| `Mode` | Unsigned integer bit field: bit 0 executable, bit 1 opaque content mode. Directories carry 0. |
187
| `OpRef` (placement or mode reference) | `[bytes(32) change hash, uint index]`; the all-zero hash means this change. |
188
| `AtomRef` | `0` for `START`, `1` for `END`, otherwise `[bytes(32) change hash, uint index]` with the all-zero hash meaning this change. |
189
| `AtomRange` | `[bytes(32) change hash, uint from, uint to]`, half-open, non-empty. |
132b884b Andy McCall ↶ 190
| `View` (in `Op`) | Key 0 bookmarks: array of `[text name, bytes(32) state ref]` sorted by name; key 1 workspaces: array of `[bytes(16) id, bytes(32) change]` sorted by id; key 2 remotes: array of `[text remote, [[text bookmark, bytes(32) state ref]]]`; key 3 policy ref or null; key 4 tags: array of `[text name, bytes(32) state ref]`. Repository format 1 stored `[name, [heads]]`; format 2 stores the state ref. |
5b1114a0 Andy McCall ↶ 191
| `Signature` | Map: 0 key id, 1 alg (1 = Ed25519 + ML-DSA-65), 2 Ed25519 bytes(64), 3 ML-DSA-65 bytes(3309). A signature list is a CBOR array of these. |
192
| Secret key file | Map: 0 version, 1 key id, 2 Ed25519 seed bytes(32), 3 ML-DSA-65 seed bytes(32). Kept only in the user's key directory. |
193
194
Atom indices count the atoms one change introduces for one node, in edit order, across all of that change's `Insert` edits on the node.
195
ff0af1ee unknown 196
## 6. The change graph
197
198
### 6.1 Change object
199
200
```
201
Change {
0f93f49c Andy McCall ↶ 202
  version   [0]: 1, 2 or 3
ff0af1ee unknown 203
  change_id [1]: bytes(16)            // random, CSPRNG; see §6.3
204
  deps      [2]: [ChangeHash]         // sorted, unique; see §8
205
  author    [3]: KeyId                // ObjectRef of an identity object
206
  hlc       [4]: { wall_ms: u64, logical: u32 }
207
  message   [5]: text
208
  tree_ops  [6]: [TreeOp]             // §7.2
209
  edits     [7]: [Edit]               // §7.4
210
  kind      [8]: 0 = normal, 1 = resolution, 2 = working-copy
6f8aeb99 Andy McCall ↶ 211
  extra     [9]: map                  // reserved; MUST be empty
ff0af1ee unknown 212
}
213
```
6f8aeb99 Andy McCall ↶ 214
0f93f49c Andy McCall ↶ 215
Writers produce version 3. The versions differ only in how an `Insert` holds its lines (§7.4): a version 1 change carries every inserted line inline; versions 2 and 3 store each large insert as a **text blob** beside it, so no change grows with the size of the files it adds, and differ in what the text blob holds. Readers MUST accept all three. A change keeps the version it was made in: rewriting its encoding would change its hash and invalidate its signatures.
ff0af1ee unknown 216
217
### 6.2 Two identifiers
218
219
- **ChangeHash** is the integrity handle (G1). All references between objects use it. Dependencies, bookmark heads, tags, and checkpoints reference changes by hash only.
220
- **ChangeId** is the stable handle. It is what users type, what tooling tracks, and what survives amend, rebase, and reorder. It carries no integrity meaning and is never used alone as a reference in any object.
221
222
A change may be *rewritten*: a new change object is created with the same ChangeId and a different hash. Changes that depended on the old hash are *orphaned* and must be rewritten in turn (§8.4). A repository may hold several changes with the same ChangeId; when more than one is reachable from the same state the ChangeId is *divergent*, and commands MUST report divergence rather than pick silently.
223
224
### 6.3 ChangeId generation
225
226
ChangeIds MUST be produced by a cryptographically secure random generator with 128 bits of entropy on Full and Lite profiles. On the Thin profile they are generated by the proxy (§22). Two changes with equal ChangeId and unequal hash are not an error; they are divergence (§6.2). An implementation MUST NOT infer anything about content from a ChangeId.
227
228
### 6.4 Timestamps
229
230
`hlc` is a hybrid logical clock: `wall_ms` is the author's wall clock in milliseconds since the Unix epoch, and `logical` is a counter. On observing a change with a later HLC, a client advances its own clock per the Kulkarni et al. HLC rules. `wall_ms` is untrusted metadata and MUST NOT be used for ordering, security, or freshness decisions. Sorting for display uses `(hlc, ChangeHash)`.
231
232
**Drift bound.** A change with a far-future `wall_ms` must not be able to drag every peer's clock forward. On observing a change, a client MUST NOT advance its own HLC more than `core.hlc-max-drift` (default 5 minutes) ahead of its local wall clock; the change itself is still accepted and stored, so whether a state is well-formed never depends on the observer's clock. A server MAY reject a push containing a change whose `wall_ms` is more than `core.hlc-max-drift` ahead of the server clock, since the author is present and can correct their clock. `logical` is a `u32`; a client whose counter would overflow MUST refuse to create the change and report clock skew.
233
234
## 7. Files, trees, and atoms
235
236
### 7.1 Tree model
237
86fe29fd Andy McCall ↶ 238
The repository tree is a set of nodes. Each node has a **NodeId**, a **kind** (`file`, `dir`, `symlink`, `subrepo`, §7.8), a **mode**, and zero or more **placements**. The mode of a file is its executable bit and its **content mode**, `text` or `opaque` (§7.5); a symlink's content is its target path and its content mode is always `opaque`; a directory has no mode. The root directory has the fixed NodeId of 16 zero bytes and no placement.
ff0af1ee unknown 239
240
A placement is `(parent NodeId, name)`. Names are byte strings that MUST be valid UTF-8, MUST NOT be empty, `.`, or `..`, and MUST NOT contain `/` or NUL. Placements that a particular file system cannot represent distinctly, such as names differing only by case, are workspace conflicts (§9.7), not errors in the state.
241
6e723b81 Andy McCall ↶ 242
Renames and moves are placement changes, so rename tracking is exact once recorded. A working copy has no way to report a move, so when a snapshot finds a removed file and an added file of the same kind whose contents are at least 50% alike (the share of the larger file's bytes found in lines both have; opaque files must be identical), it records a `Move` of the existing node rather than a removal and an addition. Beyond 10,000 candidate pairs only identical contents are paired. The same detection applies to Git import (§23).
ff0af1ee unknown 243
244
### 7.2 Tree operations
245
246
Placements are atoms in the sense of §7.3: each is introduced by exactly one change and may be deleted by later changes. Because a placement is defined relative to the parent's NodeId rather than a path, moving a directory moves its whole subtree with a single placement change; no operations are generated for descendants. The cost of this is that two concurrent moves can create a cycle, handled in §9.2.
247
248
```
249
TreeOp = AddNode  { node: NodeId, kind, mode, place: (parent, name) }   // introduces node + placement
250
       | RemoveNode { node: NodeId, placement: PlacementRef }            // deletes the placement; node dead when no live placements
251
       | Move     { node: NodeId, old: PlacementRef, place: (parent, name) } // deletes old placement, introduces new one; implicitly moves all descendants
252
       | SetMode  { node: NodeId, old: ModeRef, mode }                  // mode is an atom too
253
PlacementRef = (ChangeHash, u32 index into that change's tree_ops)
254
```
255
256
A node is **live** in a state when it has at least one live placement and its parent is live. A node with more than one live placement, or two live nodes with the same placement, is a conflict (§9.2).
257
258
### 7.3 Atoms and the file graph
259
260
The content of a text file is a directed graph whose vertices are atoms. Each atom is one line, including its terminator (bytes up to and including `\n`, or to end of file). Edges express "immediately after". An atom is introduced by exactly one change and has an `AtomId = (ChangeHash, index)`, where `index` counts atoms introduced by that change for that node, starting at 0.
261
262
Every file graph contains two virtual atoms, `START` and `END`. An atom is **live** unless a change in the state deletes it. Deleted atoms remain in the graph as tombstones; they may still be referenced as anchors.
263
264
The graph of a file in a state is the union of the atoms and edges introduced by every change in the state, minus nothing: deletion sets a flag, it does not remove vertices. **The graph is a function of the set of changes alone and not of the order they were applied.** Implementations MUST preserve this invariant; the conformance suite (§26) tests it by applying random permutations.
265
266
### 7.4 Edits
267
268
```
0f93f49c Andy McCall ↶ 269
Edit = Insert  { node: NodeId, after: AtomRef, before: AtomRef, lines: [bytes] | text: ObjectRef, count: u32, cbor?: 1 }
ff0af1ee unknown 270
     | Delete  { node: NodeId, atoms: [AtomRange] }
271
     | Replace { node: NodeId, old: ObjectRef | null, new: ObjectRef }     // opaque nodes only
272
AtomRef   = START | END | AtomId
273
AtomRange = { change: ChangeHash, from: u32, to: u32 }   // half-open
274
```
275
276
- `Insert` introduces `len(lines)` new atoms, indexed consecutively, and edges `after → first`, `last → before`. `after` and `before` MUST be adjacent in the author's view at authoring time; the pair is the insert's **anchor**.
277
- Atoms introduced by the same change are referenced with the all-zero "self" hash. This is how a change can insert a line and then insert another after it without knowing its own hash.
0f93f49c Andy McCall ↶ 278
- **Text blobs.** In a version 2 or 3 change, an insert whose lines encode as a canonical CBOR array of byte strings to at least 4096 bytes MUST be stored as `text` and `count`: `text` is a blob (§13.2) holding the lines, and `count` is the number of lines, at least 1. In version 2 the blob's content is exactly that CBOR array. In version 3 it is the lines' bytes, joined, when splitting those bytes by the line rule (every line but the last ends in a newline and holds no other; §7.3) gives the lines back, as every insert a snapshot makes does; otherwise it is the CBOR array and the insert carries `cbor: 1` (key 7). A reader rejects a version 3 insert marked `cbor` whose lines the line rule would split back. The joined form stores a file's text as its bytes, so a text blob shares chunks with any blob of the same content and carries no per-line framing (a tenth of a tree's compressed text). A shorter insert MUST be inline. The rule is deterministic, so a change has one encoding and one hash, and identical inserts share their text blob. A reader rejects an inline insert that should have been stored, a text blob shorter than 4096 bytes, and a text blob whose lines do not number `count`. A text blob and its chunks belong to their change: they are carried with it by every transfer (§16.3), kept with it by garbage collection (§21), and checked with it by `fsck`. A change cannot be read without them, so they are never withheld from a partial clone (§13.5).
ff0af1ee unknown 279
- `Replace` is the only edit for `opaque` nodes. `old` is the blob currently expected; two concurrent `Replace` operations with the same `old` and different `new` conflict (§9.3).
280
281
### 7.5 Opaque nodes
282
283
A node's content mode is `opaque` if its content is not line-mergeable. A node MUST be opaque if it matches a `binary` pattern in `.oxyattributes`, if it contains a NUL byte in its first 8 KiB, if it exceeds `core.max-text-size` (default 4 MiB), or if it is a symlink. Symlink targets are stored as small blobs and edited only by `Replace`, bypassing the line graph. Opaque content is stored as a blob and versioned by `Replace`.
284
285
Content mode is part of the node's mode atom, so a switch is a `SetMode`. A change switching a node to `opaque` MUST also `Delete` every live atom of the node and `Replace` with `old = null`; a change switching to `text` MUST also `Replace` with `new = null` and `Insert` the content between `START` and `END`. Concurrent edits made under the other content mode then conflict through the mode atom (§9.2) rather than silently combining.
286
287
### 7.6 Line endings and encoding
288
289
Oxygen stores bytes verbatim and never rewrites line endings unless `.oxyattributes` requests `eol=lf` or `eol=crlf`, in which case normalization happens on snapshot and denormalization on checkout, exactly as Git does with `text eol=`. Implementations MUST NOT guess.
290
291
### 7.7 Checkout safety
292
293
Checkout writes only inside the workspace directory, whatever the tree contains.
294
295
- Checkout MUST create a symlink without following it and MUST NOT write any file through a path that contains a symlink, whether that symlink came from the repository or already existed in the workspace. Implementations SHOULD open parent directories with `O_NOFOLLOW` or the platform equivalent and write relative to that handle.
296
- A placement named `.oxy` under the root, or any placement that resolves to a path inside `.oxy/`, MUST be refused at snapshot and at checkout.
297
- `.oxyignore` and `.oxyattributes` files that are symlinks MUST be treated as absent, with a warning from `oxy status`.
298
- On file systems without symlink support, a symlink node is written as a plain file containing the target path, and `oxy status` reports it; snapshot MUST NOT convert such a file back into a content change.
c863c7a7 Andy McCall ↶ 299
- On file systems without executable bits, snapshot keeps each file's recorded executable bit instead of reading it as cleared.
300
- Paths that a case-insensitive file system (macOS and Windows by default) cannot hold side by side (equal but for case, including a directory's case), and names Windows forbids (the reserved device names such as `CON` and `AUX` with any extension, the characters `<>:"\|?*` and control characters, a trailing dot or space), are reported by checkout on such a platform, and by `oxy doctor` (§21.2) on every platform so they can be renamed before anyone meets them.
ff0af1ee unknown 301
86fe29fd Andy McCall ↶ 302
### 7.8 Subrepositories
303
304
A **subrepo** node embeds another repository at a path, as Git's submodules do. Its content is a **pin**: which repository, and which state of it (§5.5, "Subrepo pin"). The pin is stored and versioned exactly like an opaque file, as a blob replaced by `Replace` edits, so two changes that move the same subrepo to different states produce an ordinary replace conflict (§9.3) that `oxy status` lists and a resolution settles. A subrepo has no placements of its own inside the parent beyond its one entry; its files belong to the nested repository and are never atoms of the parent.
305
306
Two pin kinds exist. An **Oxygen pin** names a repository id and a state (a sorted set of heads), and is verifiable: the nested repository's changes are fetched from its own remotes and checked under its own policy. A **Git pin** names a commit hash and is what `oxy git import` produces for a Git submodule (§23); it is checked out with Git and never converted silently.
307
308
The mapping from path to remote lives in `.oxymodules` at the repository root, a tracked file of `[path]` sections with `url` and optional `bookmark` (Oxygen) or `branch` (Git) keys, edited by `oxy submodule add|remove` and readable by hand. A pin without a `.oxymodules` entry is checked out only when the nested repository is already present.
309
1a3b6203 Andy McCall ↶ 310
In the working copy a subrepo is a directory holding its own `.oxy/` (or `.git/`). The snapshot MUST NOT descend into it; it reads the base of the nested repository's working copy (its bookmark's heads unless a parent checked out an older pin; or Git `HEAD`) as the pin, so committing the parent records whatever the nested repository is at, as in Git. A nested repository that is neither registered in `.oxymodules` nor tracked as a subrepo node is left alone entirely: not walked, not added. A tracked subrepo whose directory is absent keeps its recorded pin, as a Git gitlink does. `oxy checkout` of the parent MUST move an existing nested repository to the pinned state (`oxy checkout` of those heads, or `git checkout --detach <sha>`), fetching from the recorded remote when objects are missing. It moves the nested repository's bookmark only forward: when the bookmark has changes the pin lacks, the bookmark stays where it is and the checkout says so, as Git leaves a submodule's branch alone. A checkout that carries work (`--carry`) leaves a subrepo whose pin is part of that work as it is. It MUST leave an absent nested repository absent until `oxy submodule update --init` clones it, and MUST never delete a nested repository, even when the target state has no node for it. A nested working copy with uncommitted changes is never overwritten: the checkout reports it and continues.
86fe29fd Andy McCall ↶ 311
e7ac6c2e Andy McCall ↶ 312
`oxy submodule add URL PATH [--bookmark NAME]` clones URL into PATH (an Oxygen remote when URL answers as one, otherwise Git), records `.oxymodules`, and the next snapshot adds the node pinned at the clone's heads. `oxy submodule update [--init] [PATH..]` brings nested repositories to their pins. `oxy submodule status` lists each subrepo with its pin and whether the nested repository matches it; one registered but not committed yet is listed as added, with the pin the next commit records (`committed: false` in `--json`). `oxy submodule remove PATH` drops the `.oxymodules` entry and records that the node is to go with the next snapshot; the directory is left in place and thereafter ignored as unregistered.
86fe29fd Andy McCall ↶ 313
ff0af1ee unknown 314
## 8. Dependencies and commutation
315
316
### 8.1 Definition
317
318
The **scope** of a change `C` is the set of atoms and placements it references: every atom it deletes, every atom used as an anchor, every placement it deletes or moves, and every node it operates on.
319
62794ecc Andy McCall ↶ 320
`deps(C)` MUST contain the ChangeHash of the introducing change of every element of `scope(C)`, excluding `C` itself, START, END, and the root node. It MUST also contain, for every atom `C` deletes, the introducing change of every atom live directly after it in the author's state, since without it the deletion reads as concurrent with those insertions and materializes as a delete/insert conflict (§9.1); and, for every `Replace` whose `old` is a blob, the change(s) whose replace record made that blob current. 
ff0af1ee unknown 321
322
**Explicit (semantic) dependencies.** Changes that share no atoms commute even when they are semantically coupled, for example an API change in one file and the update of its caller in another. To prevent such changes from being separated, `deps(C)` MAY contain additional hashes that the author or tooling chose to require. `oxy commit --depends-on <change>` adds one; the argument is resolved to a ChangeHash as in §24, and a divergent ChangeId is an error. Implementations SHOULD support the `semantic-deps` hook (§20) so that build tools or language servers can inject cross-file dependencies at commit time.
323
324
An explicit dependency is a hard dependency with every consequence of §8.2 and §8.4: the change no longer commutes with its target, and rewriting the target orphans it. Tooling that injects dependencies MUST inject the minimal set and MUST be enabled per repository by the user, because injecting dependencies liberally turns history back into a linear sequence and forfeits the model's benefits. `deps(C)` MUST NOT be transitively reduced by the writer; readers MAY reduce it.
325
326
A change with `kind = resolution` MUST additionally depend on every change that contributed an atom or placement to the conflict it resolves.
327
328
### 8.2 Commutation
329
330
Two changes `A` and `B` **commute** if neither is in the transitive dependency closure of the other. By §7.3 applying `{A, B}` in either order produces the same graph, so no rewriting, re-hashing, or merge change is needed to reorder them. Commutation is a consequence of the model, not a separate rule to implement; the implementer's obligation is the invariant in §7.3.
331
332
### 8.3 Well-formedness
333
334
A state `S` is well-formed iff for every `C ∈ S`, `deps(C) ⊆ S`. Every reference to a state (bookmark, tag, checkpoint) names it by its heads, and the state is the dependency closure of the heads. Implementations MUST refuse to construct a state that is not well-formed and MUST refuse to store a change whose dependencies are unavailable unless the repository is shallow (§17).
335
336
### 8.4 Rewriting
337
6706bdcf Andy McCall ↶ 338
Rewriting a change `C` to `C'` (amend, rebase, split, squash) produces a new object with the same ChangeId. Amending an older change with the working copy's edits (`oxy amend --into C`) is allowed when every atom, placement, and node those edits reference exists in `C`'s closure; the edits are appended to `C` with references to `C` turned into self references, so `C`'s own atoms keep their indices and dependents are rewritten with an identity correspondence. Splitting a head (`oxy split C PATH..`) rebuilds it as a first change holding its effect on the named paths and a second holding the rest under the original ChangeId. Any change `D` with `hash(C) ∈ deps(D)` is orphaned relative to `C'`. `oxy` MUST automatically rewrite orphaned descendants by translating their AtomRefs and PlacementRefs from `C` to `C'` where a correspondence exists. Where none exists the reference is **reified by re-anchoring**: an insert anchored on a vanished atom is anchored instead on the nearest surviving atom of the same insert chain of `C` (the preceding one for `after`, the following one for `before`), or on that chain's own anchors when none survives, so that an order conflict (§9.1) arises exactly when `C'` placed other content there; a deletion of a vanished atom is dropped; an operation or edit on a node that vanished with `C` is dropped. Each adjustment MUST be reported to the user. Removing `C` altogether rewires each dependent to depend on `deps(C)` and reifies references into `C` the same way. Changes that do not depend on `C` are untouched, and a rewritten change by another author is stored without that author's signature, which only they can renew. Rewriting is recorded in the operation log (§12) and is undoable.
ff0af1ee unknown 339
86fe29fd Andy McCall ↶ 340
### 8.5 Folding and re-basing
341
342
Changes that commute never need re-basing to be combined, so `oxy rebase` and `oxy squash` are history-editing tools, not merge tools.
343
344
**Squash.** `oxy squash C --into D` folds change `C` into `D`, where `D` is a dependency of `C` and every atom, placement, and mode that `C` refers to in `D` is one `D` introduces or one both already know. The result `D'` carries `D`'s operations followed by `C`'s, with `C`'s references to `D` turned into self-references, keeps `D`'s ChangeId, and is signed afresh by its author; `C` is removed from the state and every change that depended on `C` or `D` is rewritten with the correspondence of §8.4. A `C` that depends on changes `D` does not depend on cannot be squashed into `D`; the command names them. `oxy amend --into` is the special case where `C` is the working copy.
345
e266b7d4 Andy McCall ↶ 346
**Rebase.** `oxy rebase C --onto HEADS` re-expresses `C` against a different base: the state `HEADS` names is materialized with and without `C` added to it (commutation makes the union meaningful, and `HEADS` must contain `C`'s dependencies), the difference is re-derived as edits against the `HEADS` state, and the new change keeps `C`'s ChangeId with dependencies computed from what it now references (plus `HEADS` as explicit dependencies). This is the move a Git user makes with `rebase --onto`: it changes what a change is said to build on. It is refused when `HEADS` conflicts in any file `C` touches, since the re-derivation would have no single text to diff against, and when the result would be empty. With `--into-working-copy`, a rebase that would be conflicted instead merges the `HEADS` state into the current bookmark, which must hold `C`: a working copy without changes follows, so the conflict is fixed there and committed as a resolution on top, as after `oxy merge` (§10.1). Dependents of `C` are rewritten per §8.4. Because commuting changes already coexist, the common reason to rebase is to drop an accidental dependency, for instance on a change that is to be abandoned.
86fe29fd Andy McCall ↶ 347
ff0af1ee unknown 348
## 9. Conflicts
349
350
### 9.1 Text conflicts
351
352
The live atoms of a file graph form a partial order. The file is **conflicted** if that order is not total. Two shapes arise:
353
354
- **Order conflict.** Two or more chains of live atoms are incomparable (concurrent inserts at the same anchor, or concurrent edits of overlapping regions). The conflict region is the minimal interval of the graph containing all incomparable chains.
355
- **Delete/insert conflict.** An atom `x` is deleted by change `D`, and a live atom `y` is anchored after `x` by a change `I`, where `D` and `I` commute. The inserted content has no live neighbour to attach to.
356
357
### 9.2 Tree conflicts
358
359
- **Name conflict.** Two live nodes share a placement.
360
- **Multi-placement.** One node has two live placements (concurrent renames).
361
- **Orphan.** A live node's parent directory is not live (concurrent delete of a directory and modification within it).
362
- **Cycle.** A directory is placed inside its own descendant, which arises when two commuting changes each move a directory under the other. Both placements are live and the tree is conflicted. For materialization the placement whose introducing change has the greater `(hlc, ChangeHash)` is treated as absent; resolution deletes one of the two placements.
363
- **Kind conflict.** Concurrent `AddNode` of the same placement with different kinds is a name conflict; concurrent `SetMode` with different modes is an order conflict on the mode atom.
364
365
### 9.3 Opaque conflicts
366
367
Two live `Replace` edits with the same `old` and different `new` are a **replace conflict**. The state exposes both candidate blobs.
368
f7510b63 Andy McCall ↶ 369
Which blobs are live: a replace record is superseded by a later replacement of its blob, a record whose `old` is its `new` and whose change depends on its change (§8.1 makes every replacement depend on the change that made its `old` current). Replacements that do not know each other both stay live, and a file reverted to an earlier blob keeps it. A replace conflict is **resolved** by a `kind = resolution` change with one `Replace` from each live blob to the chosen one, depending on every change that made a candidate live, so each is superseded and one blob remains. A working-copy snapshot never settles a replace conflict on its own (a checkout writes one candidate at the path); a tool that knows the user's choice, such as a forge's resolver, builds the resolution. A replacement recorded without that dependency (before Draft 0.8i) supersedes every record with its `new` that does not come after it, the rule until then.
62794ecc Andy McCall ↶ 370
ff0af1ee unknown 371
### 9.4 Properties
372
373
- A conflicted state is well-formed and valid. It MAY be committed, pushed, fetched, tagged, and built upon.
374
- Conflicts nest and compose automatically because they are graph shapes, not special objects. A resolution that is itself concurrent with another resolution yields a new order conflict between the two resolutions.
375
- Servers MAY refuse conflicted heads on protected bookmarks through policy (§20). Clients MUST be able to push conflicted heads to unprotected bookmarks.
376
377
### 9.5 Materialization in the working copy
378
379
Conflicted text files are written with markers that identify the conflict and each side so that tools can parse them unambiguously:
380
381
```
382
<<<<<<< oxy conflict 1 of 2, side 1: change k7m2p9 "Add retry loop"
383
... lines ...
384
======= oxy conflict 1 of 2, side 2: change q3x8w1 "Refactor client"
385
... lines ...
386
>>>>>>> oxy conflict 1 of 2 end
387
```
388
389
The short identifiers are ChangeId prefixes. Delete/insert conflicts render the deleted context as a `-------` section. Tree conflicts are materialized by writing each candidate with a suffix `.oxy-conflict-<n>` and are listed by `oxy status`. Replace conflicts materialize as `<name>.oxy-conflict-<n>` files.
390
50a046e4 Andy McCall ↶ 391
`oxy status` MUST list every conflict, MUST exit with status 2 when any conflict exists in the working-copy state, MUST exit with status 3 when only workspace conflicts (§9.7) exist, and MUST support `--json` output enumerating conflicts by node, kind, and contributing ChangeIds. The JSON object has `bookmark`, `working_copy` (ChangeId), `heads`, `clean`, `conflicted`, `paths` (`[{path, status}]`) and `conflicts`: for a text or replace conflict `{kind, path, node, changes: [ChangeId], resolved: true | false | "markers"}`, for a tree conflict `{kind: name | multi-placement | orphan | cycle | mode, nodes, path?}`.
ff0af1ee unknown 392
393
### 9.6 Resolution
394
395
A **resolution** is a change with `kind = resolution` whose deletions and insertions leave the affected graphs totally ordered and the tree conflict-free. `oxy resolve` computes it from the edited working copy by parsing the markers; a resolution MAY also be authored by any tool that can produce a change. Because a resolution depends on all contributing changes (§8.1), any peer that fetches it necessarily has the conflict it resolves.
396
397
### 9.7 Workspace conflicts
398
399
A state can be conflict-free and still not be representable on a particular file system. These are **workspace conflicts**: the state is valid, peers on other file systems see nothing, and no resolution change is involved.
400
401
- Two live placements in one directory whose names are equal under the file system's case folding (`Foo.c` and `foo.c` on default Windows and macOS volumes).
402
- Two live placements whose names are equal after the file system's Unicode normalization (NFC and NFD forms of the same name on macOS).
403
- A name the file system reserves or alters: Windows device names such as `CON` and `NUL`, names ending in a dot or a space, and names containing `:` or other characters the platform forbids.
404
- A path longer than the platform limit.
405
406
Each colliding node MUST be materialized with a `.oxy-conflict-<n>` suffix so that no content is lost, and `oxy status` MUST list the conflict with its kind. A workspace conflict is cleared by any ordinary change that renames one of the nodes; it is never counted by `allow_conflicts` (§14.5) or by server rules that forbid conflicted heads (§20).
407
408
## 10. States, bookmarks, and tags
409
410
### 10.1 References
411
412
References live in a namespace:
413
414
```
415
bookmarks/<name>                 local bookmarks
416
remotes/<remote>/bookmarks/<name>  last-seen remote bookmarks
417
tags/<name>                      tags (immutable once created)
418
policy                           the trust policy head (§14.5)
419
```
420
421
`<name>` is UTF-8, 1–255 bytes, without control characters, leading `-`, or the sequences `..`, `//`, `@{`.
422
86e682c2 Andy McCall ↶ 423
A bookmark or tag points at a **state object**:
424
425
```
426
State { version [0]: 1, heads [1]: [ChangeHash] }   // sorted, unique; may be empty
427
```
428
4fe7d604 Andy McCall ↶ 429
A repository holds any number of bookmarks. Each workspace has a **current bookmark** (`oxy bookmark switch NAME`, recorded per workspace) that its working copy sits on and that `commit`, `log`, `push`, and `pull` act on by default. Switching refuses uncommitted work unless told what to do with it: `--discard` drops it, and `--carry` takes it along when every path it touches is the same in NAME's state as in the working copy's base (so the same edits apply there), refusing and naming the paths otherwise. `oxy bookmark create feat && oxy bookmark switch feat --carry` moves work started on one bookmark onto a new one. `oxy bookmark create NAME [CHANGE..]` names a new state (default: the current working copy's base heads); given CHANGE.., it refuses when the current state has changes recorded before the latest of them that they do not contain, since a state holds its heads' dependency closure and not everything recorded before them, unless `--upto` (the current state's changes up to CHANGE.., as Git ancestry would have it) or `--only` (exactly CHANGE.. and their dependencies) says which is meant, and `bookmark switch` reports how many changes the new state lacks of the old; `set NAME CHANGE..` moves one, `delete NAME` removes it (its changes stay until unreachable, §21), `list` shows all with their heads and the remote-tracking difference, and `oxy merge NAME` unions the named bookmark's heads into the current one, which is the whole of merging in this model: no merge change, conflicts reified (§9); a working copy without changes follows the merge, whichever state it sat on. `oxy checkout NAME` materializes a bookmark without switching to it; with uncommitted work it refuses, naming each changed path and how it changed (a *removed* path is in the working copy's base but not on disk), unless `--discard` or `--carry` (as for switching) says what to do with the work. Bookmark names, then tag names, are resolved before change prefixes wherever a change is expected (a bookmark or tag names a change when its state has one head). For builds, `oxy describe` without `-m` names the working copy as `git describe` does: its base state's tag (`v1.2`), else the nearest tag whose state the base contains with the number of changes it adds and the newest head's ChangeId (`v1.2-3-8c5ea12a`), else the bookmark and that ChangeId (`develop-8c5ea12a`), with `-dirty` when the working copy has uncommitted changes; `--json` gives the parts.
86fe29fd Andy McCall ↶ 430
86e682c2 Andy McCall ↶ 431
Every reference, attestation, view, and checkpoint therefore names a state by one ObjectRef however many heads it has. Multiple heads are normal in this model: every independent change is a head until something depends on it, so a busy bookmark may have dozens. That is not a conflict in the §9 sense; `oxy log` shows the heads as parallel, and a state object keeps the cost of naming them constant for everything that refers to the bookmark. The empty state, `State { heads: [] }`, is what a new bookmark points at.
ff0af1ee unknown 432
433
### 10.2 Reference attestations (G2, G3)
434
86e682c2 Andy McCall ↶ 435
Every published bookmark update is accompanied by an attestation (§14.4) of kind `ref` that binds the repository ID, the reference name, the previous state, the new state, and a per-reference monotonically increasing `serial`. Servers MUST store the latest attestation per reference and serve it with the reference. Clients MUST verify the attestation against policy before updating a `remotes/` reference and MUST reject an attestation whose `serial` is lower than one they have already recorded for that reference (rollback protection). Clients SHOULD warn when the newest attestation's `hlc.wall_ms` is older than `remote.max-age` (default 30 days) and the remote claims no update.
ff0af1ee unknown 436
132b884b Andy McCall ↶ 437
Tags are attestations of kind `tag` and are never updated; a server MUST reject a second attestation for an existing tag name, and any receiver MUST refuse a tag whose name it already holds with a different state. `oxy tag create NAME [CHANGE..]` tags the named state (default the bookmark's heads); tags travel with `refs` and `push` (§16.3), and a policy rule for `tags/<name>` governs who may create them.
ff0af1ee unknown 438
439
## 11. Checkpoints
440
441
Materializing a state by replaying every change is linear in history length. Checkpoints make checkout proportional to state size instead.
442
443
```
444
Checkpoint {
445
  version [0]: 1
cfb7d414 Andy McCall ↶ 446
  heads   [1]: [ChangeHash]        // sorted
447
  count   [2]: u64                 // changes in the state
448
  payload [3]: ObjectRef           // blob holding the serialized state
ff0af1ee unknown 449
}
450
```
451
cfb7d414 Andy McCall ↶ 452
The payload blob is a deterministic CBOR map `{0: version, 1: changes, 2: tree, 3: files, 4: replaces}`:
453
454
- `changes`: for every change in the state, `[hash, change id, [deps], message, hlc wall, hlc logical, kind, author]`, sorted by hash. Operations are not repeated: once applied they live in the graphs, and the store keeps the change objects.
455
- `tree`: the tree graph, `{0: nodes [[node, kind, introducer]], 1: placements [[change, index, node, parent, name]], 2: retired placements [[change, index, [deleters]]], 3: modes [[change, index, node, mode]], 4: retired modes [[change, index, [deleters]]]}`.
456
- `files`: `[[node, file graph]]` where a file graph is `{0: atoms [[change, index, content or null, [deleters]]], 1: edges [[from, to]]}` with vertices encoded as in §5.5 (`0` start, `1` end, `[hash, index]`). Tombstones keep their content so conflicts can show deleted context.
457
- `replaces`: `[[node, [[change, old blob or null, new blob or null]]]]`.
afaba5f1 Andy McCall ↶ 458
459
**Payload version 2** has the same map with `version` 2, differing only in `files`: `[[node, graph or ref]]`, where a file graph whose encoding (as above) is shorter than 2048 bytes appears inline and a longer one is replaced by the ObjectRef (a byte string) of a blob holding exactly that encoding. Graph blobs are content addressed, so a file unchanged between two checkpoints shares its blob, and a reader can load only the graphs it needs. The rule is deterministic, so version 2 checkpoints are as reproducible as version 1. Writers produce version 2; readers MUST accept both. The checkpoint object is unchanged, and whatever carries a checkpoint (garbage-collection roots, the `checkpoint` fetch of §16.1) carries its graph blobs and their chunks with it. A version 2 payload's graph blobs are part of the checkpoint: one missing makes the checkpoint unusable, and a reader reports it rather than dropping the file.
ff0af1ee unknown 460
cfb7d414 Andy McCall ↶ 461
A checkpoint is valid only if it is bit-for-bit reproducible from its heads. Implementations MUST create one at least every `core.checkpoint-interval` changes (default 256) along each bookmark and MAY discard any checkpoint at any time; keeping only the newest per bookmark is enough. To load a state from heads `H`, a client walks from `H` and stops at any change the checkpoint contains; the checkpoint applies iff every one of its heads was reached, since a state is closed downward, and the walk yields exactly the changes to replay on top. Clients fetching a state SHOULD request the nearest checkpoint and replay only the difference.
ff0af1ee unknown 462
463
## 12. Operation log, workspaces, and the working copy
464
465
### 12.1 Operations
466
467
Every command that mutates local repository state (bookmarks, workspaces, the working-copy change, fetched remote references) records an operation:
468
469
```
470
Op {
471
  version [0]: 1
472
  parents [1]: [ObjectRef]            // usually one; two when merging concurrent operations
473
  hlc     [2]: HLC
474
  command [3]: text                   // e.g. "oxy rebase -s k7m2p9 -d main"
86e682c2 Andy McCall ↶ 475
  view    [4]: { bookmarks: map<name, state ObjectRef>, remotes: map, tags: map, workspaces: map<WorkspaceId, ChangeHash> }
530a0fc6 Andy McCall ↶ 476
  workspace [5]: WorkspaceId          // optional: the workspace whose command recorded it
477
  undoes  [6]: ObjectRef              // optional: the operation an in-place undo reverted
ff0af1ee unknown 478
}
479
```
480
6caacfff Andy McCall ↶ 481
`view` is the complete state of all references and workspaces after the operation. The operation log is a DAG. The head (`.oxy/ops/head`) moves by compare-and-swap under a short lock (`.oxy/ops/lock`), held only while the pointer is written: a process that finds the head is no longer the parent it built on records a merge operation with both as parents, whose view is the three-way merge of the two views against that parent. A reference changed on one side takes that side's value; changed differently on both, a bookmark keeps both sides' heads (it then has several), a change beats a deletion, a tag keeps its first value, remote references and the policy take the later process's value, and a workspace changed by both is an error: the later operation is not recorded, since the files on disk match only one of the two working-copy changes. `oxy undo` refuses when the head moved while it ran.
ff0af1ee unknown 482
530a0fc6 Andy McCall ↶ 483
`oxy undo` undoes **this workspace's** last command (oxygen#99): the newest operation recorded by this workspace (for operations older than the `workspace` field, one that moved only other workspaces' working copies is theirs), passing over working-copy snapshots, pushes (the remote keeps what it was sent) and commands an earlier undo reverted, and taking a command's own `(checkout)` with it. When no other workspace recorded an operation since, the head moves back to the command's parent. Otherwise the command is reverted in place as a new operation naming it in `undoes`: each reference it changed returns to its value before it; a bookmark another workspace moved since is put back head by head (the heads the command added go, those it replaced return, the other workspace's stay); and a reference that cannot be put back so (a later change builds on one the command added, or another workspace changed the same tag, remote reference, policy or working copy) refuses the undo, naming it. Undo reports the command it undid, what that changed, the other workspaces' operations it kept, and bookmarks whose undone changes a remote already holds. `oxy redo` and `oxy op log` expose the DAG. Operations are retained for `core.op-retention` (default 90 days) and then eligible for garbage collection.
ff0af1ee unknown 484
767ac6cd Andy McCall 485
A process reads the operation head once, when it first needs it, and answers every later read of references, bookmarks and the view from that operation (oxygenhub#249): one command, or one request a host serves, never mixes two heads, and never waits on another process's operation to read. Its own operations move it to the head each one recorded. It reads the head again only where it must see what others did meanwhile: once it holds the working-copy lock (whoever held the lock may have moved the head), when a host receives a push (the updates are checked against the head as it is now), in `oxy fsck`, and wherever the implementation asks for it explicitly (`Repo::refresh`). An operation recorded from an older head merges with the newer one as any concurrent operation does.
486
6caacfff Andy McCall ↶ 487
Because all mutation goes through the operation log and object writes are idempotent, no long-held repository lock is needed: the operation head's lock covers one pointer write. Each workspace has a **working-copy lock** (`.oxy/workspaces/<id>/lock`), held by every command that reads or writes the working copy and replaces its working-copy change (snapshot, checkout, bookmark switch, merge, commit, amend and rewrites, discard, pull), from reading the operation head until its operation is recorded. A snapshot therefore never sees a half-written checkout, and never records one as edits. The lock is re-entrant within a process thread; a waiting command gives up after a minute and says why; a lock whose process is gone is taken over, and where that cannot be told, one older than a minute.
ff0af1ee unknown 488
489
### 12.2 Workspaces
490
e02bf0cd Andy McCall ↶ 491
A repository has one or more workspaces, each with a `WorkspaceId` (128-bit random) and a checkout directory. Each workspace has exactly one **working-copy change** at any time. The primary workspace is the directory holding `.oxy/`; a secondary workspace holds a `.oxy` file instead, `oxydir: <path>` and `workspace: <id>`, pointing at the shared repository, and the repository records each workspace's directory under `.oxy/workspaces/<id>/`. `oxy workspace add PATH [CHANGE..]` creates one, `list` shows them, and `forget` drops one from the view without touching its files. Workspaces do not nest. A **bare** repository (`oxy init --bare`, marker `.oxy/bare`) has no workspace and no working-copy change; it holds objects, the operation log and references only, for hosting. Working-copy commands fail on it with a clear message; every state-based query works.
ff0af1ee unknown 492
493
### 12.3 The working-copy change
494
495
The working copy is a real change with `kind = working-copy`. Before running any command that reads repository state, `oxy` snapshots the working copy: it diffs the file system against the working-copy change's materialized state and rewrites the working-copy change to include the difference. The change therefore always describes the working directory. To prevent operation log bloat, a command that snapshots the working copy MUST NOT record a new Operation (§12.1) if the filesystem diff was empty (i.e., the working-copy change's hash did not change).
496
497
- The working-copy change is unsigned and never pushed while `kind = working-copy`.
b689779e Andy McCall ↶ 498
- A **new, untracked file larger than `snapshot.max-new-file-size`** (default 100 MiB; 0 for no limit) is left out of the snapshot and named by `oxy status` (and `status --json` as `skipped_large`), with how to ignore it or raise the limit (oxygen#98): a build's output or a downloaded image is not stored unseen by the next command. A tracked file, or one already in the working copy, is never left out for its size.
6a81dbab Andy McCall ↶ 499
- `oxy commit` (`-m MSG`, or `-F FILE` with `-` for standard input) sets the message, sets `kind = normal`, and starts a fresh working-copy change on top of it. The change is signed immediately if `core.sign = "change"`; with the default `core.sign = "seal"` it is left unsigned locally and covered by a seal at push time (§14.4).
ff0af1ee unknown 500
- `oxy new`, `oxy edit <change>`, and `oxy abandon` move which change the workspace edits.
86fe29fd Andy McCall ↶ 501
- There is no staging area and no stash. A partial commit is a split of the working copy: `oxy commit PATH..` records only the named paths (and every edit of theirs) and leaves the rest as uncommitted work; `oxy commit --interactive` offers each hunk (one `Insert` or `Delete` edit on one file, or one tree operation) for inclusion, like `git add -p`; `oxy split` does the same to an existing change. A hunk that another selected hunk refers to is selected with it, so the committed change is always well-formed (§8.3).
ff0af1ee unknown 502
503
### 12.4 Snapshot performance
504
50a046e4 Andy McCall ↶ 505
Implementations MUST support a stat-based crawl with a persistent cache keyed by at least `(path, size, mtime)` and SHOULD support a file-system monitor (Watchman or a native inotify/FSEvents/ReadDirectoryChangesW watcher) for the latency targets in §25.
ff0af1ee unknown 506
da868d5d Andy McCall ↶ 507
The reference implementation's cache is the workspace **manifest** written after every snapshot, commit, and checkout: per file its size, mtime, executable bit, content hash, and, for an opaque file, the ObjectRef of its stored blob. A text file found byte for byte equal to the base state's unconflicted rendering also records its **graph key**: a hash of the file's node, the checkpoint graph its graph was built from (a blob ref, an inline value, or none), and the set of later changes that edited it. Equal keys mean equal graphs and so equal renderings, since a rendering depends only on the graph and the changes' immutable metadata. On the next snapshot a file whose size and mtime match is not read: an opaque file contributes its recorded blob, and a text file whose recorded graph key equals the key of its base graph now is known equal to the base, so it is neither read nor rendered (a stat-clean file that differs from the base, such as an older uncommitted edit, is always read). Attributes decide how a file compares with the base, so `.oxyattributes` files are always read, and when any of them is added, removed, edited, or changed in the base, no text file is skipped. A file whose mtime lies within two seconds of the manifest's own write time is never trusted (the racy-stat window), so a modification that preserves size and lands in the same timestamp tick is still seen. An implementation MAY use a file-system monitor instead of, or in addition to, the manifest.
508
509
A commit records in the manifest the graph key of every text file of its new base (except files with an `eol` attribute, whose bytes on disk are not the rendering), from a history-only load that builds no graph, so the first snapshot after it reads and renders no unchanged file. A history-only load reads no stored insert's text blob (§7.4) until it builds that file's graph. `oxy status` answers from the manifest alone, loading no history, when the manifest records no uncommitted work and no conflict, every entry is older than the racy window, the repository has no subrepos, and every file matches its entry (size, mtime or content, executable bit) with none added or removed; anything else takes a snapshot.
b7f102fc Andy McCall ↶ 510
ff0af1ee unknown 511
## 13. Storage engine
512
513
### 13.1 Layout
514
515
```
516
.oxy/
86e682c2 Andy McCall ↶ 517
  format              "oxy repository format 2\n"
ff0af1ee unknown 518
  config              repository-local configuration (TOML)
519
  objects/
520
    pack-<id>.pack    append-only pack file
521
    pack-<id>.idx     sorted (ObjectRef → offset, length, type) index
767ac6cd Andy McCall 522
    pack-<id>.dict    the pack's zstd dictionary, when a full repack trained one
523
    pack-retired      packs and loose objects a repack replaced, kept for readers (§13.4)
5b1114a0 Andy McCall ↶ 524
    loose/            objects written since the last pack, as header || body
525
    sigs/             signature lists keyed by the ObjectRef they cover (§5.4)
ff0af1ee unknown 526
  ops/                operation objects and op heads
527
  refs/               current view cache (derived from ops; rebuildable)
528
  workspaces/<id>/    snapshot cache, fsmonitor cookie
170bb436 Andy McCall ↶ 529
  index/              derived caches, rebuilt when missing or damaged: each
530
                      workspace's state indexes, and the nodes each change touches
ff0af1ee unknown 531
  policy/             cached policy and identities
532
```
533
2b872d07 Andy McCall ↶ 534
Object writes go to a temporary file, are `fsync`ed, and are renamed into place. Pack files are `fsync`ed before their index is written. An interrupted write can never corrupt an existing object. A reader MUST validate an index before trusting it: entries sorted and unique, each `offset + length` inside the pack, and lengths below an implementation limit (1 GiB), so a damaged index can neither drive out-of-bounds reads nor huge allocations; every object read is re-hashed regardless. The refs cache is derivable from the operation log and is rebuilt if missing.
ff0af1ee unknown 535
536
### 13.2 Chunking and blobs (G5)
537
f9c98dee unknown ↶ 538
File content is split with FastCDC (Xia et al., 2016) using a 32 KiB minimum, 64 KiB target, and 256 KiB maximum chunk size, with normalized chunking level 2 and cut-point skipping from the minimum size. Files of 32 KiB or less are a single chunk, and the final chunk of a file may be shorter than the minimum. The minimum and maximum bounds guarantee that adversarial content produces at most `len / 32 KiB` chunks and at least `len / 256 KiB` chunks, bounding both index growth and dedup loss.
539
540
So that every implementation produces identical boundaries, the parameters are fixed:
541
542
- **Gear table.** `GEAR[i]` for `i` in `0..256` is the little-endian `u64` at bytes `8i..8i+8` of the BLAKE3 extendable output of the ASCII string `oxy1 fastcdc gear`.
543
- **Rolling hash.** `h = (h << 1) + GEAR[byte]` in 64-bit wrapping arithmetic, starting from zero at the minimum size.
544
- **Masks.** A mask with `k` one-bits has them at bit positions `63 - floor(i * 64 / k)` for `i` in `0..k`. The small mask has 18 one-bits (used until the target size) and the large mask has 14 (used until the maximum). A cut occurs at the first position where `h & mask == 0`.
ff0af1ee unknown 545
546
```
547
Blob { version [0]: 1, length [1]: u64, chunks [2]: [ObjectRef] }
548
```
549
6f8aeb99 Andy McCall ↶ 550
Chunks and blobs are the objects that carry content. Other objects are small in practice; a change holds its tree operations and inserts of under 4096 bytes, with larger inserts stored as text blobs (§7.4). No object may be larger than 1 GiB after encoding: a writer MUST refuse to store one and nothing of it is written.
ff0af1ee unknown 551
552
### 13.3 Compression
553
554
Each object in a pack is individually compressed with zstd (level 3 by default). Objects whose compressed size is not smaller are stored raw, with a flag. Chunks matching a `nocompress` attribute pattern (already compressed media) are stored raw.
555
556
### 13.4 Packing
557
a67914bc Andy McCall ↶ 558
An automatic trigger when the loose object count exceeds 4096 writes loose objects into a new pack and merges packs when there are more than 16; `oxy gc` repacks every surviving object into one pack, compressed harder (zstd level 9, against level 3 for everyday writes) and on every core, as `git gc` repacks. A store that is already one such pack (it has a dictionary) with nothing loose is left as it is: repacking it would write the same pack. Writers SHOULD batch: a snapshot, a fetched bundle, a Git import, or a checkpoint is one write batch, and a batch that grows beyond a small threshold (256 objects or 8 MiB) is written as a single pack with one `fsync` rather than as loose objects, each with its own; smaller batches stay loose so ordinary commits do not multiply packs. Objects of an open batch MUST be readable by the writer before the batch completes. Pack indexes MUST support lookup by ObjectRef prefix and MUST be memory-mappable. A repository with two million objects MUST be openable with under 64 MiB of resident index memory on the Full profile.
767ac6cd Andy McCall 559
560
Readers never wait for writers (oxygenhub#249). A store loads its list of packs when it opens and keeps it; a read that misses is not believed at once. When the pack an object was found in is gone, or its dictionary is, the list is loaded again and the read tried once more; when the object is in no pack and not loose, the store lists its packs, and if the listing differs from the one it loaded from, loads the new packs and tries once more. Only then is the object missing (`has` answers false). A pack whose index is gone between the listing and the read, as when a collection deletes it meanwhile, is passed over and the store listed once more, so opening a repository never fails for it. A writer asking whether it already holds many objects lists the packs at most once a second while nothing new turns up.
561
562
A repack does not delete what it replaces. The packs it folded in and the loose objects it packed are **retired**: recorded, with the time, in `pack-retired`, a reference record of lines `<wall ms> <key>` updated by compare-and-swap, so concurrent repacks and collections never lose each other's lines. A store opening, or loading its packs again, does not load a retired pack, and retired loose objects are not counted or packed again as loose; a reader that loaded its packs before the repack reads on from the old files. Each `oxy gc`, and each automatic pack check after a commit, first deletes what was retired at least an hour ago: each such pack with its index and dictionary, and each such loose object a live pack holds. A retired loose object no live pack holds (its pack copy was pruned since) is kept as an ordinary loose object. Readers that loaded their packs before the files went find the objects in the new pack by the reload above. The hour is independent of `core.gc-grace` (§21): `oxy gc --now` waives the grace period for unreachable objects, not the hour readers have.
ff0af1ee unknown 563
d02e545d Andy McCall ↶ 564
The pack and index formats are fixed so that tools can read them without the implementation:
565
566
```
567
pack:  "OXYPACK1" || entry*
568
entry: type u8 || flags u8 || stored_len u32le || raw_len u32le || ref bytes(32) || body[stored_len]
569
index: "OXYIDX01" || count u32le || ( ref bytes(32) || offset u64le || stored_len u32le || raw_len u32le || type u8 || flags u8 )*
570
```
571
572
`type` indexes the object types in the order of §5.3 (chunk 0, blob 1, change 2, checkpoint 3, op 4, attest 5, identity 6, policy 7, state 8). Flag bit 0 means the body is zstd-compressed and `raw_len` is its decompressed size; otherwise the body is stored raw and the two lengths are equal. `offset` is the position of the body, not of the entry. Index entries are sorted by `ref`. A reader MUST re-hash every object it reads from a pack and reject a mismatch.
573
ff0af1ee unknown 574
### 13.5 Lazy objects
575
b3c5a59a Andy McCall ↶ 576
A blob or chunk may be **absent** in a partial repository (§17). After a fetch with a blob filter, the repository records every blob its new changes reference but did not receive, together with the remote that withheld it (`.oxy/lazy/<ref>`), and fetches it through `objects` on first read, dropping the record once stored. Changes and checkpoints are never lazy: a repository that has a change has all of its inline data.
ff0af1ee unknown 577
4dcb8c09 Andy McCall ↶ 578
### 13.6 Durability
579
580
Every update that must survive a crash replaces a file atomically: write a uniquely named temporary file in the same directory, `fsync` it, rename it over the target, and `fsync` the directory. Loose objects and packs are written the same way, a pack before its index. The operation head (`.oxy/ops/head`) is the commit point of every command: before it moves, every object the new operation names MUST be durable, including objects still held by an open write batch (§13.4), which is flushed first. A crash therefore leaves the repository at the operation before or after the interrupted command, never between; stray temporary files may remain and are ignored. The reference implementation's tests stop `commit` and `push` at each durable step and require both repositories to open, pass `oxy fsck` (§21.1) and accept the same command again.
581
ff0af1ee unknown 582
## 14. Identity, signatures, and trust policy
583
584
### 14.1 Algorithms
585
586
| Purpose | Algorithm |
587
|---|---|
588
| Hash | BLAKE3-256 |
589
| Signature (composite) | Ed25519 **and** ML-DSA-65 (FIPS 204) |
590
| Transport AEAD | ChaCha20-Poly1305 |
591
| Transport key agreement | X25519, with ML-KEM-768 hybrid where the Noise pattern supports it |
592
| Random | OS CSPRNG |
593
594
ML-DSA-65 is chosen over ML-DSA-44 for NIST security category 3, at the cost of signatures of 3309 bytes.
595
596
### 14.2 Identity object
597
598
```
599
Identity {
600
  version    [0]: 1
601
  ed25519_pk [1]: bytes(32)
602
  mldsa65_pk [2]: bytes(1952)
603
  name       [3]: text
604
  email      [4]: text
605
  created    [5]: u64 (wall ms)
606
  expires    [6]: u64 | null
607
  previous   [7]: KeyId | null        // rotation chain
7ba155f2 Andy McCall ↶ 608
  delegates  [8]: [Delegate]          // keys allowed to act on this identity's behalf
ff0af1ee unknown 609
}
7ba155f2 Andy McCall ↶ 610
Delegate {
611
  key     [0]: KeyId
612
  scope   [1]: ["changes" | "refs" | "seal"]   // what it may do, in that order
613
  expires [2]: u64 | null                       // wall ms; null for as long as it is listed
614
}
ff0af1ee unknown 615
KeyId = ObjectRef of the Identity
616
```
617
7ba155f2 Andy McCall ↶ 618
The two public keys live in one object, so a `KeyId` names the pair. `delegates` lists keys that may act for this identity, each within a **scope**: `changes` lets the key sign changes whose `author` is this identity, `seal` lets it seal them, and `refs` lets it attest references and tags as this identity would (§14.5). A proxy acting for a Thin client (§22) or a host that commits browser edits for a user gets `changes,refs`; an AI agent gets `changes` alone, so its signature authenticates the changes it makes but it can never move a reference, whatever its owner may do. A delegation with `expires` authenticates nothing whose `hlc.wall_ms` is later than that, so an agent can be let in for a session without changing the owner's own expiry. Delegation is one level deep and is not transitive. A bare `KeyId` in `delegates`, as identities carried before scopes existed, means every scope with no expiry. Rotation is a new identity with `previous` set, plus an attestation of kind `rotate` signed by the previous key. Revocation is an attestation of kind `revoke` signed by the revoked key, by the identity that delegated to it, or by a policy root key (§14.5); the delegator's revocation marks the changes the key already signed as valid but revoked, and the delegator then re-issues their identity without it.
ff0af1ee unknown 619
e5d9b051 Andy McCall ↶ 620
**Lifecycle rules.** Because the `KeyId` is the hash of the whole object, any change to an identity, including its `delegates` or `expires`, is a rotation: a successor object with `previous` set. A successor MAY keep the same key pair (`oxy identity renew`, `oxy identity delegate add|remove`) or carry a fresh one (`oxy identity rotate`); either way earlier signatures remain verifiable under the earlier object, which stays stored. A rotation **supersedes** the previous KeyId: a policy that named it MUST be updated to name the successor (§14.5), and a rule key covers every predecessor in its `previous` chain that is not revoked, so changes signed before a rotation remain authorized. A tag's attestation (§10) signed by such a predecessor is authorized only when its `hlc.wall_ms` is earlier than the `created` of the identity that succeeded the signer, so tags made before a rotation stay valid and a superseded key makes no new ones. Revocation is for compromised or abandoned keys: a revoked key authorizes nothing, whatever its chain. `expires` bounds the changes a key can authenticate: a signature or seal over a change whose `hlc.wall_ms` is later than the identity's `expires` is invalid. A client records the revocations it makes or verifies under `.oxy/revoked/` and reports signatures by such keys as valid but revoked.
28fc23d7 Andy McCall ↶ 621
ff0af1ee unknown 622
### 14.3 Composite signature (G2, G4)
623
624
```
625
Signature {
626
  key  [0]: KeyId
627
  alg  [1]: 1                          // ed25519+mldsa65
628
  ed   [2]: bytes(64)
629
  ml   [3]: bytes(3309)
630
}
631
message = "oxy1 sig " || domain || 0x00 || KeyId || ObjectRef-being-signed
e5f8eb2a Andy McCall ↶ 632
domain  = "change" | "attest" | "policy" | "certificate" | "grant" | "checkpoint"
ff0af1ee unknown 633
```
634
635
Both signatures are computed over the same `message`, which includes the `KeyId` of the composite pair. Verification MUST require both signatures to verify with the keys of the identity named by `key`. A signature with either half missing, empty, or failing is invalid. This is the same construction as the IETF LAMPS composite signature drafts and prevents stripping the post-quantum half.
636
637
Profiles that cannot compute ML-DSA (§22) do not sign; they produce changes that a proxy signs.
638
7aa2dc93 Andy McCall ↶ 639
**Challenges.** A host that must know a client holds a key (linking a key to an account, signing in to a server) sends at least 16 random bytes; the client returns a composite signature in domain `challenge` over `BLAKE3-256("oxy1 challenge " || bytes)`, which can never equal a stored object's reference, so a challenge signature authenticates nothing but the challenge. `oxy identity sign-challenge HEX` prints it.
640
ff0af1ee unknown 641
### 14.4 Attestations
642
643
```
644
Attest {
645
  version [0]: 1
f53cbe7a Andy McCall ↶ 646
  kind    [1]: "ref" | "tag" | "seal" | "rotate" | "revoke" | "lock" | "receipt" | "approval" | "accepted"
ff0af1ee unknown 647
  repo    [2]: bytes(16)               // repository ID, chosen at init
648
  hlc     [3]: HLC
649
  body    [4]: map                     // per kind
650
}
86e682c2 Andy McCall ↶ 651
ref:    { name, old_state: ObjectRef, new_state: ObjectRef, serial: u64 }
652
tag:    { name, state: ObjectRef, message }
ff0af1ee unknown 653
seal:   { hashes: [ChangeHash] }        // sorted, unique, 1..4096 entries
654
rotate: { new: KeyId }
655
revoke: { key: KeyId, reason }
656
lock:   { node: NodeId, path, acquire: bool, expires }
fc1a466a Andy McCall ↶ 657
receipt: { receipt: ObjectRef, state: ObjectRef, run: text }   // §14.6
b582948f Andy McCall ↶ 658
approval: { run: text, job: text, job_hash: ObjectRef, state: ObjectRef,
659
            inputs: [ObjectRef], environment: text, approve: 0 | 1 }  // sorted, unique inputs
f53cbe7a Andy McCall ↶ 660
accepted: { attest: ObjectRef }        // §14.7
ff0af1ee unknown 661
```
662
b582948f Andy McCall ↶ 663
An **approval** is a person's decision about one job of one OxygenFlow run (an approval gate, OxygenFlow spec §5.3.2): the job as hashed with its sandbox, on that state, taking exactly those input blobs (for a deploy, the reviewed plan), for that environment. It approves nothing else: another run, state, plan or environment needs another approval. It is handed to the hub as `[attest bytes, [Signature], identity bytes]`, signed in domain `attest` by the approver's own key (`oxy seal --flow URL --job NAME`, `--reject` to reject), so it verifies with nothing else; whether the signer may approve is the hub's rule (a key linked to an account with the gate's role; delegated keys only where the repository allows). The hub keeps it in the repository, so it can be verified without the hub.
664
ff0af1ee unknown 665
A **seal** is one signature over a batch of change hashes. A composite signature is 3373 bytes, which is larger than a typical change, so signing every change individually roughly doubles the size of history in high-frequency workflows. A seal replaces that with one signature per push.
666
667
**Cadence.** With the default `core.sign = "seal"`, changes are left unsigned locally and `oxy push` creates one seal covering every change in the push that carries no individual signature, immediately before the reference attestation. Seals are created only at push: a seal made earlier would be invalidated by any amend, since rewriting produces a new hash. With `core.sign = "change"`, every change is signed at commit and no seal is needed. Policy (§14.5) decides which the receiving repository accepts.
668
a2387d54 Andy McCall ↶ 669
**Verification.** A sealed change `C` is authenticated iff a seal `S` exists such that `hash(C) ∈ S.body.hashes`, `S` carries a valid signature from key `K`, and `K` equals `C.author` or is listed in `C.author`'s identity as a delegate with the `seal` scope, not expired at `C.hlc.wall_ms` (§14.2). The same author-or-delegate rule applies to individual change signatures, with the `changes` scope. This rule is what stops a collaborator from sealing a change whose `author` field names someone else (§3.1). A change whose `author` is the zero KeyId names nobody (history imported from Git, §21, or made before an identity existed), so a seal from any key authenticates it: the sealer **adopts** it, and policy authorizes the sealer for the reference as usual. A push seals the pusher's own unauthenticated changes and adopts anonymous ones in the same seal; a pusher that signs for its configured author (`author.signer`: a delegate, or a short-lived key under the author's grant, §14.7) seals the author's as well, and the receiver checks it may. A seal's signer must additionally be authorized by policy for the reference being pushed.
ff0af1ee unknown 670
096f447c Andy McCall ↶ 671
**Handling.** A seal is stored and transported with the changes it covers: `fetch` (§16.1) streams the seals covering every change it returns, `push` (§16.4) MUST include a seal or signature for every change, and a seal covering any reachable change is itself reachable (§21). Seals from different keys MAY overlap; a change needs at least one valid authenticator, and policy thresholds count distinct authenticating keys.
ff0af1ee unknown 672
673
### 14.5 Trust policy (G2)
674
675
The `policy` reference points to a signed `Policy` object:
676
677
```
678
Policy {
679
  version   [0]: 1
680
  repo      [1]: bytes(16)
681
  serial    [2]: u64
682
  roots     [3]: { keys: [KeyId], threshold: u32 }
683
  rules     [4]: [ { pattern: glob over reference names, keys: [KeyId], threshold: u32,
3a6effc7 Andy McCall ↶ 684
                     require: "signed" | "sealed" | "any" /* default */, allow_conflicts: bool,
685
                     subjects [5]: [ { pattern: glob over subjects, issuer: KeyId, scope: [text] } ] /* only when any */ } ]
ff0af1ee unknown 686
  identities[5]: [Identity]           // embedded so verification needs no fetch
687
  revoked   [6]: [KeyId]
688
  expires   [7]: u64
689
}
690
```
691
28fc23d7 Andy McCall ↶ 692
- The initial policy is signed by the root keys; every update MUST be signed by `roots.threshold` root keys of the policy it replaces and have a higher `serial`. An update that a single holder cannot complete is a **proposal**: the signed policy (`[policy bytes, [Signature]]`, §16.3) is passed to other root holders, each of whom adds a signature with `oxy policy sign`; whoever completes the threshold applies it. Rotating a root key (§14.2) substitutes the successor in the roots and rules; such a policy is signed by both the previous and the successor key so a fresh clone can bootstrap from it.
693
- A change's signer is authorized by a rule if the rule lists it, or lists a successor of it, or lists the change's `author` (or a successor) when the signer is that author's delegate.
7ba155f2 Andy McCall ↶ 694
- A reference attestation's signer is authorized by a rule if the rule lists it, or lists an identity whose `delegates` name it with the `refs` scope, not expired at the attestation's `hlc.wall_ms` (§14.2). Delegation is the identity's own signed statement and the policy names that identity, so a delegate with `refs` moves references exactly as far as its delegator may; revoking either key, or the delegation expiring, ends it. Hosts use this to land and to commit browser edits on a user's behalf with their own key. A delegate without `refs` (an agent) is never authorized for a reference or tag, even when its delegator is.
f53cbe7a Andy McCall ↶ 695
- A rule may also grant **subjects an issuer certifies** (§14.7): `oxy policy grant-subject SUBJECT --issuer IDENTITY --scope SCOPES` adds `{pattern, issuer, scope}` and embeds the issuer's identity. A signer is then authorized when it is a short-lived key whose certificate verifies with that issuer, names a subject the pattern matches, gives the scope needed (`changes` for changes, `refs` for reference attestations and tags) and covers a time the signer did not choose; it counts as one signer toward the threshold. A rule with subjects only has no keys, and its threshold counts keys and subjects together. The time is the receiver's own clock when it receives a push (changes and the reference attestation), and, on a fetch, the time the certificate's issuer says it accepted the update (§14.7, *Acceptance*), or the attestation's own time when there is no such acceptance, which is weaker: a key whose certificate expired could backdate an attestation within its scope. A policy without subjects encodes as before (field 5 is left out), so its hash is unchanged.
86e682c2 Andy McCall ↶ 696
- A change or attestation is **authorized** for reference `r` if its signer is listed in the first rule whose pattern matches `r`, and the rule's threshold of distinct authorized signers is met across the attestation and each change in the closure of `new_state` not previously in the closure of `old_state`. `require` selects which authenticators count for changes: individual signatures only, seals only, or either. Every change MUST additionally satisfy the author-or-delegate rule of §14.4 regardless of `require`.
ff0af1ee unknown 697
- Clients MUST verify policy on fetch, MUST refuse to update a `remotes/` reference whose attestation is unauthorized, and MUST warn loudly on an expired policy. Servers MUST enforce the same rules on push.
83e0677b Andy McCall ↶ 698
- A repository without a `policy` reference is **unverified**; every command that talks to a remote MUST print a one-line warning until one is created. `oxy policy init` creates one with the author's key as the sole root; `oxy policy grant` and `oxy policy revoke` produce successors.
28fc23d7 Andy McCall ↶ 699
- **Propagation.** The `refs` response carries the current signed policy together with the earlier policies the server still holds, and a push MAY carry a newer policy with the intermediate ones in its bundle, applied before any bookmark update. A receiver behind by several serials verifies each step against the one before it. A receiver accepts a policy only if it is for the same repository, its serial exceeds the current one, and it is signed by the current policy's root threshold; the first policy a repository ever sees vouches for itself, which means at least one valid signature by one of its own roots (trust on first use).
83e0677b Andy McCall ↶ 700
- **Downgrade protection.** A client records the highest policy serial each remote has presented. A remote that later presents none, or a lower serial, is refused. A remote that has never presented a policy is unverified: the client fetches with a warning and cannot verify what it received. Policy signatures use the domain `policy` (§14.3).
701
- **Existing history.** A policy governs changes added after it. The receiver of a push checks the changes the new state adds beyond the state it already had, so history that predates the policy is not re-verified. A fetching client verifies the bookmark's attestation, which binds the whole state through its head hashes, and does not re-verify individual changes; it therefore trusts what an authorized attester vouched for, which is the point of the policy.
fc1a466a Andy McCall ↶ 702
703
### 14.6 Execution receipts
704
705
A runner (OxygenFlow, or any CI) records each job it ran as a `receipt` object:
706
707
```
708
Receipt {
709
  version         [0]: 1
710
  job_name        [1]: text
711
  pipeline_hash   [2]: ObjectRef      // the pipeline definition the job came from
712
  scope_tree_hash [3]: ObjectRef      // BLAKE3 over the sorted (path, content) of the files the job's scope matches; the whole tree without a scope
713
  inputs          [4]: [ObjectRef]    // artifacts and other inputs beyond the tree
714
  environment     [5]: ObjectRef      // what it ran in: isolation tier, rootfs, argv, environment
715
  outputs         [6]: [ObjectRef]    // blobs the job produced
716
  exit_code       [7]: u32            // 0 passed
717
  duration_ms     [8]: u64
718
}
719
```
ff0af1ee unknown 720
fc1a466a Andy McCall ↶ 721
A receipt names no state: the same job on the same scope in the same environment gives the same receipt, which is what lets a runner reuse it for a later state whose scope is unchanged. An attestation of kind `receipt` (§14.4) binds a receipt to the `state` it ran on and names the `run` (usually its URL). The runner signs that attestation with its own identity (§14.2), an ordinary composite key, in domain `attest`.
722
c63fed2e Andy McCall ↶ 723
**Signed receipt.** A runner hands a receipt over as `[receipt bytes, attest bytes, [Signature], identity bytes]`, so it verifies with nothing else: the attestation's `receipt` is the receipt's ObjectRef, and a signature by the included identity verifies over the attestation. Whether a runner is trusted is the reader's decision (a host lists its runners' keys); a revoked runner key is reported. A runner that signs with a short-lived key (§14.7) adds a fifth element, `[certificate bytes, issuer identity bytes]`: the receipt then verifies only if the certificate does (signed by the identity given, which its `issuer` names), certifies the key that signed, gives the `seal` scope and covers the attestation's time; a host checks the time it accepted the receipt as well, since the attestation's is the signer's claim. A repository keeps the certificate (type `certificate`, with its issuer's signature, indexed by the key it certifies) and the issuer's identity beside the receipt, and gc keeps them with it.
fc1a466a Andy McCall ↶ 724
725
**Storage.** A repository keeps receipts beside the state they name: the receipt, the attestation with its signatures, and the runner's identity, indexed by state ObjectRef. Receipts live as long as the changes of their state, and keep their outputs and inputs alive (§21). `oxy receipt list|show|export|verify|import` reads and exchanges them; a receipt for another repository is refused.
726
a39754d0 Andy McCall ↶ 727
**SLSA provenance.** For tools outside Oxygen, `oxy receipt export --format slsa` writes what a valid receipt says as an in-toto Statement v1 with an SLSA provenance v1 predicate (`buildType` `https://oxygenhub.io/oxygenflow/job/v1`), in a DSSE envelope (payload type `application/vnd.in-toto+json`). Its subjects are the job's outputs, with the digest `oxygen-blob` (their ObjectRefs); a job with none has the state as its subject (`oxygen-state`). Resolved dependencies are the state, the scope (`oxygen-scope`) and the inputs; the builder is `urn:oxygen:runner:<runner KeyId>`; `startedOn` is the attestation's wall time less `duration_ms`, `finishedOn` that time. The signed receipt itself is a byproduct (`oxygen-receipt`, media type `application/vnd.oxygen.receipt+cbor`, base64 `content`), so an Oxygen verifier can still check the runner's signature. The envelope is signed by the **exporter**, with the Ed25519 half of its signing key over DSSE's pre-authentication encoding (which cannot be mistaken for an Oxygen signature message, §14.3); `keyid` is the KeyId in hex. DSSE verifiers know no ML-DSA, so this signature is classical: it vouches that the exporter checked the receipt, and the receipt inside keeps the runner's composite signature. `oxy identity export --pem` writes the signing key's Ed25519 public key as a PEM SubjectPublicKeyInfo, as cosign's `--key` takes it. A host does the same with its own key for the artifacts it keeps (hub spec).
728
02e69188 Andy McCall ↶ 729
### 14.7 Certificates for short-lived keys
730
731
An issuer (a hub's key) may vouch for a fresh key for a few minutes, so a Flow job or an agent signs with a key that dies with it instead of one kept on disk (oxygenhub#241, oxygenhub docs/keyless.md):
732
733
```
734
Certificate {
735
  version     [0]: 1
736
  key         [1]: bytes          // the short-lived key's identity object (§14.2)
737
  issuer      [2]: KeyId          // the key that signs this certificate
738
  subject     [3]: text           // who proved itself: "repo:OWNER/NAME:bookmark:B", "agent:NAME@ACCOUNT"
739
  claims      [4]: [[text, text]] // what else the proof said, sorted by name
740
  scope       [5]: [text]         // changes, refs, seal
741
  not_before  [6]: u64            // wall milliseconds
742
  expires     [7]: u64
a2387d54 Andy McCall ↶ 743
  grant       [8]: bytes          // optional: a signed grant (below), only when present
02e69188 Andy McCall ↶ 744
}
745
```
746
747
A signed certificate is `[certificate bytes, Signature]`: the issuer's composite signature, in domain `certificate` (§14.3), over the certificate's ObjectRef (type `certificate`). It is valid when the signature verifies with the identity its `issuer` names, `expires` is after `not_before` and at most an hour later, and its scope is not empty. A signature by the short-lived key names the key's KeyId as any signature does; it counts only within the certificate's scope and window, and the time it is checked at must be one the signer did not choose (a host's record of when it accepted the signature, or the transparency log's), never the signed object's own clock. Which issuers a repository trusts, and for which subjects, its trust policy says (§14.5).
748
a2387d54 Andy McCall ↶ 749
**Grants.** An author lets the keys an issuer certifies for a subject act for it, as a delegate does but without a key of their own (an agent's fresh keys, oxygenhub docs/keyless.md):
750
751
```
752
Grant {
753
  version     [0]: 1
754
  grantor     [1]: KeyId          // the author, whose key signs the grant
755
  subject     [2]: text           // "agent:NAME@ACCOUNT"
756
  issuer      [3]: KeyId          // whose certificates for the subject count
c895bc52 Andy McCall ↶ 757
  scope       [4]: [text]         // what certificates under it may give: changes, seal, refs
a2387d54 Andy McCall ↶ 758
  not_before  [5]: u64            // wall milliseconds
759
  expires     [6]: u64
760
}
761
```
762
763
A signed grant is `[grant bytes, Signature]`: the grantor's composite signature, in domain `grant`, over the grant's ObjectRef (type `grant`). It needs no maximum lifetime: it gives nothing until an issuer certifies a key under it, and the grantor revokes it by asking the issuer to stop. The issuer puts it in each certificate it issues under it (field 8), so it travels with the certificate and a verifier needs nothing else. A signature by a key that is neither a change's author nor its delegate (§14.4) still satisfies the author-or-delegate rule when the key's certificate verifies with its issuer and carries a grant that verifies with the change's author, names the certificate's subject and issuer, gives the scope needed (`changes` for a change signature, `seal` for a seal) in both the grant and the certificate, and covers the change's `hlc.wall_ms`, as a delegation does. Whether the certificate covered the time the key signed is the policy's check (§14.5), at a time the signer did not choose.
764
f53cbe7a Andy McCall ↶ 765
**Acceptance.** A host that issues certificates records when it accepted a reference attestation signed by a short-lived key: an `accepted` attestation (§14.4) naming that attestation's ObjectRef, its `hlc` the host's clock at the push, signed in domain `attest` by the issuer's key. The host keeps it indexed by the attestation it accepts (gc keeps it beside the bookmark's latest attestation) and serves it with the bookmark (§16, `refs` key 5). A fetch checks the certificate at the acceptance's time only when the acceptance names exactly the attestation being checked and verifies with the certificate's issuer; otherwise, including when there is none, at the attestation's own time. An acceptance by any other key changes nothing: the issuer is already trusted to vouch for the key, so it adds no trust it did not have.
766
e5f8eb2a Andy McCall ↶ 767
### 14.8 The transparency log
768
769
A host keeps an append-only log of what it accepts, so it cannot show one history to some readers and another to others (oxygenhub#238, oxygenhub docs/transparency.md). Each accepted signed object (a reference or tag attestation, a seal, a rotation or revocation, a receipt, an approval, a policy, an identity, a certificate, an acceptance) becomes one entry, in the order the host accepted it:
770
771
```
772
Entry {
773
  version   [0]: 1
774
  kind      [1]: text           // "ref", "seal", "tag", "rotate", "revoke", "receipt", "approval",
775
                                //  "policy", "identity", "certificate", "accepted"
776
  repo      [2]: bytes(16)      // the repository's id; zero for identities
777
  object    [3]: ObjectRef      // the attestation, policy, identity or certificate
778
  signers   [4]: [KeyId]        // the keys whose signatures the host checked
779
  received  [5]: u64            // the host's clock, milliseconds (its claim)
780
}
781
```
782
783
**The tree** is RFC 6962's Merkle tree over the entries, with BLAKE3: a leaf's hash is `BLAKE3("oxygen-tlog-leaf\0" || cbor(Entry))`, an interior node's `BLAKE3("oxygen-tlog-node\0" || left || right)`, and the empty tree's root `BLAKE3("")`. A tree of n > 1 leaves splits at k, the largest power of two below n. Inclusion and consistency proofs are RFC 6962's (§2.1.1, §2.1.2), their hashes listed from the leaf upwards.
784
785
**Stored order.** Every hash of the tree is stored once, as soon as it is known: adding leaf n stores its hash and then the hash of each subtree it completes (one per trailing 1 bit of n). The hash of tree level L at position n is therefore at stored index `i(L, n)`, where n is first replaced L times by 2n+1, then i = n + ⌊n/2⌋ + ⌊n/4⌋ + … + L. The store is append-only.
786
787
**Tiles** (C2SP tlog-tiles, height 8). Level-L tile N holds the hashes of tree level 8L for leaves from N·256 on, 32 bytes each: 256 of them when full, fewer in a partial tile. Its path is `tile/<L>/<N>`, with `.p/<W>` appended for a partial tile of W hashes; N is written in groups of three digits, every group but the last prefixed `x` (1234067 is `x001/x234/067`). The entries of level-0 tile N are at `tile/entries/<N>[.p/<W>]`, each as a big-endian u16 length and its bytes. Full tiles never change.
788
789
**Checkpoints** (C2SP tlog-checkpoint) are C2SP signed notes. The text is the origin (the host's name and `/log`), the tree size in decimal and the root in base64, each line ending in a newline. It is signed twice under the origin's name:
790
- with the host key's Ed25519 half, as a standard note signature (type 0x01: key hash = SHA-256(name ‖ "\n" ‖ 0x01 ‖ public key)[:4], signature over the text), which note tools read;
791
- with the composite key, as type 0x4f (key hash over the identity object's bytes): the CBOR of a composite Signature in domain `checkpoint` over `BLAKE3("oxygen-tlog-note\0" || text)`.
792
0ed150d6 Andy McCall ↶ 793
A reader accepts a checkpoint only when both verify, so the log stays post-quantum. Which witnesses must cosign it is oxygenhub docs/transparency.md's.
e5f8eb2a Andy McCall ↶ 794
b0621c2f Andy McCall ↶ 795
**Witnesses** (C2SP tlog-witness) cosign a log's checkpoints with their own keys and clocks, so a log cannot show two histories without someone seeing both. A log sends a witness `POST add-checkpoint`: `old <size>` (of the last checkpoint that witness cosigned), the consistency proof's hashes in base64 one per line, a blank line, then the checkpoint note. The witness:
796
- answers 409 with its own size when `old` is not it;
797
- answers 403 when the note does not verify with the log's key;
798
- answers 422 when the checkpoint does not extend the one it cosigned;
799
- otherwise remembers the new size and root, and returns its cosignature lines, which the log adds to the checkpoint it serves.
800
801
A cosignature is two lines under the witness's name:
802
- C2SP cosignature/v1 (type 0x04): the witness's time in seconds as 8 big-endian bytes, then an Ed25519 signature over `cosignature/v1\ntime <seconds>\n` followed by the checkpoint's text;
803
- a composite line (type 0x4e): the same time, then the CBOR of a composite Signature in domain `checkpoint` over `BLAKE3("oxygen-tlog-cosig\0" || that message)`.
804
805
A reader counts a cosignature only when both lines verify and agree on the time.
806
0ed150d6 Andy McCall ↶ 807
**On fetch**, when a refs response carries a checkpoint (§16), a client:
808
- pins the host's log key on first sight (the hub's published identity), per user and hub, with the checkpoint;
809
- requires every later checkpoint to verify with that key, keep its origin, and extend the last one seen: a smaller tree, the same size with another root, or a consistency proof that fails (computed from the host's tiles) means the host showed two histories, and the fetch fails;
810
- checks each bookmark's attestation against its entry (kind `ref`, the attestation's ObjectRef and repository id) and inclusion proof, reporting one that is missing or does not verify. While logs are new this is a warning, not a refusal.
811
e59512f2 Andy McCall ↶ 812
**`oxy tlog verify [HUB]`** audits a whole log: it accepts the checkpoint as a fetch does, downloads every entry bundle, and requires the entries to make the root the checkpoint signs. It then lists this repository's entries (`--key KEYID` for a key's, `--all` for every one, `--json` for machines).
813
ff0af1ee unknown 814
## 15. Transport
815
816
Every transport carries the same wire protocol (§16). The transport provides the channel; integrity and authenticity of history come from the object model regardless of channel.
817
818
### 15.1 HTTPS (mandatory)
819
096f447c Andy McCall ↶ 820
Every server MUST offer HTTPS with TLS 1.3 at `https://<host>/<repo>/oxy/v1/`. Authentication is by bearer token, mutual TLS, or HTTP signature over a server-issued challenge with the client's identity key. HTTPS is required because it traverses corporate proxies, load balancers, and CDNs; it is the interoperability baseline. A reference server MAY speak plain HTTP behind a TLS-terminating proxy; it MUST NOT be exposed without TLS except on a loopback or otherwise trusted network. Reads MAY be open; a push MUST be authenticated when any credential is configured.
ff0af1ee unknown 821
822
### 15.2 Noise (optional)
823
824
Peer-to-peer and self-hosted deployments MAY use Noise over TCP or QUIC.
825
826
- Pattern `Noise_XXhfs_25519+kyber768_ChaChaPoly_BLAKE2s` (hybrid post-quantum handshake) for first contact, or `IKhfs` when the server's static key is known.
827
- The client's Noise static key MUST be the X25519 key derived from its Ed25519 identity key (RFC 8032 key clamping via the standard birational map), so the transport identity and the signing identity are the same KeyId.
828
- Servers publish their static key fingerprint out of band. Clients pin it on first use and MUST refuse a changed key without explicit `oxy remote trust` action.
829
830
### 15.3 Zero-Crypto Transport (ZCT, restricted)
831
832
ZCT is a plaintext, length-prefixed framing of the wire protocol over TCP or serial, for Thin clients that cannot run ChaCha20 or X25519 at usable speed.
833
834
- ZCT is disabled by default on servers and on clients. A remote MUST be explicitly configured with `transport = "zct"` and clients MUST print a warning on every use.
835
- ZCT provides no confidentiality. Over ZCT the client MUST verify every change signature or seal, every reference attestation, and every ObjectRef it receives; a client incapable of verification (Thin profile without Ed25519) MUST only use ZCT through a trusted proxy on a network it trusts, and this MUST be a documented deployment decision.
836
- Push over ZCT is permitted only with a reference attestation signed by an authorized key; there is no other authentication. Servers MAY refuse push over ZCT entirely.
837
- ZCT MUST NOT be offered on a public network interface by default.
838
839
### 15.4 Proxy
840
841
A **proxy** is a Full-profile `oxy` server that fronts one or more remotes for Thin clients, performing verification, chunking, delta computation, and signing on their behalf. Its security properties are those of the Thin client's trust in it.
842
843
## 16. Wire protocol
844
845
Messages are deterministic CBOR. Over HTTPS each message is a request body with `Content-Type: application/oxy-cbor` and responses are CBOR sequences (RFC 8742) with `Content-Encoding: zstd` when supported. Over Noise and ZCT, messages are framed as `u32 big-endian length || bytes`.
846
847
### 16.1 Endpoints
848
849
| Endpoint | Request | Response |
850
|---|---|---|
75167672 Andy McCall 851
| `GET capabilities` | — | `{ versions: [1], features }`; feature names are `seal`, and, when offered, `lock`; `max_frame` and `transports` are reserved for framed transports. The repository id is all zero for an **unbound** hosted repository, one created empty that adopts the id of the first repository pushed to it; a client MUST NOT clone or fetch from one and MAY push to one, sending its id in the `Oxy-Repo-Id` header, which the server binds before validating the push |
86e682c2 Andy McCall ↶ 852
| `GET refs` | `{ patterns }` | `[ { name, state: ObjectRef, heads, attestation: Signed<Attest> } ]` plus the current `Signed<Policy>` |
ff0af1ee unknown 853
| `POST negotiate` | `{ have: [ChangeHash], want: [ChangeHash], filter }` | stream of `ChangeHash` the client lacks, in dependency order |
854
| `POST fetch` | `{ changes: [ChangeHash], include_blobs: bool, filter }` | stream of `Signed<Change>`, then every `Signed<Attest kind=seal>` covering them, then referenced `blob`/`chunk` objects |
855
| `POST objects` | `{ refs: [ObjectRef] }` | stream of objects |
856
| `GET checkpoint` | `{ heads, filter }` | nearest `Checkpoint` and its objects, and the list of changes to replay |
857
| `POST push` | stream of `Signed<Change>`, objects, seals, then `Signed<Attest kind=ref>` per reference | `{ results: [ { name, ok, reason } ] }` |
858
| `POST lock` | `Signed<Attest kind=lock>` | `{ ok, holder, expires }` |
859
860
### 16.2 Negotiation
861
862
The client sends the heads it has for the requested references and the heads it wants. Because dependencies form a DAG and every state is its heads' closure, the server computes `closure(want) \ closure(have)` by a reverse traversal, stopping at any hash in `have`. For very large histories a server MAY ask the client for the Merkle root of its known-hash set in 2^16-hash-prefix buckets and skip buckets whose roots match. The response is ordered so that no change precedes any of its dependencies.
863
096f447c Andy McCall ↶ 864
### 16.3 Message encodings
865
866
Every message is a deterministic CBOR map. Signatures use the encoding of §5.5; a change or attestation travels as its canonical bytes so the receiver hashes exactly what was signed.
867
868
| Message | Encoding |
869
|---|---|
870
| Signed attestation | `[attest bytes, [Signature]]` |
f0a517d2 Andy McCall ↶ 871
| `Bundle` | `{0: [[change bytes, [Signature]]], 1: [signed attestation], 2: [[type name, object bytes]], 3: [signed policy]}`; changes in dependency order; objects are chunks, blobs, identities, and states only, and include the text blobs of every change in the bundle and their chunks, so a bundle can be read on its own; a push that moves a bookmark also carries the identity that signed the move and, for a delegate, the identity it acts for, so a receiver that has seen no change by that key can check it (oxygenhub#129); key 3 (optional) holds the policies between the receiver's serial and the one sent with a push, ascending; key 4 (optional, only when there are any) holds signed certificates (§14.7) of short-lived keys that signed what the bundle carries or the push's attestation, their keys' and issuers' identities among the objects; a receiver keeps each that verifies with its issuer and refuses a bundle with one that does not |
096f447c Andy McCall ↶ 872
| `capabilities` response | `{0: [versions], 1: repo id bytes(16), 2: [feature names]}` |
83e0677b Andy McCall ↶ 873
| Signed policy | `[policy bytes, [Signature]]` |
0ed150d6 Andy McCall ↶ 874
| `refs` response | `{0: [{0: name, 1: state ref, 2: [heads], 3: signed attestation or null, 4: last serial or null, 5: signed acceptance}], 1: repo id, 2: signed policy or null, 3: [tag], 4: [signed policy]}`; key 4 (optional) lists earlier policies ascending by serial; key 5 (optional, text or null) names the default bookmark, which a clone checks out; the attestation is present only when it describes the current state; an entry's key 5 (optional, only when there is one) is the host's acceptance of that attestation (§14.7); an entry's key 6 (optional) is the attestation's place in the host's transparency log, `[entry number, entry bytes, [proof hash]]`, an inclusion proof against the checkpoint in the response's key 6 (§14.8); the response's key 6 (optional, text) is the host's latest checkpoint, a signed note; a tag is `{0: name, 1: state ref, 2: [heads], 3: signed attestation or null}` |
096f447c Andy McCall ↶ 875
| `negotiate` request | `{0: [have], 1: [want]}`; response `{0: [hashes]}` in dependency order |
b3c5a59a Andy McCall ↶ 876
| `fetch` request | `{0: [hashes], 1: max blob size or null}`; response a `Bundle` without blobs above the size |
877
| `objects` request | `{0: [refs]}`; response a `Bundle` carrying only objects, a blob accompanied by its chunks |
2863456a Andy McCall ↶ 878
| `checkpoint` request | `{0: [heads]}`; response a `Bundle` of objects: the checkpoint of exactly those heads, its payload blob, and the blob's chunks |
879
| `shallow` request | `{0: [heads], 1: depth}`; response `{0: [hashes]}`, the changes within `depth` steps of the heads, dependencies first |
132b884b Andy McCall ↶ 880
| `push` request | `{0: Bundle, 1: [{0: name, 1: old state, 2: new state, 3: force, 4: signed attestation or null}], 2: signed policy or null, 3: [tag]}`; response `{0: [{0: name, 1: ok, 2: reason}]}`, where a result named `policy` reports the policy update and `tags/<name>` each tag |
096f447c Andy McCall ↶ 881
2863456a Andy McCall ↶ 882
The HTTP content type is `application/oxy-cbor`.
096f447c Andy McCall ↶ 883
884
### 16.4 Push validation
ff0af1ee unknown 885
be451356 Andy McCall ↶ 886
On push, the server MUST, in order: verify that every change has a valid signature or seal from a key embedded in policy and satisfies the author-or-delegate rule (§14.4), verify the attestation for each reference, check that `old_state` matches its current state (compare-and-swap), check policy rules for the reference including `allow_conflicts`, check that every dependency of every change is either included or already stored, and only then update references atomically. A failure rejects the whole push. A refusal's reason names the reference and, for a failure of the server's own storage, the step that failed; the reference server also logs every refusal and every failed request with its repository and operation, and answers a storage failure with 500 rather than 400.
f8c69576 Andy McCall ↶ 887
cb8b8082 Andy McCall ↶ 888
**A push in parts.** A push may carry no reference update and no tag: its changes and objects alone, which the server validates as above and stores, reporting the outcome as one result named `objects` (`ok`, and as its reason how many changes it stored or why it refused them). A client sends a long history this way, in dependency order, in parts of about `push.batch-bytes` (default 256 MiB) of changes and their texts and blobs, and the last part carries the reference updates, so that neither side holds the whole history at once and no request passes the server's body limit. A part already stored is harmless if the push stops, and a server MAY collect its changes under §21 if no reference ever reaches them.
889
8c30d814 Andy McCall ↶ 890
**Deleting a reference.** An update whose `new_state` is the zero ObjectRef deletes the bookmark (`oxy push REMOTE --delete NAME`). The bookmark MUST exist at `old_state`. A server MUST refuse to delete its default bookmark: in a bare repository that is the host's to name (the reference server's is the repository's configured bookmark, a forge's its default-bookmark setting), and in a non-bare repository it is the checked-out bookmark. The update is attested like any other: under a policy the attestation is required and its key MUST be authorized for the reference. The deletion's attestation stays the reference's latest, so an older attestation cannot be replayed to bring the bookmark back. The first attestation of a reference the remote does not list takes the client's clock in milliseconds as its serial, which is past the serial of any deleted namesake; later ones increment it. Changes of a deleted bookmark stay until unreachable (§21).
ff0af1ee unknown 891
892
## 17. Partial, sparse, and shallow repositories
893
894
A repository records a **filter** in `.oxy/config`:
895
896
```
897
[filter]
898
paths    = ["src/", "docs/"]      # sparse: only these subtrees are materialized and blobs fetched
899
max-blob = "1MiB"                 # blobs larger than this are lazy
900
depth    = 100                    # shallow: history beyond this many changes from heads is not stored
901
```
902
7421309f Andy McCall ↶ 903
- **Sparse** repositories hold the full change graph but only materialize the listed paths: the working copy contains nothing else, files outside the filter are neither reported nor removed, and changes touching other paths are stored and applied to the graph. A sparse clone sets `max-blob` to zero unless a size is given, so blobs outside the filter are fetched only if the filter later widens (§13.5). `oxy sparse PATH..` changes the filter of a clean working copy and re-checks it out.
ff0af1ee unknown 904
- **Lazy blobs** are fetched via `objects` on first read. `oxy status` and `oxy commit` MUST NOT need them.
2863456a Andy McCall ↶ 905
- **Shallow** repositories hold a checkpoint of their bookmark's heads (obtained with the `checkpoint` message) and the changes within `depth` dependency steps of those heads (the `shallow` message lists them). The *shallow boundary* is the set of dependencies of stored changes that are not stored (`.oxy/shallow`). Dependencies into the boundary are accepted as well-formed, negotiation stops at the boundary (the remote is assumed to hold it), and the log shows every change from the checkpoint's metadata. `oxy deepen N` fetches `N` more levels below the boundary; when nothing remains the repository is no longer shallow.
ff0af1ee unknown 906
- Thin clients (§22) always use `depth = 0`: a checkpoint plus zero history.
907
908
Servers MUST honour `filter` on `negotiate`, `fetch`, and `checkpoint`.
909
910
## 18. Locks for unmergeable files
911
912
Opaque files (models, textures, binary documents) cannot be merged. Paths matching a `lockable` attribute in `.oxyattributes` are read-only in the working copy until locked.
913
914
- `oxy lock <path>` sends a `lock` attestation to the remote configured as `lock-server`. The server grants it if unlocked or expired, and records the holder.
915
- A push that replaces a lockable node's blob MUST be rejected by the server unless the pushing key holds the lock.
916
- Locks expire (default 7 days) and can be broken by a policy root key.
917
- Locks are advisory in repositories with no `lock-server`.
918
919
## 19. Repository configuration files
920
921
| File | Purpose |
922
|---|---|
d4dbab97 Andy McCall ↶ 923
| `.oxyignore` | Ignore rules. Gitignore syntax and semantics, per directory, plus `.oxy/info/ignore` and the global ignore file. A directory's `.gitignore` and `.ignore` files are read too, with lower precedence than its `.oxyignore` (a `!` rule there re-includes what they exclude), so a project coming from Git leaves out what Git did. Rules apply to untracked paths only: a tracked path stays tracked, and its edits are seen, until it is deleted or removed from the state. |
cdb5beba Andy McCall ↶ 924
| `.oxyattributes` | Per-path attributes: `binary` (force opaque), `text` (text unless NUL bytes), `eol=lf|crlf` (store LF; check out as given), `nocompress` (chunks packed raw), `lockable` (read-only until `oxy lock`), `filter=<name>`. Gitattributes syntax: a pattern without `/` matches basenames below the file's directory, one with `/` is anchored to it; later rules and deeper files override. The attributes in force for a change are those in the files being committed. |
ff0af1ee unknown 925
| `.oxy/config` | Repository-local settings, TOML. |
c895bc52 Andy McCall ↶ 926
| `~/.config/oxy/config.toml` | User settings, the same keys as `.oxy/config`: `author.name`, `author.email`, `author.key`, `author.signer` (the key that signs and seals in place of `author.key`, which must list it as a delegate: an agent's own key; or `keyless`, for fresh keys the hub certifies under an agent's grant, §14.7, from the grant token `oxy auth grant` keeps; also `$OXY_SIGNER`), `keyless.hub` (the hub that certifies them; default the default remote's), `core.sign`, and `remote.<name>.token`; also `auth.helper`. Overridden by `$OXY_HOME/config.toml` when `$OXY_HOME` is set. |
ff1f62fe Andy McCall ↶ 927
| `~/.config/oxy/credentials.toml` | Tokens from `oxy auth login`, one `[scheme://host[:port]]` section per hub with `token`, `account` and `ca`. Written owner-only; ignored when others can read it. With `auth.helper` set, the token is kept by that helper (Git's credential-helper protocol, user name `oxy-token`) and the section holds the rest. |
fce44ae9 Andy McCall ↶ 928
ff1f62fe Andy McCall ↶ 929
Settings resolve in order: the repository file overrides the user file, which overrides environment variables (`OXY_AUTHOR_NAME`, `OXY_AUTHOR_EMAIL`, `OXY_TOKEN`) and built-in defaults. A remote's token and CA without `remote.<name>.token` or `.ca` come from `OXY_TOKEN` and `OXY_CA`, then from `credentials.toml` for the remote's hub, then from a remote on the same hub in an enclosing repository's `.oxy/config`. A client MUST NOT send a token over plain `http` except to a loopback address, and MUST NOT forward it across a redirect. `oxy config` writes the repository file; `oxy config --user` writes the user file. Secret keys live beside the user file in `keys/`, one `<KeyId>.secret` and one `<KeyId>.identity` per key, so any repository the user works in can store the identity object when it first signs.
ff0af1ee unknown 930
931
Filters (`filter=<name>`) are clean/smudge programs configured in user or repo config, as in Git. Implementations MUST refuse to run filters defined in `.oxy/config` of a freshly cloned repository until the user approves them.
932
933
## 20. Hooks and server policy
934
6d3eefbc Andy McCall ↶ 935
Client hooks live in `.oxy/hooks/` and are never cloned. Events: `pre-commit`, `semantic-deps`, `post-commit`, `pre-push`, `post-fetch`, `post-checkout`. A hook is an executable named after its event, run in the workspace root with `OXY_EVENT` and `OXY_ROOT` set; it receives one JSON object on stdin and blocks the operation by non-zero exit for `pre-*` events, whose stderr becomes the error message. Payloads: `pre-commit` `{event, message, paths: [{path, status}]}`; `post-commit` `{event, change_id, hash, message}`; `pre-push` `{event, remote, bookmark, heads}`; `post-fetch` `{event, remote, bookmarks: [{name, heads}]}`; `post-checkout` `{event, heads, written, removed}`. Hashes and ids are hex strings.
ff0af1ee unknown 936
6d3eefbc Andy McCall ↶ 937
The `semantic-deps` hook runs during `oxy commit` after the working copy is snapshotted and before the change is finalized. It receives `{ change_id, touched_paths, candidate_deps: [ { change_id, hash, paths } ] }`, where `candidate_deps` are the changes the new change already references plus the heads it sits on, each with the paths it touched, and it returns `{ deps: [hash hex] }` to add to `deps(C)`; hashes not in the base state are ignored. It runs only when `hooks.semantic-deps = true` is set in `.oxy/config`, which is never set by clone.
ff0af1ee unknown 938
86e682c2 Andy McCall ↶ 939
Server policy is the trust policy (§14.5) plus server configuration for per-reference rules: protected patterns that additionally require CI status, forbid conflicted heads, forbid multiple heads, or forbid force (non-fast-forward) updates. A force update is a `ref` attestation whose `old_state` heads are not all in the closure of `new_state`.
ff0af1ee unknown 940
941
## 21. Garbage collection
942
767ac6cd Andy McCall 943
An object is **reachable** if it is referenced, directly or transitively, from any bookmark, tag, policy, the current operation heads, any operation newer than `core.op-retention`, any workspace's current working-copy change, the working-copy changes of operations newer than `core.snapshot-retention` (default 7 days; `oxy gc --expire-snapshots` keeps none but the current ones, oxygen#98), or any checkpoint of a reachable state. A file that was only ever in a working copy, never committed, so goes once the snapshots that held it expire. `oxy gc` first deletes the packs and loose objects earlier repacks retired over an hour ago (§13.4), then deletes unreachable objects that are older than `core.gc-grace` (default 14 days, to protect concurrent writers) and repacks; what this repack replaces is retired in turn, for a later collection. Tombstone atoms are never collected independently of their change. Divergent changes are reachable if any of their variants is. A change's text blobs and their chunks are reachable with it (§7.4). A seal is reachable if any change it covers is reachable, and reference attestations are reachable while they are the latest for their reference or are within `core.op-retention`.
ff0af1ee unknown 944
387335ff Andy McCall ↶ 945
### 21.1 Integrity check
946
6f8aeb99 Andy McCall ↶ 947
`oxy fsck` reads everything and changes nothing. It checks (1) every stored object: header, length, ObjectRef, and that the body decodes as its type; (2) the operation log: every operation reachable from the head through all parents is present and decodes, and no operation's clock is earlier than its parent's (a warning); (3) the current view's bookmarks, tags, remote bookmarks, workspaces and policy, and the dependency closure of every change they name, with each change's author identity, text blobs, blobs and chunks, and each committed change's signature or seal (§14.4); (4) that every bookmark forms a valid state, that every checkpoint restores exactly the state of the changes it names (§11), and that reference and tag attestations carry valid signatures. A change beyond the shallow boundary (§17) may be absent; a blob absent from a partial clone (`filter.max-blob` or a sparse filter) is a warning, not an error. Each problem names the object it concerns. The exit status is 1 when any error is found and 0 otherwise; `--json` gives the counts and problems.
387335ff Andy McCall ↶ 948
3e5772cc Andy McCall ↶ 949
### 21.2 Doctor
950
951
`oxy doctor` checks what commonly stands between a user and a working repository and says, for each problem, what to do next: the operation log opens and no lock was left by a dead process; the signing identity exists, its secret key file is present, and it has not expired (a warning within 30 days); the author name and email are not the defaults; and for each remote, that plain `http://` is only used on loopback, a token is set, the CA file exists, and (unless `--offline`) the remote answers and is this repository or an unbound one. Failures of remote requests anywhere in `oxy` carry the same next step (an untrusted certificate, a rejected or insufficient token, no repository at the URL, nothing listening, an unknown host, a timeout). The exit status is 1 when a check fails; `--json` gives the checks.
952
ff0af1ee unknown 953
## 22. Client profiles and retro hardware
954
955
| | Full | Lite | Thin |
956
|---|---|---|---|
957
| Minimum hardware | 1 GiB RAM, current desktop or server | 16 MiB RAM (486, 68030, ARMv5, RISC-V RV32) | 68000 @ 8 MHz, 1 MiB RAM, block storage (Amiga 500+, Atari ST, Macintosh Plus/SE, early 386 DOS) |
958
| Local history | Full or partial | Partial (shallow + lazy) | None: one checkpoint |
959
| Change graph operations | Yes | Yes | No |
960
| BLAKE3 verification | Yes | Yes | Yes (chunks and objects received; ~150 KiB/s on 68000) |
961
| Ed25519 verification | Yes | Yes | MAY (~2 s per signature on 68000; used for attestations only) |
962
| ML-DSA-65 | Sign + verify | Verify; sign MAY | No |
963
| Chunking, delta computation, signing | Local | Local | Proxy |
964
| Transports | HTTPS, Noise, ZCT | HTTPS, Noise, ZCT | ZCT to proxy; serial or TCP |
965
| Working copy | Full | Sparse | Sparse subtree, whole-file edits |
966
967
A Thin client fetches a checkpoint for a sparse subtree, verifies each object's ObjectRef, and writes files. To commit, it sends the full content of modified files with their previous blob refs to the proxy; the proxy computes edits, assigns ChangeIds, signs or seals with its own key, which the user has listed in `delegates` on their identity (§14.2), and pushes. The threat model for Thin clients is therefore "the proxy is trusted", and `oxy` documentation MUST state this without qualification.
968
969
Machines under 512 KiB of RAM are out of scope; a serial terminal to a Lite or Full machine is the recommended path.
970
971
## 23. Git interoperability
972
973
`oxy git import <repo>` and `oxy git export <repo>` provide a lossless-enough bridge; `oxy git fetch/push` operate against a Git remote directly.
974
d6e0d5b8 Andy McCall ↶ 975
- **Import.** Each Git commit becomes one change whose edits are the diff between the union of its parents' states and its tree, so a merge commit becomes a resolution exactly when the parents conflict in the change model, and an ordinary change otherwise. The commit's author and hash are recorded as `Git-Author` and `Git-Commit` trailers; the author key is the zero KeyId, since nobody can sign on the original author's behalf. The exception is the importing identity's own history: with `git.claim` set (`oxy git import --claim`, `oxy git fetch --claim`), a commit whose Git author email is that identity's email, compared without case, gets its key as author, and is sealed by it on push like any of its changes; every other commit keeps the zero key. Submodule entries are skipped. Renames are detected on import exactly as for a snapshot (§7.1, similarity ≥ 50%), then tracked exactly. The Git commit's `Change-Id` trailer, if present and 128 bits, becomes the ChangeId; otherwise one is derived as BLAKE3-128 of the Git commit hash, so re-import is idempotent. New files' and directories' NodeIds are derived too, as BLAKE3-128 of the commit hash, the node kind and the path, so importing the same history twice, anywhere, gives the same changes. Note that a change's own closure is what it depends on, not the Git snapshot: the imported bookmark's state equals the Git tree, but checking out an individual imported change yields only the files it and its dependencies touch.
a87c7b9d Andy McCall ↶ 976
- **Directories.** Git gives a directory no identity: its path is all it is. An import therefore never makes a second node for a directory path: when a commit needs a directory where the parent's state has none, and the parent's history has a directory last placed at that (parent, name) and now placed nowhere, the change brings that node back with a `Move` from its last placement (the newest by operation key when there are several). Otherwise lines of history that placed files in the directory before it was deleted and after it came back would place them in two different directories, a conflict Git never had. Likewise a node the commit keeps is in exactly the one place its tree names: any other live placement it has in the parents' union (one left under a directory another line removed is out of sight, and no diff of the tree would reach it) is retired by the commit's change, and so is every live placement of a node the union cannot show at all (an orphan, whose only placements are under removed directories): Git's tree has no path for it.
ff0af1ee unknown 977
- **Export.** Each state along a bookmark becomes a Git commit whose tree is the materialized state. The ChangeId is written as a `Change-Id:` trailer and the ChangeHash as `Oxy-Hash:`. Conflicted states are exported with §9.5 markers and an `Oxy-Conflicted: true` trailer. Signatures are not exported; Git commits MAY be additionally signed with the Ed25519 key via SSH signature format.
30ea9c1e Andy McCall ↶ 978
- **Fetch and push against a Git remote.** `oxy git fetch URL` keeps a bare mirror under `.oxy/git/` for the remote, fetches the branch into it, and imports the commits the bookmark does not have yet, seeding the import with the commits already known: an imported commit maps to its change, and a commit this repository exported maps to the bookmark state it was made from (so a Git commit on top of exported work is based on everything that work contained, including changes Git never saw). The bookmark moves to the union of its heads and the new changes, like a pull; the working copy follows when it was clean. A fetched state whose previous state was itself a Git tree is remembered as equal to the fetched tip. `oxy git push URL` refuses until the remote's tip is known, exports the bookmark's later states as commits chained onto that tip, and pushes the branch, so the push is always a fast-forward.
4dee277c Andy McCall ↶ 979
- **Submodules.** A Git gitlink entry (mode `160000`) imports as a `subrepo` node with a Git pin (§7.8) and the matching `.gitmodules` entry becomes a `.oxymodules` entry with `branch` set to the submodule's configured branch when there is one. Export writes a Git pin back as a gitlink, keeps a tracked `.gitmodules` as is, and otherwise derives `.gitmodules` from `.oxymodules`; an Oxygen pin has no Git form and is omitted from the exported tree, which the export report counts.
62794ecc Andy McCall ↶ 980
- **Merges of opaque files.** When the union of a merge commit's parents holds several live blobs for a file or submodule, the merge's change replaces each of them with the commit's blob and is a resolution: Git's merge is the authority, unlike a working copy, where a candidate on disk is not yet a decision.
7eac74ab Andy McCall ↶ 981
- **Round trip.** Import followed by export MUST reproduce every tree hash. Export followed by import MUST reproduce every ChangeId. `oxy git verify GIT_DIR` checks an import commit by commit: for each commit, the state formed by the changes imported for it and all its ancestors must materialize to exactly that commit's tree (content, executable bits, symlink targets, submodule pins; `.oxymodules` is allowed beside submodules). `--last N` rebuilds only the last N commits, following the others, to look again at where an import went wrong.
ff0af1ee unknown 982
983
## 24. Command-line interface
984
985
Commands mirror Jujutsu where semantics match, so that muscle memory transfers.
986
987
```
e02bf0cd Andy McCall ↶ 988
oxy init [PATH] [--bare]            oxy clone URL [--sparse PATH].. [--depth N] [--max-blob SIZE]
50a046e4 Andy McCall ↶ 989
oxy status [--json]                 oxy diff                     oxy prompt [--format F]
990
oxy commit -m MSG [--depends-on CHANGE]..
0299076f Andy McCall ↶ 991
oxy log [-r REVSET] [-n N] [-v] [-L S,E:PATH]   oxy show CHANGE | -r REVSET [--stat]
6a81dbab Andy McCall ↶ 992
oxy describe [CHANGE] [-m MSG]      oxy amend [-m MSG] [--into CHANGE]
2c9ddac3 Andy McCall ↶ 993
oxy split CHANGE [PATH]..           oxy abandon CHANGE | -r REVSET [--yes]   oxy resolve
1c2bd9ab Andy McCall ↶ 994
oxy checkout [CHANGE].. [--discard|--carry] oxy undo                     oxy op log
50a046e4 Andy McCall ↶ 995
oxy tag create NAME [CHANGE].. [-m MSG]   oxy tag list
86fe29fd Andy McCall ↶ 996
oxy bookmark create|set|delete|list|switch  oxy merge BOOKMARK
e266b7d4 Andy McCall ↶ 997
oxy commit [PATH].. [--interactive]         oxy squash CHANGE --into DEST   oxy rebase CHANGE --onto HEADS.. [--into-working-copy]
86fe29fd Andy McCall ↶ 998
oxy submodule add|update|status|remove      oxy blame PATH               oxy bisect start|good|bad|run|reset
50a046e4 Andy McCall ↶ 999
oxy workspace add|list|forget       oxy sparse [PATH]..          oxy deepen [N]
1000
oxy fetch [REMOTE]                  oxy pull [REMOTE]            oxy push [REMOTE] [--force]
1001
oxy remote add|remove|list          oxy serve                    oxy lock|unlock PATH
7aa2dc93 Andy McCall ↶ 1002
oxy identity create|show|export|rotate|renew|revoke|delegate|sign-challenge
2c9ddac3 Andy McCall ↶ 1003
oxy policy init|show|grant|revoke|root|sign                      oxy verify [CHANGE | -r REVSET]
d82e4409 Andy McCall ↶ 1004
oxy seal [CHANGE].. | --proposal URL  seal your unauthenticated and anonymous changes (§14.4), or a hub proposal's changes as your approval
1005
oxy proposal show|land URL          hub proposals from a client (hub spec §7)
62794ecc Andy McCall ↶ 1006
oxy git import|export|fetch|push|verify   (fetch and push go through a bare mirror under .oxy/git/)
b689779e Andy McCall ↶ 1007
oxy checkpoint                      oxy gc [--now] [--expire-snapshots]               oxy config [--user] KEY VALUE
c6aa5bd9 Andy McCall ↶ 1008
oxy root                            oxy cat PATH [-r CHANGE]     oxy version
387335ff Andy McCall ↶ 1009
oxy fsck [--json]                   check integrity (§21.1)
3e5772cc Andy McCall ↶ 1010
oxy doctor [--offline] [--json]     check identity, config and remotes (§21.2)
d42369ab Andy McCall 1011
oxy completions SHELL               shell completion script     oxy man [--dir DIR]   man pages
c6aa5bd9 Andy McCall ↶ 1012
oxy hunks                           oxy discard PATH.. | --hunks N,M
ff0af1ee unknown 1013
```
1014
c6aa5bd9 Andy McCall ↶ 1015
**Machine interface.** `status`, `log` (with `--path`), `show`, `diff`, `hunks`, `blame`, `resolve`, `op log`, `bookmark list`, `remote list`, `fetch`, `pull`, `push`, `submodule status`, `bisect status`, `verify`, `policy show`, `root` and `version` take `--json` and then print exactly one JSON document on stdout, with ids as hex strings and times in milliseconds since the epoch; errors go to stderr with a non-zero status. `commit --hunks N,M` and `discard --hunks N,M` take hunk numbers as `hunks` prints them. Editors and other tools MUST use these rather than reading `.oxy/`.
1016
50a046e4 Andy McCall ↶ 1017
Change arguments accept a ChangeId prefix, a ChangeHash prefix (with `#`), a bookmark or tag name, and `@` for the working-copy change. Ambiguous prefixes are an error.
ff0af1ee unknown 1018
86fe29fd Andy McCall ↶ 1019
Reserved for a later version, with these names: `oxy new`, `oxy edit`, `oxy redo`, revsets (`log -r`), and `oxy remote trust` (with the Noise transport, §15).
50a046e4 Andy McCall ↶ 1020
ff0af1ee unknown 1021
## 25. Performance targets
1022
1023
Measured on the Full profile, warm cache, NVMe storage, 8-core x86-64, with a 20 GiB repository of 2 million files and 500 000 changes.
1024
1025
| Operation | Target |
1026
|---|---|
1027
| `oxy status` (no changes, fsmonitor) | < 50 ms |
1028
| `oxy status` (no changes, stat crawl) | < 2 s |
1029
| `oxy commit` touching 10 files | < 100 ms |
1030
| `oxy log -r main` first page | < 100 ms |
1031
| Checkout between adjacent checkpoints | proportional to changed bytes, ≥ 500 MiB/s |
1032
| Full clone over 1 Gbit/s | line rate; chunk verification ≥ 1 GiB/s per core |
1033
| Sparse clone of a 50 MiB subtree | < 5 s |
1034
| Rebase of 100 commuting changes | < 200 ms, zero object rewrites |
1035
| Resident memory, `status` | < 200 MiB |
1036
1037
Implementations SHOULD ship a benchmark harness that reproduces this table.
1038
1039
## 26. Conformance and test vectors
1040
1041
A conforming implementation MUST pass the `oxy-conformance` suite, which includes:
1042
1043
- Encoding vectors: canonical CBOR and ObjectRef for every object type.
1044
- Signature vectors: composite signatures, including stripped-half and wrong-KeyId negatives.
1045
- Graph invariance: for random change sets, all permutations of application yield identical graphs and checkpoints.
1046
- Conflict shapes: each shape in §9 with expected materialization and expected `status --json`.
1047
- Rewrite propagation: amend of a change with dependents, with and without correspondence.
1048
- Protocol: negotiate/fetch/push against a reference server, including rollback and unauthorized-attestation rejection.
1049
- Adversarial chunking: inputs with no gear boundaries and inputs with boundaries at every byte.
1050
- Git round trip on a corpus of public repositories.
1051
50a046e4 Andy McCall ↶ 1052
### 26.1 Reference implementation status
1053
1054
The reference implementation (`crates/oxy-core`, `crates/oxy-server`, the `oxy` binary) as of this draft:
1055
1056
| Area | Status |
1057
|---|---|
1058
| Objects, hashing, deterministic CBOR (§5) | Implemented; format 2 repositories, migration from format 1 |
1059
| Change graph, commutation, conflicts and resolution (§6–§9) | Implemented, including order, delete/insert, tree, replace, and mode conflicts; workspace conflicts (§9.7) and exit status 3 are not detected |
728da0f9 Andy McCall ↶ 1060
| Rewrites (§8.4, §8.5) | Implemented: describe, amend (also into older changes), split (any change), abandon, squash, rebase; orphaned references re-anchored |
fbda43e7 Andy McCall ↶ 1061
| Bookmarks, per-workspace current bookmark, merge by union (§10.1) | Implemented |
4dee277c Andy McCall ↶ 1062
| Subrepositories with Oxygen and Git pins, `.oxymodules`, Git gitlink import and export (§7.8, §23) | Implemented |
957fee30 Andy McCall ↶ 1063
| Partial and interactive commits (§12.3) | Implemented |
1064
| Blame and bisect (§27) | Implemented |
50a046e4 Andy McCall ↶ 1065
| Operation log, undo (§12.1) | Implemented; `redo` reserved |
1066
| Snapshots and the stat cache (§12.3, §12.4) | Implemented; no file-system monitor yet |
1067
| Storage: loose objects, packs, GC, checkpoints, batching (§11, §13, §21) | Implemented; pack indexes are read fully rather than mapped |
1068
| Identity, composite signatures, seals, attestations, policy, lifecycle (§14) | Implemented; `lock` attestations reserved |
1069
| HTTPS transport and endpoints (§16) | Implemented, HTTP only; Noise and ZCT (§15) reserved; negotiation is exact (no Merkle buckets); `refs` takes no `patterns` |
1070
| Partial, sparse, shallow (§17) | Implemented |
1071
| Locks (§18) | Local advisory locks only; server locks and the `lock` endpoint reserved |
1072
| Configuration, attributes, hooks (§19, §20) | Implemented (`.oxyattributes`: binary, text, eol, nocompress, lockable) |
1073
| Thin profile and proxy (§22) | Reserved |
1074
| Git interoperability (§23) | Implemented: import, export, fetch, push, rename detection |
1075
| Conformance suite (§26) | Partial: encoding, signature, permutation, conflict, rewrite, protocol, chunking and Git tests exist as unit and integration tests in the crates; a standalone vector suite is not published yet |
1076
86fe29fd Andy McCall ↶ 1077
## 27. Inspection: blame and bisect
1078
1079
**Blame.** Every line of a file in a state is an atom identified by the change that introduced it (§7.3), so `oxy blame PATH` is a lookup, not a heuristic: for each line it prints the introducing change's ChangeId, author, and date, and `--json` gives the ChangeHash and atom index. A line inside a conflict region is attributed to the side that introduced it. Blame follows renames for free, since the node survives them, and it MUST cost no more than materializing the file.
1080
83d61e80 Andy McCall ↶ 1081
**Line history.** For the same reason the history of a range of lines, such as a function, is exact: `oxy log -L START,END:PATH` lists every change that inserted or deleted an atom on some path in the file graph from the range's first line to its last (the lines' own atoms and those deleted from between them), newest first, with lines numbered as in the working copy's base. Changes elsewhere in the file are not in it, and edits within the range stay in it however the lines moved around them. Hosts use the same query for a function's history (tree-sitter gives the range).
1082
86fe29fd Andy McCall ↶ 1083
**Bisect.** A regression is a change, and the candidates lie between a known-good state and a known-bad one: `candidates = closure(bad) \ closure(good)`. `oxy bisect start`, then `oxy bisect good CHANGE..` and `oxy bisect bad CHANGE..` (either may name a bookmark), orders the candidates so that every prefix is dependency-closed and checks out `good ∪ prefix` at the midpoint. Each further `good`/`bad` halves the remaining range, and the search ends by naming the first change whose inclusion makes the state bad; `oxy bisect run CMD` automates it with the command's exit status (0 good, 125 skip, anything else bad). `oxy bisect reset` restores the working copy from before the search. Because prefixes are closed under dependencies, every tested state is well-formed; a change that cannot be tested in isolation because it commutes with the culprit is simply ordered after it. Bisect state lives under `.oxy/bisect/` and is not versioned.
1084
1085
## 28. Open questions
ff0af1ee unknown 1086
1087
1. **Encryption at rest.** Per-repository XChaCha20-Poly1305 with a key hierarchy is reserved for format version 2; object headers already leave room. Convergent encryption is rejected because it leaks equality.
1088
2. **Word- or character-granularity atoms** for prose and minified files. Line atoms are fixed for v1.
50a046e4 Andy McCall ↶ 1089
3. **Set reconciliation** at the scale of tens of millions of changes may need an IBLT-based negotiator; the Merkle bucket scheme in §16.2 is the v1 answer, and the reference implementation's exact negotiation is sufficient below that scale.
1090
4. **Workspace conflicts (§9.7)** on case-insensitive file systems are specified but not yet detected by the reference implementation.
1091
5. **Server-side locks (§18)** need the `lock` endpoint and attestation to be exercised by an implementation before their wording is final.
ff0af1ee unknown 1092
1093
---
1094
0f93f49c Andy McCall ↶ 1095
*Changelog.* Draft 0.8r (2026-09-29): change format version 3, text blobs as the lines' bytes (§6.1, §7.4); `oxy gc` repacks into one pack at a higher compression level (§13.4); an import brings a deleted directory back rather than making a second node for its path, and retires a kept node's other placements (§23); graph keys at commit, lazy text in history-only loads, and a clean `status` from the manifest (§12.4); a parent checkout never moves a subrepo's bookmark backwards, and the pin is read from the nested working copy (§7.8); `bookmark create --upto/--only` (§10.1); the working-copy lock covers every command that writes the working copy; a workspace changed by two concurrent operations is an error (§12.1). Draft 0.8q (2026-09-29): change format version 2 stores large inserts as text blobs, so a change does not grow with the files it adds (§6.1, §7.4); no object may exceed 1 GiB (§13.2). Draft 0.8p (2026-09-29): Git import derives new NodeIds from the commit, making imports deterministic (§23). Draft 0.8o (2026-09-29): execution receipts, the `receipt` object and attestation (§14.6). Draft 0.8n (2026-09-29): line history, `oxy log -L` (§27). Draft 0.8m (2026-09-29): resolving replace conflicts (§9.3). Draft 0.8l (2026-09-28): deleting a reference on a remote; first serials from the clock (§16.4). Draft 0.8k (2026-09-28): the manifest records graph keys, so unchanged files are neither read nor rendered (§12.4). Draft 0.8j (2026-09-28): checkpoint payload version 2, file graphs by reference (§11).
afaba5f1 Andy McCall ↶ 1096
1097
Draft 0.8i (2026-09-28): portability rules for exec bits and path names (§7.7); found by verifying a real Git import commit by commit: deletions depend on the inserters of the lines right after them, and replacements on the change that set their old blob (§8.1); replace records are superseded causally, so a reverted binary keeps its blob (§9.3); merges resolve opaque conflicts to the commit's blob, and `oxy git verify` (§23). Draft 0.8h (2026-09-28): `oxy fsck` (§21.1); durability rules and the operation head as commit point (§13.6); the operation head moves by compare-and-swap and concurrent operations are merged (§12.1); `oxy doctor` (§21.2). Draft 0.8g (2026-09-27): delegates of a rule's identities may sign reference attestations (§14.5); local-path pushes bind unbound repositories like HTTP ones. Draft 0.8f (2026-09-27): challenge signatures (§14.3) for hosts. Draft 0.8e (2026-09-27): anonymous changes (zero author) are authenticated by a seal from any key, so imported and pre-identity history can be pushed under a policy (§14.4). Draft 0.8d (2026-09-27): unbound hosted repositories adopt the first pusher's id (§16.1). Draft 0.8c (2026-09-27): bare repositories (`oxy init --bare`, §12.2: no workspace, no working copy; hosting) and the reference server as an embeddable library (§15.1). Draft 0.8b (2026-09-27): the machine interface for editors (§24): `root`, `cat`, `version`, `hunks`, `discard`, and `--json` on the reporting commands.
c6aa5bd9 Andy McCall ↶ 1098
1099
Draft 0.8a (2026-09-27): every Draft 0.8 addition is implemented in the reference implementation (§26.1). Clarifications from doing so: an unregistered nested repository is left alone and a tracked subrepo whose directory is absent keeps its pin (§7.8); Oxygen pins are omitted from Git export (§23).
728da0f9 Andy McCall ↶ 1100
1101
Draft 0.8 (2026-09-27): subrepositories with Oxygen and Git pins (§7.8, §23); squash and rebase defined as history edits (§8.5); multiple bookmarks per repository with a per-workspace current bookmark and `oxy merge` (§10.1); partial and interactive commits (§12.3); blame and bisect (§27); open questions moved to §28.
86fe29fd Andy McCall ↶ 1102
1103
Draft 0.7 (2026-09-27): review pass. `oxy status --json` shape (§9.5); stat crawl required and file-system monitor recommended (§12.4); capabilities aligned with the implementation (§16.1); §24 lists the implemented command line and names the reserved commands; new §26.1 reference implementation status; §28 records the open workspace-conflict and lock items.
50a046e4 Andy McCall ↶ 1104
1105
Draft 0.6s (2026-09-27): the manifest as a stat cache with a racy-stat window (§12.4).
b7f102fc Andy McCall ↶ 1106
1107
Draft 0.6r (2026-09-27): batched object writes (§13.4).
8f10a07a Andy McCall ↶ 1108
1109
Draft 0.6q (2026-09-27): pack index validation rules (§13.1).
2b872d07 Andy McCall ↶ 1110
1111
Draft 0.6p (2026-09-27): `oxy git fetch` and `oxy git push` against a Git remote through a mirror, with incremental import seeded by imported and exported commits (§23).
30ea9c1e Andy McCall ↶ 1112
1113
Draft 0.6o (2026-09-27): rename detection on snapshot and Git import records moves of the existing node (§7.1, §23).
6e723b81 Andy McCall ↶ 1114
1115
Draft 0.6n (2026-09-27): identity lifecycle (§14.2): every identity edit is a rotation that supersedes rather than revokes, expiry bounds authenticated changes, revocations recorded locally; policy proposals for multi-signer updates, chain-aware authorization, bootstrap rule, and policy history on `refs` and in push bundles (§14.5, §16.3).
28fc23d7 Andy McCall ↶ 1116
1117
Draft 0.6m (2026-09-27): orphaned references on rewrite are reified by re-anchoring rather than refused (§8.4).
6706bdcf Andy McCall ↶ 1118
1119
Draft 0.6l (2026-09-27): amend into older changes and split (§8.4).
23b217d7 Andy McCall ↶ 1120
1121
Draft 0.6k (2026-09-27): hook execution model and JSON payloads (§20).
6d3eefbc Andy McCall ↶ 1122
1123
Draft 0.6j (2026-09-27): `.oxyattributes` semantics (§19).
cdb5beba Andy McCall ↶ 1124
1125
Draft 0.6i (2026-09-27): secondary workspaces and their `.oxy` file (§12.2).
4e88e774 Andy McCall ↶ 1126
1127
Draft 0.6h (2026-09-27): tags in views, on the wire, and under policy (§5.5, §10.2, §16.3).
132b884b Andy McCall ↶ 1128
1129
Draft 0.6g (2026-09-27): shallow repositories as checkpoint plus history window, the `checkpoint` and `shallow` messages, deepening (§16.3, §17).
2863456a Andy McCall ↶ 1130
1131
Draft 0.6f (2026-09-27): sparse checkout semantics (§17).
7421309f Andy McCall ↶ 1132
1133
Draft 0.6e (2026-09-27): lazy blobs: the `fetch` filter, the `objects` message, and the absent-blob record (§13.5, §16.3).
b3c5a59a Andy McCall ↶ 1134
1135
Draft 0.6d (2026-09-27): §11 checkpoint format as implemented, with the loading rule.
cfb7d414 Andy McCall ↶ 1136
1137
Draft 0.6c (2026-09-26): §23 import defines the base as the union of parent states, records author and commit trailers, skips submodules, and states the closure-versus-snapshot difference.
315167f3 Andy McCall ↶ 1138
1139
Draft 0.6b (2026-09-26): §8.4 allows refusing a rewrite whose orphaned references have no specified conflict shape, defines removal, and notes the signature consequence.
f7769883 Andy McCall ↶ 1140
1141
Draft 0.6a (2026-09-26): §13.4 fixes the pack and index file formats.
d02e545d Andy McCall ↶ 1142
1143
Draft 0.6 (2026-09-26): trust policy propagation over the wire, the `policy` signing domain, per-remote downgrade protection, the bootstrap rule for unverified remotes, and the rule that policies govern only changes added after them.
83e0677b Andy McCall ↶ 1144
1145
Draft 0.5b (2026-09-26): §19 defines the user configuration file's keys, the precedence of repository over user over environment, and the key directory layout.
fce44ae9 Andy McCall ↶ 1146
1147
Draft 0.5a (2026-09-26): §15.1 allows a reference server behind TLS termination and requires authenticated pushes when credentials exist; §16.3 defines the CBOR message encodings (push validation moves to §16.4).
096f447c Andy McCall ↶ 1148
1149
Draft 0.5 (2026-09-26): bookmarks and tags point at a `state` object naming the heads, so attestations, views, and references carry one ObjectRef per bookmark however many heads it has; repository format 2 records this, and format-1 repositories migrate on open.
86e682c2 Andy McCall ↶ 1150
1151
Draft 0.4b (2026-09-26): §5.5 records the encoding conventions for variants, references, views, signatures, and secret key files; §13.1 adds the signature directory.
5b1114a0 Andy McCall ↶ 1152
1153
Draft 0.4a (2026-09-26): §13.2 pins the FastCDC gear table, rolling hash, and masks so boundaries are identical across implementations.
f9c98dee unknown ↶ 1154
1155
Draft 0.4 (2026-09-26): overview no longer describes the constrained profile by word size; HLC drift bound replaces timestamp-based rejection; node kinds reduced to file, dir, symlink with a separate content mode and defined mode switching; symlinks always opaque; checkout safety rules (§7.7); workspace conflicts (§9.7) replace the case-folding tree conflict and cover normalization, reserved names, and path length; operation log entries suppressed for empty snapshots.
ff0af1ee unknown 1156
1157
Draft 0.3 (2026-09-26): recursive directory moves with cycle handling; explicit semantic dependencies with the `semantic-deps` hook; seals redefined as push-time batch signatures carrying their hash list, with an author-or-delegate authentication rule and `delegates` on identities; `core.sign` setting; fetch, push, policy, and garbage collection updated for seals.
1158
1159
Draft 0.2 (2026-09-26): rewrote from the 0.1 outline (the original `spec.txt`, since removed). Added threat model, two-identifier model, atom coordinate system, tree operations as atoms, conflict taxonomy and materialization, operation log, checkpoints, storage layout, composite signature construction, trust policy, reference attestations, HTTPS-mandatory transport, ZCT restrictions, wire protocol, partial repositories, locks, client profiles with a named hardware floor, Git bridge, CLI, performance targets, and conformance suite.