# Benchmarks

The harness is `crates/oxy-core/tests/bench.rs` (spec §25). It builds a

repository through the library, one commit at a time, then times the

everyday operations against thresholds:

    OXY_BENCH_DIR=~/.cache/oxy-bench \

      cargo test --release -p oxy-core --test bench -- --ignored --nocapture

`OXY_BENCH_FILES` (default 10 000) and `OXY_BENCH_CHANGES` (default

5 000) set the size. The data lands under `OXY_BENCH_DIR`; `/tmp` is often

a RAM disk, which hides the cost of `fsync`, so measure on a real disk.

## 2026-09-28: first run

Machine: 8 cores, x86-64, NVMe (ext4 on LVM), release build, warm cache.

Files are 21 lines each in 100 directories; one change in ten appends a

line to `hot.txt`, the rest rewrite one ordinary file.

| Operation (10 000 files, 5 000 changes) | Time | Threshold | |

|---|---:|---:|---|

| commit: import 10 000 files | 1 031 ms | 60 000 ms | ok |

| commit: one file (mean of 4 999) | 3 761 ms | 250 ms | **over** |

| status: clean working copy (stat crawl) | 2 977 ms | 2 000 ms | **over** |

| status: 10 files modified | 3 062 ms | 2 000 ms | **over** |

| commit: 10 files | 9 089 ms | 500 ms | **over** |

| log: whole bookmark | 408 ms | 2 000 ms | ok |

| blame: hot file (500 lines) | 2 920 ms | 2 000 ms | **over** |

| blame: ordinary file | 2 927 ms | 1 000 ms | **over** |

| checkpoint | 596 ms | 10 000 ms | ok |

| status after checkpoint | 2 873 ms | 2 000 ms | **over** |

| push: everything to an empty remote | 782 ms | 60 000 ms | ok |

| push: up to date | 4 ms | 1 000 ms | ok |

| clone: everything, with checkout | 3 378 ms | 60 000 ms | ok |

| fsck | 7 955 ms | 60 000 ms | ok |

| gc | 9 489 ms | 60 000 ms | ok |

| status after gc | 2 829 ms | 2 000 ms | **over** |

Peak resident memory over the whole run: 1 405 MiB. The run took 5 h 14 min,

almost all of it building the history, because each commit costs as much

as a status.

Smaller runs, same machine:

| Files, changes | Mean commit | Clean status | Blame (hot) | Peak memory |

|---|---:|---:|---:|---:|

| 1 000, 200 | 90 ms | 62 ms | 62 ms | 70 MiB |

| 1 000, 800 | 166 ms | 130 ms | 119 ms | 118 MiB |

| 4 000, 200 | 343 ms | 235 ms | 239 ms | 254 MiB |

| 10 000, 5 000 | 3 761 ms | 2 977 ms | 2 920 ms | 1 405 MiB |

### What it shows

Sync, log, checkpoints and repository maintenance are fine. Everything

that snapshots the working copy (status, commit, blame of the working

copy's base) grows with both the number of files and the length of the

history. Timing the phases of one snapshot at 4 000 files and 300 changes:

| Phase | Time |

|---|---:|

| load the base state (checkpoint plus the changes since) | 75–130 ms |

| materialize every file of the base state | 75 ms |

| walk the working copy against the manifest | 50 ms |

| build the working-copy change | 8 ms |

A clean status should be a stat walk and nothing else. The fix is backlog

item O10: skip loading and materializing when every file's stat matches a

manifest written for the same base, make the base state lazy for the

callers that need it, and cache materialized snapshots per state. The

thresholds above stay as they are until then, so the harness reports the

gap rather than hiding it.

## 2026-09-28: after O10a

`State::maximal` compared every pair of heads, and the benchmark's

bookmark has about 4 500 of them (changes to different files are

independent). It now walks once; commit also stops loading the base state

twice. Same machine and size:

| Operation (10 000 files, 5 000 changes) | Before | After | Threshold | |

|---|---:|---:|---:|---|

| commit: one file (mean of 4 999) | 3 761 ms | 861 ms | 250 ms | **over** |

| status: clean working copy | 2 977 ms | 803 ms | 2 000 ms | ok |

| status: 10 files modified | 3 062 ms | 864 ms | 2 000 ms | ok |

| commit: 10 files | 9 089 ms | 861 ms | 500 ms | **over** |

| log: whole bookmark | 408 ms | 400 ms | 2 000 ms | ok |

| blame: hot file | 2 920 ms | 809 ms | 2 000 ms | ok |

| blame: ordinary file | 2 927 ms | 828 ms | 1 000 ms | ok |

| status after checkpoint | 2 873 ms | 770 ms | 2 000 ms | ok |

| push: everything | 782 ms | 742 ms | 60 000 ms | ok |

| clone: everything | 3 378 ms | 1 664 ms | 60 000 ms | ok |

| fsck | 7 955 ms | 6 872 ms | 60 000 ms | ok |

| gc | 9 489 ms | 9 526 ms | 60 000 ms | ok |

Peak resident memory: 1 145 MiB (was 1 405). The run took 1 h 12 min

(was 5 h 14 min). Commit time no longer grows with history; what remains

grows with the number of files: at 10 000 files a snapshot spends about

320 ms loading the base state, 190 ms materializing it, 130 ms walking the

working copy and 40 ms building the change. That is O10b.

## 2026-09-28: after O10b

Rendering the base state, building file graphs from a checkpoint and

reading the working copy now run on all cores. Measured at 10 000 files

with 60 and 300 changes (commit time no longer depends on the length of

history, so the full 5 000 was not rerun):

| Phase of one snapshot, 10 000 files | After O10a | After O10b |

|---|---:|---:|

| load the base state | 320 ms | 180–280 ms |

| materialize the base state | 190 ms | 75 ms |

| walk and read the working copy | 130 ms | 80 ms |

| build the working-copy change | 40 ms | 40 ms |

| write the manifest | – | 46 ms |

| Operation, 10 000 files | After O10a | After O10b | Threshold |

|---|---:|---:|---:|

| status: clean working copy | 803 ms | 464–581 ms | 2 000 ms |

| commit: one file (mean) | 861 ms | 550–605 ms | 250 ms |

| commit: 10 files | 861 ms | 583–635 ms | 500 ms |

Loading the whole base state is now the largest cost. O10c removes it: a

snapshot should load and render only the files whose stat changed, which

needs checkpoints that can be read per file.

## 2026-09-28: O10c, first part: the in-memory state

Profiling a clean snapshot with `OXY_TIMINGS=1` (10 000 files of 40 lines,

200 changes) showed loading the base state dominating it. Applying the

changes to the in-memory graphs cost 650 ms and reading them 160 ms. A

checkpoint did not help: decoding one took 680 ms, as long as replaying

every change, because both build the same graphs for 400 000 lines. So the

cost was in the graph structure, not in the format:

- **Hashing.** The graph maps used SipHash over 36-byte keys, three to

  five times per line. The keys begin with BLAKE3 hashes, so they now use

  a seeded multiply-rotate hasher (`fasthash.rs`). The seed is random per

  process, so keys chosen by someone else cannot be aimed at one bucket.

- **The predecessor map.** It was written for every line and read nowhere,

  so it is gone.

- **Successor sets.** A `BTreeSet` per vertex became a sorted edge list

  that keeps its single successor inline. Nearly every line has one.

- **Line bytes.** Loading moves them into the graph instead of copying

  them, and chains each new line with one lookup instead of three.

Both binaries were built from the same tree and run alternately, nine

times each, medians shown:

| 10 000 files, 200 changes | Before | After |

|---|---:|---:|

| status: clean working copy | 0.96 s | 0.52 s |

| commit: one file | 0.99 s | 0.51 s |

| peak memory (either) | 657 MB | 239 MB |

The checkpoint format is unchanged. What remains of O10c is split into

O10d and O10e in the backlog:

- **O10d.** Skip reading text files whose stat matches the manifest, when

  the manifest recorded them equal to the base.

- **O10e.** Checkpoints readable per file, so a snapshot loads only the

  graphs it needs. This one changes the checkpoint format in spec §25.

## 2026-09-28: O10g, first part: status snapshots skip unchanged files

The manifest now records, per text file equal to the base, the key of the

base graph it matched: the file's node, its checkpoint graph (blob,

inline value or none) and the set of later changes that edited it

(spec §12.4). When a file's stat and key both still match, the snapshot

neither reads nor renders it.

`status`, `log`, `show`, `tag` and `bookmark` use a partial snapshot. It

loads the base for its history and builds only the graphs of files that

are not known unchanged. `commit`, `diff` and the rest still build the

whole base.

10 000 files of 40 lines, 22 changes, a version 2 checkpoint after 21 of

them; `OXY_TIMINGS=1`, clean working copy:

| Operation | Before | After |

|---|---:|---:|

| status | 0.55 s | 0.22 s |

| status peak memory | 259 MB | 51 MB |

| log | 0.7–0.8 s | 0.38 s |

| diff (full snapshot) | 0.56 s | 0.57 s |

| Phase of a clean status snapshot | Before | After |

|---|---:|---:|

| load base (whole / history only) | 317 ms | 31 ms |

| render base | 91 ms | 25 ms |

| read working copy | 68 ms | 50 ms |

| diff against base | 41 ms | 28 ms |

| write manifest | 33 ms | 35 ms |

The same for `commit` and `diff` comes next (O10h). There, only the graphs

the change touches are needed, and materialized states can be cached.

## 2026-09-29: O10h, commit, diff and show build only what they touch

Commit, working-copy diff, hunks and discard use the partial snapshot of

O10g. Files known unchanged are neither read nor rendered, and only the

graphs of the files the working copy changes are built. Showing a

committed change (`oxy show`, `oxy diff CHANGE`) builds the graphs of the

files that change edits and leaves the rest unrendered.

Same repository as O10g (10 000 files of 40 lines):

| Operation | After O10g | After O10h |

|---|---:|---:|

| diff (working copy) | 0.57 s, 244 MB | 0.22 s, 52 MB |

| commit (one file) | 0.58 s, 245 MB | 0.23 s, 52 MB |

| show (a one-file change) | not measured | 0.53 s, 145 MB |

Caching materialized states, planned alongside, is not needed now that

these commands render almost nothing.

## 2026-09-29: P5c, importing microsoft/vscode, first part

`oxy git import` on slices of microsoft/vscode's `main` (166 412 commits in

total), taken in the importer's own order (`git log --reverse

--topo-order`). The first 5 000 commits reach May 2016 with about 4 700

files.

The importer did work proportional to the whole tree for every commit:

- It cloned the previous state.

- It rebuilt a merge's first parent from history.

- It rendered, read and compared every file.

- It scanned every entry, with an allocation each, to decide whether each

  directory was still needed.

Now:

- Files with the same Git blob and mode as in the first parent's tree are

  marked unchanged and skipped, as a clean working copy's are. Paths that a

  merge's other parents touched are excluded.

- A commit's state is kept while first-parent children remain, up to 8,

  and the last child takes it without a clone.

- Needed directories come from one set of ancestors. That code is shared

  by snapshot and commit.

| Slice | Before | After |

|---|---:|---:|

| 300 commits (35 merges) | 65.1 s | 13.6 s |

| 1 000 commits (95 merges) | 271 s | 120 s, 3.3 GB |

| 5 000 commits (360 merges) | not finished after 80 min | 412 s, 6.3 GB, 17 MB store |

Where the 5 000-commit import spends its time (summed per phase): render

base 172 s, build change 99 s, bases 65 s, `git ls-tree` 56 s, read blobs

10 s, store 5 s.

Correctness: `oxy git verify` reproduces every commit of the 300 and

1 000-commit imports (1.9 million file comparisons for 1 000). A test

covers the case that must not be skipped: a `-s ours` merge whose other

side edited a file.

Next: rendering the base is still O(files) per commit, and memory grows

with the kept states. The full import follows.

## Linux, stage 1: the tip as one commit (2026-09-29)

The Linux tree at its tip: 95 943 files, 1.8 GB, on Earth (4 cores,

31 GiB), each run capped at 14 GB.

| | Git | oxy before | oxy, change format 2 |

|---|---:|---:|---:|

| First commit | 33.7 s, 42 MB | failed, 10.9 GB | 55.3 s, 9.9 GB |

| Store | | unreadable | 442 MB |

| Clean `status` | 0.26 to 0.31 s | | killed at 14 GB |

Before format 2 the commit made one change of 1.8 GB. That was over the

1 GiB an object may be, and the repository would not open afterwards

(oxygen#33, oxygen#34). Format 2 stores each insert of 4 KiB or more as

a text blob, so the change stays small.

Memory is the next problem. Every line is kept as its own allocation,

once in the change and again in the file graph, and `status` builds

every graph (L1b).

### Status after the commit (2026-09-29, oxygen#35)

A profile with the counting allocator (`--features alloc-stats`,

`OXY_TIMINGS=1`) of the first clean `status`, before it was killed:

| Phase | Time | Live after | Allocated |

|---|---:|---:|---:|

| Load base history | 17.5 s | 3.2 GB | 16 GB in 96 M allocations |

| Read working copy | 79.5 s | 4.9 GB | 1.8 GB |

| Load and render base | killed at 14 GB | | |

It had two causes, and both are fixed:

- **History held every line.** A history-only state now reads a stored

  insert's text blob only when it builds that file's graph, and a load no

  longer hashes each change again, which for format 2 means chunking all

  its text.

- **The manifest had no graph keys.** They are now recorded at commit, so

  the next status neither reads nor renders an unchanged file.

| Clean `status` on the Linux tip | Time | Peak RSS |

|---|---:|---:|

| Before | killed at 14 GB | |

| Lazy text and keys at commit | 3.7 s | 768 MB |

| And answered from the manifest | 0.72 s | 120 MB |

| And the manifest checked by a parallel walk | 0.44 s | 123 MB |

| Git | 0.22 s | 31 MB |

A clean status is now answered from the manifest alone, loading no

history. The manifest also records the executable bit, since a mode

change alters neither size nor time. Any change, a manifest older than

the racy window, or subrepos send it back to a full snapshot.

Still to do: status to the 0.4 s target (the walk and reading the

manifest); a status without a usable manifest still builds every graph

and is killed at 14 GB; `fsck` is killed at

14 GB; the first commit takes 57 s and 10.2 GB.

### Checkout and clone (2026-09-29, oxygen#35)

A checkout built and rendered every file of the target, read the whole

working copy, and after writing walked and hashed it all again for the

manifest. Switching bookmarks or cloning the Linux tree was killed at

14 GB. Now:

- a file is skipped when the manifest shows it unchanged and its graph is

  the same in both states;

- the rest are rendered 2 048 at a time, and each batch's graphs are

  dropped once written;

- the manifest is built from what the checkout already knows.

| Linux tip | oxy before | oxy now | Git |

|---|---:|---:|---:|

| Switch bookmark (one file differs) | killed at 14 GB | 7.1 to 7.4 s, 1.3 GB | 0.18 to 0.22 s, 39 MB |

| Local clone | killed at 14 GB | 98 s, 7.9 GB | 9.8 s, 469 MB |

A history-only load now reads a change's skeleton: the inline lines of

its inserts are stepped over, not decoded, and are read from the change

only for a graph that is built. A one-file switch then takes 5.4 to 5.6 s

and 0.8 GB, and the status after it 3.3 s and 600 MB (from 3.8 s).

A checkout of a working copy that is clean by the manifest now skips the

status snapshot, and a manifest entry inside the racy window is checked

by hashing the file rather than distrusting the whole manifest. A

one-file switch then takes 3.7 to 3.9 s and 705 MB. Where it goes:

checking the manifest (a walk of 96 000 files) 0.75 s, the base's and

the target's history 0.5 s each, their tree views 0.3 s each, planning

0.2 s, and writing the manifest 0.6 s.

A checkout now needs no base state at all: after the manifest check or a

snapshot, the manifest says which graph each file on disk renders. A

file is skipped when its manifest key is the target's, and a file left

as it was keeps its stat in the new manifest instead of being statted

again. What the new state lacks is found by a walk from the old heads

that stops at the target. A one-file switch takes 2.1 to 2.2 s and 455

MB: the manifest check 0.4 s, the target's history 0.5 s and tree view

0.3 s, planning 0.16 s, and writing the manifest 0.4 s.

Before that, most of a switch was loading history: the base's and the target's

history each decode the one huge first change (1.2 s and 364 MB each).

A clone holds the whole bundle in memory.

### log, blame and fsck (2026-09-30, oxygen#35)

The first full run of `~/bench/linux-bench.sh` against this work found

`log -n 100` taking 28 s and 4.7 GB, and `blame` killed at 14 GB:

- **log.** Checking each change's signature read the whole change, and

  for the first change that meant all 1.8 GB of its text. Signature

  checks, the dependency walks of fetch and push, and the prompt now

  read a change's metadata from its skeleton, and log answers the

  working copy's state from the manifest when it proves it clean.

- **blame.** It took a full snapshot, building every graph, to blame one

  file; it now loads the base's history and builds that file's graph.

- **fsck.** A bookmark's state was checked by loading it whole. It is

  now checked a batch of graphs at a time, deleted files' graphs

  included.

| Linux tip | Before | Now | Git |

|---|---:|---:|---:|

| `log -n 100` | 28 s, 4.7 GB | 2.0 s, 435 MB | 0.16 s |

| `blame MAINTAINERS` | killed at 14 GB | 1.1 s, 315 MB | 0.31 s |

| `fsck` | killed at 14 GB | 123 s, 4.5 GB | to measure again |

### One file changed (2026-09-30, oxygen#35)

A status with one file changed (MAINTAINERS) took 4.1 s: the walk ran

on one thread (0.83 s), and the manifest was written by statting all

96 000 files again (0.57 s). The walk now runs in parallel and each

file's stat is carried from the walk into the manifest. It now takes

2.5 s: the base's history 0.5 s, its tree view and graph keys 0.38 s,

the walk 0.35 s, rendering the changed file 0.47 s, the diff 0.18 s,

and the manifest 0.14 s. (Git: 0.23 s.) The tree view is now computed once per

state and cloned for each use, which brings rendering to 0.2 s and the

status to 2.3 s.

### The first commit (2026-09-30, oxygen#35)

A profile of the first commit of the Linux tip (61 s, 10 GB RSS): read

the working copy 2.3 s (1.6 GB of content), diff against the empty base

6.2 s (4.2 GB live, a Vec per line), hash and record 18.8 s, then the

commit itself about 33 s. The text of the one huge change was chunked

and hashed three or four times on one thread (hashing the working-copy

change, storing it, storing the commit, adding it to the state), the

commit copied every line of the working copy, and the chunks were

compressed on one thread.

Now:

- a snapshot encodes the working copy once, and hashes and stores that

  one encoding;

- a whole commit moves the working copy's lines rather than copying

  them, and joins the state under the hash it was stored by;

- text blobs are made on every core, 256 inserts at a time;

- the chunks of a large change are compressed on every core

  (`Store::put_many`).

| First commit, Linux tip | Time | Peak (counted) | RSS |

|---|---:|---:|---:|

| Before | 61 s | 7.1 GB | 10.1 GB |

| After | 40.7 s | 7.4 GB | 8.7 GB |

| Texts stored a batch at a time while encoding | 41.6 s | 4.8 GB | 5.8 GB |

| Lines in one shared buffer per insert (Earth under load) | 50.1 s | 4.3 GB | 4.9 GB |

| Git | 33.7 to 37.9 s | | 42 MB |

Storing a change's texts a batch at a time as they are made (the

encoder hands each batch of 256 to the store) removed the storing peak.

The peak was then the working copy's content (1.6 GB) and its lines, a

Vec each (2.6 GB). An insert's lines are now one buffer with line ends

(`Lines`), shared by the change and by the file graph, whose atoms point

into it: the diff of the whole tree takes 3.3 s and 1.1 million

allocations (from 6.2 s and 45 million), and the peak is 4.3 GB (4.9 GB

RSS). That run shared Earth with Flow jobs (load average 6.5 on 4 cores),

so its total time is not comparable with the rows above.

### Clone (2026-09-30, oxygen#35)

A local clone of the Linux tip took 96 s and 5.2 GB RSS. Its checkout

spent most of its time rendering: working out each file's line order

allocated three sets per line (the contraction of the graph), 93 GB in

all, although every file of a fresh tree is one chain of lines. And the

import hashed each change three times, for the huge first change

chunking its 1.8 GB of text each time.

- A graph that is one chain from start to end with nothing deleted (an

  edge to End besides the next line orders nothing) is read off in

  order; any other graph takes the general algorithm, which also skips

  its contraction where a line's successors are all kept. A test checks

  the two agree.

- A bundle carries each change's hash when its maker knows it (the store

  it came from, or the received bytes); never from the wire. An import

  hashes each change once, by storing it, and checks it is the named one.

| Local clone, Linux tip | Time | RSS |

|---|---:|---:|

| Before | 96 s (81.6 s in a quieter run) | 5.2 to 5.4 GB |

| After | 61.0 s | 4.1 GB |

| The change read once for all batches | 56.8 s | 4.7 GB |

| Changes moved as stored (load average 5.2) | 48.2 s | 3.9 GB |

| Git | 9.6 s | 469 MB |

A fetch between repositories on one machine now moves changes as they

are stored, their text blobs and chunks among the objects, instead of

decoding every change's text on one side and chunking, hashing and

compressing it again on the other; the import stores all the objects at

once, compressed on every core, and refuses a change whose text blob did

not come.

### Git import: the vscode slice (2026-09-30, oxygen#30)

The 1 747-commit vscode slice (to merge 0babb4299131) had one file as a

name conflict at every commit from that merge on: 51 verify differences.

There were three causes, all from Git having no identity for a directory:

- a directory deleted on one line and made again got a second node for

  the same path;

- a placement left under a directory another line removed was out of

  sight, so no merge could retire it;

- a file whose only placements were under removed directories (an

  orphan) could not be seen either.

Now an import brings a directory back rather than making a second node,

and a merge retires a kept node's other placements and every orphan.

`oxy git verify` built and rendered every file of every commit and read

every Git blob; it now loads each commit's history and builds only the

files whose graph (by key) or blob it has not already found equal to

that Git blob.

| vscode slice, 1 747 commits | Before | After |

|---|---:|---:|

| Import | 128.6 s | 132.1 s |

| Differences | 51 | none: 3 327 291 files compared |

| Verify | 1 369 to 1 444 s, 832 MB | 114.6 s, 441 MB |

### Git import memory (2026-09-30, backlog H2)

An import kept up to eight full states (every file's graph) for commits

whose children were still to come. It now keeps history-only states and

builds the graphs of only the files a commit may change; unchanged ones

were already neither read nor rendered. An imported change joins its

base under the hash it was stored by, instead of being encoded again.

| vscode, first 5 000 commits | Import | Peak RSS | Verify |

|---|---:|---:|---:|

| Before (2026-09-29) | 412 s | 6.3 GB | |

| After | 309 s | 2.3 GB | every commit (13.4 million files) in 978 s |

Verify loaded each commit's history from scratch. It now keeps the

state of a commit whose children are still to come, as the import does,

and a child adds only the changes its parent lacks. Those states stay

history only: graphs are built beside them (`State::build_graphs_detached`)

and dropped after rendering, so changes are added without their text and

every file keeps its graph key, which the cache of verified files needs.

The same 5 000 commits verify in 166.5 s (from 978 s), 1.3 GB.

### Git import at scale (2026-09-30, oxygen#25)

An imported commit cost what the whole tree is: `git ls-tree` of every

path, a tree view worked out again, every path compared, and every

changed file rendered from its whole history. Phases timed over the

first 5 000 vscode commits (`OXY_TIMINGS`, repeated phases summed) led

the work, one cost at a time:

| Change | 5 000 commits |

|---|---:|

| Before | 312 s |

| Each commit's tree is its parent's plus the diff, from one `git diff-tree --stdin` | 245 s |

| A parent not kept has its tree listed, so its child renders only what changed | 225 s |

| An ordinary commit is built from the paths it changes alone | 178 s |

| The diff sets aside lines found on one side only, as Git's xdiff does | 91 s |

| It diffs from the base's rendering rather than rendering again | 81 s |

| Tree views updated in place; merges add the other side without text | 56 s |

| File graphs keep their order as edits splice it | 38.8 s |

Twenty text diffs took 92 of the 103 s the builder spent: files

rewritten whole (line endings changed), on which Myers' algorithm is

quadratic. Rendering a long-lived file walked every line it ever had;

it now follows its live lines. Every step verified all 5 000 commits

(13.4 million files) clean.

| vscode slice | Before | After |

|---|---:|---:|

| 5 000 commits | 312 s, 2.1 GB | 38.8 s, 1.5 GB |

| 20 000 commits | 4 530 s, 6.7 GB | 925 s, 6.4 GB |

### Store size (2026-09-30, backlog P1)

The Linux tip's store was 448 MB, and 390 MB after `oxy gc`, against

311 MB for Git after `git gc`. The file contents cut into the store's

chunks compress, at zstd level 3, to 312 MB, level with Git; at level 9

to 284 MB (12.9 s on four cores), at level 19 to 270 MB (314 s). The

rest was two things:

- `oxy gc` left the first pack as written: packs merge only above 16.

- A text blob holds its lines as a CBOR array, so every line has a

  length header: 62 MB more raw, 35 MB more compressed for the tree's

  46 308 files of 4 KiB or more.

`oxy gc` now repacks everything into one pack at zstd level 9, compressed

on every core: 357 MB (349 MB pack) in 52 s. Change format 3 then stores

a text blob as the lines' bytes:

| Linux tip | Store after gc | First commit |

|---|---:|---:|

| Format 2, gc at level 3 | 390 MB | 41 s |

| Format 2, gc repacks at level 9 | 357 MB | |

| Format 3 (Earth at load average 11) | 324 MB (objects 306 MB) | 32.6 s, 4.4 GB |

| Git | 311 MB (.git) | 33.7 to 37.9 s, 42 MB |

### fsck (2026-09-30)

On the Linux tip in change format 3, `oxy fsck` took 24.7 s and 2.8 GB:

its references pass read every change whole, the first one's 1.8 GB of

text into memory, to check dependencies, authors and blobs. It now reads

a change's skeleton and checks its text as the blobs it is stored in (the

objects pass has checked every object already): 17.2 s and 984 MB, the

references pass 1.7 s. Phases are timed (`OXY_TIMINGS`): objects 5.1 s,

references 1.7 s, states and checkpoints 10.4 s.

### The full run (2026-09-30 06:24, main-0ba9b2e4)

`~/bench/linux-bench.sh` on Earth, at load average 3.8 to 4.5 (flow jobs

were running). Times are wall clock; memory is peak resident size.

| Operation | oxy | Git |

|---|---:|---:|

| init and commit the tip | 38.50 s, 4.4 GB | 38.77 s, 42 MB |

| status, clean (three runs) | 0.68 to 0.76 s, 123 MB | 0.35 to 0.76 s, 31 MB |

| status, one file | 3.32 s, 641 MB | 0.59 s |

| diff, one file | 2.93 s | 0.24 s |

| commit, one file | 4.74 s, 752 MB | 0.45 s |

| status, 1 000 files | 3.88 s, 811 MB | 0.41 s |

| diff, 1 000 files | 2.83 s | 0.95 s |

| commit, 1 000 files | 4.85 s, 922 MB | 0.57 s |

| log -n 100 | 1.39 s, 291 MB | 0.01 s |

| blame MAINTAINERS | 1.18 s, 335 MB | 0.09 s |

| checkout the first state (run by hand) | 2.84 s | 0.42 s |

| checkout back | 2.22 to 2.72 s, 491 MB | 0.47 s |

| fsck | 22.10 s, 1.07 GB | 13.09 s |

| gc | 39.66 s, 2.8 GB | 102.79 s |

| store before gc | 414 MB | 648 MB |

| store after gc | 324 MB | 295 MB |

The script's own oxy checkout of the first state failed; its hash lacked

the `#`, and the script is fixed. The time in the table is from running

it by hand.

Commit, one file: its graph keys now come from the state the commit

extended rather than a reload of the new base (main-eff93613), which

takes it to 3.05 to 3.27 s; the status after it takes 0.43 s.

What is left for the everyday commands is mostly loading the base's

history and its tree view (about 1 s of each) and rendering what

changed. oxygen#41 proposes a base index that would make those reads

proportional to what changed.

### A state index for the working copy's base (2026-09-30, oxygen#41)

A snapshot replayed the base's history (0.70 s) and worked out its tree

view (0.31 s) every time. Each workspace now keeps its base as a local

state index: change metadata, the view, the introducers, every file's

deferred history with its graph key, and the tree's maps, decoded only

when a command needs them. The index is hashed, and a test build checks

every read against a full load. The rest of a status that grew with

the tree now follows the change: the manifest is parsed once and

rewritten only when that helps; graph keys are looked up only for files

the manifest knows; and the change is built from the changed paths.

Linux tip, MAINTAINERS edited, oxy and Git alternating under the same

load (Earth at load average 4.5, other jobs running):

| | oxy (06:24) | oxy now | Git |

|---|---:|---:|---:|

| status | 3.32 s, 641 MB | 1.26 to 1.39 s, 370 MB | 0.26 to 0.29 s |

| diff | 2.93 s | 1.19 to 1.45 s | 0.16 to 0.25 s |

| status, clean | 0.68 to 0.76 s | 0.41 to 0.43 s | 0.26 to 0.32 s |

### Baseline for backlog 4 (2026-09-30 21:43, oxygen#41)

`~/bench/linux-bench.sh`, now with three runs of each everyday command,

oxy and Git alternating, and a local clone. Earth's load average was 7.2

at the start (no Flow jobs running), falling during the run; the first run

of each set is the noisiest.

| Operation | oxy | Git |

|---|---:|---:|

| init and commit the tip | **33.3 s**, 3.1 GB | 38.2 s, 42 MB |

| status, clean | 0.43 to 0.53 s, 75 MB | 0.34 to 0.46 s |

| status, one file | 1.73 to 1.81 s (3.36 s first), 375 MB | 0.36 to 0.65 s |

| diff, one file | 1.77 to 2.24 s | 0.24 to 0.29 s |

| commit, one file | 3.10 s | 0.41 s |

| status, 1 000 files | 1.62 s (4.21 s first), 573 MB | 0.28 to 0.38 s |

| diff, 1 000 files | 1.54 to 2.40 s | 0.59 to 0.81 s |

| commit, 1 000 files | 2.47 s | 0.53 s |

| log -n 100 | 0.86 to 1.03 s | 0.00 s |

| blame MAINTAINERS | 0.99 to 1.04 s | 0.07 s |

| clone, local path | 56.3 s, 3.3 GB | 18.4 s |

| checkout the first state, and back | 2.86 s, 3.00 s | 0.39 s, 0.50 s |

| fsck | 24.4 s, 773 MB | 12.8 s |

| gc | **48.1 s** | 100.5 s |

| store before and after gc | 465 MB, 375 MB | 648 MB, 295 MB |

oxy is now ahead of Git on the first commit (33 s against 38 s) and on gc

(48 s against 100 s). The store after gc grew from 324 MB because it now

includes the workspace's state index.

### S1, first part: walk, manifest, lazy index, mimalloc (2026-09-30 22:39, oxygen#41)

The same script after the first round of S1: a lister of our own in place

of the ignore crate's walker, the manifest as a flat binary file with

hashes as bytes, each file's history in the state index decoded only when

used, maps built in path order, and mimalloc as the allocator. Earth's

load average was 4 to 5 (no Flow jobs); runs alternate with Git.

| Operation | oxy | before (21:43) | Git |

|---|---:|---:|---:|

| init and commit the tip | **24.4 s**, 3.2 GB | 33.3 s, 3.1 GB | 36.2 s |

| status, clean | **0.17 to 0.18 s**, 130 to 150 MB | 0.43 to 0.53 s | 0.22 to 0.23 s |

| status, one file | 0.47 s (1.47 s first) | 1.73 to 1.81 s | 0.22 to 0.23 s |

| diff, one file | 0.47 to 0.51 s | 1.77 to 2.24 s | 0.11 to 0.14 s |

| commit, one file | 0.96 s | 3.10 s | 0.17 s |

| status, 1 000 files | 0.65 s (1.99 s first) | 1.62 s | 0.23 to 0.24 s |

| diff, 1 000 files | 0.65 s | 1.54 to 2.40 s | 0.46 to 0.48 s |

| commit, 1 000 files | 1.20 s | 2.47 s | 0.50 s |

| log -n 100 | 0.48 to 0.53 s | 0.86 to 1.03 s | 0.00 s |

| blame MAINTAINERS | 0.59 s | 0.99 to 1.04 s | 0.07 s |

| clone, local path | 36.7 s, 3.6 GB | 56.3 s | 18.0 s |

| checkout the first state, and back | 1.48 s, 1.37 s | 2.86 s, 3.00 s | 0.35 s, 0.34 s |

| fsck | 14.0 s, 1.0 GB | 24.4 s | 12.6 s |

| gc | **29.9 s** | 48.1 s | 70.3 s |

| store after gc | 369 MB | 375 MB | 295 MB |

mimalloc alone took one-file status from 0.78 to 0.47 s and clean status

from 0.26 to 0.18 s: status allocates hundreds of thousands of small

values from every thread, and glibc's arena locks were the largest single

cost. The first status after a commit (1.47 s) rebuilds the state index.

Afterwards (22:55), log reads change metadata from the state index and a

metadata cache in the store, and its signature marks no longer read each

change whole: `log -n 100` takes 0.19 to 0.20 s, of which 9 ms is the log

itself and the rest the check that the working copy is clean.

Where one-file status goes now (0.47 s): listing the working copy about

150 ms (the syscalls alone: `find` with a stat of each file takes 0.37 s

on one core), loading the state index about 60 ms, reading the manifest

about 40 ms, rendering and diffing the changed file about 100 ms.

### S1 to S5, second round (2026-09-30 23:54, oxygen#41)

After the second round: merge joins over whole-tree maps, a commit that

works out keys only for what it changed and brings the state index up to

date by replaying rather than rewriting it, the manifest checksummed and

written without fsync, log and blame from the state index, checkout

through the index with two indexes kept. Load average 4.2 at the start,

no Flow jobs.

| Operation | oxy | 22:39 | Git |

|---|---:|---:|---:|

| init and commit the tip | **21.9 s**, 3.2 GB | 24.4 s | 33.7 s |

| status, clean | **0.17 to 0.19 s** | 0.17 to 0.18 s | 0.22 s |

| status, one file | 0.40 to 0.41 s (1.10 s first) | 0.47 s | 0.22 to 0.24 s |

| diff, one file | 0.39 to 0.48 s | 0.47 to 0.51 s | 0.10 to 0.11 s |

| commit, one file | 0.67 s | 0.96 s | 0.21 s |

| status, 1 000 files | 0.58 to 0.65 s | 0.65 s | 0.23 to 0.24 s |

| diff, 1 000 files | 0.59 to 0.62 s | 0.65 s | 0.46 to 0.48 s |

| commit, 1 000 files | 0.88 s | 1.20 s | 0.48 s |

| log -n 100 | 0.17 to 0.19 s | 0.48 to 0.53 s | 0.00 s |

| blame MAINTAINERS | 0.16 s | 0.59 s | 0.07 to 0.16 s |

| clone, local path | 32.2 s, 3.6 GB | 36.7 s | 17.0 s |

| checkout the first state, and back | 0.74 s, 0.74 s | 1.48 s, 1.37 s | 0.37 s, 0.33 s |

| fsck | 13.6 s | 14.0 s | 12.6 s |

| gc | **29.4 s** | 29.9 s | 70.2 s |

| store after gc | 369 MB | 369 MB | 295 MB |

The first one-file status after the first commit (1.10 s) builds the

state index for the first time. Most of what remains in status is listing

the tree (about 135 ms), loading the index (about 60 ms) and starting up;

commit adds storing and recording with fsync (about 100 ms).

### P5a: flow-casfs, a read-only FUSE mount of a state (2026-10-01, oxygenflow#21)

The first flow-casfs (single-threaded, fuser 0.16, no read-ahead) on the

Linux repository on Earth, with the H4 import and Flow jobs running (load

average 10 to 11), so the times are high:

| | |

|---|---:|

| mount (state from the index 284 ms, tree of 102 327 entries 40 ms) | 0.52 s |

| first read of MAINTAINERS (rendered) | 0.10 s |

| `ls -lR`, rendering every file one at a time | 201 s |

| full read afterwards (bytes in memory) | 0.77 s |

| full read, new mount, renderings cached on disk | 20.8 s |

| render cache on disk | 1.8 GB |

`diff -r` between the mount and the working copy finds no difference. To

reach the spec's targets: render in parallel and ahead of reads, take a

file added once and never edited straight from its text blob, keep sizes

in the render cache so a stat does not render, serve FUSE on several

threads, and read the state index lazily for a mount under 100 ms.

### Status with work in progress (2026-10-01, oxygen#41)

`MAINTAINERS` edited on the Linux tree on Earth (load average 2.2, one

Flow job running), oxy and Git alternating, three runs each:

| | before | after | Git |

|---|---:|---:|---:|

| status, one file edited (after a first status) | 0.42 to 0.46 s, 330 MB | **0.16 to 0.17 s**, 120 to 155 MB | 0.20 to 0.21 s |

| status, clean | 0.18 to 0.20 s | **0.15 to 0.16 s** | 0.20 to 0.24 s |

| diff, one file | 0.43 s | 0.40 to 0.41 s | 0.08 to 0.09 s |

Before, the manifest only proved a clean working copy. With uncommitted

work every status listed and snapshotted the tree again, loaded the state

index and rendered the base (listing 164 ms, index 60 ms, base 33 ms). Now

a snapshot leaves a status cache keyed by the working-copy change and the

manifest's checksum, and keeps the manifest current for the edited files.

A later status that finds every file as the manifest says answers from

the cache. Test builds check that answer against a snapshot. Diff still

renders the base; it is next.

Then diff (same day, load average 4.2, Git 0.10 s throughout): while the

status cache holds, `oxy diff` renders the stored working-copy change

against its base loaded from the state index, rather than snapshotting.

0.34 to 0.35 s and 250 MB (from 0.41 s and 360 MB). Rendering it through

`change_base`, which reads the base's changes rather than the index, was

slower than a snapshot (0.9 s) and was not kept. What remains is the

status check's listing (about 130 ms), the index (60 ms) and rendering.

Then directory times (same evening, load average 3.1): the manifest

(format 3) keeps every directory the walk listed with its time, read

before the directory was. While no directory's time has moved, the check

stats the tracked files and does not list the tree; a directory whose

entries changed sends it back to a listing.

| | before | after | Git |

|---|---:|---:|---:|

| status, one file edited | 0.16 to 0.20 s, 120 to 155 MB | **0.10 to 0.12 s, 80 MB** | 0.22 s |

| status, clean | 0.15 to 0.21 s | **0.09 to 0.12 s, 80 MB** | 0.20 to 0.26 s |

| diff, one file | 0.34 s | 0.24 to 0.25 s | 0.09 to 0.13 s |

Diff, last round (load average 2.7): a change's base renders only its

edited files without first building a set of every other file to leave

out, and `oxy diff` no longer frees its structures just before the

process ends. 0.19 to 0.21 s (from 0.25 s; Git 0.08 s). What remains:

the stat check 78 ms (Git stats the same files), the state index 58 ms

(39 of them building the tree view), the base's full file map 21 ms.

### Commit, one file (2026-10-01, oxygen#41)

`~/bench/commit-vs-git.sh`: commits go forward and both repositories are

put back at the end (an `oxy undo` between runs dropped the manifest, and

its rehash of the tree evicted the page cache Git's runs relied on, which

made both sides' numbers meaningless). Load average 3 to 4.

| | before | after | Git `commit -am` |

|---|---:|---:|---:|

| commit after a status | 0.68 s | **0.36 to 0.44 s** | 0.14 to 0.17 s |

| commit straight after the edit | 0.68 s | 0.62 to 0.69 s | 0.14 to 0.17 s |

After a status, the commit takes the status's snapshot from the cache.

Straight after an edit, the stat check now hands its stats to the

snapshot as its listing (8 ms instead of 122 ms), but checks every file

rather than stopping at the first difference, so the gain is small. What

remains is mostly durability: 18 fsyncs take 210 ms of the commit (one

alone 120 ms). Git does not sync loose objects by default.

Fewer waits on the disk (same evening): a batch's loose objects are

written together, synced in parallel, renamed and their directories

synced once each; a commit runs in one batch; packing keeps the emptied

fan-out directories, and a missing one has all 256 created with one sync

of `loose` (that sync alone took 130 ms on a busy disk). Serial syncs per

commit fall from 18 to about 7; each still waits on the same ext4 journal

commit, so the gain is modest. Old and new builds alternating, five

commits after a status each: median 0.39 s before, 0.34 s after (Git

0.14 to 0.16 s).

A commit's own snapshot (straight after an edit) no longer stores its

working-copy change or records an operation for it: the commit's

operation, recorded straight after, replaces the working copy. Old and

new builds alternating, six direct commits each (load average 4.4): median

0.61 s before, 0.58 s after (Git 0.16 to 0.27 s at that load).

### log -n 100 (2026-10-02, oxygen#41)

`oxy log` no longer checks the files on disk: its working-copy row

describes the working copy as last recorded (with edits, or not; the

manifest's conflict flag), as Git's log never looks at the working tree.

The entries come from the state index's change metadata.

| repository | before | after | Git |

|---|---:|---:|---:|

| Linux tree (Earth) | 0.17 s | **0.03 s** | 0.00 s |

| oxygen, 388 changes (workstation) | 0.06 s, 79 MB | **0.03 s, 22 MB** | |

| vscode slice, 10 000 commits (Earth, warm) | | **0.02 s** | |

What remains on oxygen is checking each row's signature (30 ms for 100).

### Against Git tuned for large repositories (2026-10-02)

Git as it comes compares an untuned Git with an oxy that uses every

cache it has. `~/bench/vs-git-tuned.sh` adds a copy of the Git repository

tuned as Git recommends for large ones: `feature.manyFiles` (index v4 and

more), `core.untrackedCache`, and a commit-graph. Not `core.fsmonitor`:

Git 2.53 has no file-watcher daemon on Linux, and Watchman is not

installed. Earth, load average 1.6, three runs each, alternating.

| | oxy | Git | Git, tuned |

|---|---:|---:|---:|

| status, clean | 0.16 s | 0.20 to 0.21 s | **0.10 s** |

| status, one file edited | 0.12 to 0.13 s | 0.21 to 0.22 s | **0.10 to 0.11 s** |

| diff, one file | 0.22 to 0.23 s | 0.09 s | **0.08 to 0.09 s** |

| log -n 100 | 0.03 s | **0.00 s** | **0.00 s** |

| blame MAINTAINERS | 0.16 s | **0.06 s** | **0.06 s** |

Tuned Git is ahead of oxy on all five. Its untracked cache is what oxy's

directory times do, and with it Git's status is a little faster than

oxy's. oxy beats Git as it comes on status only. Claims of speed against

Git should say which Git.

Then the stat check by directory (same night): tracked files are grouped

by directory, each directory opened once, and each file statted relative

to it (`fstatat`), so a stat resolves one name rather than every component

from the root (the kernel's path lookup was most of a clean status). The

stat check went from 84 ms to 73 ms; a clean status is 0.10 to 0.12 s,

level with tuned Git (0.10 to 0.11 s). What is left is the 96 000 stat