Benchmarks

The harness is crates/oxy-core/tests/bench.rs (spec §25). It builds a repository through the library, one commit at a time, then times the everyday operations against thresholds:

OXY_BENCH_DIR=~/.cache/oxy-bench \
  cargo test --release -p oxy-core --test bench -- --ignored --nocapture

OXY_BENCH_FILES (default 10 000) and OXY_BENCH_CHANGES (default 5 000) set the size. The data lands under OXY_BENCH_DIR; /tmp is often a RAM disk, which hides the cost of fsync, so measure on a real disk.

2026-09-28: first run

Machine: 8 cores, x86-64, NVMe (ext4 on LVM), release build, warm cache. Files are 21 lines each in 100 directories; one change in ten appends a line to hot.txt, the rest rewrite one ordinary file.

Peak resident memory over the whole run: 1 405 MiB. The run took 5 h 14 min, almost all of it building the history, because each commit costs as much as a status.

Smaller runs, same machine:

What it shows

Sync, log, checkpoints and repository maintenance are fine. Everything that snapshots the working copy (status, commit, blame of the working copy's base) grows with both the number of files and the length of the history. Timing the phases of one snapshot at 4 000 files and 300 changes:

A clean status should be a stat walk and nothing else. The fix is backlog item O10: skip loading and materializing when every file's stat matches a manifest written for the same base, make the base state lazy for the callers that need it, and cache materialized snapshots per state. The thresholds above stay as they are until then, so the harness reports the gap rather than hiding it.

2026-09-28: after O10a

State::maximal compared every pair of heads, and the benchmark's bookmark has about 4 500 of them (changes to different files are independent). It now walks once; commit also stops loading the base state twice. Same machine and size:

Peak resident memory: 1 145 MiB (was 1 405). The run took 1 h 12 min (was 5 h 14 min). Commit time no longer grows with history; what remains grows with the number of files: at 10 000 files a snapshot spends about 320 ms loading the base state, 190 ms materializing it, 130 ms walking the working copy and 40 ms building the change. That is O10b.

2026-09-28: after O10b

Rendering the base state, building file graphs from a checkpoint and reading the working copy now run on all cores. Measured at 10 000 files with 60 and 300 changes (commit time no longer depends on the length of history, so the full 5 000 was not rerun):

Loading the whole base state is now the largest cost. O10c removes it: a snapshot should load and render only the files whose stat changed, which needs checkpoints that can be read per file.

2026-09-28: O10c, first part: the in-memory state

Profiling a clean snapshot with OXY_TIMINGS=1 (10 000 files of 40 lines, 200 changes) showed loading the base state dominating it. Applying the changes to the in-memory graphs cost 650 ms and reading them 160 ms. A checkpoint did not help: decoding one took 680 ms, as long as replaying every change, because both build the same graphs for 400 000 lines. So the cost was in the graph structure, not in the format:

• Hashing. The graph maps used SipHash over 36-byte keys, three to five times per line. The keys begin with BLAKE3 hashes, so they now use a seeded multiply-rotate hasher (fasthash.rs). The seed is random per process, so keys chosen by someone else cannot be aimed at one bucket.
• The predecessor map. It was written for every line and read nowhere, so it is gone.
• Successor sets. A BTreeSet per vertex became a sorted edge list that keeps its single successor inline. Nearly every line has one.
• Line bytes. Loading moves them into the graph instead of copying them, and chains each new line with one lookup instead of three.

Both binaries were built from the same tree and run alternately, nine times each, medians shown:

The checkpoint format is unchanged. What remains of O10c is split into O10d and O10e in the backlog:

• O10d. Skip reading text files whose stat matches the manifest, when the manifest recorded them equal to the base.
• O10e. Checkpoints readable per file, so a snapshot loads only the graphs it needs. This one changes the checkpoint format in spec §25.

2026-09-28: O10g, first part: status snapshots skip unchanged files

The manifest now records, per text file equal to the base, the key of the base graph it matched: the file's node, its checkpoint graph (blob, inline value or none) and the set of later changes that edited it (spec §12.4). When a file's stat and key both still match, the snapshot neither reads nor renders it.

status, log, show, tag and bookmark use a partial snapshot. It loads the base for its history and builds only the graphs of files that are not known unchanged. commit, diff and the rest still build the whole base.

10 000 files of 40 lines, 22 changes, a version 2 checkpoint after 21 of them; OXY_TIMINGS=1, clean working copy:

The same for commit and diff comes next (O10h). There, only the graphs the change touches are needed, and materialized states can be cached.

2026-09-29: O10h, commit, diff and show build only what they touch

Commit, working-copy diff, hunks and discard use the partial snapshot of O10g. Files known unchanged are neither read nor rendered, and only the graphs of the files the working copy changes are built. Showing a committed change (oxy show, oxy diff CHANGE) builds the graphs of the files that change edits and leaves the rest unrendered.

Same repository as O10g (10 000 files of 40 lines):

Caching materialized states, planned alongside, is not needed now that these commands render almost nothing.

2026-09-29: P5c, importing microsoft/vscode, first part

oxy git import on slices of microsoft/vscode's main (166 412 commits in total), taken in the importer's own order (git log --reverse --topo-order). The first 5 000 commits reach May 2016 with about 4 700 files.

The importer did work proportional to the whole tree for every commit:

• It cloned the previous state.
• It rebuilt a merge's first parent from history.
• It rendered, read and compared every file.
• It scanned every entry, with an allocation each, to decide whether each directory was still needed.

Now:

• Files with the same Git blob and mode as in the first parent's tree are marked unchanged and skipped, as a clean working copy's are. Paths that a merge's other parents touched are excluded.
• A commit's state is kept while first-parent children remain, up to 8, and the last child takes it without a clone.
• Needed directories come from one set of ancestors. That code is shared by snapshot and commit.

Where the 5 000-commit import spends its time (summed per phase): render base 172 s, build change 99 s, bases 65 s, git ls-tree 56 s, read blobs 10 s, store 5 s.

Correctness: oxy git verify reproduces every commit of the 300 and 1 000-commit imports (1.9 million file comparisons for 1 000). A test covers the case that must not be skipped: a -s ours merge whose other side edited a file.

Next: rendering the base is still O(files) per commit, and memory grows with the kept states. The full import follows.

Linux, stage 1: the tip as one commit (2026-09-29)

The Linux tree at its tip: 95 943 files, 1.8 GB, on Earth (4 cores, 31 GiB), each run capped at 14 GB.

Before format 2 the commit made one change of 1.8 GB. That was over the 1 GiB an object may be, and the repository would not open afterwards (oxygen#33, oxygen#34). Format 2 stores each insert of 4 KiB or more as a text blob, so the change stays small.

Memory is the next problem. Every line is kept as its own allocation, once in the change and again in the file graph, and status builds every graph (L1b).

Status after the commit (2026-09-29, oxygen#35)

A profile with the counting allocator (--features alloc-stats, OXY_TIMINGS=1) of the first clean status, before it was killed:

It had two causes, and both are fixed:

• History held every line. A history-only state now reads a stored insert's text blob only when it builds that file's graph, and a load no longer hashes each change again, which for format 2 means chunking all its text.
• The manifest had no graph keys. They are now recorded at commit, so the next status neither reads nor renders an unchanged file.

A clean status is now answered from the manifest alone, loading no history. The manifest also records the executable bit, since a mode change alters neither size nor time. Any change, a manifest older than the racy window, or subrepos send it back to a full snapshot.

Still to do: status to the 0.4 s target (the walk and reading the manifest); a status without a usable manifest still builds every graph and is killed at 14 GB; fsck is killed at 14 GB; the first commit takes 57 s and 10.2 GB.

Checkout and clone (2026-09-29, oxygen#35)

A checkout built and rendered every file of the target, read the whole working copy, and after writing walked and hashed it all again for the manifest. Switching bookmarks or cloning the Linux tree was killed at 14 GB. Now:

• a file is skipped when the manifest shows it unchanged and its graph is the same in both states;
• the rest are rendered 2 048 at a time, and each batch's graphs are dropped once written;
• the manifest is built from what the checkout already knows.

A history-only load now reads a change's skeleton: the inline lines of its inserts are stepped over, not decoded, and are read from the change only for a graph that is built. A one-file switch then takes 5.4 to 5.6 s and 0.8 GB, and the status after it 3.3 s and 600 MB (from 3.8 s).

A checkout of a working copy that is clean by the manifest now skips the status snapshot, and a manifest entry inside the racy window is checked by hashing the file rather than distrusting the whole manifest. A one-file switch then takes 3.7 to 3.9 s and 705 MB. Where it goes: checking the manifest (a walk of 96 000 files) 0.75 s, the base's and the target's history 0.5 s each, their tree views 0.3 s each, planning 0.2 s, and writing the manifest 0.6 s.

A checkout now needs no base state at all: after the manifest check or a snapshot, the manifest says which graph each file on disk renders. A file is skipped when its manifest key is the target's, and a file left as it was keeps its stat in the new manifest instead of being statted again. What the new state lacks is found by a walk from the old heads that stops at the target. A one-file switch takes 2.1 to 2.2 s and 455 MB: the manifest check 0.4 s, the target's history 0.5 s and tree view 0.3 s, planning 0.16 s, and writing the manifest 0.4 s.

Before that, most of a switch was loading history: the base's and the target's history each decode the one huge first change (1.2 s and 364 MB each). A clone holds the whole bundle in memory.

log, blame and fsck (2026-09-30, oxygen#35)

The first full run of ~/bench/linux-bench.sh against this work found log -n 100 taking 28 s and 4.7 GB, and blame killed at 14 GB:

• log. Checking each change's signature read the whole change, and for the first change that meant all 1.8 GB of its text. Signature checks, the dependency walks of fetch and push, and the prompt now read a change's metadata from its skeleton, and log answers the working copy's state from the manifest when it proves it clean.
• blame. It took a full snapshot, building every graph, to blame one file; it now loads the base's history and builds that file's graph.
• fsck. A bookmark's state was checked by loading it whole. It is now checked a batch of graphs at a time, deleted files' graphs included.

One file changed (2026-09-30, oxygen#35)

A status with one file changed (MAINTAINERS) took 4.1 s: the walk ran on one thread (0.83 s), and the manifest was written by statting all 96 000 files again (0.57 s). The walk now runs in parallel and each file's stat is carried from the walk into the manifest. It now takes 2.5 s: the base's history 0.5 s, its tree view and graph keys 0.38 s, the walk 0.35 s, rendering the changed file 0.47 s, the diff 0.18 s, and the manifest 0.14 s. (Git: 0.23 s.) The tree view is now computed once per state and cloned for each use, which brings rendering to 0.2 s and the status to 2.3 s.

The first commit (2026-09-30, oxygen#35)

A profile of the first commit of the Linux tip (61 s, 10 GB RSS): read the working copy 2.3 s (1.6 GB of content), diff against the empty base 6.2 s (4.2 GB live, a Vec per line), hash and record 18.8 s, then the commit itself about 33 s. The text of the one huge change was chunked and hashed three or four times on one thread (hashing the working-copy change, storing it, storing the commit, adding it to the state), the commit copied every line of the working copy, and the chunks were compressed on one thread.

Now:

• a snapshot encodes the working copy once, and hashes and stores that one encoding;
• a whole commit moves the working copy's lines rather than copying them, and joins the state under the hash it was stored by;
• text blobs are made on every core, 256 inserts at a time;
• the chunks of a large change are compressed on every core (Store::put_many).

Storing a change's texts a batch at a time as they are made (the encoder hands each batch of 256 to the store) removed the storing peak. The peak was then the working copy's content (1.6 GB) and its lines, a Vec each (2.6 GB). An insert's lines are now one buffer with line ends (Lines), shared by the change and by the file graph, whose atoms point into it: the diff of the whole tree takes 3.3 s and 1.1 million allocations (from 6.2 s and 45 million), and the peak is 4.3 GB (4.9 GB RSS). That run shared Earth with Flow jobs (load average 6.5 on 4 cores), so its total time is not comparable with the rows above.

Clone (2026-09-30, oxygen#35)

A local clone of the Linux tip took 96 s and 5.2 GB RSS. Its checkout spent most of its time rendering: working out each file's line order allocated three sets per line (the contraction of the graph), 93 GB in all, although every file of a fresh tree is one chain of lines. And the import hashed each change three times, for the huge first change chunking its 1.8 GB of text each time.

• A graph that is one chain from start to end with nothing deleted (an edge to End besides the next line orders nothing) is read off in order; any other graph takes the general algorithm, which also skips its contraction where a line's successors are all kept. A test checks the two agree.
• A bundle carries each change's hash when its maker knows it (the store it came from, or the received bytes); never from the wire. An import hashes each change once, by storing it, and checks it is the named one.

A fetch between repositories on one machine now moves changes as they are stored, their text blobs and chunks among the objects, instead of decoding every change's text on one side and chunking, hashing and compressing it again on the other; the import stores all the objects at once, compressed on every core, and refuses a change whose text blob did not come.

Git import: the vscode slice (2026-09-30, oxygen#30)

The 1 747-commit vscode slice (to merge 0babb4299131) had one file as a name conflict at every commit from that merge on: 51 verify differences. There were three causes, all from Git having no identity for a directory:

• a directory deleted on one line and made again got a second node for the same path;
• a placement left under a directory another line removed was out of sight, so no merge could retire it;
• a file whose only placements were under removed directories (an orphan) could not be seen either.

Now an import brings a directory back rather than making a second node, and a merge retires a kept node's other placements and every orphan.

oxy git verify built and rendered every file of every commit and read every Git blob; it now loads each commit's history and builds only the files whose graph (by key) or blob it has not already found equal to that Git blob.

Git import memory (2026-09-30, backlog H2)

An import kept up to eight full states (every file's graph) for commits whose children were still to come. It now keeps history-only states and builds the graphs of only the files a commit may change; unchanged ones were already neither read nor rendered. An imported change joins its base under the hash it was stored by, instead of being encoded again.

Verify loaded each commit's history from scratch. It now keeps the state of a commit whose children are still to come, as the import does, and a child adds only the changes its parent lacks. Those states stay history only: graphs are built beside them (State::build_graphs_detached) and dropped after rendering, so changes are added without their text and every file keeps its graph key, which the cache of verified files needs. The same 5 000 commits verify in 166.5 s (from 978 s), 1.3 GB.

Git import at scale (2026-09-30, oxygen#25)

An imported commit cost what the whole tree is: git ls-tree of every path, a tree view worked out again, every path compared, and every changed file rendered from its whole history. Phases timed over the first 5 000 vscode commits (OXY_TIMINGS, repeated phases summed) led the work, one cost at a time:

Twenty text diffs took 92 of the 103 s the builder spent: files rewritten whole (line endings changed), on which Myers' algorithm is quadratic. Rendering a long-lived file walked every line it ever had; it now follows its live lines. Every step verified all 5 000 commits (13.4 million files) clean.

Store size (2026-09-30, backlog P1)

The Linux tip's store was 448 MB, and 390 MB after oxy gc, against 311 MB for Git after git gc. The file contents cut into the store's chunks compress, at zstd level 3, to 312 MB, level with Git; at level 9 to 284 MB (12.9 s on four cores), at level 19 to 270 MB (314 s). The rest was two things:

• oxy gc left the first pack as written: packs merge only above 16.
• A text blob holds its lines as a CBOR array, so every line has a length header: 62 MB more raw, 35 MB more compressed for the tree's 46 308 files of 4 KiB or more.

oxy gc now repacks everything into one pack at zstd level 9, compressed on every core: 357 MB (349 MB pack) in 52 s. Change format 3 then stores a text blob as the lines' bytes:

fsck (2026-09-30)

On the Linux tip in change format 3, oxy fsck took 24.7 s and 2.8 GB: its references pass read every change whole, the first one's 1.8 GB of text into memory, to check dependencies, authors and blobs. It now reads a change's skeleton and checks its text as the blobs it is stored in (the objects pass has checked every object already): 17.2 s and 984 MB, the references pass 1.7 s. Phases are timed (OXY_TIMINGS): objects 5.1 s, references 1.7 s, states and checkpoints 10.4 s.

The full run (2026-09-30 06:24, main-0ba9b2e4)

~/bench/linux-bench.sh on Earth, at load average 3.8 to 4.5 (flow jobs were running). Times are wall clock; memory is peak resident size.

The script's own oxy checkout of the first state failed; its hash lacked the #, and the script is fixed. The time in the table is from running it by hand.

Commit, one file: its graph keys now come from the state the commit extended rather than a reload of the new base (main-eff93613), which takes it to 3.05 to 3.27 s; the status after it takes 0.43 s.

What is left for the everyday commands is mostly loading the base's history and its tree view (about 1 s of each) and rendering what changed. oxygen#41 proposes a base index that would make those reads proportional to what changed.

A state index for the working copy's base (2026-09-30, oxygen#41)

A snapshot replayed the base's history (0.70 s) and worked out its tree view (0.31 s) every time. Each workspace now keeps its base as a local state index: change metadata, the view, the introducers, every file's deferred history with its graph key, and the tree's maps, decoded only when a command needs them. The index is hashed, and a test build checks every read against a full load. The rest of a status that grew with the tree now follows the change: the manifest is parsed once and rewritten only when that helps; graph keys are looked up only for files the manifest knows; and the change is built from the changed paths.

Linux tip, MAINTAINERS edited, oxy and Git alternating under the same load (Earth at load average 4.5, other jobs running):

Baseline for backlog 4 (2026-09-30 21:43, oxygen#41)

~/bench/linux-bench.sh, now with three runs of each everyday command, oxy and Git alternating, and a local clone. Earth's load average was 7.2 at the start (no Flow jobs running), falling during the run; the first run of each set is the noisiest.

oxy is now ahead of Git on the first commit (33 s against 38 s) and on gc (48 s against 100 s). The store after gc grew from 324 MB because it now includes the workspace's state index.

S1, first part: walk, manifest, lazy index, mimalloc (2026-09-30 22:39, oxygen#41)

The same script after the first round of S1: a lister of our own in place of the ignore crate's walker, the manifest as a flat binary file with hashes as bytes, each file's history in the state index decoded only when used, maps built in path order, and mimalloc as the allocator. Earth's load average was 4 to 5 (no Flow jobs); runs alternate with Git.

mimalloc alone took one-file status from 0.78 to 0.47 s and clean status from 0.26 to 0.18 s: status allocates hundreds of thousands of small values from every thread, and glibc's arena locks were the largest single cost. The first status after a commit (1.47 s) rebuilds the state index.

Afterwards (22:55), log reads change metadata from the state index and a metadata cache in the store, and its signature marks no longer read each change whole: log -n 100 takes 0.19 to 0.20 s, of which 9 ms is the log itself and the rest the check that the working copy is clean.

Where one-file status goes now (0.47 s): listing the working copy about 150 ms (the syscalls alone: find with a stat of each file takes 0.37 s on one core), loading the state index about 60 ms, reading the manifest about 40 ms, rendering and diffing the changed file about 100 ms.

S1 to S5, second round (2026-09-30 23:54, oxygen#41)

After the second round: merge joins over whole-tree maps, a commit that works out keys only for what it changed and brings the state index up to date by replaying rather than rewriting it, the manifest checksummed and written without fsync, log and blame from the state index, checkout through the index with two indexes kept. Load average 4.2 at the start, no Flow jobs.

The first one-file status after the first commit (1.10 s) builds the state index for the first time. Most of what remains in status is listing the tree (about 135 ms), loading the index (about 60 ms) and starting up; commit adds storing and recording with fsync (about 100 ms).

P5a: flow-casfs, a read-only FUSE mount of a state (2026-10-01, oxygenflow#21)

The first flow-casfs (single-threaded, fuser 0.16, no read-ahead) on the Linux repository on Earth, with the H4 import and Flow jobs running (load average 10 to 11), so the times are high:

diff -r between the mount and the working copy finds no difference. To reach the spec's targets: render in parallel and ahead of reads, take a file added once and never edited straight from its text blob, keep sizes in the render cache so a stat does not render, serve FUSE on several threads, and read the state index lazily for a mount under 100 ms.

Status with work in progress (2026-10-01, oxygen#41)

MAINTAINERS edited on the Linux tree on Earth (load average 2.2, one Flow job running), oxy and Git alternating, three runs each:

Before, the manifest only proved a clean working copy. With uncommitted work every status listed and snapshotted the tree again, loaded the state index and rendered the base (listing 164 ms, index 60 ms, base 33 ms). Now a snapshot leaves a status cache keyed by the working-copy change and the manifest's checksum, and keeps the manifest current for the edited files. A later status that finds every file as the manifest says answers from the cache. Test builds check that answer against a snapshot. Diff still renders the base; it is next.

Then diff (same day, load average 4.2, Git 0.10 s throughout): while the status cache holds, oxy diff renders the stored working-copy change against its base loaded from the state index, rather than snapshotting. 0.34 to 0.35 s and 250 MB (from 0.41 s and 360 MB). Rendering it through change_base, which reads the base's changes rather than the index, was slower than a snapshot (0.9 s) and was not kept. What remains is the status check's listing (about 130 ms), the index (60 ms) and rendering.

Then directory times (same evening, load average 3.1): the manifest (format 3) keeps every directory the walk listed with its time, read before the directory was. While no directory's time has moved, the check stats the tracked files and does not list the tree; a directory whose entries changed sends it back to a listing.

Diff, last round (load average 2.7): a change's base renders only its edited files without first building a set of every other file to leave out, and oxy diff no longer frees its structures just before the process ends. 0.19 to 0.21 s (from 0.25 s; Git 0.08 s). What remains: the stat check 78 ms (Git stats the same files), the state index 58 ms (39 of them building the tree view), the base's full file map 21 ms.

Commit, one file (2026-10-01, oxygen#41)

~/bench/commit-vs-git.sh: commits go forward and both repositories are put back at the end (an oxy undo between runs dropped the manifest, and its rehash of the tree evicted the page cache Git's runs relied on, which made both sides' numbers meaningless). Load average 3 to 4.

After a status, the commit takes the status's snapshot from the cache. Straight after an edit, the stat check now hands its stats to the snapshot as its listing (8 ms instead of 122 ms), but checks every file rather than stopping at the first difference, so the gain is small. What remains is mostly durability: 18 fsyncs take 210 ms of the commit (one alone 120 ms). Git does not sync loose objects by default.

Fewer waits on the disk (same evening): a batch's loose objects are written together, synced in parallel, renamed and their directories synced once each; a commit runs in one batch; packing keeps the emptied fan-out directories, and a missing one has all 256 created with one sync of loose (that sync alone took 130 ms on a busy disk). Serial syncs per commit fall from 18 to about 7; each still waits on the same ext4 journal commit, so the gain is modest. Old and new builds alternating, five commits after a status each: median 0.39 s before, 0.34 s after (Git 0.14 to 0.16 s).

A commit's own snapshot (straight after an edit) no longer stores its working-copy change or records an operation for it: the commit's operation, recorded straight after, replaces the working copy. Old and new builds alternating, six direct commits each (load average 4.4): median 0.61 s before, 0.58 s after (Git 0.16 to 0.27 s at that load).

log -n 100 (2026-10-02, oxygen#41)

oxy log no longer checks the files on disk: its working-copy row describes the working copy as last recorded (with edits, or not; the manifest's conflict flag), as Git's log never looks at the working tree. The entries come from the state index's change metadata.

What remains on oxygen is checking each row's signature (30 ms for 100).

Against Git tuned for large repositories (2026-10-02)

Git as it comes compares an untuned Git with an oxy that uses every cache it has. ~/bench/vs-git-tuned.sh adds a copy of the Git repository tuned as Git recommends for large ones: feature.manyFiles (index v4 and more), core.untrackedCache, and a commit-graph. Not core.fsmonitor: Git 2.53 has no file-watcher daemon on Linux, and Watchman is not installed. Earth, load average 1.6, three runs each, alternating.

Tuned Git is ahead of oxy on all five. Its untracked cache is what oxy's directory times do, and with it Git's status is a little faster than oxy's. oxy beats Git as it comes on status only. Claims of speed against Git should say which Git.

Then the stat check by directory (same night): tracked files are grouped by directory, each directory opened once, and each file statted relative to it (fstatat), so a stat resolves one name rather than every component from the root (the kernel's path lookup was most of a clean status). The stat check went from 84 ms to 73 ms; a clean status is 0.10 to 0.12 s, level with tuned Git (0.10 to 0.11 s). What is left is the 96 000 stat calls themselves, the same work Git does; doing less needs a file watcher (Git's fsmonitor), which oxy does not have.

Blame, and the tree view (2026-10-02, oxygen#41)

oxy blame writes through one buffered lock on stdout and formats each change's id, author and age once (30 219 lines of MAINTAINERS share a few changes); printing went to 4 ms. The state index's tree view is built in one step from its sorted paths rather than by an insert per path: 37 ms to 21 ms, for every command that loads the index. Quiet Earth (load average 0.4), three runs each, alternating:

Checkout, first measurement against Git (2026-10-02, oxygen#41)

~/bench/checkout-vs-git.sh: to the first state (1 001 files differ) and back, oxy, Git and tuned Git alternating, three each (load average 1.6).

Where a checkout's time goes: the manifest check 134 ms (listing: see below), planning 67 ms, the 1 001 texts read from the store (134 ms) or their graphs built (141 ms) going back, writing 65 to 82 ms, recording 115 to 143 ms, the manifest 45 ms.

Found on the way, now fixed: a manifest written by a checkout kept no directory times, and directories a checkout had just written stayed within the racy window of the manifest's time for good, so every status after a checkout listed the tree. A clean check that verified the tree now restamps the manifest with the time it began, keeping the directories it listed (Git restamps its index the same way): the first status after a checkout takes 0.15 s, the ones after it 0.10 s again.

Commit: one manifest write, parallel diffs (2026-10-02, oxygen#41)

A commit's snapshot now hands its manifest rows to the commit, which writes the manifest once with the new base's graph keys; before, the snapshot wrote it and the commit read and rewrote it. The change builder diffs edited text files in parallel before updating them one by one.

(The Git figure for 1 000 files was first taken while OxygenFlow was building on Earth, as 1.1 to 1.2 s; quiet, Git takes 0.26 s.)

Of the one-file commit, about 135 ms is the operation's fsync: on ext4 with ordered data the first sync also writes the edited file's data. Git's default commit does not sync loose objects. Of the 1 000-file commit: building the change 72 ms (from 243), one read of the Linux import's root change for the inline texts of about 90 of the files (110 ms), rendering 97 ms, graph building 67 ms, the stat check 80 ms.

Fresh store size and first commit (2026-10-02, P1 and S7)

~/bench/p1.sh: the Linux tip (95 942 files, 1.8 GB) committed fresh by oxy and by Git in scratch copies, then gc.

The bench repository's store is larger (290 MB of objects) because builds before this one also stored the working-copy change a first commit's snapshot made: an 85 MB change (21 MB packed) twice. A commit's snapshot no longer stores it. The first commit's peak memory (S7, under 2 GB) is still open: the walk holds every file's text.

Later the same night (oxygen#41): a commit's own snapshot no longer encodes the working-copy change, oxy commit reads the new change's metadata rather than the whole change to print its id (6 s here), line ends take four bytes, inline lines are encoded without a value per line, and stored texts go at most 32 MiB a batch (a directory of generated register headers made one batch of hundreds of megabytes).

Under 2 GB needs the texts out of memory once read (stored as the walk reads them, the change holding references): the design in the backlog.

Closing summary of backlog 4 (2026-10-02 04:30, Z1)

~/bench/z1.sh on a quiet Earth (load 0.5 to 1.0, no jobs), the build of 3d6b6fdd: vs-git-tuned.sh, commit-vs-git.sh, commit1000.sh, checkout-vs-git.sh, fsck twice each, and p1.sh (fresh copies of the tip). Git is 2.53; "tuned" is feature.manyFiles, core.untrackedCache and a commit-graph.

The first commit's time varies with the page cache: 12.4 s when the tree was read just before, 17.4 s here.

Where oxy is ahead of Git: the first commit, fsck, gc and the store. Level with tuned Git (and twice as fast as Git as it comes): status, clean and with an edit, by a stat check per directory against the manifest. Close: blame (0.08 against 0.07 s), log (0.03 s, under the 0.05 s target, against Git's 0.00 s).

Behind, and why:

• Diff (twice Git): the status check costs what Git's whole diff does (70 ms), and loading the state index adds 40 ms (its tree view 20 ms), plus about 25 ms of process and manifest reading. A lazily read (memory-mapped) state index is the next step.
• Commit (two to three times Git): about 120 ms of a one-file commit is the operation's fsync, which Git's default commit does not make; on ext4 it is the journal commit, not the data (starting writeback early changed nothing). The rest is the stat check (80 ms), the index load (40 ms) and the manifest written whole (30 to 60 ms). For 1 000 files, one read of the Linux import's root change for the inline texts of about 90 of them (110 ms) and rendering (97 ms) add to it.
• Checkout (three times Git): the fsync again (110 to 145 ms), every text read verified and decompressed (130 ms), the status check before it (115 ms), planning (67 ms) and the manifest (72 ms).
• First commit memory: 2.6 GB resident (2.07 GiB allocated) against the 2 GB target, because the walk holds every file's text until the change is stored. Storing texts as the walk reads them, the change holding references, is the remaining design.

With 1 000 files edited (~/bench/k1000.sh, the same night):

The first status takes the snapshot (reading, diffing and storing the 1 000 files, and recording it); later ones answer from the status cache.

Not re-measured tonight: a local clone.