Instant clone: a lazy working copy (oxygen#132)

Status: design, 2026-10-25. Nothing here is built yet beyond what it names as existing. An experimental oxy clone --lazy (Linux) follows if this measures well (§9).

1. The aim

oxy clone URL of any repository, however big, gives a working copy in about a second: ls, cd, editors and builds see every file at once, a file's bytes arrive the first time something reads it, and status, diff, commit and push work at once and cost what the edits cost, not what the repository weighs. Git's partial clone and VFS for Git do part of this (VFS needs Windows and a special server); Oxygen can do all of it with what it already has, because every object is content addressed and a state's tree is known without its file contents.

2. What exists

• Shallow clones (spec §17): oxy clone --depth 0 takes a checkpoint of the bookmark's heads and no history. The checkpoint message brings the checkpoint, its payload blob and every file's graph blob (spec §11, payload version 2; Repo::checkpoint_bundle_for, crates/oxy-core/src/repo/checkpoint.rs). A file graph holds the file's current text and its tombstones, so this is the whole state's text and more: for VS Code most of the clone's bytes.
• Lazy blobs (spec §13.5, §17): --max-blob N leaves opaque blobs larger than N to be fetched with objects on first read; status and commit never need them.
• Sparse filters (--sparse PATH): only those paths are materialized.
• flow-casfs (OxygenFlow, crates/flow-casfs): a read-only FUSE mount of a state from a local repository. The tree comes from the state index; a text file is rendered from its graph on first read or stat and cached on disk by graph key; an opaque file is its blob.
• Content addressing everywhere: an object fetched later is checked against the hash that named it, so a lazily fetched byte is as trustworthy as one cloned eagerly (spec §13).

3. The design

oxy clone --lazy URL DIR:

1. Refs and the checkpoint skeleton. Fetch refs (with the bookmark's attestation, checked against the policy as any clone does), then the checkpoint of its heads without its graph blobs: a checkpoint request with a new graphs: false field (additive; an older server ignores it and sends them, so the clone is merely not lazy). The payload's files entries name each graph blob by ObjectRef (graphs under 2 KiB are inline and come anyway), and its tree gives every path, mode and node. Clone writes the repository as a shallow one at depth 0 whose missing graph blobs are lazy objects, like lazy blobs today.
2. The mount. DIR is a FUSE mount (lower layer) of that state, built on flow-casfs: the directory tree is answered from the payload's tree at once; stat of a text file needs its length, so the payload gains (payload version 3, writer-optional) each file's rendered length, and until a server writes it the length comes with the graph on first stat. A read of a file whose graph is not stored fetches it with objects (hash checked), renders it, keeps both, and answers. Opaque files fetch their blob the same way. A small daemon (oxy mount, started by the clone, restarted by any oxy command that finds it gone) serves the mount; the repository itself stays in DIR/.oxy, which is a real directory beside the mount (bind or an overlay entry), so oxy never reads through FUSE.
3. The writable layer. Writes go to an upper directory (.oxy/overlay/): a written file is copied up whole on first write (its bytes fetched first if they were never read), a new file is created there, a removal is a whiteout, a rename is a whiteout plus a copy up. This is the ordinary overlay model, done in the daemon so it needs no privileges (kernel overlayfs over FUSE needs a user namespace and is not available everywhere).
4. Status, diff and commit read the overlay only. The upper layer is exactly the set of paths that may differ from the base state, so status lists it (comparing each copied-up file's bytes with the base rendering, which is stored, since copying up read it) and never walks the tree. diff and commit do the same, and the changes they build need only the touched files' graphs, all stored by then. commit clears the overlay entries it recorded: the mount's base state moves to the new heads.
5. Fetch, pull, checkout. A fetch brings changes as today (their blobs lazy). Moving the working copy to another state swaps the mount's base: the new state's tree is known from the changes and the stored checkpoint, the overlay is kept when its paths are unchanged between the two states (the --carry rule, §10.1), and graphs needed to render the new state are lazy again.
6. Push. As today: the pushed changes and the blobs they bring. A change built on a lazy base never needs the base's unread files.

4. Offline and failures

A file read once is kept, so it reads offline. Reading a file never read while the hub cannot be reached fails with EIO, and the daemon logs the path and the reason; oxy status says how many files are not local yet. oxy lazy fetch [PATH..] brings them in ahead (a directory, the whole state, or the files a build reads, as a background job), and oxy lazy materialize turns the working copy into an ordinary one (unmounted, every file written), after which nothing is lazy. A hash that does not match is an error for that read, never a wrong byte.

5. Without FUSE

Where FUSE is not available (macOS without macFUSE, Windows, a container without /dev/fuse), --lazy falls back to what exists: a depth-0 clone with lazy blobs, and --sparse paths when given, saying so. Windows ProjFS and macOS File Provider are later backends behind the same daemon interface (lookup, read, write-through to the upper layer).

6. The protocol

• checkpoint gains graphs: false: leave out graph blobs (and their chunks) the payload names by reference.
• objects already fetches any object by hash, with a blob's chunks; it needs batching limits only (as push.batch-bytes bounds pushes).
• Optional later: render by graph key (the rendered bytes rather than the graph with its tombstones), smaller for long-lived files; a client checks it by rendering the graph when it has it, so it is an optimization a client may refuse.
• A host should keep objects cheap: it is now on the read path of editors and builds. OxygenHub's state cache and bucket stores (Strata) already serve objects by hash.

7. Trust

Nothing is weaker than an eager clone. The clone checks the bookmark's attestation and the checkpoint against its heads (a checkpoint is valid only if it reproduces from its heads, spec §11, which a lazy client cannot check without the graphs: it trusts the attested heads and checks each graph it fetches against the checkpoint's reference). oxy verify on a lazy clone checks what is stored and says how much is not.

8. Not in scope

History (log, blame beyond the checkpoint) on a lazy clone fetches changes on demand later (oxy deepen exists); the first version answers log from the checkpoint's change list as a shallow clone does.

9. Measurements

On Earth (4 cores, SSD), 2026-10-25, the VS Code history (~/bench/vscode-full9.oxy, 166k changes; ~/bench/vscode.git for Git) served over loopback by oxy serve and git daemon, oxy at main (0.5.1 + #137), Git 2.53; one run each, a fresh directory, the server warm (its checkpoint of the heads already made). Seconds:

Before #137 the depth-0 clone failed after 11 minutes (the state's binary files' blobs came with nothing) and, before that fix's other half, wrote the checkpoint object by object: stopped after 38 minutes with 57,498 loose objects. The full oxy clone (12 GB of history) was not run tonight.

What this says:

• A depth-0 clone is five times slower than Git's full clone and fifteen times slower than its blobless one. Timed on both sides (OXY_TIMINGS=1, oxygen#137): the server spent 375 s gathering the checkpoint's 57,000 graph blobs and chunks one read at a time (239 s once read in parallel), 19 s encoding one 2 GB answer, and the client 120 s on checkout, 111 s of it rendering 19,545 files on one core. Depth 0 also took the 33,835 head changes, which it no longer does (spec §17), though that was not where the time went. A lazy clone avoids all three: no graph blobs move at clone time, and nothing is rendered until read.
• --max-blob 0 without a mount is slower, not faster: the checkout then fetches every binary file's blob one request at a time. Lazy contents only pay with a mount that reads them when asked.
• The first status after a clone is 8 s (it builds the manifest); Git's is 0.2 s. A lazy clone's status from the overlay (§3.4) would not depend on the tree at all.

So the prototype's bar: clone in seconds (refs, the checkpoint skeleton without graphs, the mount), ls -R at once from the payload's tree, a first read in tens of milliseconds, status in milliseconds; and, whatever happens to --lazy, the depth-0 clone itself should come down to Git's order (streamed checkpoint in parts, rendering on every core).

10. The prototype (L2)

1. checkpoint with graphs: false in oxy-core and oxy-server, and a repository flag for lazy graph objects (fetch on load, like lazy blobs).
2. oxy mount (Linux, fuser, as flow-casfs) with the upper layer, in a new crate oxy-mount beside the CLI, behind a cargo feature so builds without FUSE are unaffected.
3. status/diff/commit from the overlay when the working copy is mounted.
4. oxy clone --lazy, oxy lazy fetch, oxy lazy materialize.
5. Measured as in §9; numbers on oxygen#132.