OXYGENHUB

Backlog: O10e done (oxygen#4); OxygenFlow dogfooding F1-F5 first, then O10f and O10g b037ade16953

Author
Andy McCall @andymccall <[email protected]>
When
· 2026-09-28 17:10 UTC
Signature
✔ signed by Andy McCall
Hash
b037ade16953e826ad80149a11f2cad29ad3010615d795dd395e1fc3711f5463
Depends on
65aebbe49056 75036530b621 f97d3556db5f
Depended on by
c1fe1fb25021 cc7a00565161 3c5c6ad2518a 9a239f064f76 9ce6f08451b1 85df6b4c4d9c 66eb646918c2 28fee1c9eb6c 6f16ce2be33b
In bookmarks
agents-in-readme bare-hashes carried-not-removed carry-identical doctor-trust identical-sides keyless-tags main merge-remote-bookmark only-help-103 paths-for-resolutions-and-amend push-notes show-stat subrepo-pin-state switch-says
In tags
v0.1.0 v0.10.0 v0.11.0 v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.4.1 v0.4.2 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.5.0 v0.5.1 v0.6.0 v0.7.0 v0.8.0 v0.9.0
Edits
3 atoms, 0 tree operations, 1 file

Browse the tree at this change

modified docs/backlog-overnight-2.md +19 −1
…
54 unchanged lines
55 55
- [x] O12 `log_for_path_in` and `blame_in` rebuild and re-render the whole state on every call, and walk every change: take an already loaded state, stop at the first change touching the path when only the newest is wanted, and benchmark it (found in H16).
56 56
- [x] O10c Snapshot cost in the in-memory state: faster graph hashing, no unused predecessor map, inline successor sets, lines moved not copied; `OXY_TIMINGS=1` phase timings.
57 57
- [x] O10d Skip reading text files whose stat matches a manifest that recorded them equal to the base (racy entries, those modified at or after the manifest was written, are still read); test that no edit is ever missed.
58 -
- [ ] O10e Checkpoints readable per file so a snapshot loads only the file graphs it needs; cache materialized states. Changes the checkpoint format (spec §25); agreed by Andy on 2026-09-28: new oxy reads both formats and writes the new one.
58 +
- [ ] F1 **Priority (Andy, 2026-09-28): dogfood OxygenFlow.** Every push to oxygen, oxygenhub, oxygenflow and both extensions lints, tests and builds through OxygenFlow on Earth, with results as checks on OxygenHub. F1: a build sandbox for the runner: a rootfs with the Rust toolchain (stable, rustfmt, clippy) and Node, resource limits that fit a cargo build, and dependencies without open network (a crate and npm cache or vendoring); measured on Earth.
59 +
- [ ] F2 oxygen pipeline (.flow/pipeline.kdl): fmt, clippy -D warnings, tests, release build; green on a real push.
60 +
- [ ] F3 oxygenhub pipeline: needs oxygen alongside (path dependencies), so the job checks out oxygen at a pinned change; fmt, clippy, tests, build.
61 +
- [ ] F4 Extension pipelines (oxygen-vscode, oxygenhub-vscode): typecheck, bundle, tests (with oxy, and the hub binary for the hub client tests), package; and oxygenflow's own pipeline gains cargo fmt, clippy and tests.
62 +
- [ ] F5 Required checks: each repository's default bookmark requires its pipeline's checks to land, and the work loop pushes and waits for green instead of gating only locally.
63 +
- [x] O10e Checkpoint payload version 2: large file graphs as shared blobs named from an index, read and decoded in parallel; readers accept both versions (spec §11). Agreed by Andy on 2026-09-28.
64 +
- [ ] O10f Load file graphs lazily from a version 2 checkpoint: a state keeps unread graphs by reference and reads one when first needed.
65 +
- [ ] O10g Render only what a snapshot needs: the manifest records, per file, the graph blob it matched, so an unchanged file whose graph is untouched since the checkpoint is neither read nor rendered; cache materialized states.
59 66
- [x] O5 Concurrency: two processes committing, snapshotting and pushing on one repository; locks hold, nothing is lost.
60 67
- [x] O6 Error messages audit: every user-facing error says what to do next; `oxy doctor` checks identity, config, remotes, CA files, hub reachability.
61 68
- [x] O7 Shell completions (`oxy completions bash|zsh|fish`) and man pages generated from clap.
…
211 unchanged lines
273 280
    - after a commit the edits become the base.
274 281
  - Spec §12.4. Gate green; release binary rebuilt. The hub (bare repositories) is unaffected.
275 282
  - Next is O10e, which waits for Andy's agreement to change the checkpoint format.
283 +
- 2026-09-28 O10e done (oxygen#4, closed by the change). Spec §11, Draft 0.8j.
284 +
  - **Format.** Payload version 2: a file graph encoded in 2 KiB or more is a blob of its own, named from an index; smaller ones stay inline. Blobs are content addressed, so unchanged files are shared between checkpoints.
285 +
  - **Compatibility.** Readers accept both versions; writers produce version 2. gc keeps the graph blobs, fsck checks them against a full load in the payload's own version, and shallow-clone bundles carry them.
286 +
  - **Speed.** The first cut was slower than version 1, because it read 10 000 blobs one at a time. So the store gained `get_many`, which reads loose objects in parallel and packed ones in parallel after one index pass, and the graphs are decoded in parallel.
287 +
  - **Result at 10 000 files, 216 changes** (the same binary reading both versions, alternated, medians): load through a checkpoint 615 ms (v1) to 342 ms (v2). The index is 2.4 MB against a 62 MB v1 payload. A second checkpoint after a one-file edit took 0.63 s against 1.08 s.
288 +
  - **Tests.** Unit tests for the v2 round trip, determinism, missing blobs and unknown versions, and `get_many` over loose, packed and missing objects. A repository test covers a large file's graph blob through gc, fsck, a shared second checkpoint, the shallow bundle, an old v1 checkpoint still loading, and a lost blob failing loudly.
289 +
  - Gates green for oxygen and oxygenhub; release binary rebuilt; hub redeployed.
290 +
  - **Andy:** oxy older than this cannot read checkpoints written from now on. Rebuild every copy, as ~/.local/bin/oxy already is.
291 +
  - **New priority from Andy:** dogfood OxygenFlow on every push (F1–F5, ahead of O10f and O10g).
292 +
    - The runner is live on Earth, but only oxygenflow has a pipeline, of light shell jobs in a 256 MiB, one-CPU Alpine microVM.
293 +
    - No Rust or Node sandbox exists yet.