Backlog: O10e done (oxygen#4); OxygenFlow dogfooding F1-F5 first, then O10f and O10g b037ade16953
- Author
- Andy McCall @andymccall <[email protected]>
- When
- · 2026-09-28 17:10 UTC
- Signature
- ✔ signed by Andy McCall
- Hash
b037ade16953e826ad80149a11f2cad29ad3010615d795dd395e1fc3711f5463- Depends on
- 65aebbe49056 75036530b621 f97d3556db5f
- Depended on by
- c1fe1fb25021 cc7a00565161 3c5c6ad2518a 9a239f064f76 9ce6f08451b1 85df6b4c4d9c 66eb646918c2 28fee1c9eb6c 6f16ce2be33b
- In bookmarks
- agents-in-readme bare-hashes carried-not-removed carry-identical doctor-trust identical-sides keyless-tags main merge-remote-bookmark only-help-103 paths-for-resolutions-and-amend push-notes show-stat subrepo-pin-state switch-says
- In tags
- v0.1.0 v0.10.0 v0.11.0 v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.4.1 v0.4.2 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.5.0 v0.5.1 v0.6.0 v0.7.0 v0.8.0 v0.9.0
- Edits
- 3 atoms, 0 tree operations, 1 file
Browse the tree at this change
modified docs/backlog-overnight-2.md +19 −1
| … | 54 unchanged lines |
||
| 55 | 55 | - [x] O12 `log_for_path_in` and `blame_in` rebuild and re-render the whole state on every call, and walk every change: take an already loaded state, stop at the first change touching the path when only the newest is wanted, and benchmark it (found in H16). |
|
| 56 | 56 | - [x] O10c Snapshot cost in the in-memory state: faster graph hashing, no unused predecessor map, inline successor sets, lines moved not copied; `OXY_TIMINGS=1` phase timings. |
|
| 57 | 57 | - [x] O10d Skip reading text files whose stat matches a manifest that recorded them equal to the base (racy entries, those modified at or after the manifest was written, are still read); test that no edit is ever missed. |
|
| 58 | - | - [ ] O10e Checkpoints readable per file so a snapshot loads only the file graphs it needs; cache materialized states. Changes the checkpoint format (spec §25); agreed by Andy on 2026-09-28: new oxy reads both formats and writes the new one. |
|
| 58 | + | - [ ] F1 **Priority (Andy, 2026-09-28): dogfood OxygenFlow.** Every push to oxygen, oxygenhub, oxygenflow and both extensions lints, tests and builds through OxygenFlow on Earth, with results as checks on OxygenHub. F1: a build sandbox for the runner: a rootfs with the Rust toolchain (stable, rustfmt, clippy) and Node, resource limits that fit a cargo build, and dependencies without open network (a crate and npm cache or vendoring); measured on Earth. |
|
| 59 | + | - [ ] F2 oxygen pipeline (.flow/pipeline.kdl): fmt, clippy -D warnings, tests, release build; green on a real push. |
|
| 60 | + | - [ ] F3 oxygenhub pipeline: needs oxygen alongside (path dependencies), so the job checks out oxygen at a pinned change; fmt, clippy, tests, build. |
|
| 61 | + | - [ ] F4 Extension pipelines (oxygen-vscode, oxygenhub-vscode): typecheck, bundle, tests (with oxy, and the hub binary for the hub client tests), package; and oxygenflow's own pipeline gains cargo fmt, clippy and tests. |
|
| 62 | + | - [ ] F5 Required checks: each repository's default bookmark requires its pipeline's checks to land, and the work loop pushes and waits for green instead of gating only locally. |
|
| 63 | + | - [x] O10e Checkpoint payload version 2: large file graphs as shared blobs named from an index, read and decoded in parallel; readers accept both versions (spec §11). Agreed by Andy on 2026-09-28. |
|
| 64 | + | - [ ] O10f Load file graphs lazily from a version 2 checkpoint: a state keeps unread graphs by reference and reads one when first needed. |
|
| 65 | + | - [ ] O10g Render only what a snapshot needs: the manifest records, per file, the graph blob it matched, so an unchanged file whose graph is untouched since the checkpoint is neither read nor rendered; cache materialized states. |
|
| 59 | 66 | - [x] O5 Concurrency: two processes committing, snapshotting and pushing on one repository; locks hold, nothing is lost. |
|
| 60 | 67 | - [x] O6 Error messages audit: every user-facing error says what to do next; `oxy doctor` checks identity, config, remotes, CA files, hub reachability. |
|
| 61 | 68 | - [x] O7 Shell completions (`oxy completions bash|zsh|fish`) and man pages generated from clap. |
|
| … | 211 unchanged lines |
||
| 273 | 280 | - after a commit the edits become the base. |
|
| 274 | 281 | - Spec §12.4. Gate green; release binary rebuilt. The hub (bare repositories) is unaffected. |
|
| 275 | 282 | - Next is O10e, which waits for Andy's agreement to change the checkpoint format. |
|
| 283 | + | - 2026-09-28 O10e done (oxygen#4, closed by the change). Spec §11, Draft 0.8j. |
|
| 284 | + | - **Format.** Payload version 2: a file graph encoded in 2 KiB or more is a blob of its own, named from an index; smaller ones stay inline. Blobs are content addressed, so unchanged files are shared between checkpoints. |
|
| 285 | + | - **Compatibility.** Readers accept both versions; writers produce version 2. gc keeps the graph blobs, fsck checks them against a full load in the payload's own version, and shallow-clone bundles carry them. |
|
| 286 | + | - **Speed.** The first cut was slower than version 1, because it read 10 000 blobs one at a time. So the store gained `get_many`, which reads loose objects in parallel and packed ones in parallel after one index pass, and the graphs are decoded in parallel. |
|
| 287 | + | - **Result at 10 000 files, 216 changes** (the same binary reading both versions, alternated, medians): load through a checkpoint 615 ms (v1) to 342 ms (v2). The index is 2.4 MB against a 62 MB v1 payload. A second checkpoint after a one-file edit took 0.63 s against 1.08 s. |
|
| 288 | + | - **Tests.** Unit tests for the v2 round trip, determinism, missing blobs and unknown versions, and `get_many` over loose, packed and missing objects. A repository test covers a large file's graph blob through gc, fsck, a shared second checkpoint, the shallow bundle, an old v1 checkpoint still loading, and a lost blob failing loudly. |
|
| 289 | + | - Gates green for oxygen and oxygenhub; release binary rebuilt; hub redeployed. |
|
| 290 | + | - **Andy:** oxy older than this cannot read checkpoints written from now on. Rebuild every copy, as ~/.local/bin/oxy already is. |
|
| 291 | + | - **New priority from Andy:** dogfood OxygenFlow on every push (F1–F5, ahead of O10f and O10g). |
|
| 292 | + | - The runner is live on Earth, but only oxygenflow has a pipeline, of light shell jobs in a 256 MiB, one-CPU Alpine microVM. |
|
| 293 | + | - No Rust or Node sandbox exists yet. |