OXYGENHUB

Backlog 4: H4's memory, what shedding did and what is next (refs #25) bdddd0c53720

Author
Andy McCall @andymccall <[email protected]>
When
· 2026-10-01 06:23 UTC
Signature
✔ signed by Andy McCall
Hash
bdddd0c53720d9ff6deb4beecb1a3a2107aeb4c0bdecb10e8bfdd8ec4613df7c
Depends on
3110dc5bea4c 61e9ad6ddc2d
Depended on by
bbe654f95806
In bookmarks
agents-in-readme bare-hashes carried-not-removed carry-identical doctor-trust identical-sides keyless-tags main merge-remote-bookmark only-help-103 paths-for-resolutions-and-amend push-notes show-stat subrepo-pin-state switch-says
In tags
v0.10.0 v0.11.0 v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.4.1 v0.4.2 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.5.0 v0.5.1 v0.6.0 v0.7.0 v0.8.0 v0.9.0
Edits
1 atom, 0 tree operations, 1 file

Browse the tree at this change

hide whitespace changes
modified docs/backlog-overnight-4.md +2 −0
… 
186 unchanged lines
187 187 
- 2026-10-01 05:15 **P5 started.** Design in the hub spec (13.3, oxygenhub#69) and the OxygenFlow spec (5.1, oxygenflow#21): a state is mounted from its state index, text files as renderings cached by graph key, opaque files as blobs; the editor runs in a microVM on a runner and commits go to a workspace bookmark signed by the hub as the user's delegate. **P5a prototype** (`flow-casfs`, 42236046): on the Linux tip, on a loaded Earth, mount 0.5 to 0.9 s, first read 0.1 s, cold full read 52 s, warm 0.65 s, identical to the working copy under `diff -r`. To do for the targets: several FUSE threads, bounded memory, read-ahead, a lazy state index. The Flow image is rebuilt with fuser vendored and pinned in all three repositories. P5b (Open in Workspace) is next, but its microVM image (openvscode-server) and the hub proxy are a large piece that is better started with Andy's eye on the design.
188 188 

    

  
    

      189
      189
       
      
- 2026-10-01 05:40 **oxygenhub#41 parts 2b, 3 and 4** (deployed; hub runs #69 to #71 and oxygenflow #39 pass): Settings → Runners for repository admins and organisation owners; `runs-on` labels in pipelines matched at claim; the queue on Admin → Runners and the runner API. Only part 5 is left: Andy's Plex and VisionFive machines.

    

  
    

      
      190
      +
      

    

  
    

      
      191
      +
      
- 2026-10-01 07:25 **H4 again (item 6).** The full vscode import (166 425 commits) was OOM-killed at the 14 GB cap after 2 h 18 min. A per-commit trace (`OXY_IMPORT_TRACE=1`, 5765d0a8) showed why: ordinary commits run at 2 to 4 GB, but merges of long-lived branches (1 500 to 5 000 changes) applied those changes to built graphs that kept states share, copying them: 10 to 13.4 GB. `State::shed_graphs` (graphs back into deferred histories, each from a stored checkpoint-form blob) is now used before merges of over 100 commits and on kept states over 2 000 graphs (39d71b23); the 10 000-commit slice is 146 s and 1.6 GB (from 136 s and 2.2 GB), verified. On the full import that took the peak to 9.6 GB so far, but at a cost: about 5 600 commits per 30 minutes (from about 22 000 in the first 15), so the whole history would take about 14 hours, and the remaining peaks are merges onto rebuilt bases with no built graphs: the merge's touched files are built, rendered and diffed all at once. Next: build and diff a large merge's files in batches, shed only on merges (not on every kept state) to win the speed back, then run H4 to the end. The run continues on Earth meanwhile (`vscode-trace2`).