Backlog 4: S7 measured, with a design (refs #41) cb9a097def6f
- Author
- Andy McCall @andymccall <[email protected]>
- When
- · 2026-10-01 00:17 UTC
- Signature
- ✔ signed by Andy McCall
- Hash
cb9a097def6fccef38e3bf67beaf7c83af6a229edabcbfce3f72e1daf3ced564- Depends on
- 61e9ad6ddc2d 7f75b41c107d
- Depended on by
- 332c1241a11c
- In bookmarks
- agents-in-readme bare-hashes carried-not-removed carry-identical doctor-trust identical-sides keyless-tags main merge-remote-bookmark only-help-103 paths-for-resolutions-and-amend push-notes show-stat subrepo-pin-state switch-says
- In tags
- v0.10.0 v0.11.0 v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.4.1 v0.4.2 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.5.0 v0.5.1 v0.6.0 v0.7.0 v0.8.0 v0.9.0
- Edits
- 1 atom, 0 tree operations, 1 file
Browse the tree at this change
modified docs/backlog-overnight-4.md +2 −0
| … | 123 unchanged lines |
||
| 124 | 124 | - 2026-10-01 00:40 **S6 done, clone level, P1 partly** (3287e4ff to cc1f1723). fsck 8.2 to 8.7 s (from 13.6 s; Git 12.6 s): objects checked a chunk at a time on every core, graphs built with their lines in parallel, and the last four change encodings kept rather than one (a batch that took inline text from the Linux tip and a later change in turn read the tip again at each switch). Local clone 16.8 to 17.6 s (from 32 s; Git 17.0 s): files added once and never edited are written from their text without a graph (test builds compare with the rendering), new files are written in parallel, rewritten ones one at a time (six times slower in parallel on ext4). A faster local clone would copy the source's packs as Git's `--local` does; that skips the import's checks and brings every object of the source, so it waits for Andy. **P1**: a trained zstd dictionary per pack: pack 293.5 MB (from 301 MB; Git's 295.5 MB), gc 33.7 s (Git 70 s). The whole `.oxy` stays larger than `.git` by the state index (52 MB, a cache; compressing it would slow every load). |
|
| 125 | 125 | ||
| 126 | 126 | - 2026-10-01 01:20 Flow runs #150 to #153 failed: the Rust build image had no `mimalloc` (jobs build offline from vendored crates). Rebuilt the image, imported it on Earth, pinned it in oxygen, oxygenhub and oxygenflow (oxygenflow#20); runs oxygen #154, oxygenhub #61 and oxygenflow #35 pass. Lesson: watch the run after every push, a dependency change above all. |
|
| 127 | + | ||
| 128 | + | - 2026-10-01 01:35 **S7 measured, not done.** The first commit of the Linux tree peaks at 2.7 GB (counting allocator; 3.1 GB resident): the walk holds every file's text (1.6 GB, shared with the change's inserts), building the change adds 0.5 GB, and encoding and storing adds 0.5 GB at the peak. Writing the change's encoding an edit at a time instead of through one CBOR value saved only 50 MB, so it was not kept. Under 2 GB needs the text never all in memory: the walk stores each added text file as its text blob as it reads it and keeps only the reference; the builder makes skeleton inserts (lines left out, a `TextRef` beside them, as history-only loads already hold changes); encoding, the commit's re-encoding, history rewrites and adding the change to a state take the skeleton and its texts. Rename detection still reads the contents of added files when files were removed. A day's careful work across objects, snapshot, commit and rewrite: left for a fresh start, with this as the design. |