The state index in the benchmarks, and the import at scale 60622d23b9e4
refs #41, refs #25
- Author
- Andy McCall @andymccall <[email protected]>
- When
- · 2026-09-30 13:23 UTC
- Signature
- ✔ signed by Andy McCall
- Hash
60622d23b9e4349b711704ecdb5983230ef22e315c90e0a37eed44d709052395- Depends on
- 1d360c91a289 75b6c8665219 b493d993a9df da652d6392d8
- Depended on by
- e4b8bcf04a86
- In bookmarks
- agents-in-readme bare-hashes carried-not-removed carry-identical doctor-trust identical-sides keyless-tags main merge-remote-bookmark only-help-103 paths-for-resolutions-and-amend push-notes show-stat subrepo-pin-state switch-says
- In tags
- v0.1.0 v0.10.0 v0.11.0 v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.4.1 v0.4.2 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.5.0 v0.5.1 v0.6.0 v0.7.0 v0.8.0 v0.9.0
- Edits
- 2 atoms, 0 tree operations, 1 file
Browse the tree at this change
modified docs/benchmarks.md +52 −0
| … | 475 unchanged lines |
||
| 476 | 476 | every file keeps its graph key, which the cache of verified files needs. |
|
| 477 | 477 | The same 5 000 commits verify in 166.5 s (from 978 s), 1.3 GB. |
|
| 478 | 478 | ||
| 479 | + | ### Git import at scale (2026-09-30, oxygen#25) |
|
| 480 | + | ||
| 481 | + | An imported commit cost what the whole tree is: `git ls-tree` of every |
|
| 482 | + | path, a tree view worked out again, every path compared, and every |
|
| 483 | + | changed file rendered from its whole history. Phases timed over the |
|
| 484 | + | first 5 000 vscode commits (`OXY_TIMINGS`, repeated phases summed) led |
|
| 485 | + | the work, one cost at a time: |
|
| 486 | + | ||
| 487 | + | | Change | 5 000 commits | |
|
| 488 | + | |---|---:| |
|
| 489 | + | | Before | 312 s | |
|
| 490 | + | | Each commit's tree is its parent's plus the diff, from one `git diff-tree --stdin` | 245 s | |
|
| 491 | + | | A parent not kept has its tree listed, so its child renders only what changed | 225 s | |
|
| 492 | + | | An ordinary commit is built from the paths it changes alone | 178 s | |
|
| 493 | + | | The diff sets aside lines found on one side only, as Git's xdiff does | 91 s | |
|
| 494 | + | | It diffs from the base's rendering rather than rendering again | 81 s | |
|
| 495 | + | | Tree views updated in place; merges add the other side without text | 56 s | |
|
| 496 | + | | File graphs keep their order as edits splice it | 38.8 s | |
|
| 497 | + | ||
| 498 | + | Twenty text diffs took 92 of the 103 s the builder spent: files |
|
| 499 | + | rewritten whole (line endings changed), on which Myers' algorithm is |
|
| 500 | + | quadratic. Rendering a long-lived file walked every line it ever had; |
|
| 501 | + | it now follows its live lines. Every step verified all 5 000 commits |
|
| 502 | + | (13.4 million files) clean. |
|
| 503 | + | ||
| 504 | + | | vscode slice | Before | After | |
|
| 505 | + | |---|---:|---:| |
|
| 506 | + | | 5 000 commits | 312 s, 2.1 GB | 38.8 s, 1.5 GB | |
|
| 507 | + | | 20 000 commits | 4 530 s, 6.7 GB | 925 s, 6.4 GB | |
|
| 508 | + | ||
| 479 | 509 | ### Store size (2026-09-30, backlog P1) |
|
| 480 | 510 | ||
| 481 | 511 | The Linux tip's store was 448 MB, and 390 MB after `oxy gc`, against |
|
| … | 64 unchanged lines |
||
| 546 | 576 | history and its tree view (about 1 s of each) and rendering what |
|
| 547 | 577 | changed. oxygen#41 proposes a base index that would make those reads |
|
| 548 | 578 | proportional to what changed. |
|
| 579 | + | ||
| 580 | + | ### A state index for the working copy's base (2026-09-30, oxygen#41) |
|
| 581 | + | ||
| 582 | + | A snapshot replayed the base's history (0.70 s) and worked out its tree |
|
| 583 | + | view (0.31 s) every time. Each workspace now keeps its base as a local |
|
| 584 | + | state index: change metadata, the view, the introducers, every file's |
|
| 585 | + | deferred history with its graph key, and the tree's maps, decoded only |
|
| 586 | + | when a command needs them. The index is hashed, and a test build checks |
|
| 587 | + | every read against a full load. The rest of a status that grew with |
|
| 588 | + | the tree now follows the change: the manifest is parsed once and |
|
| 589 | + | rewritten only when that helps; graph keys are looked up only for files |
|
| 590 | + | the manifest knows; and the change is built from the changed paths. |
|
| 591 | + | ||
| 592 | + | Linux tip, MAINTAINERS edited, oxy and Git alternating under the same |
|
| 593 | + | load (Earth at load average 4.5, other jobs running): |
|
| 594 | + | ||
| 595 | + | | | oxy (06:24) | oxy now | Git | |
|
| 596 | + | |---|---:|---:|---:| |
|
| 597 | + | | status | 3.32 s, 641 MB | 1.26 to 1.39 s, 370 MB | 0.26 to 0.29 s | |
|
| 598 | + | | diff | 2.93 s | 1.19 to 1.45 s | 0.16 to 0.25 s | |
|
| 599 | + | | status, clean | 0.68 to 0.76 s | 0.41 to 0.43 s | 0.26 to 0.32 s | |
|
| 600 | + |