Import keeps no text for files it has not built ab4d4e31b38d
A Git import added each change it made to its base state whole, so for every file whose graph was not built the state kept the change's lines: over a long history, all the text ever inserted (the full vscode import grew past 6 GB in its first hour). The change now joins its base mixed (State::add_known_mixed): a built graph takes its edits whole, and a file not built keeps only where its text is, from the change read back out of the store. The first 5 000 vscode commits import in 314 s at 2.1 GB, every commit verified. refs #35
- Author
- Andy McCall @andymccall <[email protected]>
- When
- · 2026-09-30 03:07 UTC
- Signature
- ✔ signed by Andy McCall
- Hash
ab4d4e31b38d653758ffc3660226b92476669ef21492c178c2bd4fcadb1ee0c2- Depends on
- 79ab8528ff62 86e1824f2d0b b493d993a9df dfb23a3a8cfa ff0af1ee299d
- Depended on by
- 2306a4d24779
- In bookmarks
- agents-in-readme bare-hashes carried-not-removed carry-identical doctor-trust identical-sides keyless-tags main merge-remote-bookmark only-help-103 paths-for-resolutions-and-amend push-notes show-stat subrepo-pin-state switch-says
- In tags
- v0.1.0 v0.10.0 v0.11.0 v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.4.1 v0.4.2 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.5.0 v0.5.1 v0.6.0 v0.7.0 v0.8.0 v0.9.0
- Edits
- 3 atoms, 0 tree operations, 2 files
Browse the tree at this change
modified crates/oxy-core/src/repo/git.rs +4 −1
| … | 855 unchanged lines |
||
| 856 | 856 | heads_now.remove(d); |
|
| 857 | 857 | } |
|
| 858 | 858 | heads_now.insert(hash); |
|
| 859 | - | base.add_known(hash, change, Vec::new()); |
|
| 859 | + | // Built graphs take its edits whole; files not built keep only |
|
| 860 | + | // where its text is, read back from the store. |
|
| 861 | + | let (skeleton, texts) = self.store.get_change_lazy(&hash)?; |
|
| 862 | + | base.add_known_mixed(hash, change, skeleton, texts); |
|
| 860 | 863 | known.imported.insert(gc.sha.clone(), hash); |
|
| 861 | 864 | if waiting.get(gc.sha.as_str()).is_some_and(|n| *n > 0) { |
|
| 862 | 865 | kept.insert(gc.sha.clone(), (base, tree)); |
|
| … | 465 unchanged lines |
modified crates/oxy-core/src/state.rs +57 −0
| … | 514 unchanged lines |
||
| 515 | 515 | h |
|
| 516 | 516 | } |
|
| 517 | 517 | ||
| 518 | + | /// Add a change just stored under hash `h` to a history-only state that |
|
| 519 | + | /// has built some graphs: an edit of a built graph is applied whole |
|
| 520 | + | /// (from `full`), and one of a file not built waits without its text |
|
| 521 | + | /// (from `skeleton` and its `texts`, as the store reads it back), so the |
|
| 522 | + | /// state holds no text it has not built. A Git import adds each change |
|
| 523 | + | /// it makes this way, keeping memory to what changes. |
|
| 524 | + | pub fn add_known_mixed( |
|
| 525 | + | &mut self, |
|
| 526 | + | h: ChangeHash, |
|
| 527 | + | full: Change, |
|
| 528 | + | skeleton: Change, |
|
| 529 | + | texts: Vec<TextRef>, |
|
| 530 | + | ) { |
|
| 531 | + | if self.deferred.is_none() || self.changes.contains_key(&h) { |
|
| 532 | + | self.add_known(h, full, Vec::new()); |
|
| 533 | + | return; |
|
| 534 | + | } |
|
| 535 | + | let Change { |
|
| 536 | + | version, |
|
| 537 | + | change_id, |
|
| 538 | + | deps, |
|
| 539 | + | author, |
|
| 540 | + | hlc, |
|
| 541 | + | message, |
|
| 542 | + | tree_ops, |
|
| 543 | + | edits: full_edits, |
|
| 544 | + | kind, |
|
| 545 | + | } = full; |
|
| 546 | + | let mut texts = texts.into_iter(); |
|
| 547 | + | let mut edits = Vec::with_capacity(full_edits.len()); |
|
| 548 | + | let mut kept = Vec::new(); |
|
| 549 | + | for (f, s) in full_edits.into_iter().zip(skeleton.edits) { |
|
| 550 | + | let text = match &s { |
|
| 551 | + | Edit::Insert { lines, .. } if lines.is_empty() => texts.next(), |
|
| 552 | + | _ => None, |
|
| 553 | + | }; |
|
| 554 | + | if self.files.contains_key(&f.node()) { |
|
| 555 | + | edits.push(f); |
|
| 556 | + | } else { |
|
| 557 | + | edits.push(s); |
|
| 558 | + | kept.extend(text); |
|
| 559 | + | } |
|
| 560 | + | } |
|
| 561 | + | let c = Change { |
|
| 562 | + | version, |
|
| 563 | + | change_id, |
|
| 564 | + | deps, |
|
| 565 | + | author, |
|
| 566 | + | hlc, |
|
| 567 | + | message, |
|
| 568 | + | tree_ops, |
|
| 569 | + | edits, |
|
| 570 | + | kind, |
|
| 571 | + | }; |
|
| 572 | + | self.add_known(h, c, kept); |
|
| 573 | + | } |
|
| 574 | + | ||
| 518 | 575 | /// Add a change read from the store under hash `h`, without hashing it |
|
| 519 | 576 | /// again (for a version 2 change that means chunking all its text). |
|
| 520 | 577 | /// `texts` names, in order, the inserts left without their lines (see |
|
| … | 1403 unchanged lines |