OXYGENHUB

Commit takes a status's snapshot from the cache; the stat check lends its stats to the snapshot (refs #41) e226a6cc2f71

snapshot_from_cache builds the snapshot a commit needs while the status
cache holds: the stored working-copy change, its base from the state
index, the .oxyattributes files from disk. When no directory changed, the
manifest check's stats become the listing of the snapshot that follows,
racy files included, so a snapshot after an edit lists no directory.

Fixes the status cache this evening introduced: a mode change moves
neither size nor time, so the manifest was not rewritten when only an
edited file's executable bit changed, and reverting it replayed the old
status. The test build's cross-check found it.

On the Linux tree: commit after a status 0.36 to 0.44 s (from 0.68 s;
Git 0.14 to 0.17 s). 18 fsyncs are now 210 ms of a commit.
Author
Andy McCall @andymccall <[email protected]>
When
· 2026-10-01 20:48 UTC
Signature
✔ signed by Andy McCall
Hash
e226a6cc2f713b7b3f71469ba12d5dcb5543e151909a781928f90a4ae8df95e6
Depends on
10b4fd430eab 135212de5370 1d360c91a289 1ec3f008e96a 3604179679eb 6caacfff1d4f 77445673174b 86e1824f2d0b 8fd78eb52ad2 982864068a59 c07406bcc237 c6aa5bd9d1c8 f9466021286a
Depended on by
236f35c0baba 0389c3e82f6c e96a430c78f3 33472ccc08cc b30a880fa409
In bookmarks
agents-in-readme bare-hashes carried-not-removed carry-identical doctor-trust identical-sides keyless-tags main merge-remote-bookmark only-help-103 paths-for-resolutions-and-amend push-notes show-stat subrepo-pin-state switch-says
In tags
v0.10.0 v0.11.0 v0.3.0 v0.3.1 v0.4.0 v0.4.1 v0.4.2 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.5.0 v0.5.1 v0.6.0 v0.7.0 v0.8.0 v0.9.0
Edits
16 atoms, 0 tree operations, 5 files

Browse the tree at this change

modified CHANGELOG.md +1 −1
…
1 unchanged lines
2 2
3 3
## Unreleased
4 4
5 -
- **Status twice as fast as Git on the Linux tree (#41).** With uncommitted work, `oxy status` answers from a status cache while every file is as the last snapshot left it, and the manifest keeps every directory's time, so the tree is listed only when a directory's entries changed. One edited file is 0.10 to 0.12 s (from 0.44 s; Git 0.22 s) and a clean status 0.09 to 0.12 s (Git 0.20 to 0.26 s), in 80 MB. `oxy diff` renders the working copy from the cache too (0.20 s, from 0.41 s; Git 0.08 s). A file written within the racy window is now hashed by the prompt's check too, as it was by status.
5 +
- **Status twice as fast as Git on the Linux tree (#41).** With uncommitted work, `oxy status` answers from a status cache while every file is as the last snapshot left it, and the manifest keeps every directory's time, so the tree is listed only when a directory's entries changed. One edited file is 0.10 to 0.12 s (from 0.44 s; Git 0.22 s) and a clean status 0.09 to 0.12 s (Git 0.20 to 0.26 s), in 80 MB. `oxy diff` renders the working copy from the cache too (0.20 s, from 0.41 s; Git 0.08 s). A commit after a status takes that status's snapshot instead of walking the tree again (0.37 s, from 0.68 s; Git 0.15 s). A file written within the racy window is now hashed by the prompt's check too, as it was by status.
6 6
7 7
- **Quieter notes (#55).** Notes start with a dimmed `›` instead of `ℹ`, which many terminal fonts draw as a boxed or emoji-style "i". A spinner's final line no longer shows two ticks or runs into the next line.
8 8
…
53 unchanged lines
modified crates/oxy-core/src/repo/prompt.rs +85 −19
…
664 unchanged lines
665 665
    /// walk of the working copy when it does.
666 666
    fn disk_differs(&self, manifest: &Manifest, keep: bool) -> Result<bool> {
667 667
        // A file written within the racy window can change without its size
668 -
        // or time moving (a coarse clock): those entries are hashed.
669 -
        if self.racy_change(manifest).is_some() {
668 +
        // or time moving (a coarse clock): those entries are hashed. One
669 +
        // that changed still lets the directory check below hand its stats
670 +
        // to the snapshot that follows.
671 +
        let racy = self.racy_change(manifest).is_some();
672 +
        if let Some(differs) = self.disk_differs_by_dirs_keeping(manifest, keep, racy) {
673 +
            return Ok(differs || racy);
674 +
        }
675 +
        if racy {
670 676
            return Ok(true);
671 -
        }
672 -
        if let Some(differs) = self.disk_differs_by_dirs(manifest) {
673 -
            return Ok(differs);
674 677
        }
675 678
        self.disk_differs_listing(manifest, keep)
676 679
    }
…
4 unchanged lines
681 684
    /// recorded (and none within the racy window) the files are the
682 685
    /// manifest's, and each is checked by its stat. `None` when a
683 686
    /// directory changed or the manifest has none: the tree must be listed.
687 +
    #[cfg(test)]
684 688
    pub(crate) fn disk_differs_by_dirs(&self, manifest: &Manifest) -> Option<bool> {
689 +
        self.disk_differs_by_dirs_keeping(manifest, false, false)
690 +
    }
691 +
692 +
    /// [`Repo::disk_differs_by_dirs`]; with `keep`, when a file differs
693 +
    /// (or `racy` says one does), the stats taken become the listing of the
694 +
    /// snapshot that follows: no directory's entries changed, so the
695 +
    /// manifest's files are the tree's.
696 +
    fn disk_differs_by_dirs_keeping(
697 +
        &self,
698 +
        manifest: &Manifest,
699 +
        keep: bool,
700 +
        racy: bool,
701 +
    ) -> Option<bool> {
685 702
        use rayon::prelude::*;
686 703
        if manifest.dirs.is_empty() || !self.config.sparse_paths.is_empty() {
687 704
            return None;
…
6 unchanged lines
694 711
        });
695 712
        if !dirs_same {
696 713
            return None;
714 +
        }
715 +
        // Each tracked path as it is now: a file's stat, a symlink's target.
716 +
        enum Now {
717 +
            File(PathBuf, fs::Metadata),
718 +
            Link(Vec<u8>),
697 719
        }
698 -
        let differs = manifest.entries.par_iter().any(|(rel, me)| {
699 -
            let path = root.join(rel.as_str());
700 -
            let Ok(m) = fs::symlink_metadata(&path) else {
701 -
                return true;
702 -
            };
720 +
        let differs_from = |me: &ManifestEntry, path: &Path, now: &Now| match now {
721 +
            Now::Link(t) => *blake3::hash(t).as_bytes() != me.hash,
722 +
            Now::File(_, m) => {
723 +
                m.len() != me.size
724 +
                    || super::is_executable(m) != me.exec
725 +
                    || (mtime_ns(m) != me.mtime_ns
726 +
                        && !fs::read(path).is_ok_and(|b| *blake3::hash(&b).as_bytes() == me.hash))
727 +
            }
728 +
        };
729 +
        let stat = |rel: &str| -> Option<Now> {
730 +
            let path = root.join(rel);
731 +
            let m = fs::symlink_metadata(&path).ok()?;
703 732
            if m.file_type().is_symlink() {
704 -
                return !fs::read_link(&path).is_ok_and(|t| {
705 -
                    *blake3::hash(t.to_string_lossy().as_bytes()).as_bytes() == me.hash
706 -
                });
733 +
                let t = fs::read_link(&path).ok()?;
734 +
                Some(Now::Link(t.to_string_lossy().as_bytes().to_vec()))
735 +
            } else if m.is_file() {
736 +
                Some(Now::File(path, m))
737 +
            } else {
738 +
                None
707 739
            }
708 -
            !m.is_file()
709 -
                || m.len() != me.size
710 -
                || super::is_executable(&m) != me.exec
711 -
                || (mtime_ns(&m) != me.mtime_ns
712 -
                    && !fs::read(&path).is_ok_and(|b| *blake3::hash(&b).as_bytes() == me.hash))
713 -
        });
740 +
        };
741 +
        let differs = if !keep {
742 +
            manifest
743 +
                .entries
744 +
                .par_iter()
745 +
                .any(|(rel, me)| match stat(rel) {
746 +
                    Some(now) => differs_from(me, &root.join(rel.as_str()), &now),
747 +
                    None => true,
748 +
                })
749 +
        } else {
750 +
            let now: Vec<(&String, Option<Now>)> = manifest
751 +
                .entries
752 +
                .par_iter()
753 +
                .map(|(rel, _)| (rel, stat(rel)))
754 +
                .collect();
755 +
            // A path gone or no longer a file: its directory moved, unless
756 +
            // within a clock tick; list the tree to be sure.
757 +
            if now.iter().any(|(_, n)| n.is_none()) {
758 +
                return None;
759 +
            }
760 +
            let differs = now.par_iter().any(|(rel, n)| {
761 +
                let me = &manifest.entries[*rel];
762 +
                differs_from(me, &root.join(rel.as_str()), n.as_ref().unwrap())
763 +
            });
764 +
            if differs || racy {
765 +
                let mut listing = super::walk::Listing {
766 +
                    dirs: manifest.dirs.clone(),
767 +
                    nested: manifest.nested.iter().map(|n| root.join(n)).collect(),
768 +
                    ..Default::default()
769 +
                };
770 +
                for (rel, n) in now {
771 +
                    match n.unwrap() {
772 +
                        Now::File(path, m) => listing.files.push((rel.clone(), path, m)),
773 +
                        Now::Link(t) => listing.symlinks.push((rel.clone(), t)),
774 +
                    }
775 +
                }
776 +
                *self.listing_cache.borrow_mut() = Some(listing);
777 +
            }
778 +
            differs
779 +
        };
714 780
        if !differs {
715 781
            // Nested repositories the manifest has no entry for are not
716 782
            // subrepos: nothing in them is tracked (oxygen#43).
…
182 unchanged lines
modified crates/oxy-core/src/repo/snapshot.rs +53 −2
…
621 unchanged lines
622 622
                                .ok()
623 623
                                .map(|md| (md.len(), super::prompt::mtime_ns(&md)))
624 624
                        });
625 -
                        me.hash != hash || stat != Some((me.size, me.mtime_ns))
625 +
                        // The executable bit too: a mode change moves
626 +
                        // neither size nor time.
627 +
                        me.hash != hash || me.exec != e.exec || stat != Some((me.size, me.mtime_ns))
626 628
                    }))
627 629
            }
628 630
        };
…
173 unchanged lines
802 804
        selection: &Selection,
803 805
    ) -> Result<ChangeHash> {
804 806
        let _wc = self.lock_working_copy()?;
805 -
        let mut snap = self.snapshot_as("oxy commit", true, true)?;
807 +
        let mut snap = match self.snapshot_from_cache()? {
808 +
            Some(snap) => snap,
809 +
            None => self.snapshot_as("oxy commit", true, true)?,
810 +
        };
806 811
        if snap.wc.tree_ops.is_empty() && snap.wc.edits.is_empty() {
807 812
            return fail("nothing to commit: the working copy matches its base");
808 813
        }
…
732 unchanged lines
1541 1546
            None => self.working_copy_with_base()?,
1542 1547
        };
1543 1548
        Ok(diff_files_of(&c, &base, &base_snapshot))
1549 +
    }
1550 +
1551 +
    /// The snapshot a command would take, without taking it, when the
1552 +
    /// status cache holds: every file is as the last snapshot left it, so
1553 +
    /// that snapshot's working-copy change is the answer. Its base comes
1554 +
    /// from the state index, the `.oxyattributes` files from disk. `None`
1555 +
    /// when a snapshot must be taken, and when there is nothing to commit
1556 +
    /// (the snapshot then says so). A conflicted base never gets here: the
1557 +
    /// cache does not hold for one.
1558 +
    pub(crate) fn snapshot_from_cache(&self) -> Result<Option<SnapshotResult>> {
1559 +
        let Some(statuses) = self.status_by_manifest()? else {
1560 +
            return Ok(None);
1561 +
        };
1562 +
        if statuses.is_empty() {
1563 +
            return Ok(None);
1564 +
        }
1565 +
        let _t = crate::timing::phase("snapshot: from the status cache");
1566 +
        let (wc_hash, wc) = self.working_copy()?;
1567 +
        let base = self.load_state_indexed(&wc.deps)?;
1568 +
        let (base, base_snapshot) = self.render_base_for(&wc, base)?;
1569 +
        let attribute_files: Vec<(String, Vec<u8>)> = match self.read_manifest()? {
1570 +
            Some((_, _, m)) => m
1571 +
                .entries
1572 +
                .keys()
1573 +
                .filter(|p| is_attributes(p))
1574 +
                .filter_map(|p| fs::read(self.root.join(p)).ok().map(|b| (p.clone(), b)))
1575 +
                .collect(),
1576 +
            None => Vec::new(),
1577 +
        };
1578 +
        let attributes = crate::attributes::Attributes::from_files(
1579 +
            attribute_files
1580 +
                .iter()
1581 +
                .map(|(p, b)| (p.as_str(), b.as_slice())),
1582 +
        );
1583 +
        Ok(Some(SnapshotResult {
1584 +
            wc_hash,
1585 +
            wc,
1586 +
            base,
1587 +
            base_snapshot,
1588 +
            statuses,
1589 +
            changed: false,
1590 +
            resolves: false,
1591 +
            contributors: Vec::new(),
1592 +
            attributes,
1593 +
            untracked_nested: self.untracked_nested.borrow().clone(),
1594 +
        }))
1544 1595
    }
1545 1596
1546 1597
    /// The working-copy change with its base state and the base's
…
1259 unchanged lines
modified crates/oxy-core/src/repo/tests.rs +39 −0
…
6032 unchanged lines
6033 6033
    assert_eq!(repo.disk_differs_by_dirs(&m), None);
6034 6034
}
6035 6035
6036 +
/// oxygen#41: a commit right after a status takes the snapshot the status
6037 +
/// left (the status cache holds) rather than walking again, and commits
6038 +
/// exactly what a snapshot would have.
6039 +
#[test]
6040 +
fn a_commit_after_a_status_uses_the_cached_snapshot() {
6041 +
    hermetic();
6042 +
    let dir = tempfile::tempdir().unwrap();
6043 +
    let root = dir.path();
6044 +
    let repo = Repo::init(root).unwrap();
6045 +
    write(root, "a.txt", b"a\nb\n");
6046 +
    write(root, ".oxyattributes", b"*.bin binary\n");
6047 +
    repo.commit("base", &[]).unwrap();
6048 +
    write(root, "a.txt", b"a\nB\n");
6049 +
    write(root, "new.txt", b"new\n");
6050 +
    let snap = repo.snapshot_status("oxy status").unwrap();
6051 +
    let cached = repo
6052 +
        .snapshot_from_cache()
6053 +
        .unwrap()
6054 +
        .expect("the cache holds");
6055 +
    assert_eq!(cached.wc_hash, snap.wc_hash);
6056 +
    assert_eq!(cached.statuses, snap.statuses);
6057 +
    assert!(
6058 +
        !cached.attributes.is_empty(),
6059 +
        "the .oxyattributes file was read"
6060 +
    );
6061 +
    let h = repo.commit("edit", &[]).unwrap();
6062 +
    let c = repo.store().get_change(&h).unwrap();
6063 +
    assert_eq!(
6064 +
        c.tree_ops.len() + c.edits.len(),
6065 +
        snap.wc.tree_ops.len() + snap.wc.edits.len()
6066 +
    );
6067 +
    assert_eq!(
6068 +
        repo.status_by_manifest().unwrap(),
6069 +
        Some(vec![]),
6070 +
        "clean after the commit"
6071 +
    );
6072 +
    assert_eq!(fs::read(root.join("a.txt")).unwrap(), b"a\nB\n");
6073 +
}
6074 +
6036 6075
/// oxygen#35: a history-only state keeps no inserted text. It reads a
6037 6076
/// stored insert's text blob only when it builds that file's graph, and
6038 6077
/// builds the same graph a full load does.
…
708 unchanged lines
modified docs/benchmarks.md +19 −0
…
797 unchanged lines
798 798
the stat check 78 ms (Git stats the same files), the state index 58 ms
799 799
(39 of them building the tree view), the base's full file map 21 ms.
800 800
801 +
### Commit, one file (2026-10-01, oxygen#41)
802 +
803 +
`~/bench/commit-vs-git.sh`: commits go forward and both repositories are
804 +
put back at the end (an `oxy undo` between runs dropped the manifest, and
805 +
its rehash of the tree evicted the page cache Git's runs relied on, which
806 +
made both sides' numbers meaningless). Load average 3 to 4.
807 +
808 +
| | before | after | Git `commit -am` |
809 +
|---|---:|---:|---:|
810 +
| commit after a status | 0.68 s | **0.36 to 0.44 s** | 0.14 to 0.17 s |
811 +
| commit straight after the edit | 0.68 s | 0.62 to 0.69 s | 0.14 to 0.17 s |
812 +
813 +
After a status, the commit takes the status's snapshot from the cache.
814 +
Straight after an edit, the stat check now hands its stats to the
815 +
snapshot as its listing (8 ms instead of 122 ms), but checks every file
816 +
rather than stopping at the first difference, so the gain is small. What
817 +
remains is mostly durability: 18 fsyncs take 210 ms of the commit (one
818 +
alone 120 ms). Git does not sync loose objects by default.
819 +