OXYGENHUB

A verified clean check restamps the manifest; checkout keeps directory times (refs #41) 0389c3e82f6c

Every status after a checkout listed the whole tree: the checkout's
manifest kept no directory times, and the directories it had just written
stayed within the racy window of the manifest's own time for good. A
clean check that verified the tree now restamps the manifest with the
time it began, keeping the directories it listed (as Git restamps its
index); entries changed within two seconds of that time stay racy, which
a test checks with a same-size rewrite straight after. Checkout records
the directories' times itself when the old manifest had them. On the
Linux tree the first status after a checkout takes 0.15 s, the next ones
0.10 s, where all took 0.14 s.
Author
Andy McCall @andymccall <[email protected]>
When
· 2026-10-02 00:23 UTC
Signature
✔ signed by Andy McCall
Hash
0389c3e82f6c6455a70b858706018bb1df88650071b8ec0c1471831ada895686
Depends on
135212de5370 1d360c91a289 1ec3f008e96a 3604179679eb 69692f14b80f 75167672ebcd 86e1824f2d0b 982864068a59 b7f102fcbb17 c5ce0367eddf d0990c2bff74 d3cd7ef95da6 d4dbab97e948 da868d5d2646 e226a6cc2f71 fddf64baa3d2
Depended on by
4476cdbb0443 4f264d6f20b6 236f35c0baba
In bookmarks
agents-in-readme bare-hashes carried-not-removed carry-identical doctor-trust identical-sides keyless-tags main merge-remote-bookmark only-help-103 paths-for-resolutions-and-amend push-notes show-stat subrepo-pin-state switch-says
In tags
v0.10.0 v0.11.0 v0.3.0 v0.3.1 v0.4.0 v0.4.1 v0.4.2 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.5.0 v0.5.1 v0.6.0 v0.7.0 v0.8.0 v0.9.0
Edits
15 atoms, 0 tree operations, 5 files

Browse the tree at this change

modified crates/oxy-core/src/repo/checkout.rs +34 −1
…
311 unchanged lines
312 312
        }
313 313
314 314
        drop(_render);
315 -
        let _t = crate::timing::phase("checkout: record and manifest");
315 +
        let recording = crate::timing::phase("checkout: record");
316 316
        let (op_ref, op) = self.head_op()?;
317 317
        let wc = self.new_working_copy(heads.clone(), "");
318 318
        let wc_hash = self.store.put_change(&wc)?;
319 319
        let mut view = op.view.clone();
320 320
        view.workspaces.insert(self.workspace, wc_hash);
321 321
        self.record_op(&[op_ref], command, view)?;
322 +
        drop(recording);
323 +
        let _t = crate::timing::phase("checkout: manifest");
324 +
        // The directories the new tree has, with any the old manifest kept
325 +
        // that still exist (holding only ignored files, say), statted now,
326 +
        // after the writes: the next check then need not list the tree. A
327 +
        // directory written just now is within the racy window, so for a
328 +
        // moment the check lists anyway.
329 +
        if !old_manifest.dirs.is_empty() {
330 +
            let mut dirs: BTreeSet<String> =
331 +
                old_manifest.dirs.iter().map(|(d, _)| d.clone()).collect();
332 +
            dirs.insert(String::new());
333 +
            for p in entries.keys() {
334 +
                let mut at = p.as_str();
335 +
                while let Some((parent, _)) = at.rsplit_once('/') {
336 +
                    if !dirs.insert(parent.to_string()) {
337 +
                        break;
338 +
                    }
339 +
                    at = parent;
340 +
                }
341 +
            }
342 +
            use rayon::prelude::*;
343 +
            let root = &self.root;
344 +
            let dirs: Vec<(String, u128)> = dirs
345 +
                .into_par_iter()
346 +
                .filter_map(|d| {
347 +
                    let m = fs::metadata(root.join(&d)).ok().filter(|m| m.is_dir())?;
348 +
                    Some((d, prompt::mtime_ns(&m)))
349 +
                })
350 +
                .collect();
351 +
            let mut dirs = dirs;
352 +
            dirs.sort_unstable();
353 +
            *self.walked_dirs.borrow_mut() = Some((dirs, old_manifest.nested.clone()));
354 +
        }
322 355
        self.write_manifest_entries(report.conflicted, false, &entries, &HashMap::new())?;
323 356
        let _ = self.run_hook(
324 357
            "post-checkout",
…
134 unchanged lines
modified crates/oxy-core/src/repo/mod.rs +6 −0
…
234 unchanged lines
235 235
    /// The directories (with their times) and nested repositories of the
236 236
    /// last walk, for the manifest written from it.
237 237
    pub(crate) walked_dirs: std::cell::RefCell<Option<WalkedDirs>>,
238 +
    /// The directories a manifest check listed, for its restamp only.
239 +
    pub(crate) listed_dirs: std::cell::RefCell<Option<WalkedDirs>>,
238 240
}
239 241
240 242
/// A walk's directories with their times, and its nested repositories.
…
65 unchanged lines
306 308
            last_change: Default::default(),
307 309
            at_op: None,
308 310
            walked_dirs: Default::default(),
311 +
            listed_dirs: Default::default(),
309 312
            git_export_skipped: Default::default(),
310 313
            manifest_cache: Default::default(),
311 314
            listing_cache: Default::default(),
…
49 unchanged lines
361 364
            last_change: Default::default(),
362 365
            at_op: None,
363 366
            walked_dirs: Default::default(),
367 +
            listed_dirs: Default::default(),
364 368
            git_export_skipped: Default::default(),
365 369
            manifest_cache: Default::default(),
366 370
            listing_cache: Default::default(),
…
71 unchanged lines
438 442
                    last_change: Default::default(),
439 443
                    at_op: None,
440 444
                    walked_dirs: Default::default(),
445 +
                    listed_dirs: Default::default(),
441 446
                    git_export_skipped: Default::default(),
442 447
                    manifest_cache: Default::default(),
443 448
                    listing_cache: Default::default(),
…
26 unchanged lines
470 475
                    last_change: Default::default(),
471 476
                    at_op: None,
472 477
                    walked_dirs: Default::default(),
478 +
                    listed_dirs: Default::default(),
473 479
                    git_export_skipped: Default::default(),
474 480
                    manifest_cache: Default::default(),
475 481
                    listing_cache: Default::default(),
…
1624 unchanged lines
modified crates/oxy-core/src/repo/prompt.rs +50 −0
…
157 unchanged lines
158 158
/// came from: reused while the file is the same one.
159 159
pub(crate) type CachedManifest = ((u64, u128, u64), (bool, bool, std::sync::Arc<Manifest>));
160 160
161 +
/// Now, in nanoseconds since the epoch.
162 +
fn now_ns() -> u128 {
163 +
    std::time::SystemTime::now()
164 +
        .duration_since(UNIX_EPOCH)
165 +
        .map(|d| d.as_nanos())
166 +
        .unwrap_or(0)
167 +
}
168 +
161 169
pub(crate) fn mtime_ns(m: &fs::Metadata) -> u128 {
162 170
    m.modified()
163 171
        .ok()
…
466 unchanged lines
630 638
        if conflicted || uncommitted || self.root.join(".oxymodules").exists() {
631 639
            return not("conflicted, uncommitted or subrepos");
632 640
        }
641 +
        let started = now_ns();
633 642
        if self.disk_differs(&m, true)? {
634 643
            return not("a file differs");
644 +
        }
645 +
        // Verified clean. Entries or directories within the racy window of
646 +
        // the manifest's time were hashed or listed just now; restamped with
647 +
        // the time this check began, the next check can trust them, as
648 +
        // Git restamps its index (anything changed within two seconds of
649 +
        // that time is still racy by the same rule).
650 +
        // A check that had to list the tree leaves the directories it
651 +
        // listed: kept, the next check need not list.
652 +
        let window = 2_000_000_000;
653 +
        let listed = self.listed_dirs.take();
654 +
        let racy = m.dirs.iter().any(|(_, t)| t + window >= m.written_ns)
655 +
            || m.entries
656 +
                .values()
657 +
                .any(|e| e.mtime_ns + window >= m.written_ns);
658 +
        if (racy || (m.dirs.is_empty() && listed.is_some())) && started > m.written_ns {
659 +
            let (dirs, nested) = listed.unwrap_or_else(|| (m.dirs.clone(), m.nested.clone()));
660 +
            self.restamp_manifest(&m, conflicted, uncommitted, started, &dirs, &nested);
635 661
        }
636 662
        Ok(true)
663 +
    }
664 +
665 +
    /// Rewrite the manifest with a new time and directory list, its entries
666 +
    /// unchanged.
667 +
    fn restamp_manifest(
668 +
        &self,
669 +
        m: &Manifest,
670 +
        conflicted: bool,
671 +
        uncommitted: bool,
672 +
        written_ns: u128,
673 +
        dirs: &[(String, u128)],
674 +
        nested: &[String],
675 +
    ) {
676 +
        let _t = crate::timing::phase("status: restamp the manifest");
677 +
        let mut rows = m.entries.iter().map(|(p, e)| (p.as_str(), e.clone()));
678 +
        let bytes = encode_manifest(conflicted, uncommitted, written_ns, dirs, nested, &mut rows);
679 +
        self.manifest_cache.take();
680 +
        let _ = write_cache(&self.manifest_path(), &bytes);
637 681
    }
638 682
639 683
    /// A manifest entry written within the racy window whose file no
…
318 unchanged lines
958 1002
                .collect();
959 1003
            nested.sort();
960 1004
            *self.untracked_nested.borrow_mut() = nested;
1005 +
            // A complete list of the directories, each timed before it was
1006 +
            // read, for a restamp to keep (see `clean_by_manifest`).
1007 +
            *self.listed_dirs.borrow_mut() = Some((
1008 +
                listing.dirs.clone(),
1009 +
                listing.nested.iter().map(|d| self.rel_path(d)).collect(),
1010 +
            ));
961 1011
        }
962 1012
        if dirty && keep {
963 1013
            *self.listing_cache.borrow_mut() = Some(listing);
…
79 unchanged lines
modified crates/oxy-core/src/repo/tests.rs +23 −0
…
6071 unchanged lines
6072 6072
    assert_eq!(fs::read(root.join("a.txt")).unwrap(), b"a\nB\n");
6073 6073
}
6074 6074
6075 +
/// oxygen#41: a clean check that verified racy entries restamps the
6076 +
/// manifest, so later checks can trust them, and a same-size rewrite
6077 +
/// straight afterwards is still found: entries within two seconds of the
6078 +
/// new time stay racy.
6079 +
#[test]
6080 +
fn a_verified_clean_check_restamps_the_manifest_safely() {
6081 +
    hermetic();
6082 +
    let dir = tempfile::tempdir().unwrap();
6083 +
    let root = dir.path();
6084 +
    let repo = Repo::init(root).unwrap();
6085 +
    write(root, "a.txt", b"one\n");
6086 +
    repo.commit("one", &[]).unwrap();
6087 +
    let before = repo.read_manifest().unwrap().unwrap().2.written_ns;
6088 +
    assert!(repo.clean_by_manifest().unwrap());
6089 +
    let after = repo.read_manifest().unwrap().unwrap().2.written_ns;
6090 +
    assert!(after > before, "restamped");
6091 +
    write(root, "a.txt", b"two\n");
6092 +
    assert!(
6093 +
        !repo.clean_by_manifest().unwrap(),
6094 +
        "a same-size rewrite is still seen"
6095 +
    );
6096 +
}
6097 +
6075 6098
/// oxygen#35: a history-only state keeps no inserted text. It reads a
6076 6099
/// stored insert's text blob only when it builds that file's graph, and
6077 6100
/// builds the same graph a full load does.
…
708 unchanged lines
modified docs/benchmarks.md +23 −0
…
895 unchanged lines
896 896
| log -n 100 | 0.03 s | 0.00 s | 0.00 s |
897 897
| blame MAINTAINERS | **0.08 s** (from 0.16) | 0.06 s | 0.06 s |
898 898
899 +
### Checkout, first measurement against Git (2026-10-02, oxygen#41)
900 +
901 +
`~/bench/checkout-vs-git.sh`: to the first state (1 001 files differ) and
902 +
back, oxy, Git and tuned Git alternating, three each (load average 1.6).
903 +
904 +
| | oxy | Git | Git, tuned |
905 +
|---|---:|---:|---:|
906 +
| to the first state | 0.65 to 0.71 s | 0.26 to 0.27 s | 0.24 to 0.26 s |
907 +
| and back | 0.71 to 0.73 s | 0.23 to 0.26 s | 0.23 to 0.24 s |
908 +
909 +
Where a checkout's time goes: the manifest check 134 ms (listing: see
910 +
below), planning 67 ms, the 1 001 texts read from the store (134 ms) or
911 +
their graphs built (141 ms) going back, writing 65 to 82 ms, recording
912 +
115 to 143 ms, the manifest 45 ms.
913 +
914 +
Found on the way, now fixed: a manifest written by a checkout kept no
915 +
directory times, and directories a checkout had just written stayed
916 +
within the racy window of the manifest's time for good, so every status
917 +
after a checkout listed the tree. A clean check that verified the tree
918 +
now restamps the manifest with the time it began, keeping the
919 +
directories it listed (Git restamps its index the same way): the first
920 +
status after a checkout takes 0.15 s, the ones after it 0.10 s again.
921 +