Status with work in progress from a status cache: ahead of Git on the Linux tree (refs #41) 3604179679eb

- **Status with work in progress ahead of Git (#41).** With uncommitted work, `oxy status` answers from a status cache while every file is as the last snapshot left it: one edited file on the Linux tree is 0.16 s (from 0.44 s; Git 0.20 s), in half the memory.

/// The status cache: a magic number, the working-copy change, the

/// manifest checksum, then each status line (a status byte and a path).

const STATUS_MAGIC: &[u8; 8] = b"oxystat1";

fn status_byte(s: ChangeStatus) -> u8 {

    match s {

        ChangeStatus::Added => 0,

        ChangeStatus::Modified => 1,

        ChangeStatus::Removed => 2,

        ChangeStatus::Renamed => 3,

        ChangeStatus::ModeChanged => 4,

        ChangeStatus::ConflictEdited => 5,

        ChangeStatus::Resolved => 6,

    }

}

fn encode_status_cache(wc: &ChangeHash, stamp: &[u8; 32], statuses: &[PathStatus]) -> Vec<u8> {

    let mut out = Vec::with_capacity(80 + statuses.len() * 40);

    out.extend_from_slice(STATUS_MAGIC);

    out.extend_from_slice(&wc.0);

    out.extend_from_slice(stamp);

    out.extend_from_slice(&(statuses.len() as u32).to_le_bytes());

    for s in statuses {

        out.push(status_byte(s.status));

        out.extend_from_slice(&(s.path.len() as u32).to_le_bytes());

        out.extend_from_slice(s.path.as_bytes());

    }

    out

}

fn decode_status_cache(b: &[u8]) -> Option<(ChangeHash, [u8; 32], Vec<PathStatus>)> {

    let rest = b.strip_prefix(STATUS_MAGIC)?;

    let wc = ObjectRef(rest.get(..32)?.try_into().ok()?);

    let stamp: [u8; 32] = rest.get(32..64)?.try_into().ok()?;

    let n = u32::from_le_bytes(rest.get(64..68)?.try_into().ok()?) as usize;

    let mut at = 68;

    let mut out = Vec::with_capacity(n.min(1 << 16));

    for _ in 0..n {

        let status = match *rest.get(at)? {

            0 => ChangeStatus::Added,

            1 => ChangeStatus::Modified,

            2 => ChangeStatus::Removed,

            3 => ChangeStatus::Renamed,

            4 => ChangeStatus::ModeChanged,

            5 => ChangeStatus::ConflictEdited,

            6 => ChangeStatus::Resolved,

            _ => return None,

        };

        let len = u32::from_le_bytes(rest.get(at + 1..at + 5)?.try_into().ok()?) as usize;

        let path = std::str::from_utf8(rest.get(at + 5..at + 5 + len)?)

            .ok()?

            .to_string();

        out.push(PathStatus { path, status });

        at += 5 + len;

    }

    (at == rest.len()).then_some((wc, stamp, out))

}

        if let Some(rel) = self.racy_change(&m) {

            return not(&format!("{rel} changed within the racy window"));

        }

        if self.disk_differs(&m, true)? {

            return not("a file differs");

        }

        Ok(true)

    }

    /// A manifest entry written within the racy window whose file no

    /// longer matches it. Such entries are hashed, in parallel (a checkout

    /// just wrote them all).

    fn racy_change(&self, m: &Manifest) -> Option<String> {

        racy.par_iter()

            .find_any(|(rel, e)| {

                let path = root.join(rel.as_str());

                let same = match fs::symlink_metadata(&path) {

                    Ok(md) if md.file_type().is_symlink() => fs::read_link(&path).is_ok_and(|t| {

                        *blake3::hash(t.to_string_lossy().as_bytes()).as_bytes() == e.hash

                    }),

                    Ok(md) if md.is_file() => {

                        md.len() == e.size

                            && fs::read(&path).is_ok_and(|b| *blake3::hash(&b).as_bytes() == e.hash)

                    }

                    _ => false,

                };

                !same

            })

            .map(|(rel, _)| rel.to_string())

    }

    /// The status lines of the last snapshot, when the working copy is as

    /// that snapshot left it: the cache it wrote names the working-copy

    /// change and the manifest it recorded, and every file still matches

    /// that manifest. `None` when a snapshot must decide (no cache, a file

    /// changed, conflicts or subrepos). An empty list is a clean working

    /// copy.

    pub fn status_by_manifest(&self) -> Result<Option<Vec<PathStatus>>> {

        // A listing left by an earlier check describes the files as they

        // were then; this check makes its own.

        self.listing_cache.take();

        let Some((conflicted, uncommitted, m)) = self.read_manifest()? else {

            return Ok(None);

        };

        if !uncommitted {

            return Ok(self.clean_by_manifest()?.then(Vec::new));

        let _t = crate::timing::phase("status: by the status cache");

        if conflicted || self.root.join(".oxymodules").exists() {

            return Ok(None);

        }

        let Some((wc, stamp, statuses)) = fs::read(self.status_cache_path())

            .ok()

            .and_then(|b| decode_status_cache(&b))

        else {

            return Ok(None);

        };

        let now = self.view()?.workspaces.get(&self.workspace).copied();

        if now != Some(wc) || self.manifest_stamp() != Some(stamp) {

            return Ok(None);

        }

        if self.racy_change(&m).is_some() || self.disk_differs(&m, true)? {

            return Ok(None);

        }

        // Test builds check the cache against a snapshot of the files.

        #[cfg(test)]

        {

            let snap = self.snapshot_status("status cache check")?;

            let key = |v: &[PathStatus]| {

                let mut k: Vec<(String, u8)> = v

                    .iter()

                    .map(|p| (p.path.clone(), status_byte(p.status)))

                    .collect();

                k.sort();

                k

            };

            assert_eq!(

                key(&statuses),

                key(&snap.statuses),

                "status cache disagrees with a snapshot"

            );

        Ok(Some(statuses))

    }

    fn status_cache_path(&self) -> PathBuf {

        self.manifest_path().with_file_name("status")

    }

    /// The manifest file's checksum, which names its content.

    fn manifest_stamp(&self) -> Option<[u8; 32]> {

        use std::io::Read;

        let mut head = [0u8; MANIFEST_SUM.end];

        fs::File::open(self.manifest_path())

            .and_then(|mut f| f.read_exact(&mut head))

            .ok()?;

        head[MANIFEST_SUM].try_into().ok()

    /// Record `statuses` as the status of working-copy change `wc` against

    /// the manifest as it is now; with no uncommitted work there is nothing

    /// to record, and the cache goes.

    pub(crate) fn write_status_cache(&self, wc: Option<&ChangeHash>, statuses: &[PathStatus]) {

        let path = self.status_cache_path();

        let (Some(wc), Some(stamp)) = (wc, self.manifest_stamp()) else {

            let _ = fs::remove_file(&path);

            return;

        };

        let _ = write_cache(&path, &encode_status_cache(wc, &stamp, statuses));

    }

        Ok(Some(self.disk_differs(&manifest, keep)?))

    }

    /// Whether the working copy differs from `manifest`: a file added,

    /// gone or changed. With `keep`, the listing made is left for the next

    /// walk of the working copy when it does.

    fn disk_differs(&self, manifest: &Manifest, keep: bool) -> Result<bool> {

            return Ok(true);

        Ok(dirty)

                    // With uncommitted work, the status cache below holds

                    // only while the manifest describes the edited files as

                    // they are now.

                    || (uncommitted && statuses.iter().any(|st| {

                        let Some(e) = fs_entries.get(&st.path) else {

                            return m.entries.contains_key(&st.path);

                        };

                        let Some(me) = m.entries.get(&st.path) else {

                            return true;

                        };

                        let hash = e

                            .content_hash

                            .unwrap_or_else(|| *blake3::hash(&e.content).as_bytes());

                        let stat = e.stat.or_else(|| {

                            fs::symlink_metadata(self.root.join(&st.path))

                                .ok()

                                .map(|md| (md.len(), super::prompt::mtime_ns(&md)))

                        });

                        me.hash != hash || stat != Some((me.size, me.mtime_ns))

                    }))

        self.write_status_cache((uncommitted && !conflicted).then_some(&new_hash), &statuses);

/// oxygen#41: with uncommitted work, a status after a snapshot is answered

/// from the status cache while every file is as the snapshot left it, and

/// from a snapshot again as soon as one is not.

#[test]

fn status_with_uncommitted_work_comes_from_the_cache_until_a_file_changes() {

    hermetic();

    let dir = tempfile::tempdir().unwrap();

    let root = dir.path();

    let repo = Repo::init(root).unwrap();

    write(root, "a.txt", b"a\n");

    write(root, "b.txt", b"b\n");

    repo.commit("base", &[]).unwrap();

    assert_eq!(repo.status_by_manifest().unwrap(), Some(vec![]), "clean");

    write(root, "a.txt", b"a\nmore\n");

    assert_eq!(

        repo.status_by_manifest().unwrap(),

        None,

        "no snapshot of the edit yet"

    );

    let snap = repo.snapshot_status("oxy status").unwrap();

    assert_eq!(snap.statuses.len(), 1);

    let cached = repo

        .status_by_manifest()

        .unwrap()

        .expect("answered from the cache");

    assert_eq!(cached.len(), 1);

    assert_eq!(

        (cached[0].path.as_str(), cached[0].status),

        ("a.txt", ChangeStatus::Modified)

    );

    // Any further change sends status back to a snapshot.

    write(root, "a.txt", b"a\nmore\nand more\n");

    assert_eq!(repo.status_by_manifest().unwrap(), None, "edited again");

    repo.snapshot_status("oxy status").unwrap();

    assert!(repo.status_by_manifest().unwrap().is_some());

    write(root, "new.txt", b"n\n");

    assert_eq!(repo.status_by_manifest().unwrap(), None, "a file added");

    fs::remove_file(root.join("new.txt")).unwrap();

    assert!(

        repo.status_by_manifest().unwrap().is_some(),

        "back as it was"

    );

    fs::remove_file(root.join("b.txt")).unwrap();

    assert_eq!(repo.status_by_manifest().unwrap(), None, "a file removed");

    write(root, "b.txt", b"b\n");

    // A commit leaves nothing uncommitted: clean again, no cache needed.

    repo.snapshot_status("oxy status").unwrap();

    repo.commit("more", &[]).unwrap();

    assert_eq!(repo.status_by_manifest().unwrap(), Some(vec![]));

}

### Status with work in progress (2026-10-01, oxygen#41)

`MAINTAINERS` edited on the Linux tree on Earth (load average 2.2, one

Flow job running), oxy and Git alternating, three runs each:

| | before | after | Git |

|---|---:|---:|---:|

| status, one file edited (after a first status) | 0.42 to 0.46 s, 330 MB | **0.16 to 0.17 s**, 120 to 155 MB | 0.20 to 0.21 s |

| status, clean | 0.18 to 0.20 s | **0.15 to 0.16 s** | 0.20 to 0.24 s |

| diff, one file | 0.43 s | 0.40 to 0.41 s | 0.08 to 0.09 s |

Before, the manifest only proved a clean working copy. With uncommitted

work every status listed and snapshotted the tree again, loaded the state

index and rendered the base (listing 164 ms, index 60 ms, base 33 ms). Now

a snapshot leaves a status cache keyed by the working-copy change and the

manifest's checksum, and keeps the manifest current for the edited files.

A later status that finds every file as the manifest says answers from

the cache. Test builds check that answer against a snapshot. Diff still

renders the base; it is next.

/// The working copy's changed paths, or that it is clean.

fn print_statuses(statuses: &[oxy_core::repo::PathStatus]) {

    if statuses.is_empty() {

        println!(

            "  {} {}",

            ui::ok(ui::glyph("✔", "*")),

            ui::dim("nothing to commit, working copy clean")

        );

        return;

    }

    println!();

    for s in statuses {

        println!(

            "  {} {}",

            ui::status_mark(status_tag(s.status)),

            ui::path(&s.path)

        );

    }

}

    // Answered by the manifest alone, with no history to load (oxygen#35,

    // #41): a clean working copy, or one as the last snapshot left it.

    let known = if json {

        None

    } else {

        repo.status_by_manifest()?

    };

    if let Some(statuses) = known {

        print_statuses(&statuses);

    print_statuses(&snap.statuses);