Faster state loading: seeded fast hashing for the file graphs, no unused predecessor map, inline successor sets, lines moved rather than copied; OXY_TIMINGS=1 prints snapshot phase timings. Clean status at 10 000 files 0.96 s to 0.52 s, memory 657 to 239 MB 0bdc3f91a39f

//! A fast hasher for the in-memory graphs (spec §25): their keys begin with

//! BLAKE3 hashes, so the bytes are already uniform and SipHash's cost buys

//! nothing. A per-process random seed keeps bucket placement unpredictable,

//! so keys chosen by someone else cannot be aimed at one bucket.

use std::hash::{BuildHasher, Hasher};

use std::sync::OnceLock;

fn seed() -> u64 {

    static SEED: OnceLock<u64> = OnceLock::new();

    *SEED.get_or_init(rand::random::<u64>)

}

/// Builds [`FastHasher`]s with the process seed.

#[derive(Clone, Copy, Debug)]

pub struct FastState(u64);

impl Default for FastState {

    fn default() -> FastState {

        FastState(seed())

    }

}

impl BuildHasher for FastState {

    type Hasher = FastHasher;

    fn build_hasher(&self) -> FastHasher {

        FastHasher(self.0)

    }

}

/// Folds each 8-byte word into the state with a multiply and rotate, then

/// finishes with SplitMix64's mixer so every input bit reaches every output bit.

pub struct FastHasher(u64);

impl FastHasher {

    #[inline]

    fn mix(&mut self, word: u64) {

        self.0 = (self.0 ^ word)

            .wrapping_mul(0x9e37_79b9_7f4a_7c15)

            .rotate_left(31);

    }

}

impl Hasher for FastHasher {

    #[inline]

    fn write(&mut self, bytes: &[u8]) {

        let mut chunks = bytes.chunks_exact(8);

        for c in &mut chunks {

            self.mix(u64::from_le_bytes(c.try_into().unwrap()));

        }

        let rest = chunks.remainder();

        if !rest.is_empty() {

            let mut b = [0u8; 8];

            b[..rest.len()].copy_from_slice(rest);

            self.mix(u64::from_le_bytes(b) ^ ((rest.len() as u64) << 59));

        }

    }

    #[inline]

    fn write_u8(&mut self, n: u8) {

        self.mix(n as u64 | 0x100);

    }

    #[inline]

    fn write_u32(&mut self, n: u32) {

        self.mix(n as u64 | 0x1_0000_0000);

    }

    #[inline]

    fn write_u64(&mut self, n: u64) {

        self.mix(n);

    }

    #[inline]

    fn write_usize(&mut self, n: usize) {

        self.mix(n as u64);

    }

    #[inline]

    fn finish(&self) -> u64 {

        let mut z = self.0;

        z = (z ^ (z >> 30)).wrapping_mul(0xbf58_476d_1ce4_e5b9);

        z = (z ^ (z >> 27)).wrapping_mul(0x94d0_49bb_1331_11eb);

        z ^ (z >> 31)

    }

}

pub type FastMap<K, V> = std::collections::HashMap<K, V, FastState>;

pub type FastSet<K> = std::collections::HashSet<K, FastState>;

#[cfg(test)]

mod tests {

    use super::*;

    use std::hash::Hash;

    fn h<T: Hash + ?Sized>(state: &FastState, v: &T) -> u64 {

        state.hash_one(v)

    }

    #[test]

    fn distinct_keys_spread_and_equal_keys_agree() {

        let s = FastState(42);

        let a = ([7u8; 32], 1u32);

        assert_eq!(h(&s, &a), h(&s, &a));

        assert_ne!(h(&s, &a), h(&s, &([7u8; 32], 2u32)), "the index matters");

        assert_ne!(

            h(&s, &[1u8, 2, 3][..]),

            h(&s, &[1u8, 2, 3, 0][..]),

            "length matters"

        );

        // Consecutive indices of one change land in different buckets of a small table.

        let buckets: std::collections::HashSet<u64> =

            (0..64u32).map(|i| h(&s, &([9u8; 32], i)) & 63).collect();

        assert!(buckets.len() > 32, "{} of 64 buckets used", buckets.len());

        assert_ne!(

            h(&FastState(1), &a),

            h(&FastState(2), &a),

            "the seed matters"

        );

    }

    #[test]

    fn maps_work() {

        let mut m: FastMap<([u8; 32], u32), u32> = FastMap::default();

        for i in 0..10_000u32 {

            m.insert(([(i % 251) as u8; 32], i), i);

        }

        assert_eq!(m.len(), 10_000);

        assert_eq!(m[&([77u8; 32], 77)], 77);

    }

}

//! Opt-in phase timings (`OXY_TIMINGS=1`): each named phase prints its

//! duration to stderr, for measuring where a command spends its time

//! without a profiler.

use std::sync::OnceLock;

use std::time::Instant;

fn enabled() -> bool {

    static ON: OnceLock<bool> = OnceLock::new();

    *ON.get_or_init(|| std::env::var_os("OXY_TIMINGS").is_some_and(|v| !v.is_empty() && v != "0"))

}

/// Report a duration measured by hand (several intervals summed), if timings are on.

pub fn record(name: &str, d: std::time::Duration) {

    if enabled() {

        eprintln!("timing {:<28} {:>9.2} ms", name, d.as_secs_f64() * 1000.0);

    }

}

/// Prints `timing <name> <ms>` when dropped, if timings are on.

pub struct Phase {

    name: &'static str,

    started: Option<Instant>,

}

/// Start timing a phase; the guard reports when it goes out of scope.

pub fn phase(name: &'static str) -> Phase {

    Phase {

        name,

        started: enabled().then(Instant::now),

    }

}

impl Drop for Phase {

    fn drop(&mut self) {

        if let Some(t) = self.started {

            eprintln!(

                "timing {:<28} {:>9.2} ms",

                self.name,

                t.elapsed().as_secs_f64() * 1000.0

            );

        }

    }

}

use crate::fasthash::{FastMap as HashMap, FastSet as HashSet};

use std::collections::{BTreeMap, BTreeSet};

    /// Every vertex, with its direct successors.

    succ: HashMap<Vertex, Edges>,

}

/// A vertex's direct successors, sorted and without repeats. Nearly every

/// vertex has one, which is kept inline without an allocation.

#[derive(Clone, Debug, Default, PartialEq, Eq)]

enum Edges {

    #[default]

    None,

    One(Vertex),

    Many(Vec<Vertex>),

impl Edges {

    fn insert(&mut self, v: Vertex) {

        match self {

            Edges::None => *self = Edges::One(v),

            Edges::One(a) if *a == v => {}

            Edges::One(a) => {

                let a = *a;

                *self = Edges::Many(if a < v { vec![a, v] } else { vec![v, a] });

            }

            Edges::Many(list) => {

                if let Err(i) = list.binary_search(&v) {

                    list.insert(i, v);

                }

            }

        }

    }

    fn as_slice(&self) -> &[Vertex] {

        match self {

            Edges::None => &[],

            Edges::One(v) => std::slice::from_ref(v),

            Edges::Many(list) => list,

        }

    }

    fn iter(&self) -> std::slice::Iter<'_, Vertex> {

        self.as_slice().iter()

    }

}

            atoms: HashMap::default(),

            succ: HashMap::default(),

    ) {

        self.insert_owned(change, first_index, after, before, lines.to_vec());

    }

    /// [`FileGraph::insert`] taking the lines, so loading a state moves

    /// them in rather than copying. Each new atom's entry is made once, as

    /// the source of the next edge, and the chain's ends are ensured last.

    pub fn insert_owned(

        &mut self,

        change: ChangeHash,

        first_index: u32,

        after: Vertex,

        before: Vertex,

        lines: Vec<Vec<u8>>,

        self.atoms.reserve(lines.len());

        self.succ.reserve(lines.len() + 1);

        for (i, line) in lines.into_iter().enumerate() {

            self.atoms.entry(key).or_default().content = Some(line);

            self.succ.entry(prev).or_default().insert(v);

        self.succ.entry(prev).or_default().insert(before);

        self.succ.entry(before).or_default();

        let mut seen = HashSet::default();

        let mut kept: HashSet<Vertex> = HashSet::default();

        let mut ghosts: HashSet<AtomKey> = HashSet::default();

        let mut memo = HashMap::default();

        let mut rsucc: HashMap<Vertex, BTreeSet<Vertex>> = HashMap::default();

        let mut indeg: HashMap<Vertex, usize> = HashMap::default();

        let mut side: HashMap<Vertex, u32> = HashMap::default();

        let mut joins: HashSet<Vertex> = HashSet::default();

            atoms: HashMap::default(),

            succ: HashMap::default(),

            let mut seen = HashSet::default();

pub mod fasthash;

pub mod timing;

            let payload = {

                let _t = crate::timing::phase("load: read checkpoint");

                self.read_blob(&cp.payload)?

            };

            let base = {

                let _t = crate::timing::phase("load: decode checkpoint");

                State::from_checkpoint_payload(&payload)?

            };

            let _t = crate::timing::phase("load: apply delta and validate");

        {

            let _t = crate::timing::phase("load: read and add changes");

            let (mut read, mut add) = (std::time::Duration::ZERO, std::time::Duration::ZERO);

            while let Some(h) = stack.pop() {

                if !seen.insert(h) {

                    continue;

                }

                let t0 = std::time::Instant::now();

                let c = self.store.get_change(&h)?;

                read += t0.elapsed();

                stack.extend(c.deps.iter().copied());

                let t1 = std::time::Instant::now();

                state.add_unchecked(c);

                add += t1.elapsed();

            crate::timing::record("load:   of which reading", read);

            crate::timing::record("load:   of which applying", add);

        let _t = crate::timing::phase("load: validate");

        let _all = crate::timing::phase("snapshot");

        let base = {

            let _t = crate::timing::phase("snapshot: load base state");

            self.load_state(&old_wc.deps)?

        };

        let base_snapshot = {

            let _t = crate::timing::phase("snapshot: materialize base");

            base.materialize()

        };

        let mut fs_entries = {

            let _t = crate::timing::phase("snapshot: read working copy");

            self.walk_working_copy_with(manifest.as_ref(), &Self::tracked_kinds(&base_snapshot))?

        };

        let built = {

            let _t = crate::timing::phase("snapshot: diff against base");

            self.build_from_entries(&base, &base_snapshot, reuse, &fs_entries)?

        };

        let _t = crate::timing::phase("snapshot: write manifest");

        let meta = ChangeMeta::of(&change);

        self.apply(h, change);

        self.changes.insert(h, meta);

    fn apply(&mut self, h: ChangeHash, c: Change) {

        for e in c.edits {

                    let idx = counters.entry(node).or_default();

                    self.files.entry(node).or_default().insert_owned(

                    let g = self.files.entry(node).or_default();

                    self.replaces.entry(node).or_default().push((h, old, new));

## 2026-09-28: O10c, first part: the in-memory state

Profiling a clean snapshot with `OXY_TIMINGS=1` (10 000 files of 40 lines,

200 changes) showed loading the base state dominating it. Applying the

changes to the in-memory graphs cost 650 ms and reading them 160 ms. A

checkpoint did not help: decoding one took 680 ms, as long as replaying

every change, because both build the same graphs for 400 000 lines. So the

cost was in the graph structure, not in the format:

- **Hashing.** The graph maps used SipHash over 36-byte keys, three to

  five times per line. The keys begin with BLAKE3 hashes, so they now use

  a seeded multiply-rotate hasher (`fasthash.rs`). The seed is random per

  process, so keys chosen by someone else cannot be aimed at one bucket.

- **The predecessor map.** It was written for every line and read nowhere,

  so it is gone.

- **Successor sets.** A `BTreeSet` per vertex became a sorted edge list

  that keeps its single successor inline. Nearly every line has one.

- **Line bytes.** Loading moves them into the graph instead of copying

  them, and chains each new line with one lookup instead of three.

Both binaries were built from the same tree and run alternately, nine

times each, medians shown:

| 10 000 files, 200 changes | Before | After |

|---|---:|---:|

| status: clean working copy | 0.96 s | 0.52 s |

| commit: one file | 0.99 s | 0.51 s |

| peak memory (either) | 657 MB | 239 MB |

The checkpoint format is unchanged. What remains of O10c is split into

O10d and O10e in the backlog:

- **O10d.** Skip reading text files whose stat matches the manifest, when

  the manifest recorded them equal to the base.

- **O10e.** Checkpoints readable per file, so a snapshot loads only the

  graphs it needs. This one changes the checkpoint format in spec §25.

The repository file wins. `OXY_HOME` relocates the user directory. `OXY_TIMINGS=1` prints how long each phase of a snapshot takes (loading the base state, rendering it, reading the working copy, building the change) to standard error, for finding out where a slow command spends its time.