Commit without a second snapshot; the state index after a checkpoint 7505194116e7

A history-only state loaded through a checkpoint read the changes after
it whole, text included, and a state holding text cannot be written as
a state index: after any checkpoint no index was saved and every command
loaded history the long way. It now reads their skeletons, as a replayed
load does. oxy commit PATHS took a second snapshot to count the paths it
left; when it took every operation of the working copy nothing is left,
and the snapshot is skipped.

VS Code at 19,978 commits, a one-file commit: about 450 ms to 300 ms;
the first commit after a checkpoint 25 s to 0.44 s. The state index is
24 MB rather than 84 MB once built from a checkpoint.

refs #93

Author: Andy McCall @andymccall <[email protected]>
When: 26 days ago · 2026-10-09 21:57 UTC
Signature: ✔ signed by Andy McCall
Hash: 7505194116e722bd1e616b2cef364299e73f621e7b1804487e75f12f8a2ce925
Depends on: 135212de5370 170bb436e1d5 19303b4a06fb 252e8cb23fbc 5b4c51307b14 6caacfff1d4f 75167672ebcd 8ab265dc4a51 8fd78eb52ad2 957fee30826e 982864068a59 9aa399044d9d b30a880fa409 d4dbab97e948 fce44ae9ad29
Depended on by: b689779e9e74 79b5a90ffd43
In bookmarks: bare-hashes carried-not-removed carry-identical doctor-trust identical-sides keyless-tags main merge-remote-bookmark only-help-103 paths-for-resolutions-and-amend push-notes show-stat subrepo-pin-state switch-says
In tags: v0.10.0 v0.11.0 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.5.0 v0.5.1 v0.6.0 v0.7.0 v0.8.0 v0.9.0
Edits: 14 atoms, 0 tree operations, 5 files

Browse the tree at this change

show whitespace changes

modified CHANGELOG.md +2 −0

modified crates/oxy-core/src/repo/checkpoint.rs +16 −5

modified crates/oxy-core/src/repo/snapshot.rs +16 −2

modified src/cli/workspace.rs +3 −2

modified tests/cli_strata.rs +22 −0

		…	1 unchanged lines
2	2
3	3		## Unreleased
4	4
	5	+	- Faster commits, and the state index after a checkpoint (#93). A state loaded through a checkpoint read the changes after it whole, text included, so it could never be saved as the workspace's state index and every command after a checkpoint loaded history the long way (25 s for the first commit on VS Code at 20 000 commits). It now reads them as the index needs them. `oxy commit PATHS` no longer takes a second snapshot to count what it left when it took everything. A one-file commit on VS Code went from about 450 ms to 300 ms; 70 ms of that is the store's fsync, which stays.
	6	+
5	7		- Reading one file reads one file (#92). `oxy cat` rendered every file of the state to print one. It now builds only that file, from the workspace's state index when it reads the working copy's base or the state it is on. On VS Code at 20 000 commits, `cat package.json -r main` went from 26 s to 0.10 s and at a change from 2016 from 0.66 s to 0.16 s; on the full 166 000-change history it takes 1.9 s and 1.3 GB, where it ran for minutes and used 9 GB.
6	8
7	9		- A file's history in a fraction of a second (#91). `oxy log --path` rendered every file of the tree twice just to find the path, and both it and the `touches()` revset decoded every change of the history, its text included, to see what each one touched. They now read the path from the tree alone and keep the nodes each change touches in a cache beside the store (`.oxy/index/touched`, rebuilt if missing). On VS Code at 20 000 commits, `log --path package.json` went from 53.6 s to 0.16 s (Git: 0.15 s) and `touches("package.json")` from 3.4 s to 0.16 s, with the same output.
		…	148 unchanged lines

		…	191 unchanged lines
192	192		};
193	193		let cp_heads: BTreeSet<ChangeHash> = cp.heads.iter().copied().collect();
194	194		let mut found: BTreeSet<ChangeHash> = BTreeSet::new();
195		-	let mut delta: Vec<Change> = Vec::new();
	195	+	let mut delta: Vec<(ChangeHash, Change, Vec<crate::objects::TextRef>)> = Vec::new();
196	196		let mut seen: BTreeSet<ChangeHash> = BTreeSet::new();
197	197		let mut stack: Vec<ChangeHash> = heads.to_vec();
198	198		while let Some(h) = stack.pop() {
		…	6 unchanged lines
205	205		}
206	206		continue;
207	207		}
208		-	let c = self.store.get_change(&h)?;
	208	+	// For history, the skeleton: text is read only for the
	209	+	// graphs built later, and a state holding none can be
	210	+	// indexed (oxygen#93).
	211	+	let (c, texts) = if history {
	212	+	self.store.get_change_lazy(&h)?
	213	+	} else {
	214	+	(self.store.get_change(&h)?, Vec::new())
	215	+	};
209	216		stack.extend(c.deps.iter().copied());
210		-	delta.push(c);
	217	+	delta.push((h, c, texts));
211	218		}
212	219		if found.len() != cp_heads.len() {
213	220		continue;
214	221		}
215	222		let _t = crate::timing::phase("load: apply delta and validate");
216	223		let mut state = base;
217		-	for c in delta {
218		-	state.add_unchecked(c);
	224	+	for (h, c, texts) in delta {
	225	+	if history {
	226	+	state.add_known(h, c, texts);
	227	+	} else {
	228	+	state.add_unchecked(c);
	229	+	}
219	230		}
220	231		state.validate()?;
221	232		return Ok(Some(state));
		…	4 unchanged lines

		…	843 unchanged lines
844	844		extra_deps: &[ChangeHash],
845	845		selection: &Selection,
846	846		) -> Result<ChangeHash> {
	847	+	Ok(self
	848	+	.commit_selection_partial(message, extra_deps, selection)?
	849	+	.0)
	850	+	}
	851	+
	852	+	/// As [`Repo::commit_selection`], and whether the commit left part of
	853	+	/// the working copy uncommitted: when it did not, nothing is left, and
	854	+	/// no snapshot is needed to say so.
	855	+	pub fn commit_selection_partial(
	856	+	&self,
	857	+	message: &str,
	858	+	extra_deps: &[ChangeHash],
	859	+	selection: &Selection,
	860	+	) -> Result<(ChangeHash, bool)> {
847	861		// The commit's objects are written together, made durable at once
848	862		// before the operation that names them (spec §13.6).
849	863		let batch = self.store.begin_batch()?;
		…	30 unchanged lines
880	894		message: &str,
881	895		extra_deps: &[ChangeHash],
882	896		selection: &Selection,
883		-	) -> Result<ChangeHash> {
	897	+	) -> Result<(ChangeHash, bool)> {
884	898		let _wc = self.lock_working_copy()?;
885	899		let mut snap = match self.snapshot_from_cache()? {
886	900		Some(snap) => snap,
		…	224 unchanged lines
1111	1125		json_str(message)
1112	1126		),
1113	1127		);
1114		-	Ok(hash)
	1128	+	Ok((hash, partial))
1115	1129		}
1116	1130		}
1117	1131
		…	1915 unchanged lines

		…	313 unchanged lines
314	314		oxy_core::repo::Selection::Paths(paths.to_vec())
315	315		};
316	316		let sp = ui::spinner("Committing…");
317		-	let h = repo.commit_selection(message, &deps, &selection)?;
	317	+	let (h, partial) = repo.commit_selection_partial(message, &deps, &selection)?;
318	318		// The metadata only: a first commit's change holds the whole tree.
319	319		let meta = repo.store().get_change_meta(&h)?;
320	320		sp.done(&format!(
		…	4 unchanged lines
325	325		ui::subject(subject_of(message))
326	326		));
327	327		print_untracked_nested(&repo, &repo.untracked_nested());
328		-	if !matches!(selection, oxy_core::repo::Selection::All) {
	328	+	// Only a commit that took part of the working copy leaves anything.
	329	+	if partial {
329	330		let rest = repo.snapshot_status("oxy commit")?;
330	331		if !rest.statuses.is_empty() {
331	332		println!(
		…	2372 unchanged lines

		…	119 unchanged lines
120	120		assert_ne!(code, 0);
121	121		assert!(err.contains("different repository"), "{err}");
122	122		}
	123	+
	124	+	#[test]
	125	+	fn commit_with_paths_says_what_it_left() {
	126	+	let tmp = tempfile::tempdir().unwrap();
	127	+	let home = tmp.path().join("home");
	128	+	std::fs::create_dir_all(&home).unwrap();
	129	+	let a = tmp.path().join("a");
	130	+	ok(tmp.path(), &home, &["init", a.to_str().unwrap()]);
	131	+	std::fs::write(a.join("f.txt"), "one\n").unwrap();
	132	+	std::fs::write(a.join("g.txt"), "one\n").unwrap();
	133	+	ok(&a, &home, &["commit", "-m", "first"]);
	134	+
	135	+	std::fs::write(a.join("f.txt"), "two\n").unwrap();
	136	+	std::fs::write(a.join("g.txt"), "two\n").unwrap();
	137	+	let part = ok(&a, &home, &["commit", "-m", "f only", "f.txt"]);
	138	+	assert!(part.contains("1 path left uncommitted"), "{part}");
	139	+
	140	+	let rest = ok(&a, &home, &["commit", "-m", "g", "g.txt"]);
	141	+	assert!(!rest.contains("left uncommitted"), "{rest}");
	142	+	let status = ok(&a, &home, &["status"]);
	143	+	assert!(status.contains("working copy clean"), "{status}");
	144	+	}

`/`	Search
`g` `c`	Code
`g` `h`	History
`g` `b`	Bookmarks
`g` `t`	Trust
`y`	Permalink to this state (files and trees)
`?`	This sheet

Commit without a second snapshot; the state index after a checkpoint 7505194116e7

Keyboard