The build image prebuilds a workspace's dependencies eba589f87e4f

images/rust/recipe.py writes a workspace's dependency recipe: its local
packages' manifests (and the workspace roots path dependencies inherit
from), the lock file, real build scripts, and empty files for the other
targets. The image builds oxygenhub's recipe at /src with the pipeline's
environment for the release build, the tests and clippy, and keeps the
target directory in /opt/warm/oxygenhub (2.1 GB) for jobs to copy in.
Imported on Earth as cas://blob/47b614a6d5d9ca89bfc7d8da37cfbe3b335c28cfd846e08fd2fea8a360e39c0d.

refs oxygenhub#29

Author: Andy McCall @andymccall <[email protected]>
When: 1 month ago · 2026-09-29 03:45 UTC
Signature: ✔ signed by Andy McCall
Hash: eba589f87e4f4c79ae96da829891b0014920898930fb6e8b8631290eb9475ddb
Depends on: 50354f576ea6 eb5c4db673e5
Depended on by: f3b5b86e23ef fad6152112b1 9c8820a50d7f 87eb0f97c52d 22007df200f3 2b40cb9b87e1
In bookmarks: main newer-pipeline push-uses-login
In tags: v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.5.0 v0.5.1 v0.5.2 v0.5.3
Edits: 29 atoms, 53 tree operations, 32 files

Browse the tree at this change

hide whitespace changes

added images/rust/recipe.py +73 −0

added images/rust/warm/oxygenhub/Cargo.lock +4542 −0

added images/rust/warm/oxygenhub/Cargo.toml +65 −0

added images/rust/warm/oxygenhub/crates/hub-api/Cargo.toml +16 −0

added images/rust/warm/oxygenhub/crates/hub-api/src/bin/typescript.rs +1 −0

added images/rust/warm/oxygenhub/crates/hub-api/src/lib.rs +0 −0

No line changes.

added images/rust/warm/oxygenhub/crates/hub/Cargo.toml +63 −0

added images/rust/warm/oxygenhub/crates/hub/src/lib.rs +0 −0

No line changes.

added images/rust/warm/oxygenhub/crates/hub/src/main.rs +1 −0

added images/rust/warm/oxygenhub/crates/hub/tests/accounts.rs +1 −0

added images/rust/warm/oxygenhub/crates/hub/tests/api.rs +1 −0

added images/rust/warm/oxygenhub/crates/hub/tests/browse.rs +1 −0

added images/rust/warm/oxygenhub/crates/hub/tests/contrast.rs +1 −0

added images/rust/warm/oxygenhub/crates/hub/tests/edits.rs +1 −0

added images/rust/warm/oxygenhub/crates/hub/tests/flow.rs +1 −0

added images/rust/warm/oxygenhub/crates/hub/tests/issues.rs +1 −0

added images/rust/warm/oxygenhub/crates/hub/tests/proposals.rs +1 −0

added images/rust/warm/oxygenhub/crates/hub/tests/recovery.rs +1 −0

added images/rust/warm/oxygenhub/crates/hub/tests/releases.rs +1 −0

added images/rust/warm/oxygenhub/deps/oxygenflow/Cargo.toml +24 −0

added images/rust/warm/oxygenhub/deps/oxygenflow/crates/flow-core/Cargo.toml +17 −0

added images/rust/warm/oxygenhub/deps/oxygenflow/crates/flow-core/src/lib.rs +0 −0

No line changes.

added images/rust/warm/oxygenhub/deps/oxygenflow/deps/oxygen/Cargo.toml +35 −0

added images/rust/warm/oxygenhub/deps/oxygenflow/deps/oxygen/crates/oxy-core/Cargo.toml +27 −0

added images/rust/warm/oxygenhub/deps/oxygenflow/deps/oxygen/crates/oxy-core/src/lib.rs +0 −0

No line changes.

added images/rust/warm/oxygenhub/deps/oxygenflow/deps/oxygen/crates/oxy-core/tests/bench.rs +1 −0

added images/rust/warm/oxygenhub/deps/oxygenflow/deps/oxygen/crates/oxy-core/tests/commutativity.rs +1 −0

added images/rust/warm/oxygenhub/deps/oxygenflow/deps/oxygen/crates/oxy-server/Cargo.toml +17 −0

added images/rust/warm/oxygenhub/deps/oxygenflow/deps/oxygen/crates/oxy-server/src/lib.rs +0 −0

No line changes.

added images/rust/warm/oxygenhub/deps/oxygenflow/deps/oxygen/crates/oxy-server/tests/http.rs +1 −0

modified images/rust/Dockerfile +19 −0

modified images/rust/build.sh +7 −1

1	+	#!/usr/bin/env python3
2	+	"""Write a dependency recipe of a Cargo workspace (backlog F6b2).
3	+
4	+	The recipe is the workspace with its sources stubbed out: every local
5	+	package's Cargo.toml (and the workspace roots its path dependencies
6	+	inherit from), the lock file, real build scripts, and an empty file for
7	+	each other target. Building it compiles exactly the dependencies the real
8	+	workspace uses, with the same features, so the build image can hold them
9	+	prebuilt: jobs copy that target directory in and compile only their own
10	+	crates.
11	+
12	+	recipe.py WORKSPACE OUT
13	+	"""
14	+
15	+	import json
16	+	import os
17	+	import shutil
18	+	import subprocess
19	+	import sys
20	+
21	+
22	+	def main():
23	+	ws, out = os.path.abspath(sys.argv[1]), os.path.abspath(sys.argv[2])
24	+	meta = json.loads(
25	+	subprocess.run(
26	+	["cargo", "metadata", "--format-version", "1", "--locked", "--manifest-path", os.path.join(ws, "Cargo.toml")],
27	+	check=True,
28	+	capture_output=True,
29	+	).stdout
30	+	)
31	+	root = meta["workspace_root"]
32	+	if os.path.exists(out):
33	+	shutil.rmtree(out)
34	+
35	+	def put(path, data=None):
36	+	rel = os.path.relpath(path, root)
37	+	if rel.startswith(".."):
38	+	sys.exit(f"{path} is outside the workspace; a recipe holds only paths under it")
39	+	dest = os.path.join(out, rel)
40	+	os.makedirs(os.path.dirname(dest), exist_ok=True)
41	+	if data is None:
42	+	shutil.copy2(path, dest)
43	+	else:
44	+	with open(dest, "w") as f:
45	+	f.write(data)
46	+
47	+	for name in ("Cargo.toml", "Cargo.lock"):
48	+	put(os.path.join(root, name))
49	+	for p in meta["packages"]:
50	+	if p["source"] is not None:
51	+	continue
52	+	manifest = p["manifest_path"]
53	+	put(manifest)
54	+	# Workspace roots above a path dependency, for `workspace = true`.
55	+	d = os.path.dirname(os.path.dirname(manifest))
56	+	while d.startswith(root) and d != root:
57	+	top = os.path.join(d, "Cargo.toml")
58	+	if os.path.exists(top):
59	+	put(top)
60	+	d = os.path.dirname(d)
61	+	for t in p["targets"]:
62	+	kind = t["kind"][0]
63	+	src = t["src_path"]
64	+	if kind == "custom-build":
65	+	put(src)
66	+	elif kind in ("lib", "rlib", "proc-macro"):
67	+	put(src, "")
68	+	else:
69	+	put(src, "fn main() {}\n")
70	+
71	+
72	+	if __name__ == "__main__":
73	+	main()

1	+	[workspace]
2	+	members = ["crates/hub", "crates/hub-api"]
3	+	# oxygenflow as a pinned subrepo (oxy submodule), and oxygen as oxygenflow's
4	+	# own pinned subrepo inside it: one oxygen, the one oxygenflow was tested
5	+	# with. They are workspaces of their own.
6	+	exclude = ["deps"]
7	+	resolver = "2"
8	+
9	+	[workspace.package]
10	+	version = "0.1.0"
11	+	edition = "2021"
12	+	license = "MIT OR Apache-2.0"
13	+	authors = ["Andy McCall <[email protected]>"]
14	+	repository = "https://earth.andymccall.xyz:7343/andymccall/oxygenhub"
15	+
16	+	[workspace.dependencies]
17	+	oxy-core = { path = "deps/oxygenflow/deps/oxygen/crates/oxy-core" }
18	+	oxy-server = { path = "deps/oxygenflow/deps/oxygen/crates/oxy-server" }
19	+	hub-api = { path = "crates/hub-api" }
20	+	flow-core = { path = "deps/oxygenflow/crates/flow-core" }
21	+
22	+	anyhow = "1"
23	+	thiserror = "2"
24	+	serde = { version = "1", features = ["derive"] }
25	+	serde_json = "1"
26	+	tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "sync", "time", "signal", "fs", "process", "io-util"] }
27	+	axum = { version = "0.8", features = ["macros", "form", "query", "multipart"] }
28	+	axum-server = { version = "0.8", default-features = false, features = ["tls-rustls-no-provider"] }
29	+	rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"] }
30	+	tower-http = { version = "0.6", features = ["trace", "compression-gzip", "set-header", "limit"] }
31	+	askama = "0.14"
32	+	sqlx = { version = "0.8", default-features = false, features = ["runtime-tokio", "sqlite", "migrate", "macros"] }
33	+	tracing = "0.1"
34	+	tracing-subscriber = { version = "0.3", features = ["env-filter"] }
35	+	clap = { version = "4", features = ["derive", "env"] }
36	+	toml = "0.8"
37	+	time = { version = "0.3", features = ["formatting", "macros"] }
38	+	pulldown-cmark = { version = "0.13", default-features = false, features = ["html"] }
39	+	ammonia = "4"
40	+	syntect = { version = "5", default-features = false, features = ["default-fancy"] }
41	+	rcgen = "0.14"
42	+	hex = "0.4"
43	+	rand = "0.8"
44	+	similar = "2"
45	+	argon2 = { version = "0.5", features = ["std"] }
46	+	axum-extra = { version = "0.10", features = ["cookie", "form"] }
47	+	hmac = "0.12"
48	+	sha1 = "0.10"
49	+	base32 = "0.5"
50	+	qrcode = { version = "0.14", default-features = false, features = ["svg"] }
51	+	subtle = "2"
52	+	blake3 = "1"
53	+	tar = "0.4"
54	+	zstd = "0.13"
55	+	zip = { version = "2", default-features = false, features = ["deflate"] }
56	+	lettre = { version = "0.11", default-features = false, features = ["tokio1", "tokio1-rustls-tls", "smtp-transport", "builder"] }
57	+	regex = "1"
58	+	tempfile = "3"
59	+	reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "json"] }
60	+	schemars = "1"
61	+	futures = "0.3"
62	+	tokio-stream = { version = "0.1", features = ["sync"] }
63	+
64	+	[profile.release]
65	+	debug = 1

1	+	[package]
2	+	name = "hub-api"
3	+	description = "Types of the OxygenHub JSON API, shared by the server and its clients"
4	+	version.workspace = true
5	+	edition.workspace = true
6	+	license.workspace = true
7	+	authors.workspace = true
8	+
9	+	[dependencies]
10	+	serde.workspace = true
11	+	serde_json.workspace = true
12	+	schemars = { version = "1", features = ["derive"] }
13	+
14	+	[[bin]]
15	+	name = "hub-api-typescript"
16	+	path = "src/bin/typescript.rs"

1	+	[package]
2	+	name = "oxygenhub"
3	+	description = "A forge for Oxygen repositories: hosting, browsing, review, issues and releases"
4	+	version.workspace = true
5	+	edition.workspace = true
6	+	license.workspace = true
7	+	authors.workspace = true
8	+	default-run = "oxygenhub"
9	+
10	+	[[bin]]
11	+	name = "oxygenhub"
12	+	path = "src/main.rs"
13	+
14	+	[dependencies]
15	+	oxy-core.workspace = true
16	+	oxy-server.workspace = true
17	+	hub-api.workspace = true
18	+	flow-core.workspace = true
19	+	anyhow.workspace = true
20	+	thiserror.workspace = true
21	+	serde.workspace = true
22	+	serde_json.workspace = true
23	+	tokio.workspace = true
24	+	axum.workspace = true
25	+	axum-server.workspace = true
26	+	rustls.workspace = true
27	+	tower-http.workspace = true
28	+	askama.workspace = true
29	+	sqlx.workspace = true
30	+	tracing.workspace = true
31	+	image = { version = "0.25", default-features = false, features = ["png", "jpeg", "webp"] }
32	+	tracing-subscriber.workspace = true
33	+	clap.workspace = true
34	+	toml.workspace = true
35	+	time.workspace = true
36	+	pulldown-cmark.workspace = true
37	+	ammonia.workspace = true
38	+	syntect.workspace = true
39	+	rcgen.workspace = true
40	+	hex.workspace = true
41	+	rand.workspace = true
42	+	similar.workspace = true
43	+	argon2.workspace = true
44	+	axum-extra.workspace = true
45	+	hmac.workspace = true
46	+	sha1.workspace = true
47	+	base32.workspace = true
48	+	qrcode.workspace = true
49	+	subtle.workspace = true
50	+	blake3.workspace = true
51	+	tar.workspace = true
52	+	zstd.workspace = true
53	+	zip.workspace = true
54	+	lettre.workspace = true
55	+	regex.workspace = true
56	+	reqwest.workspace = true
57	+	futures.workspace = true
58	+	tokio-stream.workspace = true
59	+
60	+	[dev-dependencies]
61	+	tempfile.workspace = true
62	+	tower = { version = "0.5", features = ["util"] }
63	+	http-body-util = "0.1"

1	+	[workspace]
2	+	members = ["crates/flow-core", "crates/flow-init", "crates/flow"]
3	+	# oxygen as a pinned subrepo (oxy submodule): the state flow builds against.
4	+	exclude = ["deps"]
5	+	resolver = "2"
6	+
7	+	[workspace.package]
8	+	version = "0.1.0"
9	+	edition = "2021"
10	+	license = "MIT OR Apache-2.0"
11	+
12	+	[workspace.dependencies]
13	+	oxy-core = { path = "deps/oxygen/crates/oxy-core" }
14	+	flow-core = { path = "crates/flow-core" }
15	+	serde = { version = "1", features = ["derive"] }
16	+	serde_json = "1"
17	+	blake3 = "1"
18	+	hex = "0.4"
19	+	thiserror = "1"
20	+	anyhow = "1"
21	+	tempfile = "3"
22	+
23	+	[profile.release]
24	+	debug = 1

1	+	[package]
2	+	name = "flow-core"
3	+	version.workspace = true
4	+	edition.workspace = true
5	+	license.workspace = true
6	+	description = "OxygenFlow pipelines: parsing, validation, canonical job graphs and receipts"
7	+
8	+	[dependencies]
9	+	oxy-core.workspace = true
10	+	serde.workspace = true
11	+	serde_json.workspace = true
12	+	blake3.workspace = true
13	+	hex.workspace = true
14	+	thiserror.workspace = true
15	+	kdl = "6"
16	+	miette = "7"
17	+	globset = "0.4"

1	+	[workspace]
2	+	members = ["crates/oxy-core", "crates/oxy-server", "."]
3	+	resolver = "2"
4	+
5	+	[workspace.package]
6	+	version = "0.1.0"
7	+	edition = "2021"
8	+	license = "MIT OR Apache-2.0"
9	+
10	+	[package]
11	+	name = "oxy"
12	+	version.workspace = true
13	+	edition.workspace = true
14	+	license.workspace = true
15	+	description = "Oxygen: a distributed, commutative version control system"
16	+	default-run = "oxy"
17	+
18	+	[[bin]]
19	+	name = "oxy"
20	+	path = "src/main.rs"
21	+
22	+	[dependencies]
23	+	oxy-core = { path = "crates/oxy-core" }
24	+	oxy-server = { path = "crates/oxy-server" }
25	+	clap = { version = "4", features = ["derive"] }
26	+	anyhow = "1"
27	+	libc = "0.2"
28	+	console = "0.15"
29	+	indicatif = "0.17"
30	+	serde_json = "1"
31	+	clap_complete = "4"
32	+	clap_mangen = "0.3"
33	+
34	+	[dev-dependencies]
35	+	tempfile = "3"

1	+	[package]
2	+	name = "oxy-core"
3	+	version.workspace = true
4	+	edition.workspace = true
5	+	license.workspace = true
6	+	description = "Core data model, change graph, and storage for Oxygen"
7	+
8	+	[dependencies]
9	+	blake3 = "1"
10	+	hex = "0.4"
11	+	rand = "0.8"
12	+	similar = "2"
13	+	thiserror = "1"
14	+	ignore = "0.4"
15	+	ml-dsa = "0.1"
16	+	ed25519-dalek = "3"
17	+	ureq = "2"
18	+	rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"] }
19	+	rustls-pemfile = "2"
20	+	webpki-roots = "1"
21	+	zstd = "0.13"
22	+	globset = "0.4"
23	+	rayon = "1"
24	+
25	+	[dev-dependencies]
26	+	proptest = "1"
27	+	tempfile = "3"

1	+	[package]
2	+	name = "oxy-server"
3	+	version.workspace = true
4	+	edition.workspace = true
5	+	license.workspace = true
6	+	description = "Reference HTTP server for the Oxygen wire protocol"
7	+
8	+	[dependencies]
9	+	oxy-core = { path = "../oxy-core" }
10	+	axum = "0.8"
11	+	tokio = { version = "1", features = ["rt-multi-thread", "net", "macros", "sync"] }
12	+	axum-server = { version = "0.8", default-features = false, features = ["tls-rustls-no-provider"] }
13	+	rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"] }
14	+
15	+	[dev-dependencies]
16	+	rcgen = "0.14"
17	+	tempfile = "3"

		…	28 unchanged lines
29	29		# vendored crates; CARGO_HOME stays the default $HOME/.cargo (/tmp).
30	30		COPY vendor /opt/vendor
31	31		COPY cargo-config.toml /.cargo/config.toml
	32	+
	33	+	# Dependencies prebuilt (backlog F6b2): each workspace's recipe (its
	34	+	# manifests and lock file with the sources stubbed out, made by recipe.py)
	35	+	# is built at /src, where jobs build, with the environment its pipeline's
	36	+	# rust sandbox sets, for the release build, the tests and clippy. The
	37	+	# target directory is kept in /opt/warm/NAME; a job copies it onto its
	38	+	# empty target disk and compiles only its own crates. A lock file that
	39	+	# has moved on since only costs the crates that changed.
	40	+	COPY warm /tmp/warm
	41	+	RUN set -e; \
	42	+	for ws in /tmp/warm/*; do \
	43	+	name=$(basename "$ws"); rm -rf /src; cp -a "$ws" /src; cd /src; \
	44	+	export CARGO_INCREMENTAL=0 CARGO_PROFILE_DEV_DEBUG=0 CARGO_PROFILE_TEST_DEBUG=0 CARGO_PROFILE_RELEASE_DEBUG=0; \
	45	+	cargo build --release --locked; \
	46	+	cargo test --workspace --locked --no-run; \
	47	+	cargo clippy --workspace --all-targets --locked; \
	48	+	mkdir -p /opt/warm; mv /src/target "/opt/warm/$name"; cd /; \
	49	+	done; \
	50	+	rm -rf /src /tmp/warm /root/.cargo/registry /root/.cargo/git; mkdir /src

		…	24 unchanged lines
25	25		rm -rf "${tmp:?}"
26	26		done
27	27		rm -rf "${here:?}/npm-cache/_logs"
	28	+	# Dependency recipes of the workspaces whose dependencies the image
	29	+	# prebuilds (see the Dockerfile).
	30	+	rm -rf "${here:?}/warm"
	31	+	for ws in oxygenhub; do
	32	+	python3 "$here/recipe.py" "$dev/$ws" "$here/warm/$ws"
	33	+	done
28	34		docker build -q -t oxygenflow-build \
29	35		--build-arg NODE_VERSION=$NODE_VERSION --build-arg NODE_SHA256=$NODE_SHA256 "$here"
30	36		id=$(docker create oxygenflow-build)
31	37		docker export "$id" -o "$out"
32	38		docker rm "$id" >/dev/null
33		-	rm -rf "${here:?}/vendor" "${here:?}/npm-cache"
	39	+	rm -rf "${here:?}/vendor" "${here:?}/npm-cache" "${here:?}/warm"
34	40		echo "$out"

`/`	Search
`g` `c`	Code
`g` `h`	History
`g` `b`	Bookmarks
`g` `t`	Trust
`y`	Permalink to this state (files and trees)
`?`	This sheet

1	+	# This file is automatically @generated by Cargo.
2	+	# It is not intended for manual editing.
3	+	version = 4
4	+
5	+	[[package]]
6	+	name = "adler2"
7	+	version = "2.0.1"
8	+	source = "registry+https://github.com/rust-lang/crates.io-index"
9	+	checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
10	+
11	+	[[package]]
12	+	name = "aho-corasick"
13	+	version = "1.1.5"
14	+	source = "registry+https://github.com/rust-lang/crates.io-index"
15	+	checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba"
16	+	dependencies = [
17	+	"memchr",
18	+	]
19	+
20	+	[[package]]
21	+	name = "allocator-api2"
22	+	version = "0.2.21"
23	+	source = "registry+https://github.com/rust-lang/crates.io-index"
24	+	checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
25	+
26	+	[[package]]
27	+	name = "ammonia"
28	+	version = "4.2.0"
29	+	source = "registry+https://github.com/rust-lang/crates.io-index"
30	+	checksum = "3f7ed3ac4252be4d3e36fb589ed5659356842587b21a31596a9dc89d3b2f5e2a"
31	+	dependencies = [
32	+	"cssparser",
33	+	"html5ever",
34	+	"maplit",
35	+	"url",
36	+	]
37	+
38	+	[[package]]
39	+	name = "anstream"
40	+	version = "1.0.0"

The build image prebuilds a workspace's dependencies eba589f87e4f

Keyboard