OXYGENHUB

Compare agents-in-readme … main

⇄

187 changes in main not in agents-in-readme. 120 files differ between the two states.

Release 0.11.0 mainv0.11.0 head
Andy McCall via agent:claude@andymccall agent ·
Name moved subrepo pins in status and commit; changelog on the WASM tier (closes #162) mainv0.11.0 head
Andy McCall via agent:claude@andymccall agent ·
A stale author.key can no longer fork the identity (closes #163)
Andy McCall via agent:claude@andymccall agent ·
oxy 0.10.0 v0.10.0
Andy McCall via agent:claude@andymccall agent ·
HaikuPorts recipes: x86_64 only and no minimum rustc, as the haikuports review asked (refs #157) mainv0.10.0v0.11.0 head
Andy McCall via agent:claude@andymccall agent ·
HaikuPorts recipes: copyright 2026, as the LICENSE says (refs #157) mainv0.10.0v0.11.0 head
Andy McCall via agent:claude@andymccall agent ·
oxy 0.9.0 v0.9.0
Andy McCall via agent:claude@andymccall agent ·
The source tarball leaves out the job disk's lost+found (refs #157) mainv0.10.0v0.11.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
oxy 0.8.0 v0.8.0
Andy McCall via agent:claude@andymccall agent ·
CHANGELOG: one #148 entry (the merge kept both sides' copy) mainsubrepo-pin-statev0.10.0v0.11.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
Merge main into subrepo-pin-state: keep both changelog entries resolution
Andy McCall via agent:claude@andymccall agent ·
Merge main into identical-sides: keep both changelog entries identical-sides resolution
Andy McCall via agent:claude@andymccall agent ·
Merge main into subrepo-pin-state: keep both changelog entries resolution
Andy McCall via agent:claude@andymccall agent ·
oxy-server: a host can add a note to a reference a push moved (closes #148) push-notes
Andy McCall via agent:claude@andymccall agent ·
A shallow clone fetches its checkpoint in parts (refs #137)
Andy McCall via agent:claude@andymccall agent ·
oxy 0.7.0 v0.7.0
Andy McCall via agent:claude@andymccall agent ·
Read many objects in pack and offset order
Andy McCall via agent:claude@andymccall agent ·
Pass the workspace directory itself, as clippy asks (refs #143) identical-sidesmainpush-notessubrepo-pin-statev0.10.0v0.11.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
Windows paths print plainly (closes #143)
Andy McCall via agent:claude@andymccall agent ·
CI on the Rust image with cargo-nextest (refs #139) identical-sidesmainpush-notessubrepo-pin-statev0.10.0v0.11.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
renameat2's flags are unsigned on Android (refs #141) identical-sidesmainpush-notessubrepo-pin-statev0.10.0v0.11.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
The object store works where hard links are refused (closes #141)
Andy McCall via agent:claude@andymccall agent ·
oxy serve takes a push of more than 2 MB (closes #138) identical-sidesmainpush-notessubrepo-pin-statev0.10.0v0.11.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
oxy 0.6.0 v0.6.0
Andy McCall via agent:claude@andymccall agent ·
Depth 0 takes no change, and a checkpoint is gathered in parallel (refs #137) v0.6.0v0.7.0
Andy McCall via agent:claude@andymccall agent ·
Instant clone: the design, with tonight's measurements (refs #132)
Andy McCall via agent:claude@andymccall agent ·
A push no longer passes a proxy's request limit (closes #133)
Andy McCall via agent:claude@andymccall agent ·
A tag can be withdrawn, by a signed withdrawal (oxygenhub#301) identical-sidespush-notessubrepo-pin-statev0.6.0v0.7.0v0.8.0v0.9.0
Andy McCall via agent:claude@andymccall agent ·
oxy 0.5.1 v0.5.1
Andy McCall via agent:claude@andymccall agent ·
WIP #129: a tag's signer travels with the push (part 1 of 3) keyless-tags
Andy McCall via agent:claude@andymccall agent ·
oxy 0.5.0 keyless-tagsv0.5.0
Andy McCall via agent:claude@andymccall agent ·
Merge #116 (pack entries) into main: keep both Unreleased lines resolution
Andy McCall via agent:claude@andymccall agent ·
Fold the older one-list keys on gc and stop reading them
Andy McCall via agent:claude@andymccall agent ·
oxy-server: a host may refuse a push by what it adds (Backend::vet_push) keyless-tagsv0.5.0
Andy McCall via agent:claude@andymccall agent ·
Changelog per change: a check in the gate, and Unreleased caught up
Andy McCall via agent:claude@andymccall agent ·
Fetch: compute the attestation signers once per operation head identical-sideskeyless-tagsmainpush-notessubrepo-pin-statev0.10.0v0.11.0v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
riscv64 check: init before the identity, and cache the target directory identical-sideskeyless-tagsmainpush-notessubrepo-pin-statev0.10.0v0.11.0v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
oxy doctor checks the trust policy doctor-trustidentical-sideskeyless-tagsmainpush-notessubrepo-pin-statev0.10.0v0.11.0v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
A native RISC-V check on narvi-1: oxy built and used on riscv64 doctor-trust
Andy McCall via agent:claude@andymccall agent ·
Trusted time: judge a fetched attestation at its integrated time
Andy McCall via agent:claude@andymccall agent ·
Merge main into #125: keep both synopsis lines doctor-trustkeyless-tagspaths-for-resolutions-and-amendv0.5.0v0.5.1v0.6.0 resolution
Andy McCall via agent:claude@andymccall agent ·
Marker-like text no longer blocks a conflict resolution doctor-trustkeyless-tagsv0.5.0v0.5.1v0.6.0v0.7.0
Andy McCall via agent:claude@andymccall agent ·
The same file added on both sides, identically, is one file doctor-trustidentical-sideskeyless-tagsmainpaths-for-resolutions-and-amendpush-notessubrepo-pin-statev0.10.0v0.11.0v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
Commit a resolution by paths, and amend only some paths
Andy McCall via agent:claude@andymccall agent ·
oxy show --stat: the files a change touched, with lines added and removed show-stat
Andy McCall via agent:claude@andymccall agent ·
A carried switch no longer says carried files were removed carried-not-removeddoctor-trustkeyless-tagspaths-for-resolutions-and-amendv0.5.0v0.5.1v0.6.0v0.7.0
Andy McCall via agent:claude@andymccall agent ·
oxy merge takes a remote's bookmark doctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notessubrepo-pin-statev0.10.0v0.11.0v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
Readers never wait: one head per handle, retry on a miss, retire before deleting carried-not-removeddoctor-trustmerge-remote-bookmarkpaths-for-resolutions-and-amendshow-stat
Andy McCall via agent:claude@andymccall agent ·
Opening a bucket store sends no request per pack (refs #116)
Andy McCall via agent:claude@andymccall agent ·
Bucket stores count pack listings and kept listings apart carried-not-removeddoctor-trustmerge-remote-bookmarkpaths-for-resolutions-and-amendshow-stat
Andy McCall via agent:claude@andymccall agent ·
oxy witness serve: a transparency-log witness anyone can run carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
Witnesses for the transparency log: cosignatures and the witness rule carried-not-removedmerge-remote-bookmarkpaths-for-resolutions-and-amendshow-stat
Andy McCall via agent:claude@andymccall agent ·
A hub's new log key is a note, not a fork carried-not-removedmerge-remote-bookmarkpaths-for-resolutions-and-amendshow-stat
Andy McCall via agent:claude@andymccall agent ·
oxy tlog verify: audit a hub's whole transparency log
Andy McCall via agent:claude@andymccall agent ·
Spec: the transparency log in refs answers and what fetch checks
Andy McCall via agent:claude@andymccall agent ·
Fetch checks the host's transparency log (spec §14.8)
Andy McCall via agent:claude@andymccall agent ·
The transparency log as a library (tlog, spec §14.8)
Andy McCall via agent:claude@andymccall agent ·
Release 0.4.6: agents push as themselves carried-not-removeddoctor-trustmerge-remote-bookmarkpaths-for-resolutions-and-amendshow-statv0.4.6
Andy McCall via agent:claude@andymccall agent ·
Grant tokens reach hubs; unsetting core.sign works carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via agent:claude@andymccall agent ·
Release 0.4.5: keyless agents v0.4.5
Andy McCall via Claude Code agent ·
cosign commands say --insecure-ignore-tlog carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via Claude Code agent ·
oxy 0.4.4 v0.4.4
Andy McCall via Claude Code agent ·
A credential helper that does not read its input is no error carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via Claude Code agent ·
Cloud stores remember a missing key for 30 s, until this process writes it v0.4.4v0.4.5v0.4.6
Andy McCall via Claude Code agent ·
gc leaves a store that is already one full pack as it is v0.4.4v0.4.5v0.4.6
Andy McCall via Claude Code agent ·
Test: a state index's long changes block reads back whole and in order carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via Claude Code agent ·
Log entries are built in one sized step carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via Claude Code agent ·
Log takes the state index's changes in order; a clean diff loads no base carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via Claude Code agent ·
Read a packed object from its pack before asking for a loose copy v0.4.4v0.4.5v0.4.6
Andy McCall via Claude Code agent ·
Strata counts its requests by kind
Andy McCall via Claude Code agent ·
Tags made before a key rotation verify again v0.4.4v0.4.5v0.4.6
Andy McCall via Claude Code agent ·
Caching store: keep the ranges read of large objects, and their sizes carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via Claude Code agent ·
Approvals are kept in the repository: indexed by state, kept by gc carried-not-removeddoctor-trustmerge-remote-bookmarkpaths-for-resolutions-and-amendshow-statv0.4.4v0.4.5v0.4.6
Andy McCall via Claude Code agent ·
oxy config never prints a secret whole carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall via Claude Code agent ·
ci: keep each build job's target from its last passing run (refs andymccall/oxygenflow#28) carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall ·
build: watch FLOW_HEADS only when OXY_BUILD does not name the build (refs andymccall/oxygenflow#52) carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkpaths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall ·
ci: the build image with the swarm's crates vendored (refs andymccall/oxygenflow#28) carried-not-removeddoctor-trustkeyless-tagsmerge-remote-bookmarkpaths-for-resolutions-and-amendshow-statv0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0
Andy McCall ·
bookmark create/set --only: say that it can leave a file partly edited carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkonly-help-103paths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall ·
A hosted change can move a subrepo's pin only-help-103v0.4.4v0.4.5v0.4.6
Andy McCall ·
Cylinder: Draft 0.3 of the proposal carried-not-removeddoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkonly-help-103paths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-statev0.10.0v0.11.0v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall ·
Merge main into bare-hashes bare-hashesonly-help-103 resolution
Andy McCall ·
Merge main into switch-says switch-says resolution
Andy McCall ·
Merge main into switch-says resolution
Andy McCall ·
Merge main into bare-hashes resolution
Andy McCall ·
Merge main into carry-identical carry-identical resolution
Andy McCall ·
Release 0.4.3 v0.4.3
Andy McCall ·
Log without walking history when the index is the closure bare-hashescarry-identicalonly-help-103switch-saysv0.4.3
Andy McCall ·
Release 0.4.2 v0.4.2
Andy McCall ·
git verify: --max-differences, and what differs by kind bare-hashescarried-not-removedcarry-identicaldoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkonly-help-103paths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-stateswitch-saysv0.10.0v0.11.0v0.4.2v0.4.3v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall ·
A binary file's new contents and new executable bit are both kept bare-hashescarry-identicalonly-help-103switch-saysv0.4.2v0.4.3v0.4.4
Andy McCall ·
deb: name the machine's architecture in the apt source bare-hashescarried-not-removedcarry-identicaldoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkonly-help-103paths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-stateswitch-saysv0.10.0v0.11.0v0.4.2v0.4.3v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall ·
Release 0.4.1 v0.4.1
Andy McCall ·
git verify of an unfinished import; no SIGPIPE when an import stops early bare-hashescarried-not-removedcarry-identicaldoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkonly-help-103paths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-stateswitch-saysv0.10.0v0.11.0v0.4.1v0.4.2v0.4.3v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall ·
bookmark set: fast-forward onto work built on the newest head bare-hashescarried-not-removedcarry-identicaldoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkonly-help-103paths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-stateswitch-saysv0.10.0v0.11.0v0.4.1v0.4.2v0.4.3v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall ·
cat: say -r CHANGE reads the change's dependency closure bare-hashescarried-not-removedcarry-identicaldoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkonly-help-103paths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-stateswitch-saysv0.10.0v0.11.0v0.4.1v0.4.2v0.4.3v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall ·
README: the coverage badge bare-hashescarried-not-removedcarry-identicaldoctor-trustidentical-sideskeyless-tagsmainmerge-remote-bookmarkonly-help-103paths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-stateswitch-saysv0.10.0v0.11.0v0.4.1v0.4.2v0.4.3v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0 head
Andy McCall ·
Tests report coverage to the hub bare-hashescarry-identicalonly-help-103switch-saysv0.4.1v0.4.2v0.4.3
Andy McCall ·
A cloud store can be used from inside an async runtime bare-hashescarry-identicalonly-help-103switch-saysv0.4.1v0.4.2v0.4.3
Andy McCall ·
List storage.cache in oxy config's help. refs #68 bare-hashescarried-not-removedcarry-identicaldoctor-trustidentical-sideskeyless-tagsmerge-remote-bookmarkonly-help-103paths-for-resolutions-and-amendpush-notesshow-statsubrepo-pin-stateswitch-saysv0.4.1v0.4.2v0.4.3v0.4.4v0.4.5v0.4.6v0.5.0v0.5.1v0.6.0v0.7.0v0.8.0v0.9.0
Andy McCall ·
hide whitespace changes
added .config/nextest.toml +8 −0
1 +
# The CI profile (oxygen#139): every test runs, and a JUnit report is
2 +
# written to target/nextest/ci/junit.xml for the hub's test report
3 +
# (the job's `junit` artifact).
4 +
[profile.ci]
5 +
fail-fast = false
6 +
7 +
[profile.ci.junit]
8 +
path = "junit.xml"
added .flow/checks.wasm +0 −0

No line changes.

modified .flow/pipeline.kdl +107 −3
…
2 unchanged lines
3 3
// Earth; the jobs appear as checks on the change.
4 4
schema-version 1
5 5
6 +
// Cheap checks on the WASM tier (oxygenflow#96): OxygenFlow's checks module,
7 +
// copied from oxygenflow/.flow/checks.wasm (built there from
8 +
// crates/flow-checks; copy it again when it changes). No image and no KVM,
9 +
// so any runner with the WASM tier takes it: a phone, plex, windows-1.
10 +
sandbox "wasm" {
11 +
    runtime "wasm"
12 +
    memory-limit "64MiB"
13 +
}
14 +
15 +
// The changelog's shape, and a section for the version Cargo.toml says, so a
16 +
// release never finds its notes missing.
17 +
job "changelog" {
18 +
    sandbox "wasm"
19 +
    scope "CHANGELOG.md" "Cargo.toml" ".flow/checks.wasm"
20 +
    exec ".flow/checks.wasm" "changelog" "CHANGELOG.md" "Cargo.toml"
21 +
}
22 +
6 23
// The OxygenFlow build image: Rust with rustfmt and clippy, Node, and every
7 24
// crate of Cargo.lock vendored (oxygenflow/images/rust). Rebuild and re-pin
8 25
// it when Cargo.lock changes.
9 26
sandbox "rust" {
10 27
    runtime "microvm"
11 -
    rootfs "cas://blob/affd50628428966adc46670047468d4471cd589b4b92ef889981b2587a4eeed1"
28 +
    rootfs "cas://blob/836461b613019f62399a960c61a25c482eb74e30054ff2cd44d5594c3305872b"
12 29
    memory-limit "8GiB"
13 30
    cpu-limit 4
14 31
    // Tests serve and connect on 127.0.0.1; nothing leaves the guest.
…
10 unchanged lines
25 42
26 43
sandbox "rust-small" {
27 44
    runtime "microvm"
28 -
    rootfs "cas://blob/affd50628428966adc46670047468d4471cd589b4b92ef889981b2587a4eeed1"
45 +
    rootfs "cas://blob/836461b613019f62399a960c61a25c482eb74e30054ff2cd44d5594c3305872b"
29 46
    memory-limit "1GiB"
30 47
    cpu-limit 1
31 48
    network "isolated"
32 49
}
33 50
51 +
// The riscv64 build image on narvi-1 (oxygenflow#69): Ubuntu 24.04 riscv64
52 +
// with Rust 1.97.1 and the same vendored crates, in the syscall tier (the
53 +
// board has no hardware virtualisation, so no microVM). `runs-on` keeps it
54 +
// on a RISC-V runner.
55 +
sandbox "rust-riscv64" {
56 +
    runtime "syscall"
57 +
    rootfs "cas://blob/b4edba9173ddf63aa1df46222e6a59b265071b19f2815c89d0153b593835c616"
58 +
    runs-on "riscv64"
59 +
    memory-limit "4GiB"
60 +
    cpu-limit 6
61 +
    network "loopback"
62 +
    env {
63 +
        CARGO_INCREMENTAL "0"
64 +
        CARGO_PROFILE_RELEASE_DEBUG "0"
65 +
        CARGO_BUILD_JOBS "4"
66 +
    }
67 +
}
68 +
34 69
job "fmt" {
35 70
    sandbox "rust-small"
36 71
    exec "cargo" "fmt" "--all" "--check"
…
6 unchanged lines
43 78
        source "ephemeral://disk"
44 79
        target "/src/target"
45 80
        size "24GiB"
81 +
        // Kept from this job's last passing run (oxygenflow#28): cargo
82 +
        // builds only what changed since.
83 +
        cache "target"
84 +
        key "Cargo.lock"
46 85
    }
86 +
    // One build identity for every CI state, so the cached target stays
87 +
    // fresh from one push to the next (oxygenflow#52).
88 +
    env {
89 +
        OXY_BUILD "ci"
90 +
    }
47 91
    exec "sh" ".flow/warm-target.sh" "cargo" "clippy" "--workspace" "--all-targets" "--locked" "--" "-D" "warnings"
48 92
}
49 93
94 +
// The tests, run under cargo-nextest and cargo-llvm-cov: a test report
95 +
// (JUnit, kept whether or not they pass; #139) for the hub's Tests tab and
96 +
// line coverage (LCOV, oxygenhub#113) for its Coverage tab, from one run.
50 97
job "test" {
51 98
    sandbox "rust"
52 99
    depends-on "fmt"
…
1 unchanged lines
54 101
        source "ephemeral://disk"
55 102
        target "/src/target"
56 103
        size "24GiB"
104 +
        // Kept from this job's last passing run (oxygenflow#28): cargo
105 +
        // builds only what changed since.
106 +
        cache "target"
107 +
        key "Cargo.lock"
57 108
    }
58 -
    exec "sh" ".flow/warm-target.sh" "cargo" "test" "--workspace" "--locked"
109 +
    // One build identity for every CI state, so the cached target stays
110 +
    // fresh from one push to the next (oxygenflow#52).
111 +
    env {
112 +
        OXY_BUILD "ci"
113 +
    }
114 +
    exec "cargo" "llvm-cov" "nextest" "--workspace" "--locked" "--profile" "ci" "--lcov" "--output-path" "/src/target/lcov.info"
115 +
    output {
116 +
        artifact "junit" {
117 +
            path "/src/target/nextest/ci/junit.xml"
118 +
            when "always"
119 +
        }
120 +
        artifact "coverage" {
121 +
            path "/src/target/lcov.info"
122 +
        }
123 +
    }
59 124
}
60 125
61 126
job "build" {
…
3 unchanged lines
65 130
        source "ephemeral://disk"
66 131
        target "/src/target"
67 132
        size "24GiB"
133 +
        // Kept from this job's last passing run (oxygenflow#28): cargo
134 +
        // builds only what changed since.
135 +
        cache "target"
136 +
        key "Cargo.lock"
68 137
    }
69 138
    exec "sh" ".flow/warm-target.sh" "cargo" "build" "--release" "--locked"
70 139
    // The binary, kept as a blob of this repository and named by the
…
19 unchanged lines
90 159
        source "ephemeral://disk"
91 160
        target "/src/target"
92 161
        size "24GiB"
162 +
        // Kept from this job's last passing run (oxygenflow#28): cargo
163 +
        // builds only what changed since.
164 +
        cache "target"
165 +
        key "Cargo.lock"
93 166
    }
94 167
    env {
95 168
        DIST_FLAT "/src/target/dist-flat"
…
28 unchanged lines
124 197
        artifact "oxy_riscv64.deb" {
125 198
            path "/src/target/dist-flat/oxy_riscv64.deb"
126 199
        }
200 +
        artifact "oxy-src.tar.gz" {
201 +
            path "/src/target/dist-flat/oxy-src.tar.gz"
202 +
        }
127 203
        artifact "SHA256SUMS" {
128 204
            path "/src/target/dist-flat/SHA256SUMS"
129 205
        }
130 206
    }
131 207
}
208 +
209 +
// The native RISC-V check (oxygenflow#69): oxy built and used on narvi-1,
210 +
// a real riscv64 board, so each release is known to run there and not
211 +
// only to cross-compile (dist). Like dist, a push runs it only when it
212 +
// touches these paths; "Run pipeline" and reruns always do. Its target
213 +
// directory is cached between passing runs (oxygenflow#70).
214 +
job "riscv64" {
215 +
    sandbox "rust-riscv64"
216 +
    depends-on "fmt"
217 +
    scope "Cargo.toml" "scripts/dist.sh" "packaging/**" ".flow/riscv64-check.sh"
218 +
    timeout "3h"
219 +
    mount {
220 +
        source "ephemeral://disk"
221 +
        target "/src/target"
222 +
        size "16GiB"
223 +
        // Kept from this job's last passing run in the syscall tier too
224 +
        // (oxygenflow#70): the first build is cold (~40 min on narvi-1),
225 +
        // later ones build only what changed.
226 +
        cache "target"
227 +
        key "Cargo.lock"
228 +
    }
229 +
    exec "sh" ".flow/riscv64-check.sh"
230 +
    output {
231 +
        artifact "oxy-linux-riscv64-native" {
232 +
            path "/src/target/release/oxy"
233 +
        }
234 +
    }
235 +
}
added .flow/riscv64-check.sh +29 −0
1 +
#!/bin/sh
2 +
# The native RISC-V check (oxygenflow#69): on narvi-1, a riscv64 board,
3 +
# build oxy natively and use it, so a release is known to work on real
4 +
# RISC-V hardware, not only to cross-compile for it (dist does that).
5 +
set -eu
6 +
uname -m
7 +
cargo build --release --locked -p oxy
8 +
oxy=/src/target/release/oxy
9 +
"$oxy" --version
10 +
# A repository's life in miniature: an identity, a signed change, a
11 +
# bookmark, a merge, and a full integrity check.
12 +
export OXY_HOME="$(mktemp -d)"
13 +
cd "$(mktemp -d)"
14 +
"$oxy" --plain init
15 +
"$oxy" --plain identity create --name "narvi-1 check" --email [email protected]
16 +
"$oxy" --plain config --user core.sign change
17 +
echo one > a.txt
18 +
"$oxy" --plain commit -m "first"
19 +
"$oxy" --plain bookmark create side
20 +
"$oxy" --plain bookmark switch side
21 +
echo two > b.txt
22 +
"$oxy" --plain commit -m "second, on side"
23 +
"$oxy" --plain bookmark switch main
24 +
"$oxy" --plain merge side
25 +
"$oxy" --plain log
26 +
"$oxy" --plain verify
27 +
"$oxy" --plain fsck
28 +
test -f b.txt
29 +
echo "oxy works natively on $(uname -m)"
added .oxyattributes +2 −0
1 +
# oxygen#145: two changes adding changelog entries keep both.
2 +
CHANGELOG.md merge=union
modified CHANGELOG.md +217 −1
1 1
# Changelog
2 2
3 -
## Unreleased
3 +
## 0.11.0 (2026-11-04)
4 +
5 +
- **Changed: the changelog is checked on the WASM tier.** A `changelog` job in the pipeline runs OxygenFlow's checks module on any runner with the WASM tier, failing when the changelog's sections are out of shape or Cargo.toml's version has none.
6 +
- **Changed: moved subrepo pins are named (#162).** A pin moved by work inside a subrepo is a pin bump the parent's next commit records, and it used to go in unsaid. `oxy status` now lists it under *Subrepo pins* (old → new), and a subrepo's own uncommitted edits, which the parent does not record; a bare `oxy commit` that records a moved pin says so and how to leave it out (`oxy commit PATHS`).
7 +
- **Added: `oxy submodule remove --delete` (#164).** It deletes the subrepo's directory too, read-only object files and all, and refuses while the subrepo has uncommitted edits (`--force` to delete them anyway); without it, removal says the directory is left and how to delete it. A directory moved by hand used to fail halfway, after the copy, on the read-only files.
8 +
- **Changed: an agent's commit says "awaiting seal", not "unsigned" (#165).** In a checkout that signs keyless under an agent grant, a change of its own author that the next push will seal shows `◌ awaiting seal` in `oxy log` and `oxy show`; "unsigned" is kept for changes nothing will sign.
9 +
- **Fixed: a stale `author.key` can no longer fork your identity (#163).** A checkout that pinned `author.key` to a key a rotation made elsewhere had already succeeded built a second successor of it on `oxy identity delegate add`, splitting the identity in two. `rotate`, `renew` and `delegate add`/`remove` now refuse to start from a key that already has a successor in your home, naming the newest key and the `oxy config` that points at it; their report says which key they started from and which config named it; and `oxy identity show` warns when the configured key is not the newest, and names the key where an identity forks.
10 +
11 +
## 0.10.0 (2026-11-01)
12 +
13 +
- **Fixed: fetch, pull and clone are far faster with many bookmarks (#158).** They fetched each bookmark in its own exchange with the hub, and changes that bookmarks share were sent again for each one; a pull with nothing new still asked for a bundle per bookmark. Now every bookmark's heads are fetched in one exchange, and none when nothing is new. On a 47 MB repository with 160 bookmarks: a pull with nothing new 169 s → 3.5 s, a clone 324 s → 26 s. `OXY_TIMINGS` now also shows the refs, transparency log and per-bookmark checks of a fetch.
14 +
- **Added: `oxy git colocate`, a `.git` beside `.oxy` (oxygenhub#355).** Your editor's Git features (blame, history, gutter marks) and other Git tools now work on an Oxygen checkout: oxy keeps a `.git` there with the current bookmark exported after every command, Git's index at its tree so `git status` shows only your uncommitted work, and `.oxy/` excluded. One way: commit with oxy. `--off` stops it.
15 +
- **Changed: `.git` is never content.** oxy no longer picks up a `.git` directory at a checkout's top, and checkout refuses a path with a `.git` component at any depth (as Git does), so a change cannot plant Git hooks.
16 +
- **Added: a Git export from the change graph (oxygenhub#355).** oxy-core's `git_export_graph` writes every bookmark and tag to a bare Git repository so that the same changes give the same commits wherever it runs: each change a commit whose parents are its dependencies, each bookmark a branch (a merge of its heads when it has several), tags as tags, incrementally. It is what OxygenHub serves to `git clone`. (`oxy git export` still follows the checkout's own history.)
17 +
- **Added: `oxy mcp`, Oxygen for AI agents (#159).** An MCP server over stdio for a repository: an agent calls typed tools (status, diff, log, show, cat, blame, bookmarks, tags, conflicts, op log, identity, commit, amend, describe, fetch, pull, push, tlog verify) and gets each command's JSON back instead of reading a shell. It signs as the identity its environment configures, so started with an agent home every change is sealed via the agent with nothing to forget per command, and says who in its `initialize` reply. Commits name their paths (or `all: true`); undo and forced pushes are off unless started with `--allow-destructive`. The protocol itself is oxy-core's `mcp` module, for `oh mcp` to reuse. See docs/mcp.md.
18 +
- **Icons for HaikuDepot (#157).** `packaging/haikuports/icons/` has oxy's and oh's icons as HVIF (Haiku's vector format) with 16, 32 and 64 px PNGs, written by `make-icons.py`: the Oxygen mark on a pale tile for oxy, on OxygenHub's dark tile for oh, the same family as the VS Code extensions.
19 +
20 +
## 0.9.0 (2026-10-29)
21 +
22 +
- **Releases carry their source (#157).** Each release now has `oxy-VERSION-src.tar.gz`, the source as released, reproducible (sorted, fixed times and owners, gzip without a timestamp), listed in SHA256SUMS, for packagers to fetch at a fixed checksum. `packaging/haikuports/make-recipe.py oxy|oh TARBALL VERSION` writes the haikuports recipe from it (oh's from OxygenHub's source tarball); the 0.8.0 recipe, which used the on-demand archive, is gone.
23 +
- **Added: a haikuports recipe for oxy (#156).** `packaging/haikuports/` holds the recipe that builds oxy for HaikuDepot from the release source and every crate in `Cargo.lock`, and the script that writes it for a new release; the 0.8.0 recipe builds with haikuporter on Haiku R1/beta6.
24 +
- **Fixed: an expired checkout token no longer hides your sign-in (#155).** A checkout's own token for a remote (`remote.NAME.token`) is used first, so when it expired every fetch, pull and push answered 401 although `oxy auth login` had a good sign-in for the same hub. A checkout token the hub no longer knows (401) now falls back to that sign-in, with a warning naming `oxy config --unset remote.NAME.token` to drop the stale one; never to another checkout's token, and not when the token is refused for permission (403, 404), so a narrowly scoped token is never swapped for a wider sign-in. When both are refused, the error says to sign in again and unset it.
25 +
- **Added: the `artifact` object type (Cylinder C1, #108).** An artifact is now an object of its own in the store (oxy-core's `artifact` module), named by its object hash as every object is, and `oxy fsck` checks it; its tree stays a blob. Stores written before read as they did: the type is a new one at the end of the list.
26 +
- **Added: artifacts (Cylinder, step C1, #108).** `oxy artifact pack DIR --kind archive|package|container|runner-image|report [--arch --os --entrypoint --env K=V --label K=V]` packs a directory into an artifact: each file an ordinary blob of the repository's store, so files and chunks it holds already cost nothing, under a tree and an artifact encoding in canonical CBOR, named by the BLAKE3 of that encoding (docs/cylinder.md §4.1). `oxy artifact list | show NAME | unpack NAME DEST | verify NAME` (any unique prefix of six characters or more; `--json` on list, show and pack), and `oxy gc` keeps every artifact's blobs. The new `cylinder` crate holds the model.
27 +
- **Added: oxy on Haiku (#154).** oxy builds and runs on Haiku R1/beta6 (x86_64) with HaikuPorts' Rust; `oxy-VERSION-haiku-x86_64.tar.gz` is on the releases page from 0.8.0, with its own `SHA256SUMS-haiku`. The README says how to install it and how to build it (`ZSTD_SYS_USE_PKG_CONFIG=1`).
28 +
29 +
- **Fixed: a subrepo put back after its removal is pinned in the change that registers it (#152).** `oxy submodule remove` leaves a marker so the next snapshot drops the node while the directory stays; the snapshot cleared it only after reading it, so when nothing looked at the working copy between the removal's commit and the subrepo's return, the change that registered it again in `.oxymodules` left it out and a second commit was needed. A marker is now dropped before it is read once the removal is recorded or the path is registered again.
30 +
- **Fixed: `oxy identity create` needs no repository (#144).** Outside one it makes a user identity: the secret and identity go to the oxy home's keys and the user file names it as the author, so a new user or machine sets up an identity first; a repository made or cloned afterwards signs with it. Only `--local` needs a repository, and inside one it is kept there too, as before.
31 +
32 +
## 0.8.0 (2026-10-27)
33 +
34 +
- **A server can add a note to a reference a push moved (#148).** `oxy-server`'s `Backend` has a `push_notes` step: each note is added to that accepted reference's result, which `oxy push` shows in place of "updated". OxygenHub will use it to say when a push moves the bookmark of a proposal that has already landed (oxygenhub#329).
35 +
36 +
- **Identical sides settle by themselves (#146).** Two bookmarks that each resolved the same conflict to the same text conflicted again when merged, both sides the identical lines. A conflict whose sides are byte-identical now settles as `merge=union` adjacent insertions do (#145), in any file and keeping one side: a checkout writes it once ("kept one of its identical sides") and the next commit records the resolution, and a host landing a proposal settles it too.
37 +
38 +
- **A subrepo at its pinned state is unmodified on any bookmark (#147).** A subrepo whose heads were exactly its pin's but which sat on another bookmark than the pin named counted as modified, so `oxy bookmark switch` refused ("the working copy has changes"). Only the state counts now: the same repository at the same heads, or the same Git commit; the recorded pin stands.
39 +
40 +
- **A shallow clone fetches its checkpoint in parts (#137).** The checkpoint of a big state came as one answer, the whole state's graph blobs in one body (about 2 GB for the VS Code history), built and held whole on both sides. A server now answers `checkpoint-index` (the checkpoint and its history, no graphs), and the client fetches the graph blobs it lacks with `objects`, 2,048 at a time; `objects` reads them in pack order. An older server answers 404 and gets the one-piece request as before.
41 +
42 +
- **A keyless agent follows its owner's newest identity (#82).** Every delegation rotates its owner's identity, so the `author.key` an agent's home was given goes stale, and its changes named an identity two rotations old (verifying only because a grant from before a rotation still counts). With `author.for = ACCOUNT` in the home, the hub sends the account's newest linked identity with each certificate, with its chain back to the grant's key; oxy keeps it in the home (`keys/author.for`), stores the chain beside the change so the grant checks, and authors as it from then on. A repository's own `author.key` pin still wins.
43 +
44 +
- **Changelogs stop conflicting (#145).** A file marked `merge=union` in `.oxyattributes` (`CHANGELOG.md merge=union`) has its adjacent insertions settled by the checkout itself: both sides kept, the earlier change's first, as `oxy resolve --both` does, and the next commit records the resolution. Only adjacent insertions: where two changes changed the same lines, the markers stay. Oxygen's own changelog is marked so.
45 +
46 +
- **Two changes adding lines at the same place are settled with one command (#121).** Like Git, oxy leaves a conflict when two changes each add lines after the same line, since either order can be wrong for code; settling it meant editing out the markers by hand. `oxy resolve --both [PATH]…` now keeps every side of each such *adjacent insertion*, the earlier change's first (`--reverse` for the other order), and `oxy status` says which conflicts qualify. It leaves the markers where the sides changed the same lines (keeping both would keep two versions of them) or where the lines between the markers were edited. Conflict markers now list their sides earliest change first, not in hash order.
47 +
48 +
## 0.7.0 (2026-10-26)
49 +
50 +
- **Many objects are read in pack order (#137):** `Store::get_many` (a clone's checkpoint, a fetch's objects) reads packed objects by pack and offset, each thread a run of them, so a spinning disk sweeps instead of seeking across the packs; results keep the asked order. The server's gathering of a VS Code-sized checkpoint (tens of thousands of objects in ~95 packs) was the slow part of a depth-0 clone on Earth.
51 +
- **Windows paths print plainly (#143):** a repository's root showed as `\\?\C:\work\demo` in every message, Windows' verbatim form from `canonicalize`; oxy now drops the prefix where the plain path says the same (`oxy_core::portable::canonical`). Found running the shipped Windows build on a Windows 11 VM.
52 +
- **Oxygen's CI writes a test report beside its coverage (#139):** the test job runs under cargo-nextest (`.config/nextest.toml`, profile `ci`) as well as cargo-llvm-cov, keeping a JUnit report whether or not the tests pass, for the hub's Tests tab.
53 +
- **Pushing a new bookmark sends only what the remote lacks (#142).** A push was worked out against the remote's heads for that bookmark alone, so a bookmark new to the remote sent its whole history again: slow on a big repository, and OxygenHub's push protection scanned all of it (and refused a proposal's bookmark for an example key in a change main had had for weeks). The heads of every bookmark the remote lists now count as already there.
54 +
- **A binary file's or subrepo pin's conflict can be settled from the command line (#140).** Two changes that replaced the same binary file, or moved the same subrepo pin, leave a replace conflict with no markers to edit, and every commit was refused until it was settled, which only a hosted edit could do. `oxy resolve PATH…` now settles it with what the working copy holds at the path (the file, or the subrepo's pin): the next commit is a resolution replacing every version with it. A path with no replace conflict is refused, and a choice is forgotten once committed.
55 +
- **The object store works where hard links are refused (#141).** Android forbids apps hard links in their own storage, so the OxygenHub app's phone runner could clone nothing (`Permission denied`). Where a link is refused, objects are renamed into place with `renameat2(RENAME_NOREPLACE)`: as atomic, and still never over a name that is taken.
56 +
- **Rendering a big tree is many times faster, and clone, pull, switch and `oxy workspace add` stop rendering it twice (#137).** Measured on the VS Code history's depth-0 clone (19,545 files, 166,425 changes), `oxy workspace add` took 467 s on Earth and 176 s on an 8-core workstation, 11 GB at its peak; it now takes 7.5 s and 1.8 GB on the workstation. Three causes: (1) whether one change depends on another, which rendering asks about every deleted line, was answered by a depth-first walk that compared a change with the one sought only on coming back up to it, so it went through thousands of changes first although the answer is nearly always a direct dependency (spec §8.1): 262,678 questions took 4.4 billion steps and 838 CPU-seconds, and now take 5 CPU-seconds, with the same answers; (2) finding the subrepos rendered every file of the tree, after every clone, pull, switch and `workspace add`, where reading the tree and the subrepos' pins is enough; (3) a new workspace started on the target state with an empty directory, so the checkout's first snapshot found the whole tree deleted and rendered it to say so. `OXY_TIMINGS=1` now also says how a checkout's files were ordered (`graph: single order`…) and how its blobs were read.
57 +
- **`oxy serve` takes a push of more than 2 MB (#138).** It set no request limit of its own, so axum's default of 2 MB applied, and a larger push part was cut off with `Broken pipe` at the client and nothing in the server's log. It now takes up to 4 GiB a request (`oxy_server::BODY_LIMIT`), OxygenHub's default `push_body_limit`; OxygenHub was not affected.
58 +
- **A push no longer sends a replaced or removed binary file's old content again (#136).** A bundle carried both the old and the new blob of every binary file a change replaces, so a change removing ~1,100 tracked cache files (oxygenhub-android's set-up) pushed all of them again, over 100 MB, though the hub stored each when it was added. The old blob is always the new blob of a change the replacement depends on, which the receiver already holds or which the same bundle carries, so pushes and fetch answers now carry only the new one.
59 +
60 +
## 0.6.0 (2026-10-25)
61 +
62 +
- **A shallow clone of a big repository works, and quickly (#137).** `oxy clone --depth 0` of the VS Code history failed after eleven minutes ("object … not found"): the blobs of the state's binary files are named only by the changes that set them, which a depth-0 clone does not take, so nothing brought them. They are now fetched with the checkpoint (in parts), or left to be fetched on read under `--max-blob`. The checkpoint itself was stored object by object, each written loose and flushed (57,498 objects at 27 a second); it is now stored in one batch, as a fetch's objects are.
63 +
- **A checkpoint is served faster, and says where its time goes (#137).** A server gathered a checkpoint's graph blobs and their chunks one read at a time: six minutes for the VS Code state's 57,000. They are now read in two parallel batches. `OXY_TIMINGS=1` now times the checkpoint's phases on both sides (`checkpoint: gather its objects`, `serve: encode the checkpoint`, `remote: checkpoint answer`, `shallow: store the checkpoint` and the rest).
64 +
- **A depth-0 clone takes no change (#137).** `oxy clone --depth 0` took the head changes themselves (the VS Code import's main has 33,835 heads), though the checkpoint already holds every change's metadata, so depth 0 now takes none (spec §17's "a checkpoint plus zero history"): its heads become the shallow boundary, and commit, push and pull on top work as before (a shallow repository takes and builds on changes below what it cloned, which the remote holds).
65 +
- **A push no longer passes a proxy's request limit (#133).** A push to a hub behind Cloudflare, which takes at most 100 MB in one request, was refused `HTTP 413` with Cloudflare's HTML page printed raw: parts of a long push were about 256 MiB, and one change's blobs went in one request however many there were. Parts are now about 64 MiB (`push.batch-bytes`), a part whose blobs pass that sends them first in parts of objects alone, an HTML error page is shown by its title, and a 413 says how to push in smaller parts.
66 +
- **A keyless agent's grant survives its owner's key rotation (#135).** Every delegation an owner adds or removes renews their identity, and a repository that names the newest key as `author.key` read every change their agent sealed as INVALID ("key … is neither the author nor a delegate"), and its pushes were refused, because the agent's grant was signed by the owner's earlier key. A grant by a key the author's identity was rotated or renewed from now counts when it was made before that rotation; one an old key made after it does not.
67 +
- **A tag can be withdrawn, by a signed withdrawal (oxygenhub#301).** There was no way to remove a tag anywhere: a mistaken release tag stayed for good. `oxy tag delete NAME [-r REASON]` records an `untag` attestation naming the tag's state, signed like a tag (a ten-minute keyless key too), and the next `oxy push` sends it to a remote that still has the tag; `oxy push --delete tags/NAME` does both at once. A receiver under a policy accepts it only from a key that may create the tag, keeps it beside the tag's creation, and lists withdrawn tags in its refs answer (key 8), so a `fetch` withdraws the tag there too (`tag v1 was withdrawn on origin, and here`). A withdrawn name is never used again, so a tag name never names two states. `oxy-server`'s `Backend` gains `vet_tag_withdrawal` (OxygenHub refuses while a release uses the tag), and `Repo::host_tag_create` and `host_tag_withdrawal` let a host make or withdraw a tag for a signed-in user. The wire change is additive, but an older server refuses a withdrawal (wrong attestation kind), and an older client keeps a withdrawn tag.
68 +
- **A keyless agent's changes verify in a repository that has never seen its owner (#134).** An agent's ten-minute key is certified under its owner's signed grant, and a change it signed only verified where the owner's identity was already stored: in a new repository, or one the owner had not pushed to since a key rotation, `oxy log --verify` and the hub said INVALID ("grant is by an unknown author"). OxygenHub now sends the grant's signer with the certificate (`grantor_identity`, additive), `oxy` keeps it (else uses `<oxy home>/keys/<key>.identity`, or refuses with how to put it there), and a push brings it beside the certificate, so the receiver can check the grant.
69 +
70 +
## 0.5.1 (2026-10-20)
71 +
72 +
- **oxy 0.5.0 and older can clone a repository again after a keyless agent tagged it (#131).** Since #129 a release tag may be signed by a ten-minute certified key, which released clients do not know how to authorize: they refused the tag (`key … may not create tags/v0.5.0 under this repository's policy`), and with it the whole clone, fetch or `oxy submodule update`. Clients now send `User-Agent: oxy/<version>` and declare what they understand in `Oxy-Features` (`certified-tags` so far; docs/protocol.md, *Client capabilities*). Under a policy, a server leaves out of its refs answer to a client that does not declare `certified-tags` every tag that only a certificate authorizes, so that client clones the rest as if the tag did not exist. The rule is `Repo::refs_for_client` in oxy-core; `oxy-server` gains a `Backend::adjust_refs` hook, given the request headers, whose default applies it, so `oxy serve` and every embedding host do.
73 +
- **Keyless agents can create release tags (#129).** `oxy tag create` and `oxy push` under a keyless home was refused "unknown key": the push did not carry the tag signer's identity and certificate, and tags were authorized only by keys the policy names, never by a certificate under a subject grant. The push now carries them, and a receiver authorizes a short-lived key's tag by a subject grant at its own clock, as it does a bookmark move, and records when it accepted it (OxygenHub signs that with its key). A fetch checks such a tag at that acceptance, or at its integrated time when the hub's log gives one, so a release tag made with a ten-minute key is still fetched months later. A fetch never checks it at the time the tag claims: without an acceptance the tag is left out with a warning (`tags/NAME not updated`), and the rest is fetched. A client keeps the acceptance and serves and pushes it with the tag, so a tag passed on to another remote, or cloned from a clone, keeps its time. The wire change is additive: tags in `refs` and `push` gain keys 4 to 6 (acceptance, log place, cosigned checkpoints), which older readers ignore.
74 +
75 +
## 0.5.0 (2026-10-20)
76 +
77 +
- **A repository kept in a bucket finds signatures and seals without a LIST each (#116).** Every lookup of an object's signatures, seals, receipts, approvals or certificate listed a prefix in the bucket, and most of these lookups found nothing. Writers now also add each such entry to a journal (`pack-entries`, updated by compare-and-swap), and `oxy gc` folds them all into one list. Once a store has been folded, a reader reads the lists and the journal when it opens it, and again after 30 seconds (as long as the pack listing is kept), and looks everything up from them. Another writer's entry is seen within 30 seconds, as its pack would be, even by a handle a server keeps open. The first `oxy gc` also copies the one-list keys of much older stores (`sigs/`, `seals/`, `certs/`, …) into the current layout and marks them folded, so readers stop requesting them; until now nearly every lookup made one of these requests and found nothing. On the test store, 40 lookups went from 85 requests (40 LISTs) to 7 (one journal read and no LIST). Older readers are unaffected, because every entry still has its own key. Entries written by oxy 0.4.6 or older after the fold are missed until the next `oxy gc`, so upgrade every client that writes to the bucket first, then run `oxy gc` once. A server's `/health` counts journal reads as `read_journal`. A repository's own directory works as before.
78 +
79 +
- **A server can refuse a push by what it adds (oxygenhub#268).** `oxy-server`'s `Backend` has a `vet_push` step, called with the push's changes before anything is stored or moved; OxygenHub uses it for push protection (secret scanning). The refusal reaches `oxy push` as the server's own text, several lines included, and a refusal that mentions a certificate or a timeout no longer picks up a misleading hint.
80 +
81 +
- **Fetch checks the hub's transparency log (oxygenhub#238).** A hub that keeps a log sends its latest checkpoint with every refs answer, and each bookmark's attestation with its log entry and inclusion proof. The first fetch pins the hub's log key in `<oxy home>/tlog/<hub>` with the last checkpoint seen. A later checkpoint must be signed by that key and extend the last one: a smaller log, another root at the same size or a failed consistency proof fails the fetch, saying the hub showed two histories. An attestation not logged yet, or not proved, is only a note for now, printed by `fetch` and `pull` and in `--json`. A hub that comes back with a new log key (a new key, a reused address, a rename) is a loud note, and the new key is pinned. Older hubs and clients ignore the new fields.
82 +
83 +
- **`oxy tlog verify` audits a hub's whole log (oxygenhub#238).** It accepts the hub's checkpoint by the same rules as fetch, downloads every entry and requires them to make the root the checkpoint signs, then lists this repository's entries, a key's (`--key`) or all of them (`--all`); `--json` too. It also shows which witnesses cosigned and when.
84 +
85 +
- **Trusted time from witnesses (oxygenhub#240).** An entry's integrated time is the earliest time a trusted witness cosigned a checkpoint that holds it, by the witness's clock, so a hub can only make it later, never earlier. A fetch pins the hub's witnesses beside its log key (`<oxy home>/tlog/<hub>.witnesses`; a new or changed witness key is a note), counts a cosignature only when it verifies against a pinned key, and checks a delegate's delegation and a short-lived key's certificate at that time. Without one it checks them as before, at the hub's acceptance or the attestation's own time. A claimed time more than 10 minutes off the integrated time is noted, not refused, for now. The wire change is additive.
86 +
87 +
- **`oxy witness serve`: a witness anyone can run (oxygenhub#238).** `oxy witness serve --name HOST --log ORIGIN=FILE …` serves the C2SP tlog-witness `add-checkpoint` call. It cosigns, with this oxy home's key and its own clock, each checkpoint that verifies with the log's key and extends the last one it cosigned, and refuses a fork. What it cosigned is kept per log in `--state`, so it survives a restart.
88 +
89 +
- **`oxy show --stat` (#119).** Lists the files a change touched instead of its diff: each path with lines added and removed and its kind, renames as from → to, and a total. `--json` gives each file's counts.
90 +
91 +
- **`oxy merge` takes a remote's bookmark (#117).** `oxy merge oxygenhub/develop` said "no bookmark": merge looked only at local bookmarks, so merging what a remote has meant switching to it, pulling and switching back. It now takes a remote's tracking bookmark as last fetched, as revsets do, and says where it looked when neither exists.
92 +
93 +
- **Commit a resolution by paths, and amend only some paths (#118).** A conflict resolution can be committed by paths, leaving other work uncommitted on top of the resolved state, as long as the paths include every file it resolves, whole; otherwise the refusal names them. `oxy amend PATH…` folds only those paths into the head and leaves the rest uncommitted.
94 +
95 +
- **`oxy doctor` checks the trust policy (#127).** A repository with no policy was "Everything looks right" while every fetch warned that nothing was verified. Doctor now has a trust line: no policy is a warning that says who sets one and how; a policy shows its serial, roots, rules and expiry, warns within 30 days of expiry and fails once it has expired.
96 +
97 +
- **The same file added on both sides is one file (#101).** The same path added on two bookmarks with identical content (files, symlinks, or subrepos at the same pin) now merges with no conflict and nothing to resolve. Every peer and the hub show it the same way whatever order the changes came in, and it stays one file through later edits, moves, removals and partial commits. Different content or a different mode is still a name conflict, and directories are not unified. The stored and signed formats are unchanged.
98 +
99 +
- **A bookmark nobody can verify no longer breaks every fetch (oxygenhub#266, oxygenhub#271).** A delegate or short-lived key that moved a bookmark without signing any of its changes left every client failing the whole fetch with "unknown key". A fetch now carries the identities and certificates of the keys that signed every bookmark's latest attestation and every tag's, so clients, older ones included, can verify the move. A client that still cannot verify a bookmark other than the current one leaves that bookmark as it was, warns naming it and why, and fetches the rest. The current bookmark (the one a pull takes or a clone checks out) still fails, as does a missing, unauthorised or rolled-back attestation. The signers are gathered once per operation, not on every fetch call: a repository in a bucket with 145 bookmarks had taken 14 s a fetch.
100 +
101 +
- **Readers never wait for a push or gc (oxygenhub#249).** A repository can now be read safely while another process writes it. A handle reads the operation head once and keeps it, so one request never mixes two views. A read that misses reloads the pack list and tries once more. A repack retires the packs and loose objects it replaces instead of deleting them, and `gc` deletes them only an hour later; `oxy gc` says what it deleted or kept for readers. A forge no longer needs to hold readers back while it pushes or collects garbage.
102 +
103 +
- **A repository kept in a bucket lists its packs less often (#116).** Every open listed the bucket's packs, and a forge opens the store once per request. The pack listing is now kept for 30 seconds and forgotten the moment this process writes or deletes a pack, so its own packs are never hidden; another writer's new pack is seen within 30 seconds. Opening a store no longer sends a request per pack, and a bucket store's stat is a HEAD of the key, not a listing. A server's `/health` counts pack listings, kept listings and other listings apart.
104 +
105 +
- **Text that looks like a conflict marker no longer blocks a resolution (#126).** A file that showed oxy's own conflict markers, such as documentation or a test fixture, could never be committed as resolved ("markers remain"). A file now still has markers only when it holds a marker line oxy wrote for its conflicts. A real marker left behind is still caught.
106 +
107 +
- **A carried switch no longer says carried files were removed (#120).** After a switch that carried the working copy's changes, oxy said new files were "removed from disk" although they were still there. Carried files are now left out of what was removed, and the switch names what it carried.
108 +
109 +
## 0.4.6 (2026-10-17)
110 +
111 +
- **A keyless agent's grant token is its only credential (oxygenhub#241).** A home holding an agent grant token and no hub token now offers the grant to that hub, which accepts it for fetch and push as an agent's token of the owner's account (writing when the grant gives `refs`). The agent pushes its own work, so its seal is its own; it needs no personal access token.
112 +
113 +
- **Agents' signatures say whose they are.** A short-lived key is named by the subject it was certified for, so `oxy log` shows `via agent:claude@andymccall` and `oxy verify` shows `signed by agent:claude@andymccall`, rather than the author's name with an unfamiliar key.
114 +
115 +
- **`oxy config --unset core.sign` works.** It failed with "core.sign must be change or seal"; unsetting now returns to the default.
116 +
117 +
## 0.4.5 (2026-10-17)
118 +
119 +
- **Agents sign with no key of their own (oxygenhub#241).** With `author.signer = keyless`, oxy signs with a fresh key the hub certifies for ten minutes whenever it must sign, kept in memory only. The agent proves itself with a grant token its owner made with `oh agent grant NAME` and `oxy auth grant` keeps. The owner's own key signs the grant, which travels inside every certificate issued under it, so the agent's commits count in the owner's name without trusting the hub for authorship. The owner revokes the grant on the hub, and there is no key left to rotate or leak. `keyless.hub` names the hub (default: the default remote's).
120 +
121 +
- **Certificates for short-lived keys (spec §14.7).** A hub's key certifies a fresh key for a subject such as `repo:OWNER/NAME:bookmark:B` or `agent:NAME@ACCOUNT`, with a scope and a window of at most an hour. Pushes and fetches carry the certificates of the keys that signed what they bring. `oxy receipt verify` reads receipts that OxygenFlow runners sign with such keys.
122 +
123 +
- **Policies grant subjects an issuer certifies.** `oxy policy grant-subject SUBJECT --issuer IDENTITY --scope SCOPES` lets keys certified for a subject sign changes and move bookmarks. A push is checked at the receiver's clock. A fetch is checked at the time the issuer says it accepted the update, a countersigned `accepted` attestation the hub serves with the bookmark, so an expired key cannot backdate one.
124 +
125 +
- **A push seals for the author it signs for.** A delegate or a certified key set as `author.signer` now seals the author's unsigned changes at push, so it no longer has to sign each change at commit (`core.sign = change`).
126 +
127 +
- **gc keeps short-lived keys' certificates** for every change and attestation it keeps, not only receipts', so a fresh clone can still check a certified bookmark after a gc.
128 +
129 +
## 0.4.4 (2026-10-17)
130 +
131 +
- **A fresh clone takes tags made before a key rotation (#114).** A tag's signature was checked against the current trust policy with no regard for rotation, so every tag signed by a key since rotated away from was refused, and a fresh `oxy clone` of a repository with such a tag failed (this repository's own v0.1.0). A tag signed by a predecessor of a key the policy names is accepted when it is dated before that key's successor was made; a superseded key makes no new ones, and revoked keys never count.
132 +
133 +
- **SLSA provenance of a receipt (oxygenhub#239).** `oxy receipt export --format slsa` writes what a valid receipt says as an in-toto Statement with an SLSA provenance v1 predicate, in a DSSE envelope signed with your key, the runner's signed receipt riding along inside it. `oxy identity export --pem` writes the public key cosign and other DSSE verifiers check it with.
134 +
135 +
- **Repositories kept in a bucket answer quickly (oxygenhub#228, #234).** A forge serving a repository from GCS or S3 opened a new client for every request, read large packs by range without keeping what it read, asked the bucket for a loose copy of every packed object it read, and asked again and again about objects it lacked. Clients are now shared per bucket and credentials, ranges and sizes are kept on disk, packs are looked in first, and a key found missing is taken as missing for 30 seconds unless this process writes it. On a hosted repository in GCS, negotiate went from 7-10 s to well under a second and bucket requests from about 540 a minute to about 30. The store counts its requests by kind, for a server to show.
136 +
137 +
- **A quicker log and an instant clean diff (#94).** The workspace's state index keeps its changes newest first, and the log takes them as a list, with no map of every change and no sort. A diff whose working copy changes nothing is empty without loading the base. On VS Code at 20 000 commits, `log -n 20` went from 34 ms to 30 ms and a clean `diff` from 48 ms to 30 ms.
138 +
139 +
- **A credential helper that does not read its input works (#115).** An `erase` that exits without reading what oxy sends it failed now and then with Broken pipe; a closed pipe is no error now, as in Git.
140 +
141 +
- **gc leaves a fully packed store as it is.** gc repacks at its harder compression level, and a store that was already one such pack with nothing loose was rewritten every time, the same pack again: 37 seconds for this repository's 48 000 objects on a forge, nightly, under the repository's write lock. Such a store is now left alone.
142 +
143 +
- **A switch names what it takes off disk (#102).** `oxy bookmark switch` and `oxy checkout` printed only how many files they removed, easy to miss when a switch to a bookmark lacking 80 files took an editor's project file with them. They now name the first few removed files, with a warning when twenty or more go.
144 +
145 +
- **Binary files' current versions found without whole closures (#107).** Which replacement of a binary file is live was decided by building each replacing change's entire closure; on a history as long as VS Code's (166,000 commits) each is a hundred thousand changes, the memo could not keep them, and `oxy git verify` spent its day rebuilding them. A bounded walk that keeps nothing answers the same.
146 +
147 +
- **`--carry` takes work the target already has (#100).** Switching bookmarks or checking out with `--carry` refused when uncommitted files were byte-for-byte what the target state holds, as after a proposal of that work landed, leaving only `--discard`. Such work is now already applied: no conflict, nothing to carry, and the rest of the work comes along.
148 +
149 +
- **Full hashes as `--json` prints them (#103).** A change named by its full 64-digit hash without the `#`, as `oxy bookmark show --json` and the hub's API print heads, resolves wherever a change is named (`bookmark set`, `bookmark create`, `checkout`, …), so JSON output feeds back into commands.
150 +
151 +
- **Large new files are not snapshotted unseen, and past snapshots expire (#98).** A 1 GB image left untracked in a working tree was taken in by the next `status` and kept by the operation log for good: deleting the file and running `gc --now` freed nothing. Now a new, untracked file over `snapshot.max-new-file-size` (default 100 MiB) is left out and named by `status`, with how to ignore it or raise the limit; tracked files and files already in the working copy are unaffected. Past working-copy snapshots keep their files only for `core.snapshot-retention` (default 7 days), so `gc` prunes what only they held, and `oxy gc --expire-snapshots --now` does it at once. `undo` and `op restore` to an operation whose working copy has expired keep the files as they are.
152 +
153 +
- **`oxy undo` undoes this workspace's last command (#99).** It undid the repository's newest operation, whichever workspace made it: in a repository with several workspaces an undo could take back another workspace's commit or merge, and it said only which operation it went back to. Operations now record their workspace, and undo takes back this workspace's last command, leaving other workspaces' work as it is. On a bookmark both share, only its own heads go; a change another workspace built on it, or the workspace it would take away, refuses the undo and names it. Undo now says which command it undid and what that changed, passes over pushes (the remote keeps what it was sent), and warns when an undone commit is already on a remote. `oxy amend` warns when it rewrites a change a remote already has, so the next push will need `--force`.
154 +
155 +
- **gc can keep what a host names (oxygenhub#126).** `Repo::gc_keeping` keeps extra blobs and their chunks beside what history reaches, so a forge's job artifacts, referenced only from its own database, survive garbage collection.
156 +
157 +
- **`oxy remote remove` removes all of a remote (#86).** It blanked the URL alone: the token, CA and store settings stayed in the config and the remote's tracking bookmarks in the view, read later as if current. Now every `remote.NAME.*` key goes, with its tracking bookmarks and a store remote's mirror; an unknown name is refused.
158 +
159 +
## 0.4.3 (2026-10-10)
160 +
161 +
- **Claim your own commits in a Git import (#97).** `oxy git import --claim` and `oxy git fetch --claim` set `git.claim`: a commit whose Git author email is your identity's email is authored with your key (and sealed by you on push), so a forge credits it to your account; anyone else's commits keep the zero key, as before.
162 +
163 +
- **Git imports of long histories much faster (#96).** Rendering a file asks whether the changes that deleted and inserted neighbouring lines depend on each other; in a deep history each answer was a walk through thousands of changes, asked again for every line, under a lock every rendering thread queued on. Answers are now remembered for the file being rendered, the shared memos are read without excluding other threads, and a merge renders its files on every core. On VS Code past 66 000 commits, 500 commits imported in 261 s rather than 734 s, with the same history.
164 +
165 +
- **A long push goes in parts (#95).** `oxy push` sent a whole history in one request, held in memory on both sides; the full VS Code import would have passed a hub's 4 GiB limit. It now sends parts of about `push.batch-bytes` (default 256 MiB) in dependency order, objects alone, and the last part moves the bookmark; the spinner says which part is going. A server stores a part and reports it as `objects`.
166 +
167 +
- **A state index read only as far as it is used (#94).** Each file's history in the workspace's state index is stored after the rest with a hash of its own, and is read and checked only when a command builds that file, so a diff or a one-file cat no longer reads and hashes the histories of every file (67 MB of an 84 MB index on VS Code at 20 000 commits). Changes added on top of an index no longer decode the histories they touch. A history found damaged is an error that says so, and the index is removed and rebuilt by the next command. Under load on that repository, `diff` went from 85 ms to 63 ms and `cat`/`blame` from 102 ms to 80 ms.
168 +
169 +
- **Removing a file removes every copy of it (#88).** When two lines of history each put a file back in the same place, it has two placements there and shows as one file. Removing or moving it retired only one, so the other stayed behind, and once its directory went it was an orphan: a conflict in the state. It is the one difference the Git import of VS Code had in its first 31 000 commits. Both now go together, in a commit and in a Git import.
170 +
171 +
- **A quicker log and diff on long histories (#94, first part).** `oxy log` no longer reads the working copy's metadata through the store's cache, walks the history with a fast hash and without copying each change's metadata, and works out the shortest unique id prefixes for the changes it shows rather than for every change. The state index is read without copying the file histories in it, and its change list is checked on every core. On VS Code at 20 000 commits, `log -n 20` went from 74 ms to 40 ms and loading the state index from 43 ms to 28 ms, with the same output.
172 +
173 +
- **Faster commits, and the state index after a checkpoint (#93).** A state loaded through a checkpoint read the changes after it whole, text included, so it could never be saved as the workspace's state index and every command after a checkpoint loaded history the long way (25 s for the first commit on VS Code at 20 000 commits). It now reads them as the index needs them. `oxy commit PATHS` no longer takes a second snapshot to count what it left when it took everything. A one-file commit on VS Code went from about 450 ms to 300 ms; 70 ms of that is the store's fsync, which stays.
174 +
175 +
- **Reading one file reads one file (#92).** `oxy cat` rendered every file of the state to print one. It now builds only that file, from the workspace's state index when it reads the working copy's base or the state it is on. On VS Code at 20 000 commits, `cat package.json -r main` went from 26 s to 0.10 s and at a change from 2016 from 0.66 s to 0.16 s; on the full 166 000-change history it takes 1.9 s and 1.3 GB, where it ran for minutes and used 9 GB.
176 +
177 +
- **A file's history in a fraction of a second (#91).** `oxy log --path` rendered every file of the tree twice just to find the path, and both it and the `touches()` revset decoded every change of the history, its text included, to see what each one touched. They now read the path from the tree alone and keep the nodes each change touches in a cache beside the store (`.oxy/index/touched`, rebuilt if missing). On VS Code at 20 000 commits, `log --path package.json` went from 53.6 s to 0.16 s (Git: 0.15 s) and `touches("package.json")` from 3.4 s to 0.16 s, with the same output.
178 +
179 +
## 0.4.2 (2026-10-09)
180 +
181 +
- **`pull` brings a working copy that fell behind up to its bookmark (#89).** The working copy followed a pull only from exactly the bookmark's old heads, and not at all when the pull brought nothing new, so one left behind by `bookmark set`, a landing or a pull from another workspace stayed on old code while `status` said clean. It now follows whenever its base is part of the bookmark's state, clean or carrying uncommitted work, and `status` says when the working copy is behind its bookmark and by how many changes.
182 +
183 +
- **Faster merges in a Git import (#80).** A merge is built from only the paths it can change, as an ordinary commit is: on VS Code at 139 000 commits merges import 3.2 times faster (9.5 s against 30.9 s on average) and the import as a whole 2.7 times faster, with exactly the history the whole-tree path builds. `OXY_IMPORT_WHOLE_MERGES=1` takes the old path.
184 +
185 +
- **A binary file changed and made executable at once keeps both (#87).** `oxy commit` and the Git import dropped the executable bit when a binary file got new contents in the same change.
186 +
187 +
- **Several remotes and no origin (#83).** `core.remote` names the default remote; `bookmark list` compares with a remote that has the bookmark and names it; the error without a default says what to do.
188 +
189 +
- **No i386 notice from apt (#85).** The .deb's apt source names the machine's own architecture, and an upgrade adds it to a source written before, so a system with i386 enabled no longer warns on every update.
190 +
191 +
- **Clearer messages.** A missing author identity says which config set it and how to pin the right one (#82); `oxy git verify --max-differences N` and a summary of what differs by kind.
192 +
193 +
## 0.4.1 (2026-10-08)
194 +
195 +
- **The prompt agrees with status in a repository with subrepos (#75).** Straight after a commit or a merge, `oxy prompt` took a subrepo's directory for a recently changed file and showed `*` until a full `oxy status` ran. Subrepos are now compared by their pins.
196 +
197 +
- **Catching up a stale bookmark (#74).** `oxy bookmark switch` brings a bookmark that is only behind its remote up to it first, rather than checking out the stale state. `oxy pull` takes uncommitted work along when the pulled changes leave its files alone, instead of leaving the working copy behind, and `checkout --carry` no longer counts a subrepo already at the target's pin as a change.
198 +
199 +
- **`bookmark set` fast-forwards (#78).** Moving a bookmark onto work built on its newest head keeps the older changes it held that the new one doesn't depend on, as `--upto` does, and says so. A head the bookmark already holds is still refused without `--upto` or `--only`.
200 +
201 +
- **A bookmark's heads by name (#73).** `oxy bookmark show [NAME]` lists each head, newest first; `bookmark list -v` does it for every bookmark, and `bookmark list` names the newest head's subject. `bookmark(NAME)` in a revset is a local or remote bookmark's heads and nothing else.
202 +
203 +
- **Clearer messages.** `submodule status` and `update` name `.oxymodules` entries that were never added, with the command that pins them, and `update` exits 1 (#72). `oxy cat -r CHANGE`'s help and error say that a change means its dependency closure, and suggest `-r BOOKMARK` or `--at-op` (#76).
204 +
205 +
- **A long Git import can stop and carry on (#77, #79).** An import moves its bookmark to the commits done so far every 2 000 commits, so `oxy git fetch` resumes it. `oxy git recover` rescues an import that stopped before its first checkpoint, or ran on an older oxy: it indexes the objects already written (each checked, a torn last one cut off) and points the bookmark at them. It recovered 139 086 commits of VS Code in under two minutes. `oxy git fetch ../path` with a relative path works.
206 +
207 +
- **`oxy git verify` in bounded memory (#81).** It held the ancestry of every commit and ran out of memory on VS Code; it now holds only the commits whose children are still to come, and checks 120 000 commits in under 1 GB. On an import not finished yet it stops at the last imported commit, counts `--last` from there, and says how many are still to come.
208 +
209 +
- **Measuring an import (#80).** `OXY_IMPORT_STOP_AFTER=N` ends an import cleanly after N commits with its timings; the trace gives each commit's time and which path each merge took. Merges built from only the paths they can change are in, behind `OXY_IMPORT_PARTIAL_MERGES=1` until verified at depth: on VS Code a small merge goes from about 18 s to under a second. An import stopping early no longer dies of SIGPIPE.
210 +
211 +
- **Cloud stores take credentials handed over (#68).** `oxy_strata::Credentials` (an S3 access key, a Google service account key, an Azure service principal) can be given when a store is opened, and a program can register a lookup for the stores it opens by URL. With them, nothing from the environment or the providers' files is used, which lets a hub keep each repository's credentials itself. Their debug form never shows a secret.
212 +
213 +
- **A repository kept on a store (#68).** `oxy init --store URL` keeps a repository's objects and operation log on `dir://`, S3, GCS or Azure, behind a local cache of `storage.cache` bytes (2 GiB by default) that can be deleted at any time. Checked against the real services: a commit takes 0.5 to 0.9 seconds, and reopening with no cache about half a second.
214 +
215 +
- **A caching store, and the operation head on a store (#68).** `strata::CachingStore` puts a local directory in front of any store: reads are kept, writes go to the remote first, the cache stays under a size limit by removing what was used least recently, and references always come from the remote. A repository kept on a store other than this disk's directory now holds its operation head there, moved by compare-and-swap with merge and retry.
216 +
217 +
- **A large merge's file no longer imports empty (#71).** When a merge rendered its files in batches, a file whose graph the import had already built came out empty, so the merge recorded the whole file as inserted again and conflicted with itself: VS Code's history failed verification from 2016 on. Built graphs are now copied. The 10 412-commit reproducer imports and verifies every commit.
218 +
219 +
- **Git import holds a budget of graph memory (#25).** A long-lived file's graph keeps every line it ever had, and the import shed a kept state's graphs only past 2 000 of them; it now also sheds them past 1.5 million atoms, and gives freed memory back after each merge. A 28 637-commit slice of VS Code imports at 5.1 GB instead of 16.9 GB, in 36 minutes instead of 61, with the same verify result. The dependency check no longer memoises every queried change's whole ancestry.
4 220
5 221
- **A rotation follows you everywhere (#69).** `delegate add`, `delegate remove`, `rotate` and `renew` record the successor key in the user config when it named the old key, and in the repository's own file when that pinned it, so every checkout moves together. `oxy identity show` warns when a repository pins a key other than the user config's; `oxy config --unset KEY` drops a key; `oxy identity delegate add` takes several identity files in one rotation and reminds you that agents configured with your `author.key` need the new one.
6 222
…
103 unchanged lines
modified Cargo.lock +19 −4
…
571 unchanged lines
572 572
]
573 573
574 574
[[package]]
575 +
name = "cylinder"
576 +
version = "0.11.0"
577 +
dependencies = [
578 +
 "blake3",
579 +
 "oxy-core",
580 +
 "tempfile",
581 +
 "thiserror 1.0.69",
582 +
]
583 +
584 +
[[package]]
575 585
name = "data-encoding"
576 586
version = "2.11.1"
577 587
source = "registry+https://github.com/rust-lang/crates.io-index"
…
45 unchanged lines
623 633
checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
624 634
dependencies = [
625 635
 "block-buffer",
636 +
 "const-oid",
626 637
 "crypto-common 0.2.2",
627 638
]
628 639
…
859 unchanged lines
1488 1499
1489 1500
[[package]]
1490 1501
name = "oxy"
1491 -
version = "0.4.0"
1502 +
version = "0.11.0"
1492 1503
dependencies = [
1493 1504
 "anyhow",
1494 1505
 "clap",
1495 1506
 "clap_complete",
1496 1507
 "clap_mangen",
1497 1508
 "console",
1509 +
 "cylinder",
1498 1510
 "indicatif",
1499 1511
 "libc",
1500 1512
 "mimalloc",
…
6 unchanged lines
1507 1519
1508 1520
[[package]]
1509 1521
name = "oxy-core"
1510 -
version = "0.4.0"
1522 +
version = "0.11.0"
1511 1523
dependencies = [
1512 1524
 "blake3",
1513 1525
 "ed25519-dalek",
…
8 unchanged lines
1522 1534
 "rayon",
1523 1535
 "rustls",
1524 1536
 "rustls-pemfile",
1537 +
 "serde_json",
1538 +
 "sha2",
1525 1539
 "similar",
1526 1540
 "tempfile",
1527 1541
 "thiserror 1.0.69",
…
4 unchanged lines
1532 1546
1533 1547
[[package]]
1534 1548
name = "oxy-server"
1535 -
version = "0.4.0"
1549 +
version = "0.11.0"
1536 1550
dependencies = [
1537 1551
 "axum",
1538 1552
 "axum-server",
…
2 unchanged lines
1541 1555
 "rustls",
1542 1556
 "tempfile",
1543 1557
 "tokio",
1558 +
 "tower",
1544 1559
]
1545 1560
1546 1561
[[package]]
1547 1562
name = "oxy-strata"
1548 -
version = "0.4.0"
1563 +
version = "0.11.0"
1549 1564
dependencies = [
1550 1565
 "async-trait",
1551 1566
 "bytes",
…
1419 unchanged lines
modified Cargo.toml +3 −2
1 1
[workspace]
2 -
members = ["crates/oxy-core", "crates/oxy-server", "crates/oxy-strata", "."]
2 +
members = ["crates/oxy-core", "crates/oxy-server", "crates/oxy-strata", "crates/cylinder", "."]
3 3
resolver = "2"
4 4
5 5
[workspace.package]
6 -
version = "0.4.0"
6 +
version = "0.11.0"
7 7
edition = "2021"
8 8
license = "MIT OR Apache-2.0"
9 9
…
13 unchanged lines
23 23
oxy-core = { path = "crates/oxy-core" }
24 24
oxy-server = { path = "crates/oxy-server" }
25 25
oxy-strata = { path = "crates/oxy-strata" }
26 +
cylinder = { path = "crates/cylinder" }
26 27
clap = { version = "4", features = ["derive"] }
27 28
anyhow = "1"
28 29
libc = "0.2"
…
19 unchanged lines
modified README.md +10 −2
1 1
# Oxygen
2 2
3 -
[![OxygenFlow](https://oxygenhub.io/andymccall/oxygen/flow/badge.svg)](https://oxygenhub.io/andymccall/oxygen/flow)
3 +
[![OxygenFlow](https://oxygenhub.io/andymccall/oxygen/flow/badge.svg)](https://oxygenhub.io/andymccall/oxygen/flow) [![coverage](https://oxygenhub.io/andymccall/oxygen/coverage/badge.svg)](https://oxygenhub.io/andymccall/oxygen/coverage)
4 4
5 5
Oxygen (`oxy`) is a distributed, commutative version control system written in Rust for large repositories. See [docs/spec.md](docs/spec.md) for the specification, [docs/guide.md](docs/guide.md) for a user guide by task, and [docs/protocol.md](docs/protocol.md) if you are implementing a client or server.
6 6
…
9 unchanged lines
16 16
17 17
Every change the agent makes is attributed to you and signed by the agent. `oxy log`, `oxy verify` and the hub therefore show who did the work, such as "Andy McCall · via Claude Code", and a policy can require that a person has approved it before it lands.
18 18
19 +
An agent can also work through `oxy mcp`, an MCP server for the repository: typed tools instead of a shell, signing as the identity it was started with, and nothing destructive unless you allow it (see [docs/mcp.md](docs/mcp.md)).
20 +
19 21
See [docs/ai_native_agent_workflows.md](docs/ai_native_agent_workflows.md) for the design and [docs/guide.md](docs/guide.md) §9 for the setup.
20 22
21 23
## What is different
…
13 unchanged lines
35 37
36 38
## Install
37 39
38 -
Builds for Linux, macOS and Windows are on the [releases page](https://oxygenhub.io/andymccall/oxygen/releases). These are early releases: the macOS and Windows builds are not signed and have had little testing.
40 +
Builds for Linux, macOS, Windows and Haiku are on the [releases page](https://oxygenhub.io/andymccall/oxygen/releases). These are early releases: the macOS and Windows builds are not signed and have had little testing.
39 41
40 42
- **Debian, Ubuntu and other Linux with apt** (x86_64, arm64 or riscv64): add the repository at https://oxygenhub.io/apt and install with apt, so `apt upgrade` keeps oxy current:
41 43
…
13 unchanged lines
55 57
- **Other Linux:** unpack `oxy-VERSION-linux-x86_64.tar.gz` (or `-linux-arm64`) and put `oxy` on your `PATH`.
56 58
- **macOS** (Apple Silicon or Intel): unpack `oxy-VERSION-macos-arm64.tar.gz` (or `-macos-x86_64`), then `xattr -d com.apple.quarantine oxy`, since the build is not notarised, and put it on your `PATH`.
57 59
- **Windows:** unzip `oxy-VERSION-windows-x86_64.zip` and put the folder on your `PATH`. SmartScreen may warn on first run, since the build is not signed.
60 +
- **Haiku** (R1/beta6, x86_64): unpack `oxy-VERSION-haiku-x86_64.tar.gz` and copy `oxy` to `~/config/non-packaged/bin`. It is built in a Haiku guest rather than by OxygenFlow, so it has its own `SHA256SUMS-haiku` and no build attestation.
58 61
59 62
`oh`, the OxygenHub command line, is released the same way from [OxygenHub's releases](https://oxygenhub.io/andymccall/oxygenhub/releases).
60 63
…
8 unchanged lines
69 72
70 73
The binary is `target/release/oxy`.
71 74
75 +
On Haiku, install `rust_bin` and `zstd_devel` with pkgman and build with `ZSTD_SYS_USE_PKG_CONFIG=1 cargo build --release`: zstd-sys then links Haiku's libzstd, since a system library also references zstd and ld refuses the bundled copy's hidden symbols.
76 +
77 +
**Changelog:** a commit that changes code adds a line under `## Unreleased` in CHANGELOG.md (Added, Changed, Fixed or Security, with the issue number); versions change only at release. `scripts/changelog-check.sh [PATH...]` checks it for the paths a commit takes (the working copy's changes without any) and runs in the gate; `NO_CHANGELOG=1` for code changes users won't notice.
78 +
72 79
## Quick start
73 80
74 81
```
…
75 unchanged lines
150 157
| `docs/strata.md` | Keeping a repository on your own storage: S3, GCS, Azure, a directory |
151 158
| `docs/protocol.md` | Wire protocol for other implementers |
152 159
| `docs/prompt.md` | Shell prompt integration |
160 +
| `docs/mcp.md` | `oxy mcp`: the repository as tools for AI agents |
153 161
154 162
## Licence
155 163
…
1 unchanged lines
modified build.rs +5 −1
…
2 unchanged lines
3 3
//! building, else "local build".
4 4
fn main() {
5 5
    println!("cargo:rerun-if-env-changed=OXY_BUILD");
6 -
    println!("cargo:rerun-if-env-changed=FLOW_HEADS");
6 +
    // The heads name the build only when OXY_BUILD does not: a CI job that
7 +
    // sets it stays fresh from one state to the next (oxygenflow#52).
8 +
    if !std::env::var("OXY_BUILD").is_ok_and(|b| !b.trim().is_empty()) {
9 +
        println!("cargo:rerun-if-env-changed=FLOW_HEADS");
10 +
    }
7 11
    let build = std::env::var("OXY_BUILD")
8 12
        .ok()
9 13
        .map(|b| b.trim().to_string())
…
18 unchanged lines
added crates/cylinder/Cargo.toml +14 −0
1 +
[package]
2 +
name = "cylinder"
3 +
version.workspace = true
4 +
edition.workspace = true
5 +
license.workspace = true
6 +
description = "Cylinder: artifacts as trees of Oxygen chunks (docs/cylinder.md)"
7 +
8 +
[dependencies]
9 +
oxy-core = { path = "../oxy-core" }
10 +
blake3 = "1"
11 +
thiserror = "1"
12 +
13 +
[dev-dependencies]
14 +
tempfile = "3"
added crates/cylinder/src/lib.rs +454 −0
1 +
//! Cylinder, step C1 (docs/cylinder.md §4.1): an **artifact** is an
2 +
//! immutable, content-addressed object naming a tree of files, each kept as
3 +
//! an ordinary Oxygen blob of FastCDC BLAKE3 chunks. An image rebuilt with
4 +
//! one changed file moves only that file's changed chunks, and a binary in
5 +
//! a release archive and in a runner image is the same chunks.
6 +
//!
7 +
//! The model is oxy-core's ([`oxy_core::artifact`]): an `artifact` object
8 +
//! in canonical CBOR, named by its object hash, whose tree is a blob. This
9 +
//! crate packs a directory into one, opens, verifies and unpacks it, and
10 +
//! keeps it: `.oxy/artifacts/NAME` records each artifact this repository
11 +
//! keeps, which garbage collection keeps with its tree and files
12 +
//! ([`roots`]).
13 +
14 +
use oxy_core::hash::{ObjectRef, ObjectType};
15 +
use oxy_core::repo::Repo;
16 +
use std::collections::BTreeMap;
17 +
use std::path::{Path, PathBuf};
18 +
19 +
pub use oxy_core::artifact::{
20 +
    Artifact, DirTree, Entry, Kind, MODE_EXEC, MODE_FILE, MODE_LINK, VERSION,
21 +
};
22 +
23 +
#[derive(Debug, thiserror::Error)]
24 +
pub enum Error {
25 +
    #[error(transparent)]
26 +
    Repo(#[from] oxy_core::repo::RepoError),
27 +
    #[error("io: {0}")]
28 +
    Io(#[from] std::io::Error),
29 +
    #[error("not an artifact: {0}")]
30 +
    Object(#[from] oxy_core::objects::ObjectError),
31 +
    #[error("store: {0}")]
32 +
    Store(#[from] oxy_core::store::StoreError),
33 +
    #[error("{0}")]
34 +
    Bad(String),
35 +
}
36 +
37 +
pub type Result<T> = std::result::Result<T, Error>;
38 +
39 +
fn bad<T>(msg: impl Into<String>) -> Result<T> {
40 +
    Err(Error::Bad(msg.into()))
41 +
}
42 +
43 +
/// What to pack a directory as.
44 +
#[derive(Clone, Debug)]
45 +
pub struct Spec {
46 +
    pub kind: Kind,
47 +
    pub arch: Option<String>,
48 +
    pub os: Option<String>,
49 +
    pub entrypoint: Option<Vec<String>>,
50 +
    pub env: BTreeMap<String, String>,
51 +
    pub labels: BTreeMap<String, String>,
52 +
}
53 +
54 +
impl Spec {
55 +
    pub fn new(kind: Kind) -> Spec {
56 +
        Spec {
57 +
            kind,
58 +
            arch: None,
59 +
            os: None,
60 +
            entrypoint: None,
61 +
            env: BTreeMap::new(),
62 +
            labels: BTreeMap::new(),
63 +
        }
64 +
    }
65 +
}
66 +
67 +
/// An artifact kept in a repository, with its name.
68 +
#[derive(Clone, Debug)]
69 +
pub struct Kept {
70 +
    pub name: String,
71 +
    pub artifact: Artifact,
72 +
    /// Its object (its name, as a reference).
73 +
    pub object: ObjectRef,
74 +
}
75 +
76 +
fn kept_dir(repo: &Repo) -> PathBuf {
77 +
    repo.oxy_dir().join("artifacts")
78 +
}
79 +
80 +
/// Every file under `dir`, by relative path: regular files and links, in
81 +
/// path order. `.oxy` and `.git` directories are not part of an artifact.
82 +
fn walk(dir: &Path) -> Result<Vec<(String, PathBuf, std::fs::Metadata)>> {
83 +
    let mut out = Vec::new();
84 +
    let mut stack = vec![(dir.to_path_buf(), String::new())];
85 +
    while let Some((d, rel)) = stack.pop() {
86 +
        for e in std::fs::read_dir(&d)? {
87 +
            let e = e?;
88 +
            let name = e
89 +
                .file_name()
90 +
                .into_string()
91 +
                .map_err(|n| Error::Bad(format!("{n:?} is not UTF-8")))?;
92 +
            let path = if rel.is_empty() {
93 +
                name.clone()
94 +
            } else {
95 +
                format!("{rel}/{name}")
96 +
            };
97 +
            let md = std::fs::symlink_metadata(e.path())?;
98 +
            if md.is_dir() {
99 +
                if name != ".oxy" && name != ".git" {
100 +
                    stack.push((e.path(), path));
101 +
                }
102 +
            } else if md.is_file() || md.file_type().is_symlink() {
103 +
                out.push((path, e.path(), md));
104 +
            }
105 +
        }
106 +
    }
107 +
    out.sort_by(|a, b| a.0.cmp(&b.0));
108 +
    Ok(out)
109 +
}
110 +
111 +
#[cfg(unix)]
112 +
fn mode_of(md: &std::fs::Metadata) -> u32 {
113 +
    use std::os::unix::fs::PermissionsExt;
114 +
    if md.file_type().is_symlink() {
115 +
        MODE_LINK
116 +
    } else if md.permissions().mode() & 0o111 != 0 {
117 +
        MODE_EXEC
118 +
    } else {
119 +
        MODE_FILE
120 +
    }
121 +
}
122 +
123 +
#[cfg(not(unix))]
124 +
fn mode_of(md: &std::fs::Metadata) -> u32 {
125 +
    if md.file_type().is_symlink() {
126 +
        MODE_LINK
127 +
    } else {
128 +
        MODE_FILE
129 +
    }
130 +
}
131 +
132 +
/// Pack the files under `dir` into an artifact of `spec`'s kind in
133 +
/// `repo`'s store: each file a blob, so files and chunks the store holds
134 +
/// already cost nothing; then its tree and its encoding, kept under its
135 +
/// name.
136 +
pub fn pack(repo: &Repo, dir: &Path, spec: &Spec) -> Result<Kept> {
137 +
    if !dir.is_dir() {
138 +
        return bad(format!("{} is not a directory", dir.display()));
139 +
    }
140 +
    let mut tree = DirTree::default();
141 +
    for (path, full, md) in walk(dir)? {
142 +
        let mode = mode_of(&md);
143 +
        let bytes = if mode == MODE_LINK {
144 +
            std::fs::read_link(&full)?
145 +
                .to_string_lossy()
146 +
                .into_owned()
147 +
                .into_bytes()
148 +
        } else {
149 +
            std::fs::read(&full)?
150 +
        };
151 +
        let blob = repo.store_blob(&bytes)?;
152 +
        tree.entries.push(Entry {
153 +
            path,
154 +
            mode,
155 +
            blob,
156 +
            size: bytes.len() as u64,
157 +
        });
158 +
    }
159 +
    let artifact = Artifact {
160 +
        kind: spec.kind,
161 +
        arch: spec.arch.clone(),
162 +
        os: spec.os.clone(),
163 +
        tree: repo.store_blob(&tree.encode())?,
164 +
        entrypoint: spec.entrypoint.clone(),
165 +
        env: spec.env.clone(),
166 +
        labels: spec.labels.clone(),
167 +
        total_size: tree.entries.iter().map(|e| e.size).sum(),
168 +
        receipt: None,
169 +
    };
170 +
    let object = repo.store().put(ObjectType::Artifact, &artifact.encode())?;
171 +
    let name = object.to_hex();
172 +
    std::fs::create_dir_all(kept_dir(repo))?;
173 +
    std::fs::write(kept_dir(repo).join(&name), b"")?;
174 +
    Ok(Kept {
175 +
        name,
176 +
        artifact,
177 +
        object,
178 +
    })
179 +
}
180 +
181 +
/// The names of the artifacts kept in `repo`, sorted.
182 +
pub fn list(repo: &Repo) -> Result<Vec<String>> {
183 +
    let mut out = Vec::new();
184 +
    match std::fs::read_dir(kept_dir(repo)) {
185 +
        Ok(rd) => {
186 +
            for e in rd {
187 +
                let n = e?.file_name().to_string_lossy().into_owned();
188 +
                if n.len() == 64 && n.bytes().all(|b| b.is_ascii_hexdigit()) {
189 +
                    out.push(n);
190 +
                }
191 +
            }
192 +
        }
193 +
        Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
194 +
        Err(e) => return Err(e.into()),
195 +
    }
196 +
    out.sort();
197 +
    Ok(out)
198 +
}
199 +
200 +
/// The kept artifact `name` names, or a unique prefix of it (at least six
201 +
/// characters), read from the store, which checks it against its name.
202 +
pub fn open(repo: &Repo, name: &str) -> Result<Kept> {
203 +
    let name = name.to_ascii_lowercase();
204 +
    if name.len() < 6 {
205 +
        return bad("name an artifact by at least six characters of its name");
206 +
    }
207 +
    let found: Vec<String> = list(repo)?
208 +
        .into_iter()
209 +
        .filter(|n| n.starts_with(&name))
210 +
        .collect();
211 +
    let full = match found.as_slice() {
212 +
        [one] => one.clone(),
213 +
        [] => {
214 +
            return bad(format!(
215 +
                "no artifact {name} here; `oxy artifact list` shows them"
216 +
            ))
217 +
        }
218 +
        _ => {
219 +
            return bad(format!(
220 +
                "{name} names {} artifacts: give more of the name",
221 +
                found.len()
222 +
            ))
223 +
        }
224 +
    };
225 +
    let object = ObjectRef::from_hex(&full).map_or_else(
226 +
        || bad(format!("artifact {full}: its record is damaged")),
227 +
        Ok,
228 +
    )?;
229 +
    // The store checks the bytes against the object's name as it reads.
230 +
    let bytes = repo.store().get_typed(&object, ObjectType::Artifact)?;
231 +
    Ok(Kept {
232 +
        name: full,
233 +
        artifact: Artifact::decode(&bytes)?,
234 +
        object,
235 +
    })
236 +
}
237 +
238 +
/// An artifact's tree.
239 +
pub fn tree(repo: &Repo, a: &Artifact) -> Result<DirTree> {
240 +
    Ok(DirTree::decode(&repo.read_blob(&a.tree)?)?)
241 +
}
242 +
243 +
/// Check that every file of `k` is in the store whole: its bytes read, at
244 +
/// the size its tree says. Answers how many files were read.
245 +
pub fn verify(repo: &Repo, k: &Kept) -> Result<usize> {
246 +
    let t = tree(repo, &k.artifact)?;
247 +
    let mut total = 0;
248 +
    for e in &t.entries {
249 +
        let n = repo.read_blob(&e.blob)?.len() as u64;
250 +
        if n != e.size {
251 +
            return bad(format!(
252 +
                "{}: {n} bytes, but the tree says {}",
253 +
                e.path, e.size
254 +
            ));
255 +
        }
256 +
        total += n;
257 +
    }
258 +
    if total != k.artifact.total_size {
259 +
        return bad(format!(
260 +
            "the files are {total} bytes together, but the artifact says {}",
261 +
            k.artifact.total_size
262 +
        ));
263 +
    }
264 +
    Ok(t.entries.len())
265 +
}
266 +
267 +
/// Write `k`'s files under `dest`, which must not exist or be empty.
268 +
pub fn unpack(repo: &Repo, k: &Kept, dest: &Path) -> Result<usize> {
269 +
    if dest.exists() && std::fs::read_dir(dest)?.next().is_some() {
270 +
        return bad(format!("{} is not empty", dest.display()));
271 +
    }
272 +
    let t = tree(repo, &k.artifact)?;
273 +
    for e in &t.entries {
274 +
        let out = dest.join(&e.path);
275 +
        if let Some(p) = out.parent() {
276 +
            std::fs::create_dir_all(p)?;
277 +
        }
278 +
        let bytes = repo.read_blob(&e.blob)?;
279 +
        if e.mode == MODE_LINK {
280 +
            #[cfg(unix)]
281 +
            std::os::unix::fs::symlink(String::from_utf8_lossy(&bytes).as_ref(), &out)?;
282 +
            #[cfg(not(unix))]
283 +
            std::fs::write(&out, &bytes)?;
284 +
            continue;
285 +
        }
286 +
        std::fs::write(&out, &bytes)?;
287 +
        #[cfg(unix)]
288 +
        if e.mode == MODE_EXEC {
289 +
            use std::os::unix::fs::PermissionsExt;
290 +
            std::fs::set_permissions(&out, std::fs::Permissions::from_mode(0o755))?;
291 +
        }
292 +
    }
293 +
    Ok(t.entries.len())
294 +
}
295 +
296 +
/// The blobs garbage collection must keep for the kept artifacts: each
297 +
/// one's encoding, its tree and its files (`Repo::gc_keeping`).
298 +
pub fn roots(repo: &Repo) -> Result<Vec<ObjectRef>> {
299 +
    let mut out = Vec::new();
300 +
    for name in list(repo)? {
301 +
        let k = open(repo, &name)?;
302 +
        out.push(k.object);
303 +
        out.push(k.artifact.tree);
304 +
        out.extend(tree(repo, &k.artifact)?.entries.iter().map(|e| e.blob));
305 +
    }
306 +
    Ok(out)
307 +
}
308 +
309 +
#[cfg(test)]
310 +
mod tests {
311 +
    use super::*;
312 +
    use oxy_core::cbor::{self, Value};
313 +
314 +
    fn repo(dir: &Path) -> Repo {
315 +
        Repo::init(&dir.join("repo")).unwrap()
316 +
    }
317 +
318 +
    #[test]
319 +
    fn a_directory_packs_into_an_artifact_and_back() {
320 +
        let t = tempfile::tempdir().unwrap();
321 +
        let r = repo(t.path());
322 +
        let src = t.path().join("out");
323 +
        std::fs::create_dir_all(src.join("bin")).unwrap();
324 +
        std::fs::write(src.join("README"), b"hello\n").unwrap();
325 +
        std::fs::write(src.join("bin/tool"), vec![7u8; 300_000]).unwrap();
326 +
        #[cfg(unix)]
327 +
        {
328 +
            use std::os::unix::fs::PermissionsExt;
329 +
            std::fs::set_permissions(src.join("bin/tool"), std::fs::Permissions::from_mode(0o755))
330 +
                .unwrap();
331 +
            std::os::unix::fs::symlink("bin/tool", src.join("tool")).unwrap();
332 +
        }
333 +
        let mut spec = Spec::new(Kind::Archive);
334 +
        spec.arch = Some("x86_64".into());
335 +
        spec.labels.insert("version".into(), "0.1.0".into());
336 +
        let k = pack(&r, &src, &spec).unwrap();
337 +
        assert_eq!(k.name, k.artifact.name().to_hex());
338 +
        assert_eq!(
339 +
            k.artifact.total_size,
340 +
            300_006 + if cfg!(unix) { 8 } else { 0 }
341 +
        );
342 +
        assert_eq!(list(&r).unwrap(), vec![k.name.clone()]);
343 +
344 +
        let again = open(&r, &k.name[..10]).unwrap();
345 +
        assert_eq!(again.artifact, k.artifact);
346 +
        let paths: Vec<(String, u32)> = tree(&r, &again.artifact)
347 +
            .unwrap()
348 +
            .entries
349 +
            .into_iter()
350 +
            .map(|e| (e.path, e.mode))
351 +
            .collect();
352 +
        assert_eq!(paths[0], ("README".into(), MODE_FILE));
353 +
        assert_eq!(
354 +
            paths[1],
355 +
            (
356 +
                "bin/tool".into(),
357 +
                if cfg!(unix) { MODE_EXEC } else { MODE_FILE }
358 +
            )
359 +
        );
360 +
        assert_eq!(verify(&r, &again).unwrap(), paths.len());
361 +
362 +
        let dest = t.path().join("back");
363 +
        unpack(&r, &again, &dest).unwrap();
364 +
        assert_eq!(
365 +
            std::fs::read(dest.join("bin/tool")).unwrap(),
366 +
            vec![7u8; 300_000]
367 +
        );
368 +
        #[cfg(unix)]
369 +
        assert_eq!(
370 +
            std::fs::read_link(dest.join("tool")).unwrap(),
371 +
            Path::new("bin/tool")
372 +
        );
373 +
        assert!(unpack(&r, &again, &dest)
374 +
            .unwrap_err()
375 +
            .to_string()
376 +
            .contains("not empty"));
377 +
378 +
        // The same files pack to the same name; garbage collection keeps it.
379 +
        assert_eq!(pack(&r, &src, &spec).unwrap().name, k.name);
380 +
        let keep = roots(&r).unwrap();
381 +
        assert!(keep.contains(&k.object) && keep.contains(&k.artifact.tree));
382 +
        r.gc_keeping(true, &keep, &mut |_| {}).unwrap();
383 +
        // An artifact is an object of its own, which fsck reads and accepts.
384 +
        let report = r.fsck().unwrap();
385 +
        let errors: Vec<_> = report
386 +
            .problems
387 +
            .iter()
388 +
            .filter(|p| p.severity == oxy_core::repo::Severity::Error)
389 +
            .collect();
390 +
        assert!(errors.is_empty(), "{errors:?}");
391 +
        assert_eq!(
392 +
            verify(&r, &open(&r, &k.name).unwrap()).unwrap(),
393 +
            paths.len()
394 +
        );
395 +
    }
396 +
397 +
    #[test]
398 +
    fn encodings_are_canonical_and_checked() {
399 +
        let a = Artifact {
400 +
            kind: Kind::RunnerImage,
401 +
            arch: Some("riscv64".into()),
402 +
            os: Some("linux".into()),
403 +
            tree: ObjectRef([1; 32]),
404 +
            entrypoint: Some(vec!["/sbin/init".into()]),
405 +
            env: [("PATH".to_string(), "/bin".to_string())].into(),
406 +
            labels: BTreeMap::new(),
407 +
            total_size: 42,
408 +
            receipt: Some(ObjectRef([2; 32])),
409 +
        };
410 +
        let bytes = a.encode();
411 +
        assert_eq!(Artifact::decode(&bytes).unwrap(), a);
412 +
        assert_eq!(
413 +
            a.name(),
414 +
            oxy_core::hash::hash_object(ObjectType::Artifact, &bytes)
415 +
        );
416 +
        assert!(Kind::parse("tarball")
417 +
            .unwrap_err()
418 +
            .to_string()
419 +
            .contains("runner-image"));
420 +
421 +
        let wrong = |v: Value| Artifact::decode(&cbor::encode(&v)).unwrap_err().to_string();
422 +
        assert!(wrong(Value::map([(0, Value::Uint(2))])).contains("version"));
423 +
        assert!(wrong(Value::map([(0, Value::Uint(1)), (10, Value::Null)])).contains("key 10"));
424 +
425 +
        let t = DirTree {
426 +
            entries: vec![Entry {
427 +
                path: "a".into(),
428 +
                mode: MODE_FILE,
429 +
                blob: ObjectRef([3; 32]),
430 +
                size: 1,
431 +
            }],
432 +
        };
433 +
        assert_eq!(DirTree::decode(&t.encode()).unwrap(), t);
434 +
        let entry = |p: &str, mode: u32| {
435 +
            Value::Array(vec![
436 +
                Value::Text(p.into()),
437 +
                Value::Uint(mode as u64),
438 +
                Value::Bytes(vec![0; 32]),
439 +
                Value::Uint(0),
440 +
            ])
441 +
        };
442 +
        let tree = |items: Vec<Value>| {
443 +
            DirTree::decode(&cbor::encode(&Value::map([
444 +
                (0, Value::Uint(1)),
445 +
                (1, Value::Array(items)),
446 +
            ])))
447 +
            .unwrap_err()
448 +
            .to_string()
449 +
        };
450 +
        assert!(tree(vec![entry("../x", MODE_FILE)]).contains("clean"));
451 +
        assert!(tree(vec![entry("b", MODE_FILE), entry("a", MODE_FILE)]).contains("order"));
452 +
        assert!(tree(vec![entry("a", 0o040000)]).contains("mode"));
453 +
    }
454 +
}
modified crates/oxy-core/Cargo.toml +2 −0
…
5 unchanged lines
6 6
description = "Core data model, change graph, and storage for Oxygen"
7 7
8 8
[dependencies]
9 +
serde_json = "1"
10 +
sha2 = "0.11"
9 11
blake3 = { version = "1", features = ["rayon"] }
10 12
hex = "0.4"
11 13
rand = "0.8"
…
20 unchanged lines
modified crates/oxy-core/src/alloc_stats.rs +13 −0
…
76 unchanged lines
77 77
        count: COUNT.load(Relaxed),
78 78
    })
79 79
}
80 +
81 +
/// Give pages the program has freed back to the system. glibc keeps freed
82 +
/// memory in each thread's arena; after a large merge renders thousands of
83 +
/// files on many threads, that kept memory is most of the process's size
84 +
/// (#25). Elsewhere it does nothing.
85 +
pub fn release_freed_memory() {
86 +
    #[cfg(all(target_os = "linux", target_env = "gnu"))]
87 +
    // SAFETY: malloc_trim only returns free pages to the system; it is
88 +
    // safe to call at any time from any thread.
89 +
    unsafe {
90 +
        libc::malloc_trim(0);
91 +
    }
92 +
}
added crates/oxy-core/src/approval.rs +228 −0
1 +
//! A person's approval of one job of one run, signed (oxygenhub#235): the
2 +
//! approval attestation, the signer's signature over it, and the signer's
3 +
//! identity, so it verifies with nothing else. The hub decides whether the
4 +
//! signer may approve; the signature makes the decision theirs, and binds
5 +
//! it to exactly that job, state, plan and environment.
6 +
7 +
use crate::attest::{Attest, AttestBody};
8 +
use crate::cbor::{self, Value};
9 +
use crate::hash::ObjectRef;
10 +
use crate::identity::{verify, Domain, Identity, SecretKey, Signature};
11 +
use crate::objects::{array, bytes, Hlc, ObjectError};
12 +
13 +
/// What is approved.
14 +
#[derive(Clone, Debug, PartialEq, Eq)]
15 +
pub struct Statement {
16 +
    pub repo: [u8; 16],
17 +
    /// The run, as its URL on the hub.
18 +
    pub run: String,
19 +
    pub job: String,
20 +
    /// The job's hash with its sandbox (OxygenFlow spec 6.2).
21 +
    pub job_hash: ObjectRef,
22 +
    pub state: ObjectRef,
23 +
    /// The blobs the job takes as inputs (a plan), any order.
24 +
    pub inputs: Vec<ObjectRef>,
25 +
    pub environment: String,
26 +
    pub approve: bool,
27 +
}
28 +
29 +
#[derive(Debug, thiserror::Error)]
30 +
pub enum ApprovalError {
31 +
    #[error("the attestation is not an approval")]
32 +
    NotApproval,
33 +
    #[error("no signature by the signer {0}")]
34 +
    Unsigned(String),
35 +
    #[error("the signature does not verify: {0}")]
36 +
    BadSignature(String),
37 +
    #[error("{0}")]
38 +
    Schema(String),
39 +
}
40 +
41 +
impl From<ObjectError> for ApprovalError {
42 +
    fn from(e: ObjectError) -> Self {
43 +
        ApprovalError::Schema(e.to_string())
44 +
    }
45 +
}
46 +
47 +
#[derive(Clone, Debug, PartialEq, Eq)]
48 +
pub struct SignedApproval {
49 +
    pub attest: Attest,
50 +
    pub sigs: Vec<Signature>,
51 +
    pub signer: Identity,
52 +
}
53 +
54 +
impl SignedApproval {
55 +
    pub fn sign(s: &Statement, signer: &Identity, sk: &SecretKey) -> SignedApproval {
56 +
        let mut inputs = s.inputs.clone();
57 +
        inputs.sort();
58 +
        inputs.dedup();
59 +
        let attest = Attest {
60 +
            repo: s.repo,
61 +
            hlc: Hlc::now(),
62 +
            body: AttestBody::Approval {
63 +
                run: s.run.clone(),
64 +
                job: s.job.clone(),
65 +
                job_hash: s.job_hash,
66 +
                state: s.state,
67 +
                inputs,
68 +
                environment: s.environment.clone(),
69 +
                approve: s.approve,
70 +
            },
71 +
        };
72 +
        let sig = sk.sign(Domain::Attest, &signer.hash(), &attest.hash());
73 +
        SignedApproval {
74 +
            attest,
75 +
            sigs: vec![sig],
76 +
            signer: signer.clone(),
77 +
        }
78 +
    }
79 +
80 +
    /// What was approved, read back from the attestation.
81 +
    pub fn statement(&self) -> Result<Statement, ApprovalError> {
82 +
        let AttestBody::Approval {
83 +
            run,
84 +
            job,
85 +
            job_hash,
86 +
            state,
87 +
            inputs,
88 +
            environment,
89 +
            approve,
90 +
        } = &self.attest.body
91 +
        else {
92 +
            return Err(ApprovalError::NotApproval);
93 +
        };
94 +
        Ok(Statement {
95 +
            repo: self.attest.repo,
96 +
            run: run.clone(),
97 +
            job: job.clone(),
98 +
            job_hash: *job_hash,
99 +
            state: *state,
100 +
            inputs: inputs.clone(),
101 +
            environment: environment.clone(),
102 +
            approve: *approve,
103 +
        })
104 +
    }
105 +
106 +
    /// It is an approval and the signer signed it. Whether the signer may
107 +
    /// approve is the reader's call.
108 +
    pub fn verify(&self) -> Result<(), ApprovalError> {
109 +
        self.statement()?;
110 +
        let key = self.signer.hash();
111 +
        let sig = self
112 +
            .sigs
113 +
            .iter()
114 +
            .find(|s| s.key == key)
115 +
            .ok_or_else(|| ApprovalError::Unsigned(key.short(12)))?;
116 +
        verify(&self.signer, sig, Domain::Attest, &self.attest.hash())
117 +
            .map_err(|e| ApprovalError::BadSignature(e.to_string()))
118 +
    }
119 +
120 +
    /// Whether `s` is what this approves (the time it was made aside).
121 +
    pub fn covers(&self, s: &Statement) -> bool {
122 +
        let mut want = s.inputs.clone();
123 +
        want.sort();
124 +
        want.dedup();
125 +
        self.statement().is_ok_and(|mine| {
126 +
            mine == Statement {
127 +
                inputs: want,
128 +
                ..s.clone()
129 +
            }
130 +
        })
131 +
    }
132 +
133 +
    /// `[attest, [Signature], identity]`, each object as its bytes.
134 +
    pub fn encode(&self) -> Vec<u8> {
135 +
        cbor::encode(&Value::Array(vec![
136 +
            Value::Bytes(self.attest.encode()),
137 +
            Value::Array(self.sigs.iter().map(|s| s.to_value()).collect()),
138 +
            Value::Bytes(self.signer.encode()),
139 +
        ]))
140 +
    }
141 +
142 +
    pub fn decode(b: &[u8]) -> Result<SignedApproval, ApprovalError> {
143 +
        let v = cbor::decode(b).map_err(ObjectError::from)?;
144 +
        let a = array(&v, "signed approval")?;
145 +
        if a.len() != 3 {
146 +
            return Err(ApprovalError::Schema(
147 +
                "signed approval: expected [attest, sigs, identity]".into(),
148 +
            ));
149 +
        }
150 +
        Ok(SignedApproval {
151 +
            attest: Attest::decode(bytes(&a[0], "attest")?)?,
152 +
            sigs: array(&a[1], "sigs")?
153 +
                .iter()
154 +
                .map(Signature::from_value)
155 +
                .collect::<Result<_, _>>()?,
156 +
            signer: Identity::decode(bytes(&a[2], "identity")?)?,
157 +
        })
158 +
    }
159 +
}
160 +
161 +
#[cfg(test)]
162 +
mod tests {
163 +
    use super::*;
164 +
    use crate::identity::SecretKey;
165 +
166 +
    fn statement() -> Statement {
167 +
        Statement {
168 +
            repo: [7; 16],
169 +
            run: "https://hub.example/andy/infra/flow/8".into(),
170 +
            job: "apply".into(),
171 +
            job_hash: ObjectRef([1; 32]),
172 +
            state: ObjectRef([2; 32]),
173 +
            inputs: vec![ObjectRef([4; 32]), ObjectRef([3; 32])],
174 +
            environment: "prod".into(),
175 +
            approve: true,
176 +
        }
177 +
    }
178 +
179 +
    #[test]
180 +
    fn an_approval_signs_exactly_one_job_state_plan_and_environment() {
181 +
        let sk = SecretKey::generate();
182 +
        let id = sk.identity("Andy", "[email protected]", 1);
183 +
        let sa = SignedApproval::sign(&statement(), &id, &sk);
184 +
        sa.verify().unwrap();
185 +
        let back = SignedApproval::decode(&sa.encode()).unwrap();
186 +
        assert_eq!(back, sa, "round trip");
187 +
        assert!(back.covers(&statement()), "inputs in any order");
188 +
        for other in [
189 +
            Statement {
190 +
                inputs: vec![ObjectRef([9; 32])],
191 +
                ..statement()
192 +
            },
193 +
            Statement {
194 +
                state: ObjectRef([8; 32]),
195 +
                ..statement()
196 +
            },
197 +
            Statement {
198 +
                job_hash: ObjectRef([8; 32]),
199 +
                ..statement()
200 +
            },
201 +
            Statement {
202 +
                run: "https://hub.example/andy/infra/flow/9".into(),
203 +
                ..statement()
204 +
            },
205 +
            Statement {
206 +
                environment: "dev".into(),
207 +
                ..statement()
208 +
            },
209 +
            Statement {
210 +
                approve: false,
211 +
                ..statement()
212 +
            },
213 +
        ] {
214 +
            assert!(!back.covers(&other), "{other:?}");
215 +
        }
216 +
        // Changed after signing: the signature no longer holds.
217 +
        let mut forged = sa.clone();
218 +
        if let AttestBody::Approval { inputs, .. } = &mut forged.attest.body {
219 +
            inputs[0] = ObjectRef([9; 32]);
220 +
        }
221 +
        assert!(forged.verify().is_err());
222 +
        // Signed by someone else than the identity it carries.
223 +
        let other_sk = SecretKey::generate();
224 +
        let mut stolen = SignedApproval::sign(&statement(), &id, &other_sk);
225 +
        stolen.signer = id.clone();
226 +
        assert!(stolen.verify().is_err());
227 +
    }
228 +
}
added crates/oxy-core/src/artifact.rs +303 −0
1 +
//! Artifacts (Cylinder, docs/cylinder.md §4.1, oxygen#108): an immutable
2 +
//! object naming a tree of files, each an ordinary blob of FastCDC BLAKE3
3 +
//! chunks, so an image rebuilt with one changed file moves only that file's
4 +
//! changed chunks, and a binary in an archive and in an image is the same
5 +
//! chunks. An artifact is an object of its own (`artifact`) in canonical
6 +
//! CBOR; its tree is a blob. The `cylinder` crate packs, unpacks and keeps
7 +
//! them.
8 +
9 +
use crate::cbor::{self, Value};
10 +
use crate::hash::{hash_object, ObjectRef, ObjectType};
11 +
use crate::objects::{schema, ObjectError};
12 +
use std::collections::BTreeMap;
13 +
14 +
type R<T> = std::result::Result<T, ObjectError>;
15 +
16 +
/// The encoding's version this crate writes and reads.
17 +
pub const VERSION: u64 = 1;
18 +
19 +
/// What an artifact is for.
20 +
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
21 +
pub enum Kind {
22 +
    Container,
23 +
    RunnerImage,
24 +
    Package,
25 +
    Archive,
26 +
    Report,
27 +
}
28 +
29 +
impl Kind {
30 +
    pub const ALL: [Kind; 5] = [
31 +
        Kind::Container,
32 +
        Kind::RunnerImage,
33 +
        Kind::Package,
34 +
        Kind::Archive,
35 +
        Kind::Report,
36 +
    ];
37 +
38 +
    pub fn as_str(self) -> &'static str {
39 +
        match self {
40 +
            Kind::Container => "container",
41 +
            Kind::RunnerImage => "runner-image",
42 +
            Kind::Package => "package",
43 +
            Kind::Archive => "archive",
44 +
            Kind::Report => "report",
45 +
        }
46 +
    }
47 +
48 +
    pub fn parse(s: &str) -> R<Kind> {
49 +
        Kind::ALL.into_iter().find(|k| k.as_str() == s).map_or_else(
50 +
            || {
51 +
                schema(format!(
52 +
                    "{s:?} is not a kind of artifact: {}",
53 +
                    Kind::ALL.map(|k| k.as_str()).join(", ")
54 +
                ))
55 +
            },
56 +
            Ok,
57 +
        )
58 +
    }
59 +
}
60 +
61 +
/// A file's mode in a tree: a regular file, an executable one, or a
62 +
/// symbolic link (whose blob is its target).
63 +
pub const MODE_FILE: u32 = 0o100644;
64 +
pub const MODE_EXEC: u32 = 0o100755;
65 +
pub const MODE_LINK: u32 = 0o120000;
66 +
67 +
/// A file of an artifact's tree.
68 +
#[derive(Clone, Debug, PartialEq, Eq)]
69 +
pub struct Entry {
70 +
    /// `/`-separated, relative, never `.` or `..` parts.
71 +
    pub path: String,
72 +
    pub mode: u32,
73 +
    pub blob: ObjectRef,
74 +
    pub size: u64,
75 +
}
76 +
77 +
/// An artifact's files, in path order: paths, modes, blobs and sizes.
78 +
#[derive(Clone, Debug, Default, PartialEq, Eq)]
79 +
pub struct DirTree {
80 +
    pub entries: Vec<Entry>,
81 +
}
82 +
83 +
fn path_ok(p: &str) -> bool {
84 +
    !p.is_empty()
85 +
        && !p.starts_with('/')
86 +
        && p.split('/').all(|s| !s.is_empty() && s != "." && s != "..")
87 +
}
88 +
89 +
fn obj(r: &ObjectRef) -> Value {
90 +
    Value::Bytes(r.0.to_vec())
91 +
}
92 +
93 +
fn get_obj(v: &Value, what: &str) -> R<ObjectRef> {
94 +
    match v.as_bytes() {
95 +
        Some(b) if b.len() == 32 => {
96 +
            let mut a = [0u8; 32];
97 +
            a.copy_from_slice(b);
98 +
            Ok(ObjectRef(a))
99 +
        }
100 +
        _ => schema(format!("{what} is not an object reference")),
101 +
    }
102 +
}
103 +
104 +
fn get_text(v: &Value, what: &str) -> R<String> {
105 +
    v.as_text()
106 +
        .map(str::to_string)
107 +
        .map_or_else(|| schema(format!("{what} is not text")), Ok)
108 +
}
109 +
110 +
fn get_uint(v: &Value, what: &str) -> R<u64> {
111 +
    v.as_uint()
112 +
        .map_or_else(|| schema(format!("{what} is not a number")), Ok)
113 +
}
114 +
115 +
impl DirTree {
116 +
    pub fn encode(&self) -> Vec<u8> {
117 +
        let entries = self
118 +
            .entries
119 +
            .iter()
120 +
            .map(|e| {
121 +
                Value::Array(vec![
122 +
                    Value::Text(e.path.clone()),
123 +
                    Value::Uint(e.mode as u64),
124 +
                    obj(&e.blob),
125 +
                    Value::Uint(e.size),
126 +
                ])
127 +
            })
128 +
            .collect();
129 +
        cbor::encode(&Value::map([
130 +
            (0, Value::Uint(VERSION)),
131 +
            (1, Value::Array(entries)),
132 +
        ]))
133 +
    }
134 +
135 +
    /// Read and check a tree: a known version, clean paths, each once, in
136 +
    /// order, with a known mode.
137 +
    pub fn decode(bytes: &[u8]) -> R<DirTree> {
138 +
        let v = cbor::decode(bytes)?;
139 +
        let m = v.as_map().map_or_else(|| schema("a tree is a map"), Ok)?;
140 +
        if m.get(&0).and_then(Value::as_uint) != Some(VERSION) {
141 +
            return schema(format!(
142 +
                "tree version {:?} is not known; this reads {VERSION}",
143 +
                m.get(&0)
144 +
            ));
145 +
        }
146 +
        let mut entries = Vec::new();
147 +
        for item in m.get(&1).and_then(Value::as_array).unwrap_or_default() {
148 +
            let [p, mode, blob, size] = item.as_array().unwrap_or_default() else {
149 +
                return schema("a tree entry is [path, mode, blob, size]");
150 +
            };
151 +
            let path = get_text(p, "a path")?;
152 +
            let mode = get_uint(mode, "a mode")? as u32;
153 +
            if !path_ok(&path) {
154 +
                return schema(format!("{path:?} is not a clean relative path"));
155 +
            }
156 +
            if ![MODE_FILE, MODE_EXEC, MODE_LINK].contains(&mode) {
157 +
                return schema(format!(
158 +
                    "{path}: mode {mode:o} is not a file, an executable or a link"
159 +
                ));
160 +
            }
161 +
            if entries.last().is_some_and(|l: &Entry| l.path >= path) {
162 +
                return schema(format!(
163 +
                    "{path}: entries are not in path order, or one is there twice"
164 +
                ));
165 +
            }
166 +
            entries.push(Entry {
167 +
                path,
168 +
                mode,
169 +
                blob: get_obj(blob, "a blob")?,
170 +
                size: get_uint(size, "a size")?,
171 +
            });
172 +
        }
173 +
        Ok(DirTree { entries })
174 +
    }
175 +
}
176 +
177 +
/// An artifact (docs/cylinder.md §4.1).
178 +
#[derive(Clone, Debug, PartialEq, Eq)]
179 +
pub struct Artifact {
180 +
    pub kind: Kind,
181 +
    /// `x86_64`, `riscv64`, `aarch64`.
182 +
    pub arch: Option<String>,
183 +
    /// `linux`, `darwin`, `none`.
184 +
    pub os: Option<String>,
185 +
    /// The [`DirTree`]'s blob.
186 +
    pub tree: ObjectRef,
187 +
    /// The default command, for a container or an image.
188 +
    pub entrypoint: Option<Vec<String>>,
189 +
    pub env: BTreeMap<String, String>,
190 +
    pub labels: BTreeMap<String, String>,
191 +
    /// The files' bytes together.
192 +
    pub total_size: u64,
193 +
    /// The receipt of the Flow job that built it, when one did.
194 +
    pub receipt: Option<ObjectRef>,
195 +
}
196 +
197 +
fn pairs(m: &BTreeMap<String, String>) -> Value {
198 +
    if m.is_empty() {
199 +
        return Value::Null;
200 +
    }
201 +
    Value::Array(
202 +
        m.iter()
203 +
            .map(|(k, v)| Value::Array(vec![Value::Text(k.clone()), Value::Text(v.clone())]))
204 +
            .collect(),
205 +
    )
206 +
}
207 +
208 +
fn get_pairs(v: Option<&Value>, what: &str) -> R<BTreeMap<String, String>> {
209 +
    let mut out = BTreeMap::new();
210 +
    let Some(v) = v.filter(|v| !v.is_null()) else {
211 +
        return Ok(out);
212 +
    };
213 +
    for p in v
214 +
        .as_array()
215 +
        .map_or_else(|| schema(format!("{what} is a list of pairs")), Ok)?
216 +
    {
217 +
        let [k, val] = p.as_array().unwrap_or_default() else {
218 +
            return schema(format!("{what}: a pair is [key, value]"));
219 +
        };
220 +
        let k = get_text(k, what)?;
221 +
        if out.insert(k.clone(), get_text(val, what)?).is_some() {
222 +
            return schema(format!("{what}: {k} is there twice"));
223 +
        }
224 +
    }
225 +
    Ok(out)
226 +
}
227 +
228 +
fn opt_text(v: &Option<String>) -> Value {
229 +
    v.as_ref().map_or(Value::Null, |s| Value::Text(s.clone()))
230 +
}
231 +
232 +
impl Artifact {
233 +
    /// Its canonical CBOR, keys as in docs/cylinder.md §4.1.
234 +
    pub fn encode(&self) -> Vec<u8> {
235 +
        let entry = self.entrypoint.as_ref().map_or(Value::Null, |a| {
236 +
            Value::Array(a.iter().cloned().map(Value::Text).collect())
237 +
        });
238 +
        cbor::encode(&Value::map([
239 +
            (0, Value::Uint(VERSION)),
240 +
            (1, Value::Text(self.kind.as_str().into())),
241 +
            (2, opt_text(&self.arch)),
242 +
            (3, opt_text(&self.os)),
243 +
            (4, obj(&self.tree)),
244 +
            (5, entry),
245 +
            (6, pairs(&self.env)),
246 +
            (7, pairs(&self.labels)),
247 +
            (8, Value::Uint(self.total_size)),
248 +
            (9, self.receipt.as_ref().map_or(Value::Null, obj)),
249 +
        ]))
250 +
    }
251 +
252 +
    pub fn decode(bytes: &[u8]) -> R<Artifact> {
253 +
        let v = cbor::decode(bytes)?;
254 +
        let m = v
255 +
            .as_map()
256 +
            .map_or_else(|| schema("an artifact is a map"), Ok)?;
257 +
        if m.get(&0).and_then(Value::as_uint) != Some(VERSION) {
258 +
            return schema(format!(
259 +
                "artifact version {:?} is not known; this reads {VERSION}",
260 +
                m.get(&0)
261 +
            ));
262 +
        }
263 +
        if let Some(k) = m.keys().find(|k| **k > 9) {
264 +
            return schema(format!("key {k} is not part of an artifact"));
265 +
        }
266 +
        let text = |k: u64, what: &str| -> R<Option<String>> {
267 +
            match m.get(&k) {
268 +
                None | Some(Value::Null) => Ok(None),
269 +
                Some(v) => get_text(v, what).map(Some),
270 +
            }
271 +
        };
272 +
        let entrypoint = match m.get(&5) {
273 +
            None | Some(Value::Null) => None,
274 +
            Some(v) => Some(
275 +
                v.as_array()
276 +
                    .unwrap_or_default()
277 +
                    .iter()
278 +
                    .map(|a| get_text(a, "the entrypoint"))
279 +
                    .collect::<R<_>>()?,
280 +
            ),
281 +
        };
282 +
        let receipt = match m.get(&9) {
283 +
            None | Some(Value::Null) => None,
284 +
            Some(v) => Some(get_obj(v, "the receipt")?),
285 +
        };
286 +
        Ok(Artifact {
287 +
            kind: Kind::parse(&text(1, "the kind")?.unwrap_or_default())?,
288 +
            arch: text(2, "the architecture")?,
289 +
            os: text(3, "the system")?,
290 +
            tree: get_obj(m.get(&4).unwrap_or(&Value::Null), "the tree")?,
291 +
            entrypoint,
292 +
            env: get_pairs(m.get(&6), "env")?,
293 +
            labels: get_pairs(m.get(&7), "labels")?,
294 +
            total_size: get_uint(m.get(&8).unwrap_or(&Value::Null), "the size")?,
295 +
            receipt,
296 +
        })
297 +
    }
298 +
299 +
    /// Its name: its object's hash, as every Oxygen object is named.
300 +
    pub fn name(&self) -> ObjectRef {
301 +
        hash_object(ObjectType::Artifact, &self.encode())
302 +
    }
303 +
}
modified crates/oxy-core/src/attest.rs +101 −0
…
25 unchanged lines
26 26
        state: ObjectRef,
27 27
        message: String,
28 28
    },
29 +
    /// Tag `name`, naming `state`, is withdrawn (spec §10.1): signed by a
30 +
    /// key the policy lets create the tag. The name is never used again.
31 +
    Untag {
32 +
        name: String,
33 +
        state: ObjectRef,
34 +
        reason: String,
35 +
    },
29 36
    /// The signing key's identity was replaced by `new` (spec §14.2).
30 37
    Rotate { new: KeyId },
31 38
    /// `key` must no longer be trusted (spec §14.2).
…
5 unchanged lines
37 44
        state: ObjectRef,
38 45
        run: String,
39 46
    },
47 +
    /// A person approved (or rejected) one job of one run (OxygenFlow
48 +
    /// approval gates; oxygenhub#235): the job as hashed with its sandbox,
49 +
    /// on that state, taking exactly those input blobs (a reviewed plan),
50 +
    /// for that environment. It approves nothing else.
51 +
    Approval {
52 +
        run: String,
53 +
        job: String,
54 +
        job_hash: ObjectRef,
55 +
        state: ObjectRef,
56 +
        inputs: Vec<ObjectRef>,
57 +
        environment: String,
58 +
        approve: bool,
59 +
    },
60 +
    /// A host accepted the reference attestation `attest`, at this
61 +
    /// attestation's own time, the host's clock (§14.7): the time a
62 +
    /// certificate of a short-lived key is checked at on a fetch.
63 +
    Accepted { attest: ObjectRef },
40 64
}
41 65
42 66
#[derive(Clone, Debug, PartialEq, Eq)]
…
10 unchanged lines
53 77
            AttestBody::Ref { .. } => "ref",
54 78
            AttestBody::Seal { .. } => "seal",
55 79
            AttestBody::Tag { .. } => "tag",
80 +
            AttestBody::Untag { .. } => "untag",
56 81
            AttestBody::Rotate { .. } => "rotate",
57 82
            AttestBody::Revoke { .. } => "revoke",
58 83
            AttestBody::Receipt { .. } => "receipt",
84 +
            AttestBody::Approval { .. } => "approval",
85 +
            AttestBody::Accepted { .. } => "accepted",
59 86
        }
60 87
    }
61 88
…
33 unchanged lines
95 122
                (1, Value::Bytes(state.0.to_vec())),
96 123
                (2, Value::Text(message.clone())),
97 124
            ]),
125 +
            AttestBody::Untag {
126 +
                name,
127 +
                state,
128 +
                reason,
129 +
            } => Value::map([
130 +
                (0, Value::Text(name.clone())),
131 +
                (1, Value::Bytes(state.0.to_vec())),
132 +
                (2, Value::Text(reason.clone())),
133 +
            ]),
98 134
            AttestBody::Rotate { new } => Value::map([(0, Value::Bytes(new.0.to_vec()))]),
99 135
            AttestBody::Revoke { key, reason } => Value::map([
100 136
                (0, Value::Bytes(key.0.to_vec())),
…
8 unchanged lines
109 145
                (1, Value::Bytes(state.0.to_vec())),
110 146
                (2, Value::Text(run.clone())),
111 147
            ]),
148 +
            AttestBody::Accepted { attest } => Value::map([(0, Value::Bytes(attest.0.to_vec()))]),
149 +
            AttestBody::Approval {
150 +
                run,
151 +
                job,
152 +
                job_hash,
153 +
                state,
154 +
                inputs,
155 +
                environment,
156 +
                approve,
157 +
            } => Value::map([
158 +
                (0, Value::Text(run.clone())),
159 +
                (1, Value::Text(job.clone())),
160 +
                (2, Value::Bytes(job_hash.0.to_vec())),
161 +
                (3, Value::Bytes(state.0.to_vec())),
162 +
                (
163 +
                    4,
164 +
                    Value::Array(inputs.iter().map(|i| Value::Bytes(i.0.to_vec())).collect()),
165 +
                ),
166 +
                (5, Value::Text(environment.clone())),
167 +
                (6, Value::Uint(u64::from(*approve))),
168 +
            ]),
112 169
        };
113 170
        Value::map([
114 171
            (0, Value::Uint(FORMAT_VERSION)),
…
49 unchanged lines
164 221
                state: oref(field(b, 1, "state")?, "state")?,
165 222
                message: text(field(b, 2, "message")?, "message")?.to_owned(),
166 223
            },
224 +
            "untag" => AttestBody::Untag {
225 +
                name: text(field(b, 0, "name")?, "name")?.to_owned(),
226 +
                state: oref(field(b, 1, "state")?, "state")?,
227 +
                reason: text(field(b, 2, "reason")?, "reason")?.to_owned(),
228 +
            },
167 229
            "rotate" => AttestBody::Rotate {
168 230
                new: oref(field(b, 0, "new")?, "new")?,
169 231
            },
…
6 unchanged lines
176 238
                state: oref(field(b, 1, "state")?, "state")?,
177 239
                run: text(field(b, 2, "run")?, "run")?.to_owned(),
178 240
            },
241 +
            "approval" => {
242 +
                let inputs = array(field(b, 4, "inputs")?, "inputs")?
243 +
                    .iter()
244 +
                    .map(|h| oref(h, "input"))
245 +
                    .collect::<Result<Vec<_>, _>>()?;
246 +
                if !inputs.windows(2).all(|w| w[0] < w[1]) {
247 +
                    return Err(ObjectError::Schema(
248 +
                        "approval: inputs must be sorted and unique".into(),
249 +
                    ));
250 +
                }
251 +
                AttestBody::Approval {
252 +
                    run: text(field(b, 0, "run")?, "run")?.to_owned(),
253 +
                    job: text(field(b, 1, "job")?, "job")?.to_owned(),
254 +
                    job_hash: oref(field(b, 2, "job_hash")?, "job_hash")?,
255 +
                    state: oref(field(b, 3, "state")?, "state")?,
256 +
                    inputs,
257 +
                    environment: text(field(b, 5, "environment")?, "environment")?.to_owned(),
258 +
                    approve: match uint(field(b, 6, "approve")?, "approve")? {
259 +
                        0 => false,
260 +
                        1 => true,
261 +
                        _ => return Err(ObjectError::Schema("approval: approve is 0 or 1".into())),
262 +
                    },
263 +
                }
264 +
            }
265 +
            "accepted" => AttestBody::Accepted {
266 +
                attest: oref(field(b, 0, "attest")?, "attest")?,
267 +
            },
179 268
            other => {
180 269
                return Err(ObjectError::Schema(format!(
181 270
                    "unknown attestation kind {other:?}"
…
45 unchanged lines
227 316
        };
228 317
        assert_eq!(Attest::decode(&t.encode()).unwrap(), t);
229 318
        assert_eq!(t.kind(), "tag");
319 +
        let u = Attest {
320 +
            repo: [1; 16],
321 +
            hlc: Hlc::default(),
322 +
            body: AttestBody::Untag {
323 +
                name: "v1".into(),
324 +
                state: ObjectRef([5; 32]),
325 +
                reason: "tagged the wrong change".into(),
326 +
            },
327 +
        };
328 +
        assert_eq!(Attest::decode(&u.encode()).unwrap(), u);
329 +
        assert_eq!(u.kind(), "untag");
330 +
        assert_ne!(u.hash(), t.hash());
230 331
    }
231 332
232 333
    #[test]
…
17 unchanged lines
modified crates/oxy-core/src/attributes.rs +13 −2
1 1
//! `.oxyattributes` (spec §19): per-path attributes in gitattributes syntax.
2 2
//! Each line is `pattern attr...` where an attribute is `binary`, `text`
3 -
//! (or `-binary`), `eol=lf`, `eol=crlf`, `nocompress`, or `lockable`. A
3 +
//! (or `-binary`), `eol=lf`, `eol=crlf`, `nocompress`, `lockable`, or
4 +
//! `merge=union`. A
4 5
//! pattern without a slash matches a basename anywhere below the file's
5 6
//! directory; one with a slash is anchored to that directory. Later rules
6 7
//! override earlier ones per attribute.
…
15 unchanged lines
22 23
    pub eol: Option<Eol>,
23 24
    pub nocompress: bool,
24 25
    pub lockable: bool,
26 +
    /// `merge=union` (oxygen#145): two changes adding lines at the same
27 +
    /// place keep both, without asking (`oxy resolve --both`, spec §9.5).
28 +
    pub union: bool,
25 29
}
26 30
27 31
struct Rule {
…
80 unchanged lines
108 112
                    ("-nocompress", None) => a.nocompress = false,
109 113
                    ("lockable", None) => a.lockable = true,
110 114
                    ("-lockable", None) => a.lockable = false,
115 +
                    ("merge", Some("union")) => a.union = true,
116 +
                    ("merge", Some(_)) | ("-merge", None) => a.union = false,
111 117
                    _ => {}
112 118
                }
113 119
            }
…
52 unchanged lines
166 172
                binary: Some(true),
167 173
                eol: None,
168 174
                nocompress: true,
169 -
                lockable: false
175 +
                lockable: false,
176 +
                union: false
170 177
            }
171 178
        );
172 179
        assert_eq!(a.lookup("notes.txt").eol, Some(Eol::Lf));
173 180
        assert_eq!(a.lookup("docs/notes.txt").eol, Some(Eol::Crlf));
174 181
        assert!(a.lookup("models/x/y.bin").lockable);
182 +
        let u = Attributes::from_files([(".oxyattributes", &b"CHANGELOG.md merge=union\n"[..])]);
183 +
        assert!(u.lookup("CHANGELOG.md").union);
184 +
        assert!(u.lookup("docs/CHANGELOG.md").union);
185 +
        assert!(!u.lookup("README.md").union);
175 186
        assert_eq!(a.lookup("src/deep/x.dat").binary, Some(true));
176 187
        assert_eq!(a.lookup("other/x.dat").binary, None);
177 188
        assert_eq!(a.lookup("src/gen.txt").eol, None, "deeper file overrides");
…
11 unchanged lines
added crates/oxy-core/src/cert.rs +514 −0
1 +
//! Certificates for short-lived keys (spec §14.7, oxygenhub#241,
2 +
//! oxygenhub docs/keyless.md). An issuer, a hub's key, binds a fresh key
3 +
//! to the identity that proved itself (an OIDC subject) for minutes: a Flow
4 +
//! job or an agent signs with that key and throws it away. The certificate
5 +
//! is signed with the issuer's composite key, so it is post-quantum like
6 +
//! every other signature in Oxygen.
7 +
//!
8 +
//! Whether to trust an issuer, and for which subjects, is the trust
9 +
//! policy's (§14.5); whether a signature was made inside the certificate's
10 +
//! window needs time the signer does not choose, which the caller passes
11 +
//! in (the transparency log's integrated time, §14.8, else a host's
12 +
//! acceptance).
13 +
14 +
use crate::cbor::{self, Value};
15 +
use crate::hash::{hash_object, KeyId, ObjectRef, ObjectType};
16 +
use crate::identity::{verify, Domain, Identity, Scope, SecretKey, Signature};
17 +
use crate::objects::{array, bytes, check_version, field, map, oref, text, uint, ObjectError};
18 +
19 +
/// The longest a certificate may be valid. Jobs get ten minutes; an hour
20 +
/// leaves room for an issuer that rounds up, and no more.
21 +
pub const MAX_LIFETIME_MS: u64 = 60 * 60 * 1000;
22 +
23 +
#[derive(Clone, Debug, PartialEq, Eq)]
24 +
pub struct Certificate {
25 +
    /// The short-lived key's public identity.
26 +
    pub key: Identity,
27 +
    /// The issuer's KeyId (a hub's key).
28 +
    pub issuer: KeyId,
29 +
    /// Who proved it: `repo:OWNER/NAME:bookmark:B`, `agent:NAME@ACCOUNT`.
30 +
    pub subject: String,
31 +
    /// What else the proof said (environment, run, job), sorted by name.
32 +
    pub claims: Vec<(String, String)>,
33 +
    /// What the key may sign: changes, refs, seals.
34 +
    pub scope: Scope,
35 +
    /// Valid from and until, in wall milliseconds.
36 +
    pub not_before: u64,
37 +
    pub expires: u64,
38 +
    /// The grant that lets the subject act for an author, when an agent
39 +
    /// asked with one (field 8, only when present).
40 +
    pub grant: Option<SignedGrant>,
41 +
}
42 +
43 +
impl Certificate {
44 +
    pub fn to_value(&self) -> Value {
45 +
        let mut claims = self.claims.clone();
46 +
        claims.sort();
47 +
        let mut v = Value::map([
48 +
            (0, Value::Uint(1)),
49 +
            (1, Value::Bytes(self.key.encode())),
50 +
            (2, Value::Bytes(self.issuer.0.to_vec())),
51 +
            (3, Value::Text(self.subject.clone())),
52 +
            (
53 +
                4,
54 +
                Value::Array(
55 +
                    claims
56 +
                        .into_iter()
57 +
                        .map(|(k, v)| Value::Array(vec![Value::Text(k), Value::Text(v)]))
58 +
                        .collect(),
59 +
                ),
60 +
            ),
61 +
            (
62 +
                5,
63 +
                Value::Array(
64 +
                    self.scope
65 +
                        .names()
66 +
                        .into_iter()
67 +
                        .map(|n| Value::Text(n.to_string()))
68 +
                        .collect(),
69 +
                ),
70 +
            ),
71 +
            (6, Value::Uint(self.not_before)),
72 +
            (7, Value::Uint(self.expires)),
73 +
        ]);
74 +
        if let (Some(g), Value::Map(m)) = (&self.grant, &mut v) {
75 +
            m.insert(8, Value::Bytes(g.encode()));
76 +
        }
77 +
        v
78 +
    }
79 +
80 +
    pub fn encode(&self) -> Vec<u8> {
81 +
        cbor::encode(&self.to_value())
82 +
    }
83 +
84 +
    pub fn hash(&self) -> ObjectRef {
85 +
        hash_object(ObjectType::Certificate, &self.encode())
86 +
    }
87 +
88 +
    /// The short-lived key's KeyId: what its signatures name.
89 +
    pub fn key_id(&self) -> KeyId {
90 +
        self.key.hash()
91 +
    }
92 +
93 +
    pub fn decode(b: &[u8]) -> Result<Certificate, ObjectError> {
94 +
        let v = cbor::decode(b)?;
95 +
        let m = map(&v, "certificate")?;
96 +
        check_version(m)?;
97 +
        let key = Identity::decode(bytes(field(m, 1, "key")?, "key")?)?;
98 +
        let mut claims = Vec::new();
99 +
        for c in array(field(m, 4, "claims")?, "claims")? {
100 +
            let pair = array(c, "claim")?;
101 +
            if pair.len() != 2 {
102 +
                return Err(ObjectError::Schema("claim: expected [name, value]".into()));
103 +
            }
104 +
            claims.push((
105 +
                text(&pair[0], "claim name")?.to_owned(),
106 +
                text(&pair[1], "claim value")?.to_owned(),
107 +
            ));
108 +
        }
109 +
        let mut scope = Scope::NONE;
110 +
        for n in array(field(m, 5, "scope")?, "scope")? {
111 +
            let n = text(n, "scope")?;
112 +
            scope = scope
113 +
                | Scope::named(n).ok_or_else(|| {
114 +
                    ObjectError::Schema(format!("certificate: unknown scope {n:?}"))
115 +
                })?;
116 +
        }
117 +
        Ok(Certificate {
118 +
            key,
119 +
            issuer: oref(field(m, 2, "issuer")?, "issuer")?,
120 +
            subject: text(field(m, 3, "subject")?, "subject")?.to_owned(),
121 +
            claims,
122 +
            scope,
123 +
            not_before: uint(field(m, 6, "not_before")?, "not_before")?,
124 +
            expires: uint(field(m, 7, "expires")?, "expires")?,
125 +
            grant: match m.get(&8) {
126 +
                Some(g) => Some(
127 +
                    SignedGrant::decode(bytes(g, "grant")?)
128 +
                        .map_err(|e| ObjectError::Schema(format!("grant: {e}")))?,
129 +
                ),
130 +
                None => None,
131 +
            },
132 +
        })
133 +
    }
134 +
135 +
    /// Whether the key may sign within `scope` at `at_ms`, a time the
136 +
    /// signer did not choose.
137 +
    pub fn covers(&self, scope: Scope, at_ms: u64) -> bool {
138 +
        !scope.is_empty()
139 +
            && self.scope.has(scope)
140 +
            && self.not_before <= at_ms
141 +
            && at_ms <= self.expires
142 +
    }
143 +
144 +
    /// The claim called `name`, if the certificate has one.
145 +
    pub fn claim(&self, name: &str) -> Option<&str> {
146 +
        self.claims
147 +
            .iter()
148 +
            .find(|(k, _)| k == name)
149 +
            .map(|(_, v)| v.as_str())
150 +
    }
151 +
}
152 +
153 +
/// An author's grant (spec §14.7): it lets the keys an issuer certifies
154 +
/// for `subject` act for the author, as a delegate does, without a key of
155 +
/// their own. An agent's fresh keys then sign and seal changes in the
156 +
/// author's name while the grant lasts; the grant travels inside their
157 +
/// certificates, so a verifier needs nothing else. Signed by the author.
158 +
#[derive(Clone, Debug, PartialEq, Eq)]
159 +
pub struct Grant {
160 +
    /// The author's KeyId, which signs the grant.
161 +
    pub grantor: KeyId,
162 +
    /// The subject whose certified keys act for it: `agent:NAME@ACCOUNT`.
163 +
    pub subject: String,
164 +
    /// The issuer whose certificates count.
165 +
    pub issuer: KeyId,
166 +
    /// What they may do for the author: changes, seal.
167 +
    pub scope: Scope,
168 +
    /// Valid from and until, in wall milliseconds.
169 +
    pub not_before: u64,
170 +
    pub expires: u64,
171 +
}
172 +
173 +
impl Grant {
174 +
    pub fn to_value(&self) -> Value {
175 +
        Value::map([
176 +
            (0, Value::Uint(1)),
177 +
            (1, Value::Bytes(self.grantor.0.to_vec())),
178 +
            (2, Value::Text(self.subject.clone())),
179 +
            (3, Value::Bytes(self.issuer.0.to_vec())),
180 +
            (
181 +
                4,
182 +
                Value::Array(
183 +
                    self.scope
184 +
                        .names()
185 +
                        .into_iter()
186 +
                        .map(|n| Value::Text(n.to_string()))
187 +
                        .collect(),
188 +
                ),
189 +
            ),
190 +
            (5, Value::Uint(self.not_before)),
191 +
            (6, Value::Uint(self.expires)),
192 +
        ])
193 +
    }
194 +
195 +
    pub fn encode(&self) -> Vec<u8> {
196 +
        cbor::encode(&self.to_value())
197 +
    }
198 +
199 +
    pub fn hash(&self) -> ObjectRef {
200 +
        hash_object(ObjectType::Grant, &self.encode())
201 +
    }
202 +
203 +
    pub fn decode(b: &[u8]) -> Result<Grant, ObjectError> {
204 +
        let v = cbor::decode(b)?;
205 +
        let m = map(&v, "grant")?;
206 +
        check_version(m)?;
207 +
        let mut scope = Scope::NONE;
208 +
        for n in array(field(m, 4, "scope")?, "scope")? {
209 +
            let n = text(n, "scope")?;
210 +
            scope = scope
211 +
                | Scope::named(n)
212 +
                    .ok_or_else(|| ObjectError::Schema(format!("grant: unknown scope {n:?}")))?;
213 +
        }
214 +
        Ok(Grant {
215 +
            grantor: oref(field(m, 1, "grantor")?, "grantor")?,
216 +
            subject: text(field(m, 2, "subject")?, "subject")?.to_owned(),
217 +
            issuer: oref(field(m, 3, "issuer")?, "issuer")?,
218 +
            scope,
219 +
            not_before: uint(field(m, 5, "not_before")?, "not_before")?,
220 +
            expires: uint(field(m, 6, "expires")?, "expires")?,
221 +
        })
222 +
    }
223 +
224 +
    /// Whether it lets its subject do `scope` for the author at `at_ms`
225 +
    /// (the change's own time, as for a delegation).
226 +
    pub fn covers(&self, scope: Scope, at_ms: u64) -> bool {
227 +
        !scope.is_empty()
228 +
            && self.scope.has(scope)
229 +
            && self.not_before <= at_ms
230 +
            && at_ms <= self.expires
231 +
    }
232 +
}
233 +
234 +
/// A grant with its author's signature over its ObjectRef.
235 +
#[derive(Clone, Debug, PartialEq, Eq)]
236 +
pub struct SignedGrant {
237 +
    pub grant: Grant,
238 +
    pub sig: Signature,
239 +
}
240 +
241 +
impl SignedGrant {
242 +
    /// Grant `grant`, signed by the author `grantor` holds the secret of.
243 +
    pub fn sign(grant: Grant, grantor: &Identity, sk: &SecretKey) -> SignedGrant {
244 +
        let sig = sk.sign(Domain::Grant, &grantor.hash(), &grant.hash());
245 +
        SignedGrant { grant, sig }
246 +
    }
247 +
248 +
    /// Signed by `grantor`, which it names, and well formed.
249 +
    pub fn verify(&self, grantor: &Identity) -> Result<(), CertError> {
250 +
        let g = &self.grant;
251 +
        if grantor.hash() != g.grantor || self.sig.key != g.grantor {
252 +
            return Err(CertError::WrongIssuer(g.grantor.short(12)));
253 +
        }
254 +
        verify(grantor, &self.sig, Domain::Grant, &g.hash())
255 +
            .map_err(|e| CertError::BadSignature(e.to_string()))?;
256 +
        if g.expires <= g.not_before {
257 +
            return Err(CertError::BadLifetime);
258 +
        }
259 +
        if g.scope.is_empty() {
260 +
            return Err(CertError::NoScope);
261 +
        }
262 +
        Ok(())
263 +
    }
264 +
265 +
    /// `[grant bytes, signature]`.
266 +
    pub fn encode(&self) -> Vec<u8> {
267 +
        cbor::encode(&Value::Array(vec![
268 +
            Value::Bytes(self.grant.encode()),
269 +
            self.sig.to_value(),
270 +
        ]))
271 +
    }
272 +
273 +
    pub fn decode(b: &[u8]) -> Result<SignedGrant, CertError> {
274 +
        let v = cbor::decode(b).map_err(ObjectError::from)?;
275 +
        let a = array(&v, "signed grant")?;
276 +
        if a.len() != 2 {
277 +
            return Err(CertError::Schema(
278 +
                "signed grant: expected [grant, signature]".into(),
279 +
            ));
280 +
        }
281 +
        Ok(SignedGrant {
282 +
            grant: Grant::decode(bytes(&a[0], "grant")?)?,
283 +
            sig: Signature::from_value(&a[1])?,
284 +
        })
285 +
    }
286 +
}
287 +
288 +
/// What a fresh key signs (a challenge, §14.3) to show its holder asks for
289 +
/// the certificate: bound to the credential it is asked with, so the
290 +
/// proof cannot be replayed with another.
291 +
pub fn certify_challenge(credential: &str) -> Vec<u8> {
292 +
    let mut c = b"oxy1 certify ".to_vec();
293 +
    c.extend_from_slice(blake3::hash(credential.as_bytes()).as_bytes());
294 +
    c
295 +
}
296 +
297 +
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
298 +
pub enum CertError {
299 +
    #[error("the certificate names issuer {0}, not this one")]
300 +
    WrongIssuer(String),
301 +
    #[error("the issuer's signature does not verify: {0}")]
302 +
    BadSignature(String),
303 +
    #[error("the certificate is valid for no time, or for longer than an hour")]
304 +
    BadLifetime,
305 +
    #[error("the certificate gives no scope")]
306 +
    NoScope,
307 +
    #[error("{0}")]
308 +
    Schema(String),
309 +
}
310 +
311 +
impl From<ObjectError> for CertError {
312 +
    fn from(e: ObjectError) -> Self {
313 +
        CertError::Schema(e.to_string())
314 +
    }
315 +
}
316 +
317 +
/// A certificate with its issuer's signature over its ObjectRef.
318 +
#[derive(Clone, Debug, PartialEq, Eq)]
319 +
pub struct SignedCertificate {
320 +
    pub cert: Certificate,
321 +
    pub sig: Signature,
322 +
}
323 +
324 +
impl SignedCertificate {
325 +
    /// Issue `cert`, signed by the issuer `issuer` holds the secret of.
326 +
    pub fn sign(mut cert: Certificate, issuer: &Identity, sk: &SecretKey) -> SignedCertificate {
327 +
        // Claims in their canonical order, as encoded.
328 +
        cert.claims.sort();
329 +
        let sig = sk.sign(Domain::Certificate, &issuer.hash(), &cert.hash());
330 +
        SignedCertificate { cert, sig }
331 +
    }
332 +
333 +
    /// Signed by `issuer`, which it names, and well formed. Whether to
334 +
    /// trust the issuer for its subject is the policy's question.
335 +
    pub fn verify(&self, issuer: &Identity) -> Result<(), CertError> {
336 +
        let c = &self.cert;
337 +
        if issuer.hash() != c.issuer || self.sig.key != c.issuer {
338 +
            return Err(CertError::WrongIssuer(c.issuer.short(12)));
339 +
        }
340 +
        verify(issuer, &self.sig, Domain::Certificate, &c.hash())
341 +
            .map_err(|e| CertError::BadSignature(e.to_string()))?;
342 +
        if c.expires <= c.not_before || c.expires - c.not_before > MAX_LIFETIME_MS {
343 +
            return Err(CertError::BadLifetime);
344 +
        }
345 +
        if c.scope.is_empty() {
346 +
            return Err(CertError::NoScope);
347 +
        }
348 +
        Ok(())
349 +
    }
350 +
351 +
    /// `[certificate bytes, signature]`.
352 +
    pub fn encode(&self) -> Vec<u8> {
353 +
        cbor::encode(&Value::Array(vec![
354 +
            Value::Bytes(self.cert.encode()),
355 +
            self.sig.to_value(),
356 +
        ]))
357 +
    }
358 +
359 +
    pub fn decode(b: &[u8]) -> Result<SignedCertificate, CertError> {
360 +
        let v = cbor::decode(b).map_err(ObjectError::from)?;
361 +
        let a = array(&v, "signed certificate")?;
362 +
        if a.len() != 2 {
363 +
            return Err(CertError::Schema(
364 +
                "signed certificate: expected [certificate, signature]".into(),
365 +
            ));
366 +
        }
367 +
        Ok(SignedCertificate {
368 +
            cert: Certificate::decode(bytes(&a[0], "certificate")?)?,
369 +
            sig: Signature::from_value(&a[1])?,
370 +
        })
371 +
    }
372 +
}
373 +
374 +
#[cfg(test)]
375 +
mod tests {
376 +
    use super::*;
377 +
378 +
    fn issue(
379 +
        issuer_sk: &SecretKey,
380 +
        issuer: &Identity,
381 +
        minutes: u64,
382 +
    ) -> (SecretKey, SignedCertificate) {
383 +
        let job_sk = SecretKey::generate();
384 +
        let cert = Certificate {
385 +
            key: job_sk.identity("job 12 build", "[email protected]", 1_000),
386 +
            issuer: issuer.hash(),
387 +
            subject: "repo:andy/demo:bookmark:main".into(),
388 +
            claims: vec![
389 +
                ("run".into(), "12".into()),
390 +
                ("environment".into(), "prod".into()),
391 +
            ],
392 +
            scope: Scope::CHANGES | Scope::REFS,
393 +
            not_before: 1_000,
394 +
            expires: 1_000 + minutes * 60_000,
395 +
            grant: None,
396 +
        };
397 +
        (job_sk, SignedCertificate::sign(cert, issuer, issuer_sk))
398 +
    }
399 +
400 +
    #[test]
401 +
    fn a_certificate_round_trips_and_verifies_with_its_issuer_only() {
402 +
        let hub_sk = SecretKey::generate();
403 +
        let hub = hub_sk.identity("oxygenhub.io", "[email protected]", 1);
404 +
        let (_, sc) = issue(&hub_sk, &hub, 10);
405 +
        let back = SignedCertificate::decode(&sc.encode()).unwrap();
406 +
        assert_eq!(back, sc);
407 +
        assert_eq!(back.cert.claims[0].0, "environment", "claims kept sorted");
408 +
        assert_eq!(back.cert.claim("run"), Some("12"));
409 +
        assert!(back.verify(&hub).is_ok());
410 +
        let other_sk = SecretKey::generate();
411 +
        let other = other_sk.identity("elsewhere", "[email protected]", 1);
412 +
        assert!(matches!(
413 +
            back.verify(&other),
414 +
            Err(CertError::WrongIssuer(_))
415 +
        ));
416 +
        // Changed after signing: the signature no longer covers it.
417 +
        let mut forged = back.clone();
418 +
        forged.cert.subject = "repo:andy/demo:bookmark:release".into();
419 +
        assert!(matches!(
420 +
            forged.verify(&hub),
421 +
            Err(CertError::BadSignature(_))
422 +
        ));
423 +
        // Signed by another key in the issuer's name.
424 +
        let mut stolen = back.clone();
425 +
        stolen.sig = other_sk.sign(Domain::Certificate, &hub.hash(), &back.cert.hash());
426 +
        assert!(matches!(
427 +
            stolen.verify(&hub),
428 +
            Err(CertError::BadSignature(_))
429 +
        ));
430 +
    }
431 +
432 +
    #[test]
433 +
    fn a_grant_rides_in_its_certificate_and_verifies_with_its_author_only() {
434 +
        let hub_sk = SecretKey::generate();
435 +
        let hub = hub_sk.identity("oxygenhub.io", "[email protected]", 1);
436 +
        let ada_sk = SecretKey::generate();
437 +
        let ada = ada_sk.identity("Ada", "[email protected]", 1);
438 +
        let g = SignedGrant::sign(
439 +
            Grant {
440 +
                grantor: ada.hash(),
441 +
                subject: "agent:claude@ada".into(),
442 +
                issuer: hub.hash(),
443 +
                scope: Scope::CHANGES | Scope::SEAL,
444 +
                not_before: 1_000,
445 +
                expires: 2_000,
446 +
            },
447 +
            &ada,
448 +
            &ada_sk,
449 +
        );
450 +
        assert!(g.verify(&ada).is_ok());
451 +
        assert!(g.verify(&hub).is_err(), "only its author's");
452 +
        assert!(g.grant.covers(Scope::SEAL, 1_500));
453 +
        assert!(!g.grant.covers(Scope::REFS, 1_500));
454 +
        assert!(!g.grant.covers(Scope::SEAL, 2_001));
455 +
        let (_, plain) = issue(&hub_sk, &hub, 10);
456 +
        assert!(
457 +
            !matches!(cbor::decode(&plain.cert.encode()).unwrap(), Value::Map(m) if m.contains_key(&8)),
458 +
            "no grant, no field 8"
459 +
        );
460 +
        let mut with = plain.cert.clone();
461 +
        with.grant = Some(g.clone());
462 +
        let sc = SignedCertificate::sign(with, &hub, &hub_sk);
463 +
        let back = SignedCertificate::decode(&sc.encode()).unwrap();
464 +
        assert_eq!(back.cert.grant, Some(g));
465 +
        assert!(back.verify(&hub).is_ok());
466 +
        assert_ne!(back.cert.hash(), plain.cert.hash());
467 +
    }
468 +
469 +
    #[test]
470 +
    fn it_covers_its_scope_and_window_only() {
471 +
        let hub_sk = SecretKey::generate();
472 +
        let hub = hub_sk.identity("oxygenhub.io", "[email protected]", 1);
473 +
        let (_, sc) = issue(&hub_sk, &hub, 10);
474 +
        let c = &sc.cert;
475 +
        assert!(c.covers(Scope::CHANGES, 1_000) && c.covers(Scope::REFS, 1_000 + 10 * 60_000));
476 +
        assert!(!c.covers(Scope::SEAL, 2_000), "not a scope it gives");
477 +
        assert!(!c.covers(Scope::CHANGES, 999), "before it");
478 +
        assert!(
479 +
            !c.covers(Scope::CHANGES, 1_000 + 10 * 60_000 + 1),
480 +
            "after it"
481 +
        );
482 +
        assert!(!c.covers(Scope::NONE, 2_000));
483 +
        let (_, long) = issue(&hub_sk, &hub, 61);
484 +
        assert_eq!(long.verify(&hub), Err(CertError::BadLifetime));
485 +
    }
486 +
487 +
    #[test]
488 +
    fn the_proof_of_possession_is_bound_to_its_credential() {
489 +
        let sk = SecretKey::generate();
490 +
        let id = sk.identity("job", "[email protected]", 1);
491 +
        let proof = sk
492 +
            .sign_challenge(&id.hash(), &certify_challenge("flowjob_aaa"))
493 +
            .unwrap();
494 +
        assert!(
495 +
            crate::identity::verify_challenge(&id, &proof, &certify_challenge("flowjob_aaa"))
496 +
                .is_ok()
497 +
        );
498 +
        assert!(
499 +
            crate::identity::verify_challenge(&id, &proof, &certify_challenge("flowjob_bbb"))
500 +
                .is_err()
501 +
        );
502 +
    }
503 +
504 +
    #[test]
505 +
    fn a_signature_by_the_short_lived_key_names_it() {
506 +
        let hub_sk = SecretKey::generate();
507 +
        let hub = hub_sk.identity("oxygenhub.io", "[email protected]", 1);
508 +
        let (job_sk, sc) = issue(&hub_sk, &hub, 10);
509 +
        let target = ObjectRef([5; 32]);
510 +
        let sig = job_sk.sign(Domain::Change, &sc.cert.key_id(), &target);
511 +
        assert_eq!(sig.key, sc.cert.key_id());
512 +
        assert!(verify(&sc.cert.key, &sig, Domain::Change, &target).is_ok());
513 +
    }
514 +
}
modified crates/oxy-core/src/credentials.rs +122 −8
…
82 unchanged lines
83 83
    pub token: Option<String>,
84 84
    /// Extra roots to trust for this hub (a self-signed hub).
85 85
    pub ca: Option<PathBuf>,
86 +
    /// An agent's grant token for keyless signing (oxygenhub#241), when it
87 +
    /// is in the file (not with a helper).
88 +
    pub grant: Option<String>,
86 89
}
87 90
88 91
/// Where a saved token went.
…
71 unchanged lines
160 163
                entry.token = run_helper(h, "get", hub, None)?;
161 164
            }
162 165
        }
163 -
        if entry.token.is_none() && entry.account.is_none() && entry.ca.is_none() {
166 +
        if entry.token.is_none()
167 +
            && entry.account.is_none()
168 +
            && entry.ca.is_none()
169 +
            && entry.grant.is_none()
170 +
        {
164 171
            return Ok(None);
165 172
        }
166 173
        Ok(Some(entry))
…
33 unchanged lines
200 207
        Ok(stored)
201 208
    }
202 209
210 +
    /// Save an agent's grant token for `hub` (keyless signing, spec §14.7):
211 +
    /// in the file, or the helper under its own user name.
212 +
    pub fn save_grant(&self, hub: &str, token: &str) -> Result<Stored, String> {
213 +
        let mut all = self.read()?;
214 +
        let entry = all.entry(hub.to_string()).or_insert_with(|| HubEntry {
215 +
            hub: hub.to_string(),
216 +
            ..HubEntry::default()
217 +
        });
218 +
        let stored = match self.helper() {
219 +
            Some(h) => {
220 +
                run_helper_as(&h, "store", hub, GRANT_USER, Some(token))?;
221 +
                entry.grant = None;
222 +
                Stored::Helper(h)
223 +
            }
224 +
            None => {
225 +
                entry.grant = Some(token.to_string());
226 +
                Stored::File(self.path())
227 +
            }
228 +
        };
229 +
        self.write(&all)?;
230 +
        Ok(stored)
231 +
    }
232 +
233 +
    /// The agent grant token kept for `hub`, if any.
234 +
    pub fn grant(&self, hub: &str) -> Result<Option<String>, String> {
235 +
        if let Some(g) = self.read()?.get(hub).and_then(|e| e.grant.clone()) {
236 +
            return Ok(Some(g));
237 +
        }
238 +
        match self.helper() {
239 +
            Some(h) => run_helper_as(&h, "get", hub, GRANT_USER, None),
240 +
            None => Ok(None),
241 +
        }
242 +
    }
243 +
203 244
    /// Remove `hub` from the file and the helper; returns what it held,
204 245
    /// with the token, so the caller can revoke it.
205 246
    pub fn remove(&self, hub: &str) -> Result<Option<HubEntry>, String> {
…
53 unchanged lines
259 300
                "account" => e.account = Some(v),
260 301
                "token" => e.token = Some(v),
261 302
                "ca" => e.ca = Some(PathBuf::from(v)),
303 +
                "grant" => e.grant = Some(v),
262 304
                // Unknown keys are kept out of lookups, not fatal.
263 305
                _ => {}
264 306
            }
…
18 unchanged lines
283 325
            if let Some(c) = &e.ca {
284 326
                text.push_str(&format!("ca = {}\n", quote(&c.to_string_lossy())));
285 327
            }
328 +
            if let Some(g) = &e.grant {
329 +
                text.push_str(&format!("grant = {}\n", quote(g)));
330 +
            }
286 331
        }
287 332
        let path = self.path();
288 333
        std::fs::create_dir_all(&self.home)
…
21 unchanged lines
310 355
            (None, None)
311 356
        } else {
312 357
            match self.get(&hub) {
313 -
                Ok(Some(e)) => (e.token, e.ca),
358 +
                // An agent's grant token serves when the home holds no
359 +
                // other: the hub takes it for the owner's repositories
360 +
                // (keyless agents, oxygenhub#241).
361 +
                Ok(Some(e)) => (
362 +
                    e.token
363 +
                        .or(e.grant)
364 +
                        .or_else(|| self.grant(&hub).ok().flatten()),
365 +
                    e.ca,
366 +
                ),
314 367
                Ok(None) => (None, None),
315 368
                Err(e) => {
316 369
                    warn_once(&format!("warning: credentials for {hub}: {e}"));
…
17 unchanged lines
334 387
        RemoteOptions {
335 388
            token: env_token.or(token),
336 389
            ca: env_ca.or(ca),
337 -
            repo_id: None,
338 -
            cache_dir: None,
390 +
            ..RemoteOptions::default()
339 391
        }
340 392
    }
341 393
}
…
9 unchanged lines
351 403
    cache().lock().unwrap().clear();
352 404
}
353 405
354 -
fn warn_once(message: &str) {
406 +
pub(crate) fn warn_once(message: &str) {
355 407
    static SEEN: OnceLock<Mutex<std::collections::HashSet<String>>> = OnceLock::new();
356 408
    if SEEN
357 409
        .get_or_init(Default::default)
…
47 unchanged lines
405 457
    hub: &str,
406 458
    token: Option<&str>,
407 459
) -> Result<Option<String>, String> {
460 +
    run_helper_as(helper, action, hub, HELPER_USER, token)
461 +
}
462 +
463 +
/// The helper's user name for an agent's grant token, beside the hub's
464 +
/// own token.
465 +
const GRANT_USER: &str = "oxy-agent-grant";
466 +
467 +
fn run_helper_as(
468 +
    helper: &str,
469 +
    action: &str,
470 +
    hub: &str,
471 +
    user: &str,
472 +
    token: Option<&str>,
473 +
) -> Result<Option<String>, String> {
408 474
    let (protocol, host) = hub
409 475
        .split_once("://")
410 476
        .ok_or_else(|| format!("{hub} is not a hub URL"))?;
…
10 unchanged lines
421 487
        c.arg(format!("credential-{helper}")).arg(action);
422 488
        c
423 489
    };
424 -
    let mut input = format!("protocol={protocol}\nhost={host}\nusername={HELPER_USER}\n");
490 +
    let mut input = format!("protocol={protocol}\nhost={host}\nusername={user}\n");
425 491
    if let Some(t) = token {
426 492
        input.push_str(&format!("password={t}\n"));
427 493
    }
…
4 unchanged lines
432 498
        .stderr(Stdio::inherit())
433 499
        .spawn()
434 500
        .map_err(|e| format!("running credential helper {helper:?}: {e}"))?;
435 -
    child
501 +
    // A helper need not read what it is told (an `erase` may just remove
502 +
    // its file and exit); as Git does, a closed pipe is no error. Whether
503 +
    // the helper succeeded is its exit status.
504 +
    match child
436 505
        .stdin
437 506
        .take()
438 507
        .expect("piped")
439 508
        .write_all(input.as_bytes())
440 -
        .map_err(|e| format!("credential helper {helper:?}: {e}"))?;
509 +
    {
510 +
        Err(e) if e.kind() != std::io::ErrorKind::BrokenPipe => {
511 +
            return Err(format!("credential helper {helper:?}: {e}"));
512 +
        }
513 +
        _ => {}
514 +
    }
441 515
    let out = child
442 516
        .wait_with_output()
443 517
        .map_err(|e| format!("credential helper {helper:?}: {e}"))?;
…
14 unchanged lines
458 532
mod tests {
459 533
    use super::*;
460 534
535 +
    /// oxygenhub#241: an agent's grant token is kept beside the hub's own
536 +
    /// token, owner-only, and read back without disturbing it.
537 +
    #[test]
538 +
    fn a_grant_token_is_kept_beside_the_hub_token() {
539 +
        let dir = tempfile::tempdir().unwrap();
540 +
        let store = Credentials::at(dir.path());
541 +
        let hub = "https://oxygenhub.io";
542 +
        assert_eq!(store.grant(hub).unwrap(), None);
543 +
        store.save(hub, "oxyhub_t", Some("andy"), None).unwrap();
544 +
        store.save_grant(hub, "oxyagent_g").unwrap();
545 +
        assert_eq!(store.grant(hub).unwrap().as_deref(), Some("oxyagent_g"));
546 +
        let e = store.get(hub).unwrap().unwrap();
547 +
        assert_eq!(e.token.as_deref(), Some("oxyhub_t"));
548 +
        assert_eq!(e.account.as_deref(), Some("andy"));
549 +
        // A grant alone is an entry too.
550 +
        store
551 +
            .save_grant("https://elsewhere.example", "oxyagent_h")
552 +
            .unwrap();
553 +
        assert!(store.get("https://elsewhere.example").unwrap().is_some());
554 +
        // The grant is the credential a remote there is reached with;
555 +
        // beside a hub token, the hub token is.
556 +
        assert_eq!(
557 +
            store.lookup("https://elsewhere.example/a/b").0.as_deref(),
558 +
            Some("oxyagent_h")
559 +
        );
560 +
        assert_eq!(
561 +
            store.lookup("https://oxygenhub.io/a/b").0.as_deref(),
562 +
            Some("oxyhub_t")
563 +
        );
564 +
        #[cfg(unix)]
565 +
        {
566 +
            use std::os::unix::fs::PermissionsExt;
567 +
            let mode = std::fs::metadata(store.path())
568 +
                .unwrap()
569 +
                .permissions()
570 +
                .mode();
571 +
            assert_eq!(mode & 0o077, 0, "owner only");
572 +
        }
573 +
    }
574 +
461 575
    #[test]
462 576
    fn hubs_are_scheme_host_and_port() {
463 577
        assert_eq!(
…
114 unchanged lines
added crates/oxy-core/src/dsse.rs +151 −0
1 +
//! DSSE envelopes, the signature wrapper in-toto, SLSA and cosign read, and
2 +
//! the PEM public key cosign's `--key` takes: how Oxygen's attestations
3 +
//! reach tools outside it (spec §14.6, oxygenhub#239). Only the Ed25519 half
4 +
//! of a composite key signs here, since DSSE verifiers know Ed25519 and not
5 +
//! ML-DSA-65; DSSE's own pre-authentication encoding separates the domain
6 +
//! from Oxygen's signatures (which begin `oxy1 sig`).
7 +
8 +
use crate::identity::{Identity, SecretKey, VerifyError};
9 +
10 +
/// DSSE's payload type for in-toto statements.
11 +
pub const IN_TOTO: &str = "application/vnd.in-toto+json";
12 +
13 +
/// DSSE's pre-authentication encoding: what the signature covers.
14 +
pub fn pae(payload_type: &str, payload: &[u8]) -> Vec<u8> {
15 +
    let mut out = format!(
16 +
        "DSSEv1 {} {} {} ",
17 +
        payload_type.len(),
18 +
        payload_type,
19 +
        payload.len()
20 +
    )
21 +
    .into_bytes();
22 +
    out.extend_from_slice(payload);
23 +
    out
24 +
}
25 +
26 +
/// A DSSE envelope over `payload`, signed by `sk`, as JSON. `keyid` names
27 +
/// the key to a reader; DSSE gives it no meaning.
28 +
pub fn envelope(payload_type: &str, payload: &[u8], keyid: &str, sk: &SecretKey) -> String {
29 +
    let sig = sk.sign_ed25519(&pae(payload_type, payload));
30 +
    format!(
31 +
        "{{\"payloadType\":{},\"payload\":\"{}\",\"signatures\":[{{\"keyid\":{},\"sig\":\"{}\"}}]}}",
32 +
        crate::repo::json_str(payload_type),
33 +
        base64(payload),
34 +
        crate::repo::json_str(keyid),
35 +
        base64(&sig)
36 +
    )
37 +
}
38 +
39 +
/// Check one envelope signature, its parts already decoded.
40 +
pub fn verify(
41 +
    identity: &Identity,
42 +
    payload_type: &str,
43 +
    payload: &[u8],
44 +
    sig: &[u8],
45 +
) -> Result<(), VerifyError> {
46 +
    identity.verify_ed25519(&pae(payload_type, payload), sig)
47 +
}
48 +
49 +
/// The identity's Ed25519 public key as a PEM SubjectPublicKeyInfo.
50 +
pub fn public_pem(identity: &Identity) -> String {
51 +
    // SEQUENCE { SEQUENCE { OID 1.3.101.112 }, BIT STRING { key } }
52 +
    let mut der = vec![
53 +
        0x30, 0x2a, 0x30, 0x05, 0x06, 0x03, 0x2b, 0x65, 0x70, 0x03, 0x21, 0x00,
54 +
    ];
55 +
    der.extend_from_slice(&identity.ed25519_pk);
56 +
    format!(
57 +
        "-----BEGIN PUBLIC KEY-----\n{}\n-----END PUBLIC KEY-----\n",
58 +
        base64(&der)
59 +
    )
60 +
}
61 +
62 +
const ALPHABET: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
63 +
64 +
/// Standard base64 with padding, as DSSE and PEM use.
65 +
pub fn base64(bytes: &[u8]) -> String {
66 +
    let mut out = String::with_capacity(bytes.len().div_ceil(3) * 4);
67 +
    for chunk in bytes.chunks(3) {
68 +
        let n = chunk.iter().fold(0u32, |acc, b| (acc << 8) | *b as u32) << (8 * (3 - chunk.len()));
69 +
        for i in 0..4 {
70 +
            if i <= chunk.len() {
71 +
                out.push(ALPHABET[((n >> (18 - 6 * i)) & 63) as usize] as char);
72 +
            } else {
73 +
                out.push('=');
74 +
            }
75 +
        }
76 +
    }
77 +
    out
78 +
}
79 +
80 +
/// [`base64`] back to bytes; `None` for anything that is not.
81 +
pub fn base64_decode(text: &str) -> Option<Vec<u8>> {
82 +
    let text = text.trim_end_matches('=');
83 +
    let mut out = Vec::with_capacity(text.len() * 3 / 4);
84 +
    let (mut acc, mut bits) = (0u32, 0);
85 +
    for c in text.bytes() {
86 +
        let v = ALPHABET.iter().position(|a| *a == c)? as u32;
87 +
        acc = (acc << 6) | v;
88 +
        bits += 6;
89 +
        if bits >= 8 {
90 +
            bits -= 8;
91 +
            out.push((acc >> bits) as u8);
92 +
            acc &= (1 << bits) - 1;
93 +
        }
94 +
    }
95 +
    Some(out)
96 +
}
97 +
98 +
#[cfg(test)]
99 +
mod tests {
100 +
    use super::*;
101 +
102 +
    #[test]
103 +
    fn pae_and_base64_are_the_specs() {
104 +
        // DSSE's own example: PAE("http://example.com/HelloWorld", "hello world").
105 +
        assert_eq!(
106 +
            pae("http://example.com/HelloWorld", b"hello world"),
107 +
            b"DSSEv1 29 http://example.com/HelloWorld 11 hello world".to_vec()
108 +
        );
109 +
        for (plain, coded) in [
110 +
            ("", ""),
111 +
            ("f", "Zg=="),
112 +
            ("fo", "Zm8="),
113 +
            ("foo", "Zm9v"),
114 +
            ("foobar", "Zm9vYmFy"),
115 +
        ] {
116 +
            assert_eq!(base64(plain.as_bytes()), coded);
117 +
            assert_eq!(base64_decode(coded).unwrap(), plain.as_bytes());
118 +
        }
119 +
        assert!(base64_decode("Zm9v!").is_none());
120 +
    }
121 +
122 +
    #[test]
123 +
    fn an_envelope_verifies_with_its_key_and_nothing_else() {
124 +
        let sk = SecretKey::generate();
125 +
        let id = sk.identity("Ada", "[email protected]", 1);
126 +
        let other = SecretKey::generate().identity("Eve", "[email protected]", 1);
127 +
        let payload = br#"{"_type":"https://in-toto.io/Statement/v1"}"#;
128 +
        let env = envelope(IN_TOTO, payload, "k", &sk);
129 +
        let sig_b64 = env
130 +
            .rsplit("\"sig\":\"")
131 +
            .next()
132 +
            .unwrap()
133 +
            .trim_end_matches("\"}]}");
134 +
        let sig = base64_decode(sig_b64).unwrap();
135 +
        assert!(env.contains(&format!("\"payload\":\"{}\"", base64(payload))));
136 +
        assert!(verify(&id, IN_TOTO, payload, &sig).is_ok());
137 +
        assert!(verify(&other, IN_TOTO, payload, &sig).is_err());
138 +
        assert!(verify(&id, IN_TOTO, b"{}", &sig).is_err());
139 +
        assert!(verify(&id, "text/plain", payload, &sig).is_err());
140 +
    }
141 +
142 +
    #[test]
143 +
    fn the_pem_holds_the_ed25519_key() {
144 +
        let id = SecretKey::generate().identity("Ada", "[email protected]", 1);
145 +
        let pem = public_pem(&id);
146 +
        let body: String = pem.lines().filter(|l| !l.starts_with("-----")).collect();
147 +
        let der = base64_decode(&body).unwrap();
148 +
        assert_eq!(der.len(), 44);
149 +
        assert_eq!(&der[12..], &id.ed25519_pk);
150 +
    }
151 +
}
modified crates/oxy-core/src/graph.rs +77 −0
…
24 unchanged lines
25 25
    fn depends_on(&self, a: &ChangeHash, b: &ChangeHash) -> bool;
26 26
}
27 27
28 +
/// Another oracle's answers, each asked once: rendering one file asks about
29 +
/// the same few pairs over and over (a change deleting a block of lines
30 +
/// another inserted), and in a long history each answer is a walk through
31 +
/// thousands of changes (oxygen#96).
32 +
struct MemoDeps<'a> {
33 +
    inner: &'a dyn DepOracle,
34 +
    known: std::cell::RefCell<crate::fasthash::FastMap<(ChangeHash, ChangeHash), bool>>,
35 +
}
36 +
37 +
impl DepOracle for MemoDeps<'_> {
38 +
    fn depends_on(&self, a: &ChangeHash, b: &ChangeHash) -> bool {
39 +
        if let Some(v) = self.known.borrow().get(&(*a, *b)) {
40 +
            return *v;
41 +
        }
42 +
        let v = self.inner.depends_on(a, b);
43 +
        self.known.borrow_mut().insert((*a, *b), v);
44 +
        v
45 +
    }
46 +
}
47 +
28 48
/// No change depends on any other: every pair commutes.
29 49
pub struct NoDeps;
30 50
impl DepOracle for NoDeps {
…
342 unchanged lines
373 393
            .flat_map(|a| a.deleted_by.iter())
374 394
    }
375 395
396 +
    /// Whether the conflict at `m.segments[i]` is an adjacent insertion
397 +
    /// (oxygen#121): its sides only add lines between two lines that were
398 +
    /// next to each other, so keeping every side repeats nothing. It is not
399 +
    /// when a line there was deleted by a change only some sides build on:
400 +
    /// two changes that each replaced the same line conflict the same way,
401 +
    /// and keeping both would keep two versions of it. A line deleted in
402 +
    /// history every side shares does not count.
403 +
    pub fn adjacent_insertion(&self, m: &Materialized, i: usize, deps: &dyn DepOracle) -> bool {
404 +
        let Some(Segment::Conflict(sides)) = m.segments.get(i) else {
405 +
            return false;
406 +
        };
407 +
        let before = match i.checked_sub(1).map(|j| &m.segments[j]) {
408 +
            None => Vertex::Start,
409 +
            Some(Segment::Lines(a)) => match a.last() {
410 +
                Some(k) => Vertex::Atom(*k),
411 +
                None => return false,
412 +
            },
413 +
            Some(_) => return false,
414 +
        };
415 +
        let on_side: HashSet<AtomKey> = sides.iter().flatten().copied().collect();
416 +
        let side_changes: Vec<BTreeSet<ChangeHash>> = sides
417 +
            .iter()
418 +
            .map(|s| s.iter().map(|a| a.0).collect())
419 +
            .collect();
420 +
        let shared = |d: &ChangeHash| {
421 +
            side_changes
422 +
                .iter()
423 +
                .all(|s| s.iter().any(|c| deps.depends_on(c, d)))
424 +
        };
425 +
        // From the line before, through the sides' lines and deleted ones,
426 +
        // up to the next live line.
427 +
        let mut seen: HashSet<AtomKey> = HashSet::default();
428 +
        let mut todo = vec![before];
429 +
        while let Some(v) = todo.pop() {
430 +
            for &w in self.succ.get(&v).map(Edges::as_slice).unwrap_or(&[]) {
431 +
                let Vertex::Atom(k) = w else { continue };
432 +
                if !seen.insert(k) {
433 +
                    continue;
434 +
                }
435 +
                if !on_side.contains(&k) {
436 +
                    if self.is_live(&k) {
437 +
                        continue;
438 +
                    }
439 +
                    if self.deleters(&k).any(|d| !shared(d)) {
440 +
                        return false;
441 +
                    }
442 +
                }
443 +
                todo.push(w);
444 +
            }
445 +
        }
446 +
        true
447 +
    }
448 +
376 449
    /// Number of atoms introduced by `change` in this graph (for index bookkeeping).
377 450
    pub fn atom_count(&self) -> usize {
378 451
        self.atoms.len()
…
155 unchanged lines
534 607
535 608
    /// Order the live atoms and locate conflicts.
536 609
    pub fn materialize(&self, deps: &dyn DepOracle) -> Materialized {
610 +
        let deps = &MemoDeps {
611 +
            inner: deps,
612 +
            known: Default::default(),
613 +
        };
537 614
        let lines = |order: Vec<AtomKey>| Materialized {
538 615
            segments: if order.is_empty() {
539 616
                vec![]
…
1319 unchanged lines
modified crates/oxy-core/src/hash.rs +14 −1
…
63 unchanged lines
64 64
    Policy,
65 65
    State,
66 66
    Receipt,
67 +
    /// A short-lived key's certificate (oxygenhub#241).
68 +
    Certificate,
69 +
    /// An author's grant to a certified subject (§14.7).
70 +
    Grant,
71 +
    /// A tree of files as one immutable thing: an image, a package, an
72 +
    /// archive (Cylinder, docs/cylinder.md §4.1).
73 +
    Artifact,
67 74
}
68 75
69 76
impl ObjectType {
70 -
    pub const ALL: [ObjectType; 10] = [
77 +
    pub const ALL: [ObjectType; 13] = [
71 78
        ObjectType::Chunk,
72 79
        ObjectType::Blob,
73 80
        ObjectType::Change,
…
4 unchanged lines
78 85
        ObjectType::Policy,
79 86
        ObjectType::State,
80 87
        ObjectType::Receipt,
88 +
        ObjectType::Certificate,
89 +
        ObjectType::Grant,
90 +
        ObjectType::Artifact,
81 91
    ];
82 92
83 93
    pub fn name(self) -> &'static str {
…
8 unchanged lines
92 102
            ObjectType::Policy => "policy",
93 103
            ObjectType::State => "state",
94 104
            ObjectType::Receipt => "receipt",
105 +
            ObjectType::Certificate => "certificate",
106 +
            ObjectType::Grant => "grant",
107 +
            ObjectType::Artifact => "artifact",
95 108
        }
96 109
    }
97 110
…
59 unchanged lines
modified crates/oxy-core/src/identity.rs +25 −1
…
59 unchanged lines
60 60
            .map(|(_, n)| *n)
61 61
            .collect()
62 62
    }
63 -
    fn named(name: &str) -> Option<Scope> {
63 +
    pub(crate) fn named(name: &str) -> Option<Scope> {
64 64
        Self::NAMES
65 65
            .iter()
66 66
            .find(|(_, n)| *n == name)
…
226 unchanged lines
293 293
        }
294 294
    }
295 295
296 +
    /// Check a [`SecretKey::sign_ed25519`] signature by this identity.
297 +
    pub fn verify_ed25519(&self, msg: &[u8], sig: &[u8]) -> Result<(), VerifyError> {
298 +
        let sig = EdSignature::from_slice(sig).map_err(|_| VerifyError::Ed25519)?;
299 +
        self.ed_key()?
300 +
            .verify_strict(msg, &sig)
301 +
            .map_err(|_| VerifyError::Ed25519)
302 +
    }
303 +
296 304
    fn ed_key(&self) -> Result<EdVerifyingKey, VerifyError> {
297 305
        EdVerifyingKey::from_bytes(&self.ed25519_pk).map_err(|_| VerifyError::BadKey)
298 306
    }
…
12 unchanged lines
311 319
    /// A server-issued challenge, proving possession of a key to a host
312 320
    /// (hub spec §5.3, §5.4). Never a stored object.
313 321
    Challenge,
322 +
    /// A certificate an issuer gives a short-lived key (§14.7).
323 +
    Certificate,
324 +
    /// An author's grant to a certified subject (§14.7).
325 +
    Grant,
326 +
    /// A transparency log's checkpoint note (§14.8).
327 +
    Checkpoint,
314 328
}
315 329
316 330
impl Domain {
…
3 unchanged lines
320 334
            Domain::Attest => "attest",
321 335
            Domain::Policy => "policy",
322 336
            Domain::Challenge => "challenge",
337 +
            Domain::Certificate => "certificate",
338 +
            Domain::Grant => "grant",
339 +
            Domain::Checkpoint => "checkpoint",
323 340
        }
324 341
    }
325 342
}
…
153 unchanged lines
479 496
        }
480 497
    }
481 498
499 +
    /// Ed25519 alone over `msg`, for a format that separates its own
500 +
    /// domain and whose verifiers know only Ed25519 (DSSE: see
501 +
    /// [`crate::dsse`]). Oxygen's own objects are signed with [`Self::sign`].
502 +
    pub fn sign_ed25519(&self, msg: &[u8]) -> [u8; 64] {
503 +
        self.ed().sign(msg).to_bytes()
504 +
    }
505 +
482 506
    pub fn sign(&self, domain: Domain, key: &KeyId, target: &ObjectRef) -> Signature {
483 507
        let msg = signing_message(domain, key, target);
484 508
        let ed = self.ed().sign(&msg).to_bytes();
…
267 unchanged lines
modified crates/oxy-core/src/lib.rs +6 −0
…
5 unchanged lines
6 6
pub const VERSION: &str = env!("CARGO_PKG_VERSION");
7 7
8 8
pub mod alloc_stats;
9 +
pub mod approval;
10 +
pub mod artifact;
9 11
pub mod attest;
10 12
pub mod attributes;
11 13
pub mod cbor;
14 +
pub mod cert;
12 15
pub mod chunk;
13 16
pub mod credentials;
14 17
pub mod diff;
18 +
pub mod dsse;
15 19
pub mod durable;
16 20
pub mod fasthash;
17 21
pub mod flat;
…
3 unchanged lines
21 25
pub mod ids;
22 26
pub mod lanes;
23 27
pub mod lines;
28 +
pub mod mcp;
24 29
pub mod objects;
25 30
pub mod oplog;
26 31
pub mod policy;
…
8 unchanged lines
35 40
pub mod strata;
36 41
pub mod subrepo;
37 42
pub mod timing;
43 +
pub mod tlog;
38 44
pub mod tree;
added crates/oxy-core/src/mcp.rs +353 −0
1 +
//! A Model Context Protocol server over stdio (oxygen#159): JSON-RPC 2.0,
2 +
//! one message per line, as MCP's stdio transport has it. This is only the
3 +
//! protocol: `initialize`, `ping`, `tools/list` and `tools/call`, with the
4 +
//! tools themselves supplied by the caller (`oxy mcp`, and `oh mcp` in the
5 +
//! hub, which reaches this module through its pinned oxygen).
6 +
//!
7 +
//! Nothing is written to stdout but protocol messages: a tool's own
8 +
//! output goes into its result.
9 +
10 +
use serde_json::{json, Value};
11 +
use std::io::{BufRead, Write};
12 +
13 +
/// The protocol versions this server speaks, newest first. A client asking
14 +
/// for one of them gets it back; any other gets the newest.
15 +
pub const PROTOCOL_VERSIONS: &[&str] = &["2025-06-18", "2025-03-26", "2024-11-05"];
16 +
17 +
/// One tool as `tools/list` describes it.
18 +
#[derive(Clone, Debug)]
19 +
pub struct Tool {
20 +
    pub name: String,
21 +
    pub description: String,
22 +
    /// A JSON Schema object for the arguments.
23 +
    pub input_schema: Value,
24 +
    /// Reads only: no change to the repository, the remote or the hub.
25 +
    pub read_only: bool,
26 +
    /// May lose work or history (undo, abandon, deletes).
27 +
    pub destructive: bool,
28 +
}
29 +
30 +
impl Tool {
31 +
    fn describe(&self) -> Value {
32 +
        json!({
33 +
            "name": self.name,
34 +
            "description": self.description,
35 +
            "inputSchema": self.input_schema,
36 +
            "annotations": {
37 +
                "readOnlyHint": self.read_only,
38 +
                "destructiveHint": self.destructive,
39 +
            },
40 +
        })
41 +
    }
42 +
}
43 +
44 +
/// What a tool call produced: text for the model to read, the same as JSON
45 +
/// when there is some, and whether it failed. A failed tool is a result, not
46 +
/// a protocol error, so the model sees why.
47 +
#[derive(Clone, Debug, Default, PartialEq)]
48 +
pub struct ToolOutput {
49 +
    pub text: String,
50 +
    pub structured: Option<Value>,
51 +
    pub is_error: bool,
52 +
}
53 +
54 +
impl ToolOutput {
55 +
    pub fn ok(text: impl Into<String>) -> ToolOutput {
56 +
        ToolOutput {
57 +
            text: text.into(),
58 +
            structured: None,
59 +
            is_error: false,
60 +
        }
61 +
    }
62 +
63 +
    pub fn error(text: impl Into<String>) -> ToolOutput {
64 +
        ToolOutput {
65 +
            text: text.into(),
66 +
            structured: None,
67 +
            is_error: true,
68 +
        }
69 +
    }
70 +
71 +
    fn to_json(&self) -> Value {
72 +
        let mut out = json!({
73 +
            "content": [{ "type": "text", "text": self.text }],
74 +
            "isError": self.is_error,
75 +
        });
76 +
        // Structured content must be an object; an array is wrapped.
77 +
        if let Some(s) = &self.structured {
78 +
            let obj = if s.is_object() {
79 +
                s.clone()
80 +
            } else {
81 +
                json!({ "result": s })
82 +
            };
83 +
            out["structuredContent"] = obj;
84 +
        }
85 +
        out
86 +
    }
87 +
}
88 +
89 +
/// The tools a server offers.
90 +
pub trait Tools {
91 +
    fn list(&self) -> Vec<Tool>;
92 +
    /// Run a tool. `None` when no tool has that name.
93 +
    fn call(&self, name: &str, args: &Value) -> Option<ToolOutput>;
94 +
}
95 +
96 +
/// Who the server is, for `initialize`.
97 +
#[derive(Clone, Debug)]
98 +
pub struct ServerInfo {
99 +
    pub name: String,
100 +
    pub version: String,
101 +
    /// Shown to the model: what the server is for and how to use it safely.
102 +
    pub instructions: String,
103 +
}
104 +
105 +
const PARSE_ERROR: i64 = -32700;
106 +
const INVALID_REQUEST: i64 = -32600;
107 +
const METHOD_NOT_FOUND: i64 = -32601;
108 +
const INVALID_PARAMS: i64 = -32602;
109 +
110 +
fn error(id: Value, code: i64, message: &str) -> Value {
111 +
    json!({ "jsonrpc": "2.0", "id": id, "error": { "code": code, "message": message } })
112 +
}
113 +
114 +
fn result(id: Value, result: Value) -> Value {
115 +
    json!({ "jsonrpc": "2.0", "id": id, "result": result })
116 +
}
117 +
118 +
/// Answer one message; `None` for a notification (no reply is due).
119 +
pub fn handle(info: &ServerInfo, tools: &dyn Tools, message: &Value) -> Option<Value> {
120 +
    let Some(obj) = message.as_object() else {
121 +
        return Some(error(
122 +
            Value::Null,
123 +
            INVALID_REQUEST,
124 +
            "a message is a JSON object",
125 +
        ));
126 +
    };
127 +
    let method = obj.get("method").and_then(Value::as_str);
128 +
    let id = obj.get("id").cloned();
129 +
    let Some(method) = method else {
130 +
        // A response to something we never sent, or nonsense: a request
131 +
        // without a method gets an error, anything else is ignored.
132 +
        return id.map(|id| error(id, INVALID_REQUEST, "a request needs a method"));
133 +
    };
134 +
    // Notifications (initialized, cancelled, …) carry no id and get no reply.
135 +
    let id = id?;
136 +
    let params = obj.get("params").cloned().unwrap_or(Value::Null);
137 +
    Some(match method {
138 +
        "initialize" => {
139 +
            let asked = params.get("protocolVersion").and_then(Value::as_str);
140 +
            let version = asked
141 +
                .filter(|v| PROTOCOL_VERSIONS.contains(v))
142 +
                .unwrap_or(PROTOCOL_VERSIONS[0]);
143 +
            result(
144 +
                id,
145 +
                json!({
146 +
                    "protocolVersion": version,
147 +
                    "capabilities": { "tools": { "listChanged": false } },
148 +
                    "serverInfo": { "name": info.name, "version": info.version },
149 +
                    "instructions": info.instructions,
150 +
                }),
151 +
            )
152 +
        }
153 +
        "ping" => result(id, json!({})),
154 +
        "tools/list" => result(
155 +
            id,
156 +
            json!({ "tools": tools.list().iter().map(Tool::describe).collect::<Vec<_>>() }),
157 +
        ),
158 +
        "tools/call" => {
159 +
            let Some(name) = params.get("name").and_then(Value::as_str) else {
160 +
                return Some(error(
161 +
                    id,
162 +
                    INVALID_PARAMS,
163 +
                    "tools/call needs the tool's name",
164 +
                ));
165 +
            };
166 +
            let args = params
167 +
                .get("arguments")
168 +
                .cloned()
169 +
                .unwrap_or_else(|| json!({}));
170 +
            if !args.is_object() {
171 +
                return Some(error(
172 +
                    id,
173 +
                    INVALID_PARAMS,
174 +
                    "a tool's arguments are a JSON object",
175 +
                ));
176 +
            }
177 +
            match tools.call(name, &args) {
178 +
                Some(out) => result(id, out.to_json()),
179 +
                None => error(id, INVALID_PARAMS, &format!("no tool named {name}")),
180 +
            }
181 +
        }
182 +
        other => error(id, METHOD_NOT_FOUND, &format!("{other} is not supported")),
183 +
    })
184 +
}
185 +
186 +
/// Serve until the input ends: read a message per line, write a reply per
187 +
/// request. A line that is not JSON gets a parse error; blank lines are
188 +
/// skipped.
189 +
pub fn serve(
190 +
    info: &ServerInfo,
191 +
    tools: &dyn Tools,
192 +
    input: impl BufRead,
193 +
    mut output: impl Write,
194 +
) -> std::io::Result<()> {
195 +
    for line in input.lines() {
196 +
        let line = line?;
197 +
        if line.trim().is_empty() {
198 +
            continue;
199 +
        }
200 +
        let reply = match serde_json::from_str::<Value>(&line) {
201 +
            Ok(Value::Array(batch)) => {
202 +
                let replies: Vec<Value> = batch
203 +
                    .iter()
204 +
                    .filter_map(|m| handle(info, tools, m))
205 +
                    .collect();
206 +
                (!replies.is_empty()).then_some(Value::Array(replies))
207 +
            }
208 +
            Ok(message) => handle(info, tools, &message),
209 +
            Err(e) => Some(error(Value::Null, PARSE_ERROR, &format!("not JSON: {e}"))),
210 +
        };
211 +
        if let Some(reply) = reply {
212 +
            writeln!(output, "{reply}")?;
213 +
            output.flush()?;
214 +
        }
215 +
    }
216 +
    Ok(())
217 +
}
218 +
219 +
/// A string argument, if given and a string.
220 +
pub fn arg_str<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
221 +
    args.get(key).and_then(Value::as_str)
222 +
}
223 +
224 +
/// A list of strings, if given; a single string counts as a list of one.
225 +
pub fn arg_strings(args: &Value, key: &str) -> Result<Vec<String>, String> {
226 +
    match args.get(key) {
227 +
        None | Some(Value::Null) => Ok(vec![]),
228 +
        Some(Value::String(s)) => Ok(vec![s.clone()]),
229 +
        Some(Value::Array(a)) => a
230 +
            .iter()
231 +
            .map(|v| {
232 +
                v.as_str()
233 +
                    .map(str::to_string)
234 +
                    .ok_or_else(|| format!("{key} must be a list of strings"))
235 +
            })
236 +
            .collect(),
237 +
        Some(_) => Err(format!("{key} must be a list of strings")),
238 +
    }
239 +
}
240 +
241 +
#[cfg(test)]
242 +
mod tests {
243 +
    use super::*;
244 +
245 +
    struct Echo;
246 +
247 +
    impl Tools for Echo {
248 +
        fn list(&self) -> Vec<Tool> {
249 +
            vec![Tool {
250 +
                name: "echo".into(),
251 +
                description: "says it back".into(),
252 +
                input_schema: json!({"type": "object"}),
253 +
                read_only: true,
254 +
                destructive: false,
255 +
            }]
256 +
        }
257 +
        fn call(&self, name: &str, args: &Value) -> Option<ToolOutput> {
258 +
            (name == "echo").then(|| ToolOutput {
259 +
                text: args.to_string(),
260 +
                structured: Some(json!([1, 2])),
261 +
                is_error: false,
262 +
            })
263 +
        }
264 +
    }
265 +
266 +
    fn info() -> ServerInfo {
267 +
        ServerInfo {
268 +
            name: "t".into(),
269 +
            version: "1".into(),
270 +
            instructions: "be careful".into(),
271 +
        }
272 +
    }
273 +
274 +
    fn talk(lines: &str) -> Vec<Value> {
275 +
        let mut out = Vec::new();
276 +
        serve(&info(), &Echo, lines.as_bytes(), &mut out).unwrap();
277 +
        String::from_utf8(out)
278 +
            .unwrap()
279 +
            .lines()
280 +
            .map(|l| serde_json::from_str(l).unwrap())
281 +
            .collect()
282 +
    }
283 +
284 +
    #[test]
285 +
    fn a_session_initializes_lists_and_calls() {
286 +
        let r = talk(concat!(
287 +
            r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"c","version":"0"}}}"#,
288 +
            "\n",
289 +
            r#"{"jsonrpc":"2.0","method":"notifications/initialized"}"#,
290 +
            "\n\n",
291 +
            r#"{"jsonrpc":"2.0","id":2,"method":"tools/list"}"#,
292 +
            "\n",
293 +
            r#"{"jsonrpc":"2.0","id":"three","method":"tools/call","params":{"name":"echo","arguments":{"a":1}}}"#,
294 +
            "\n",
295 +
        ));
296 +
        assert_eq!(r.len(), 3, "the notification gets no reply: {r:?}");
297 +
        assert_eq!(
298 +
            r[0]["result"]["protocolVersion"], "2025-03-26",
299 +
            "a version we speak is kept"
300 +
        );
301 +
        assert_eq!(r[0]["result"]["instructions"], "be careful");
302 +
        assert_eq!(r[1]["result"]["tools"][0]["name"], "echo");
303 +
        assert_eq!(
304 +
            r[1]["result"]["tools"][0]["annotations"]["readOnlyHint"],
305 +
            true
306 +
        );
307 +
        assert_eq!(r[2]["id"], "three");
308 +
        assert_eq!(r[2]["result"]["content"][0]["text"], r#"{"a":1}"#);
309 +
        assert_eq!(
310 +
            r[2]["result"]["structuredContent"],
311 +
            json!({"result": [1, 2]}),
312 +
            "an array is wrapped in an object"
313 +
        );
314 +
        assert_eq!(r[2]["result"]["isError"], false);
315 +
    }
316 +
317 +
    #[test]
318 +
    fn errors_are_protocol_errors_and_unknown_versions_get_the_newest() {
319 +
        let r = talk(concat!(
320 +
            "not json\n",
321 +
            r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"1999-01-01"}}"#,
322 +
            "\n",
323 +
            r#"{"jsonrpc":"2.0","id":2,"method":"resources/list"}"#,
324 +
            "\n",
325 +
            r#"{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"nope"}}"#,
326 +
            "\n",
327 +
            r#"{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"echo","arguments":[1]}}"#,
328 +
            "\n",
329 +
            r#"[{"jsonrpc":"2.0","id":5,"method":"ping"},{"jsonrpc":"2.0","method":"notifications/cancelled"}]"#,
330 +
            "\n",
331 +
        ));
332 +
        assert_eq!(r[0]["error"]["code"], PARSE_ERROR);
333 +
        assert_eq!(r[1]["result"]["protocolVersion"], PROTOCOL_VERSIONS[0]);
334 +
        assert_eq!(r[2]["error"]["code"], METHOD_NOT_FOUND);
335 +
        assert_eq!(r[3]["error"]["code"], INVALID_PARAMS);
336 +
        assert_eq!(r[4]["error"]["code"], INVALID_PARAMS);
337 +
        assert_eq!(
338 +
            r[5],
339 +
            json!([{"jsonrpc": "2.0", "id": 5, "result": {}}]),
340 +
            "a batch is answered as a batch"
341 +
        );
342 +
    }
343 +
344 +
    #[test]
345 +
    fn string_arguments_and_lists() {
346 +
        let a = json!({"one": "x", "many": ["a", "b"], "bad": [1]});
347 +
        assert_eq!(arg_str(&a, "one"), Some("x"));
348 +
        assert_eq!(arg_strings(&a, "one").unwrap(), vec!["x"]);
349 +
        assert_eq!(arg_strings(&a, "many").unwrap(), vec!["a", "b"]);
350 +
        assert_eq!(arg_strings(&a, "missing").unwrap(), Vec::<String>::new());
351 +
        assert!(arg_strings(&a, "bad").is_err());
352 +
    }
353 +
}
modified crates/oxy-core/src/objects.rs +4 −0
…
44 unchanged lines
45 45
    Cbor(#[from] cbor::CborError),
46 46
    #[error("schema: {0}")]
47 47
    Schema(String),
48 +
    /// A cache this repository derived and can derive again (a state
49 +
    /// index) does not match the hash it was written with.
50 +
    #[error("damaged index: {0}")]
51 +
    DamagedIndex(String),
48 52
}
49 53
50 54
pub(crate) fn schema<T>(msg: impl Into<String>) -> Result<T, ObjectError> {
…
1624 unchanged lines
modified crates/oxy-core/src/oplog.rs +33 −2
…
40 unchanged lines
41 41
    pub hlc: Hlc,
42 42
    pub command: String,
43 43
    pub view: View,
44 +
    /// The workspace whose command recorded it (oxygen#99); absent in
45 +
    /// operations written before it was kept.
46 +
    pub workspace: Option<WorkspaceId>,
47 +
    /// For an undo that reverted an older operation in place: that
48 +
    /// operation, so a further undo goes on past it.
49 +
    pub undoes: Option<ObjectRef>,
44 50
}
45 51
46 52
impl View {
…
253 unchanged lines
300 306
    }
301 307
302 308
    pub fn to_value(&self) -> Value {
303 -
        Value::map([
309 +
        let mut fields = vec![
304 310
            (0, Value::Uint(FORMAT_VERSION)),
305 311
            (
306 312
                1,
…
7 unchanged lines
314 320
            (2, self.hlc.to_value()),
315 321
            (3, Value::Text(self.command.clone())),
316 322
            (4, self.view.to_value()),
317 -
        ])
323 +
        ];
324 +
        // Absent when unknown, so older operations keep their encoding.
325 +
        if let Some(w) = &self.workspace {
326 +
            fields.push((5, Value::Bytes(w.0.to_vec())));
327 +
        }
328 +
        if let Some(u) = &self.undoes {
329 +
            fields.push((6, Value::Bytes(u.0.to_vec())));
330 +
        }
331 +
        Value::map(fields)
318 332
    }
319 333
    pub fn encode(&self) -> Vec<u8> {
320 334
        cbor::encode(&self.to_value())
…
15 unchanged lines
336 350
            hlc: Hlc::from_value(field(m, 2, "hlc")?)?,
337 351
            command: text(field(m, 3, "command")?, "command")?.to_owned(),
338 352
            view: View::from_value(field(m, 4, "view")?)?,
353 +
            workspace: match m.get(&5) {
354 +
                Some(v) => Some(
355 +
                    WorkspaceId::from_slice(bytes(v, "workspace")?).ok_or_else(|| {
356 +
                        ObjectError::Schema("workspace: expected 16 bytes".into())
357 +
                    })?,
358 +
                ),
359 +
                None => None,
360 +
            },
361 +
            undoes: m.get(&6).map(|v| oref(v, "undoes")).transpose()?,
339 362
        })
340 363
    }
341 364
…
41 unchanged lines
383 406
            },
384 407
            command: "oxy commit".into(),
385 408
            view,
409 +
            workspace: None,
410 +
            undoes: None,
386 411
        };
387 412
        let d = Op::decode(&op.encode()).unwrap();
388 413
        assert_eq!(d, op);
414 +
        // The workspace and an undo's target travel when known.
415 +
        let mut known = op.clone();
416 +
        known.workspace = Some(WorkspaceId([3; 16]));
417 +
        known.undoes = Some(ObjectRef([1; 32]));
418 +
        assert_eq!(Op::decode(&known.encode()).unwrap(), known);
419 +
        assert_ne!(known.hash(), op.hash());
389 420
        assert_eq!(d.hash(), Op::decode(&d.encode()).unwrap().hash());
390 421
        // A format-1 op does not decode as format 2, but does through the migration reader.
391 422
        let mut v1 = Format1View::default();
…
18 unchanged lines
modified crates/oxy-core/src/policy.rs +162 −4
…
53 unchanged lines
54 54
    pub threshold: u32,
55 55
    pub require: Require,
56 56
    pub allow_conflicts: bool,
57 +
    /// Who else may sign, by what an issuer certified (§14.7): a short-lived
58 +
    /// key whose certificate is the issuer's, for a subject the pattern
59 +
    /// matches, counts as one signer within the scope given.
60 +
    pub subjects: Vec<SubjectGrant>,
57 61
}
58 62
63 +
/// A rule's grant to the holders of certificates an issuer gives (§14.7):
64 +
/// `repo:andy/demo:bookmark:main`, `agent:claude@andy`, with `*` as in
65 +
/// reference patterns.
59 66
#[derive(Clone, Debug, PartialEq, Eq)]
67 +
pub struct SubjectGrant {
68 +
    pub pattern: String,
69 +
    pub issuer: KeyId,
70 +
    pub scope: identity::Scope,
71 +
}
72 +
73 +
impl SubjectGrant {
74 +
    fn to_value(&self) -> Value {
75 +
        Value::map([
76 +
            (0, Value::Text(self.pattern.clone())),
77 +
            (1, Value::Bytes(self.issuer.0.to_vec())),
78 +
            (
79 +
                2,
80 +
                Value::Array(
81 +
                    self.scope
82 +
                        .names()
83 +
                        .into_iter()
84 +
                        .map(|n| Value::Text(n.to_string()))
85 +
                        .collect(),
86 +
                ),
87 +
            ),
88 +
        ])
89 +
    }
90 +
91 +
    fn from_value(v: &Value) -> Result<SubjectGrant, ObjectError> {
92 +
        let m = map(v, "subject")?;
93 +
        let mut scope = identity::Scope::NONE;
94 +
        for n in array(field(m, 2, "subject scope")?, "subject scope")? {
95 +
            let n = text(n, "subject scope")?;
96 +
            scope = scope
97 +
                | identity::Scope::named(n)
98 +
                    .ok_or_else(|| ObjectError::Schema(format!("subject: unknown scope {n:?}")))?;
99 +
        }
100 +
        Ok(SubjectGrant {
101 +
            pattern: text(field(m, 0, "subject pattern")?, "subject pattern")?.to_owned(),
102 +
            issuer: oref(field(m, 1, "subject issuer")?, "subject issuer")?,
103 +
            scope,
104 +
        })
105 +
    }
106 +
}
107 +
108 +
#[derive(Clone, Debug, PartialEq, Eq)]
60 109
pub struct Policy {
61 110
    pub repo: [u8; 16],
62 111
    pub serial: u64,
…
50 unchanged lines
113 162
                    self.rules
114 163
                        .iter()
115 164
                        .map(|r| {
116 -
                            Value::map([
165 +
                            let mut v = Value::map([
117 166
                                (0, Value::Text(r.pattern.clone())),
118 167
                                (1, Self::keys_value(&r.keys)),
119 168
                                (2, Value::Uint(r.threshold as u64)),
120 169
                                (3, Value::Uint(r.require.tag())),
121 170
                                (4, Value::Bool(r.allow_conflicts)),
122 -
                            ])
171 +
                            ]);
172 +
                            // Only when there are any, so a policy without
173 +
                            // them encodes, and hashes, as before.
174 +
                            if !r.subjects.is_empty() {
175 +
                                if let Value::Map(m) = &mut v {
176 +
                                    m.insert(
177 +
                                        5,
178 +
                                        Value::Array(
179 +
                                            r.subjects.iter().map(SubjectGrant::to_value).collect(),
180 +
                                        ),
181 +
                                    );
182 +
                                }
183 +
                            }
184 +
                            v
123 185
                        })
124 186
                        .collect(),
125 187
                ),
…
36 unchanged lines
162 224
                allow_conflicts: field(rm, 4, "allow_conflicts")?
163 225
                    .as_bool()
164 226
                    .ok_or_else(|| ObjectError::Schema("allow_conflicts: expected bool".into()))?,
227 +
                subjects: match rm.get(&5) {
228 +
                    None => vec![],
229 +
                    Some(v) => array(v, "subjects")?
230 +
                        .iter()
231 +
                        .map(SubjectGrant::from_value)
232 +
                        .collect::<Result<_, _>>()?,
233 +
                },
165 234
            });
166 235
        }
167 236
        let identities = array(field(m, 5, "identities")?, "identities")?
…
18 unchanged lines
186 255
            ));
187 256
        }
188 257
        for r in &p.rules {
189 -
            if r.threshold == 0 || r.threshold as usize > r.keys.len() {
258 +
            if r.threshold == 0 || r.threshold as usize > r.keys.len() + r.subjects.len() {
190 259
                return Err(ObjectError::Schema(format!(
191 -
                    "rule {:?}: threshold must be between 1 and the number of keys",
260 +
                    "rule {:?}: threshold must be between 1 and the number of keys and subjects",
192 261
                    r.pattern
193 262
                )));
194 263
            }
…
20 unchanged lines
215 284
            .find(|r| glob_matches(&r.pattern, reference))
216 285
    }
217 286
287 +
    /// Whether the rule for `reference` lets a certificate from `issuer`
288 +
    /// for `subject` sign within `scope` (§14.7). The certificate itself,
289 +
    /// and the time it is checked at, are the caller's to verify.
290 +
    pub fn subject_grant(
291 +
        &self,
292 +
        reference: &str,
293 +
        issuer: &KeyId,
294 +
        subject: &str,
295 +
        scope: identity::Scope,
296 +
    ) -> bool {
297 +
        !self.is_revoked(issuer)
298 +
            && self.rule_for(reference).is_some_and(|r| {
299 +
                r.subjects.iter().any(|g| {
300 +
                    g.issuer == *issuer && g.scope.has(scope) && glob_matches(&g.pattern, subject)
301 +
                })
302 +
            })
303 +
    }
304 +
218 305
    /// Whether `key` may update `reference`: listed in the matching rule and not revoked.
219 306
    pub fn authorizes(&self, key: &KeyId, reference: &str) -> bool {
220 307
        !self.is_revoked(key)
…
71 unchanged lines
292 379
                threshold: 1,
293 380
                require: Require::Any,
294 381
                allow_conflicts: true,
382 +
                subjects: vec![],
295 383
            }],
296 384
            identities: vec![root],
297 385
            revoked: vec![],
…
81 unchanged lines
379 467
        assert!(glob_matches("", ""));
380 468
    }
381 469
470 +
    /// Subjects an issuer certifies (oxygenhub#241): kept through
471 +
    /// encoding, absent from a policy without them, and granted only for
472 +
    /// their issuer, pattern and scope, and not when the issuer is revoked.
382 473
    #[test]
474 +
    fn a_rule_grants_certified_subjects() {
475 +
        use crate::identity::Scope;
476 +
        let root = sk(1).identity("Root", "r@x", 0);
477 +
        let hub = sk(3).identity("Hub", "h@x", 0);
478 +
        let mut p = Policy::single_root([9; 16], root.clone(), 10);
479 +
        let plain = p.encode();
480 +
        p.rules[0].subjects.push(SubjectGrant {
481 +
            pattern: "repo:andy/demo:bookmark:main".into(),
482 +
            issuer: hub.hash(),
483 +
            scope: Scope::REFS | Scope::CHANGES,
484 +
        });
485 +
        p.rules[0].subjects.push(SubjectGrant {
486 +
            pattern: "agent:*@andy".into(),
487 +
            issuer: hub.hash(),
488 +
            scope: Scope::CHANGES,
489 +
        });
490 +
        let back = Policy::decode(&p.encode()).unwrap();
491 +
        assert_eq!(back, p);
492 +
        let mut again = back.clone();
493 +
        again.rules[0].subjects.clear();
494 +
        assert_eq!(
495 +
            again.encode(),
496 +
            plain,
497 +
            "no subjects, the same bytes as before"
498 +
        );
499 +
500 +
        let main = "repo:andy/demo:bookmark:main";
501 +
        assert!(p.subject_grant("main", &hub.hash(), main, Scope::REFS));
502 +
        assert!(p.subject_grant("main", &hub.hash(), "agent:claude@andy", Scope::CHANGES));
503 +
        assert!(
504 +
            !p.subject_grant("main", &hub.hash(), "agent:claude@andy", Scope::REFS),
505 +
            "not its scope"
506 +
        );
507 +
        assert!(
508 +
            !p.subject_grant(
509 +
                "main",
510 +
                &hub.hash(),
511 +
                "repo:andy/demo:bookmark:dev",
512 +
                Scope::REFS
513 +
            ),
514 +
            "not its pattern"
515 +
        );
516 +
        assert!(
517 +
            !p.subject_grant("main", &root.hash(), main, Scope::REFS),
518 +
            "not its issuer"
519 +
        );
520 +
        p.revoked.push(hub.hash());
521 +
        assert!(
522 +
            !p.subject_grant("main", &hub.hash(), main, Scope::REFS),
523 +
            "a revoked issuer grants nothing"
524 +
        );
525 +
526 +
        // A rule of subjects alone is a rule.
527 +
        let mut only = Policy::single_root([9; 16], root, 10);
528 +
        only.rules[0].keys.clear();
529 +
        only.rules[0].subjects.push(SubjectGrant {
530 +
            pattern: "*".into(),
531 +
            issuer: hub.hash(),
532 +
            scope: Scope::REFS,
533 +
        });
534 +
        assert!(Policy::decode(&only.encode()).is_ok());
535 +
        only.rules[0].subjects.clear();
536 +
        assert!(Policy::decode(&only.encode()).is_err(), "nobody to sign");
537 +
    }
538 +
539 +
    #[test]
383 540
    fn round_trip_and_rules() {
384 541
        let root = sk(1).identity("Root", "r@x", 0);
385 542
        let dev = sk(2).identity("Dev", "d@x", 0);
…
6 unchanged lines
392 549
                threshold: 1,
393 550
                require: Require::Signed,
394 551
                allow_conflicts: false,
552 +
                subjects: vec![],
395 553
            },
396 554
        );
397 555
        p.rules[1].keys.push(dev.hash());
…
54 unchanged lines
modified crates/oxy-core/src/portable.rs +65 −0
…
99 unchanged lines
100 100
    out
101 101
}
102 102
103 +
/// `path.canonicalize()`, but on Windows without the verbatim `\\?\`
104 +
/// prefix when the plain form names the same file (oxygen#143): a root
105 +
/// shown as `\\?\C:\work\demo` in every message otherwise.
106 +
pub fn canonical(path: &std::path::Path) -> std::io::Result<std::path::PathBuf> {
107 +
    path.canonicalize().map(plain)
108 +
}
109 +
110 +
/// [`canonical`]'s rule alone: `\\?\C:\…` as `C:\…` and `\\?\UNC\…` as
111 +
/// `\\…`, when the result is under Windows' old length limit and has no
112 +
/// component the plain form would read differently (`.`, `..`, a trailing
113 +
/// dot or space, a reserved name). Anything else is left as it is.
114 +
pub fn plain(p: std::path::PathBuf) -> std::path::PathBuf {
115 +
    let Some(s) = p.to_str() else { return p };
116 +
    let unchanged_by_plain = |rest: &str| {
117 +
        rest.len() < 260
118 +
            && !rest.split('\\').any(|c| {
119 +
                c == "."
120 +
                    || c == ".."
121 +
                    || c.ends_with('.')
122 +
                    || c.ends_with(' ')
123 +
                    || (!c.is_empty() && windows_name_problem(c).is_some())
124 +
            })
125 +
    };
126 +
    if let Some(rest) = s.strip_prefix(r"\\?\UNC\") {
127 +
        if unchanged_by_plain(rest) {
128 +
            return std::path::PathBuf::from(format!(r"\\{rest}"));
129 +
        }
130 +
    } else if let Some(rest) = s.strip_prefix(r"\\?\") {
131 +
        let b = rest.as_bytes();
132 +
        let drive = b.len() >= 3 && b[0].is_ascii_alphabetic() && b[1] == b':' && b[2] == b'\\';
133 +
        if drive && unchanged_by_plain(&rest[3..]) {
134 +
            return std::path::PathBuf::from(rest);
135 +
        }
136 +
    }
137 +
    p
138 +
}
139 +
103 140
/// Why Windows would refuse a path, if it would.
104 141
pub fn windows_name_problem(path: &str) -> Option<&'static str> {
105 142
    const RESERVED: &[&str] = &[
…
22 unchanged lines
128 165
mod tests {
129 166
    use super::*;
130 167
168 +
    /// oxygen#143: Windows' verbatim prefix is dropped where the plain path
169 +
    /// says the same, and kept where it would not.
170 +
    #[test]
171 +
    fn a_verbatim_windows_path_is_shown_plain_when_it_can_be() {
172 +
        use std::path::PathBuf;
173 +
        let p = |s: &str| plain(PathBuf::from(s)).to_string_lossy().into_owned();
174 +
        assert_eq!(p(r"\\?\C:\work\demo"), r"C:\work\demo");
175 +
        assert_eq!(p(r"\\?\UNC\server\share\repo"), r"\\server\share\repo");
176 +
        assert_eq!(
177 +
            p(r"\\?\C:\work\CON"),
178 +
            r"\\?\C:\work\CON",
179 +
            "a reserved name stays verbatim"
180 +
        );
181 +
        assert_eq!(p(r"\\?\C:\work\trailing."), r"\\?\C:\work\trailing.");
182 +
        assert_eq!(
183 +
            p(r"\\?\Volume{1234}\x"),
184 +
            r"\\?\Volume{1234}\x",
185 +
            "not a drive path"
186 +
        );
187 +
        let long = format!(r"\\?\C:\{}", "a".repeat(300));
188 +
        assert_eq!(p(&long), long, "past the old length limit");
189 +
        assert_eq!(
190 +
            p("/home/andy/repo"),
191 +
            "/home/andy/repo",
192 +
            "nothing to do elsewhere"
193 +
        );
194 +
    }
195 +
131 196
    #[test]
132 197
    fn collisions_and_names() {
133 198
        assert_eq!(
…
24 unchanged lines
modified crates/oxy-core/src/protocol.rs +613 −23
…
55 unchanged lines
56 56
    pub history: Vec<SignedPolicy>,
57 57
    /// The bookmark a clone checks out (spec §16.1), when the remote names one.
58 58
    pub default_bookmark: Option<String>,
59 +
    /// The host's latest transparency-log checkpoint, a signed note (spec
60 +
    /// §14.8), against which `RemoteRef::logged` proofs are made (key 6,
61 +
    /// only when the host keeps a log).
62 +
    pub log: Option<String>,
63 +
    /// The witnesses the host asks to cosign its checkpoints: each one's
64 +
    /// name in cosignatures and its identity's bytes (key 7, only when it
65 +
    /// has any), which a client pins (spec §14.8, *Integrated time*).
66 +
    pub witnesses: Vec<(String, Vec<u8>)>,
67 +
    /// Tags withdrawn here (spec §10.1; key 8, only when there are any):
68 +
    /// each with its `untag` attestation as `attestation`, and that
69 +
    /// attestation's acceptance as `accepted` when a short-lived key
70 +
    /// signed it. Older clients ignore them and keep the tag they have.
71 +
    pub withdrawn: Vec<TagRef>,
72 +
}
73 +
74 +
/// Where a reference attestation sits in the host's transparency log
75 +
/// (spec §14.8): its entry number, the entry, and the inclusion proof
76 +
/// against the checkpoint served beside it.
77 +
#[derive(Clone, Debug, PartialEq, Eq)]
78 +
pub struct Logged {
79 +
    pub index: u64,
80 +
    pub entry: Vec<u8>,
81 +
    pub proof: Vec<[u8; 32]>,
82 +
}
83 +
84 +
/// A checkpoint witnesses cosigned that includes a logged entry (spec
85 +
/// §14.8, *Integrated time*): the cosigned note, and the entry's inclusion
86 +
/// proof against it.
87 +
#[derive(Clone, Debug, PartialEq, Eq)]
88 +
pub struct Integration {
89 +
    pub note: String,
90 +
    pub proof: Vec<[u8; 32]>,
59 91
}
60 92
61 93
/// A tag as a remote reports it (spec §10.1).
…
3 unchanged lines
65 97
    pub state: ObjectRef,
66 98
    pub heads: Vec<ChangeHash>,
67 99
    pub attestation: Option<SignedAttest>,
100 +
    /// The acceptance of `attestation` by its signer's certificate issuer
101 +
    /// when a short-lived key signed it (spec §14.7; key 4, only when there
102 +
    /// is one): the time a fetch checks the certificate at.
103 +
    pub accepted: Option<SignedAttest>,
104 +
    /// The attestation's place in the host's transparency log (key 5,
105 +
    /// only when it has one), as a bookmark's.
106 +
    pub logged: Option<Logged>,
107 +
    /// The cosigned checkpoints that first included `logged`'s entry (key
108 +
    /// 6, only when there are any).
109 +
    pub integration: Vec<Integration>,
110 +
}
111 +
112 +
impl TagRef {
113 +
    /// Whether this is a tag's withdrawal (its attestation an `untag`), as
114 +
    /// a push carries one (oxygenhub#301).
115 +
    pub fn is_withdrawal(&self) -> bool {
116 +
        self.attestation
117 +
            .as_ref()
118 +
            .is_some_and(|a| matches!(a.attest.body, crate::attest::AttestBody::Untag { .. }))
119 +
    }
120 +
121 +
    /// A tag with no acceptance or log place (yet).
122 +
    pub fn new(
123 +
        name: String,
124 +
        state: ObjectRef,
125 +
        heads: Vec<ChangeHash>,
126 +
        attestation: Option<SignedAttest>,
127 +
    ) -> TagRef {
128 +
        TagRef {
129 +
            name,
130 +
            state,
131 +
            heads,
132 +
            attestation,
133 +
            accepted: None,
134 +
            logged: None,
135 +
            integration: vec![],
136 +
        }
137 +
    }
68 138
}
69 139
70 140
/// One reference as a remote reports it.
…
8 unchanged lines
79 149
    /// Serial of the latest attestation ever recorded, stale or not, so the
80 150
    /// next attestation continues the sequence.
81 151
    pub last_serial: Option<u64>,
152 +
    /// The host's acceptance of `attestation` when a short-lived key signed
153 +
    /// it (spec §14.7): the time its certificate is checked at.
154 +
    pub accepted: Option<SignedAttest>,
155 +
    /// The attestation's place in the host's transparency log (key 6, only
156 +
    /// when it has one).
157 +
    pub logged: Option<Logged>,
158 +
    /// The cosigned checkpoints that first included `logged`'s entry (key
159 +
    /// 7, only when there are any): what its integrated time is read from.
160 +
    pub integration: Vec<Integration>,
82 161
}
83 162
84 163
/// Everything needed to add a set of changes to a repository.
…
16 unchanged lines
101 180
    /// fetch between repositories on one machine: their text blobs and
102 181
    /// chunks are among `objects`. Never on the wire.
103 182
    pub raw_changes: Vec<(Vec<u8>, Vec<Signature>)>,
183 +
    /// Certificates of short-lived keys that signed what the bundle carries
184 +
    /// (spec §14.7), their issuers' identities among `objects`.
185 +
    pub certificates: Vec<crate::cert::SignedCertificate>,
104 186
}
105 187
106 188
impl Bundle {
…
33 unchanged lines
140 222
    /// A checkpoint of exactly `heads` (spec §16.1 `checkpoint`): the
141 223
    /// checkpoint object, its payload blob, and the blob's chunks.
142 224
    fn checkpoint(&self, heads: &[ChangeHash]) -> Result<Bundle>;
225 +
    /// A checkpoint of exactly `heads` without its graph blobs (spec §16.1
226 +
    /// `checkpoint-index`, oxygen#137), which the client then asks for in
227 +
    /// parts with `objects`. A remote that cannot answer it sends the
228 +
    /// whole checkpoint, which does as well.
229 +
    fn checkpoint_index(&self, heads: &[ChangeHash]) -> Result<Bundle> {
230 +
        self.checkpoint(heads)
231 +
    }
143 232
    /// The changes within `depth` dependency steps of `heads` (spec §17).
144 233
    fn shallow(&self, heads: &[ChangeHash], depth: u32) -> Result<Vec<ChangeHash>>;
145 234
    /// Push changes, bookmark updates, new tags, and optionally a newer
…
27 unchanged lines
173 262
    }
174 263
}
175 264
265 +
// Bundles a local remote served on this thread, for tests that count
266 +
// round trips (oxygen#158).
267 +
#[cfg(test)]
268 +
thread_local! {
269 +
    pub(crate) static LOCAL_FETCHES: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
270 +
}
271 +
176 272
impl Remote for LocalRemote {
177 273
    fn repo_id(&self) -> Result<[u8; 16]> {
178 274
        self.repo.id()
…
5 unchanged lines
184 280
        self.repo.negotiate(have, want)
185 281
    }
186 282
    fn fetch(&self, changes: &[ChangeHash], max_blob: Option<u64>) -> Result<Bundle> {
283 +
        #[cfg(test)]
284 +
        LOCAL_FETCHES.with(|n| n.set(n.get() + 1));
187 285
        self.repo.bundle_raw_for(changes, max_blob)
188 286
    }
189 287
    fn objects(&self, refs: &[ObjectRef]) -> Result<Vec<(ObjectType, Vec<u8>)>> {
…
2 unchanged lines
192 290
    fn checkpoint(&self, heads: &[ChangeHash]) -> Result<Bundle> {
193 291
        self.repo.checkpoint_bundle_for(heads)
194 292
    }
293 +
    fn checkpoint_index(&self, heads: &[ChangeHash]) -> Result<Bundle> {
294 +
        self.repo.checkpoint_index_for(heads)
295 +
    }
195 296
    fn shallow(&self, heads: &[ChangeHash], depth: u32) -> Result<Vec<ChangeHash>> {
196 297
        self.repo.shallow_changes(heads, depth)
197 298
    }
…
37 unchanged lines
235 336
    /// `.oxy/remotes`. None (a clone, before there is a repository) means
236 337
    /// a temporary directory.
237 338
    pub cache_dir: Option<PathBuf>,
339 +
    /// Other tokens for the same hub, tried in turn when `token` is
340 +
    /// refused (401, 403 or 404): the credential store's sign-in when
341 +
    /// `token` is the remote's own (oxygen#155), or those of enclosing
342 +
    /// checkouts when it came from the credential store (oxygen#112).
343 +
    pub fallback_tokens: Vec<String>,
344 +
    /// `token` is the remote's own: fall back only when the hub says it is
345 +
    /// not a token at all (HTTP 401: expired or unknown), never when it is
346 +
    /// one that may not do this (403, 404), so a narrowly scoped token is
347 +
    /// not quietly swapped for a wider sign-in (oxygen#155).
348 +
    pub fallback_on_401_only: bool,
238 349
}
239 350
240 351
/// Opens a store for a URL it understands, or passes (`None`). Stores that
…
32 unchanged lines
273 384
/// The id of a hosted repository that has not adopted one yet.
274 385
pub const UNBOUND_ID: [u8; 16] = [0; 16];
275 386
387 +
/// The request header in which a client declares the protocol features it
388 +
/// understands beyond the 0.5.0-era set: a comma-separated list, which a
389 +
/// server may tailor its answers to (docs/protocol.md, *Client
390 +
/// capabilities*). A request without it is a client of that era.
391 +
pub const FEATURES_HEADER: &str = "Oxy-Features";
392 +
393 +
/// The feature of a client that authorizes a tag signed by a short-lived,
394 +
/// certified key by its certificate at the time the host accepted it
395 +
/// (oxygen#129). A client without it refuses such a tag, and with it the
396 +
/// whole fetch, so servers leave those tags out of its `refs` (oxygen#131).
397 +
pub const FEATURE_CERTIFIED_TAGS: &str = "certified-tags";
398 +
399 +
/// The features this client declares in [`FEATURES_HEADER`].
400 +
pub const CLIENT_FEATURES: &[&str] = &[FEATURE_CERTIFIED_TAGS];
401 +
402 +
/// The `User-Agent` every request of this client sends: `oxy/<version>`.
403 +
pub fn user_agent() -> String {
404 +
    format!("oxy/{}", env!("CARGO_PKG_VERSION"))
405 +
}
406 +
407 +
/// Whether a [`FEATURES_HEADER`] value (absent: `None`) declares `feature`.
408 +
/// Entries are comma-separated, compared without surrounding blanks and
409 +
/// regardless of case; unknown ones are ignored.
410 +
pub fn declares_feature(header: Option<&str>, feature: &str) -> bool {
411 +
    header.is_some_and(|h| h.split(',').any(|f| f.trim().eq_ignore_ascii_case(feature)))
412 +
}
413 +
276 414
/// A remote served by `oxy-server` at `<base>/oxy/v1/...`.
277 415
pub struct HttpRemote {
278 416
    base: String,
279 -
    token: Option<String>,
417 +
    /// The token in use, and those still to try when it is refused.
418 +
    token: Mutex<Option<String>>,
419 +
    fallbacks: Mutex<Vec<String>>,
420 +
    fallback_on_401_only: bool,
280 421
    repo_id: Option<[u8; 16]>,
281 422
    agent: ureq::Agent,
282 423
}
…
36 unchanged lines
319 460
        refuse_plain_token(base, options)?;
320 461
        let agent = ureq::AgentBuilder::new()
321 462
            .tls_config(tls_client_config(options.ca.as_deref())?)
463 +
            .user_agent(&user_agent())
322 464
            .build();
323 465
        Ok(HttpRemote {
324 466
            base: base.trim_end_matches('/').to_string(),
325 -
            token: options.token.clone(),
467 +
            token: Mutex::new(options.token.clone()),
468 +
            fallbacks: Mutex::new(options.fallback_tokens.iter().rev().cloned().collect()),
469 +
            fallback_on_401_only: options.fallback_on_401_only,
326 470
            repo_id: options.repo_id,
327 471
            agent,
328 472
        })
329 473
    }
330 474
475 +
    /// A request, tried again with the next fallback token while the
476 +
    /// hub refuses the one in use (oxygen#112).
331 477
    fn call(&self, post: bool, endpoint: &str, body: Option<&[u8]>) -> Result<Vec<u8>> {
478 +
        loop {
479 +
            let token = self.token.lock().unwrap_or_else(|p| p.into_inner()).clone();
480 +
            match self.call_once(post, endpoint, body, token.as_deref()) {
481 +
                Err(RepoError::Msg(m))
482 +
                    if token.is_some()
483 +
                        && refused(&m)
484 +
                        && (!self.fallback_on_401_only || m.contains(": HTTP 401")) =>
485 +
                {
486 +
                    let next = self
487 +
                        .fallbacks
488 +
                        .lock()
489 +
                        .unwrap_or_else(|p| p.into_inner())
490 +
                        .pop();
491 +
                    let Some(next) = next else {
492 +
                        return Err(RepoError::Msg(m));
493 +
                    };
494 +
                    crate::credentials::warn_once(&format!(
495 +
                        "{}: the token in use was refused; trying another this computer has for the hub \
496 +
                         (a stale remote.NAME.token goes with `oxy config --unset remote.NAME.token`)",
497 +
                        self.base
498 +
                    ));
499 +
                    *self.token.lock().unwrap_or_else(|p| p.into_inner()) = Some(next);
500 +
                }
501 +
                other => return other,
502 +
            }
503 +
        }
504 +
    }
505 +
506 +
    fn call_once(
507 +
        &self,
508 +
        post: bool,
509 +
        endpoint: &str,
510 +
        body: Option<&[u8]>,
511 +
        token: Option<&str>,
512 +
    ) -> Result<Vec<u8>> {
332 513
        let url = format!("{}/oxy/v1/{endpoint}", self.base);
333 514
        let mut req = if post {
334 515
            self.agent.post(&url)
335 516
        } else {
336 517
            self.agent.get(&url)
337 518
        };
338 -
        req = req.set("Accept", CONTENT_TYPE);
339 -
        if let Some(t) = &self.token {
519 +
        req = req
520 +
            .set("Accept", CONTENT_TYPE)
521 +
            .set(FEATURES_HEADER, &CLIENT_FEATURES.join(","));
522 +
        if let Some(t) = token {
340 523
            req = req.set("Authorization", &format!("Bearer {t}"));
341 524
        }
342 525
        if let Some(id) = &self.repo_id {
…
13 unchanged lines
356 539
            }
357 540
            Err(ureq::Error::Status(code, r)) => {
358 541
                let text = r.into_string().unwrap_or_default();
359 -
                let msg = format!("{url}: HTTP {code}: {}", text.trim());
360 -
                Err(RepoError::Msg(with_hint(msg)))
542 +
                // The hint comes from the status alone: the server's text
543 +
                // (a host's reason for refusing a push, several lines
544 +
                // perhaps) is shown as it is, never read for hints. A
545 +
                // proxy's HTML error page is shown by its title only
546 +
                // (oxygen#133).
547 +
                let status = format!("{url}: HTTP {code}");
548 +
                let msg = format!("{status}: {}", plain_error_text(&text));
549 +
                Err(RepoError::Msg(match hint_for(&status) {
550 +
                    Some(h) => format!("{msg}\n  hint: {h}"),
551 +
                    None => msg,
552 +
                }))
361 553
            }
362 554
            // The transport error already names the URL.
363 555
            Err(e) => Err(RepoError::Msg(with_hint(e.to_string()))),
…
1 unchanged lines
365 557
    }
366 558
}
367 559
560 +
/// Whether a failed request's message says the hub refused its token:
561 +
/// HTTP 401, 403, or 404 (a hub hides a private repository from a token
562 +
/// that may not read it).
563 +
fn refused(message: &str) -> bool {
564 +
    [": HTTP 401", ": HTTP 403", ": HTTP 404"]
565 +
        .iter()
566 +
        .any(|c| message.contains(c))
567 +
}
568 +
368 569
/// Refuse to send a token where anyone on the way could read it: plain
369 570
/// `http` to another machine (see [`crate::credentials::token_allowed`]).
370 571
fn refuse_plain_token(url: &str, options: &RemoteOptions) -> Result<()> {
…
12 unchanged lines
383 584
        "the hub needs you signed in, or did not accept the token: run `oxy auth login HUB` (pushing needs repo:write)"
384 585
    } else if m.contains("http 403") {
385 586
        "the token or account lacks permission: pushing needs a token with repo:write and write access to the repository"
587 +
    } else if m.contains("http 413") {
588 +
        "the request was larger than the server, or a proxy in front of it, takes at once (Cloudflare's limit is 100 MB): push in smaller parts with `oxy config push.batch-bytes 33554432`, then push again"
386 589
    } else if m.contains("http 404") {
387 590
        "nothing at this URL that you can see: check the owner and repository name; a private repository needs you signed in (`oxy auth login HUB`)"
388 591
    } else if m.contains("unknownissuer")
…
17 unchanged lines
406 609
    Some(hint)
407 610
}
408 611
612 +
/// An error response's text as it is, unless it is an HTML page (a proxy's,
613 +
/// such as Cloudflare's 413): then its title, or a note that it was one.
614 +
fn plain_error_text(text: &str) -> String {
615 +
    let t = text.trim();
616 +
    let lower = t.to_ascii_lowercase();
617 +
    if !(lower.starts_with("<!doctype html") || lower.starts_with("<html")) {
618 +
        return t.to_string();
619 +
    }
620 +
    let title = lower
621 +
        .find("<title>")
622 +
        .and_then(|i| {
623 +
            lower[i + 7..]
624 +
                .find("</title>")
625 +
                .map(|j| t[i + 7..i + 7 + j].trim())
626 +
        })
627 +
        .filter(|s| !s.is_empty());
628 +
    match title {
629 +
        Some(s) => format!("{s} (an HTML error page, from a proxy in front of the server perhaps)"),
630 +
        None => "an HTML error page (from a proxy in front of the server perhaps)".into(),
631 +
    }
632 +
}
633 +
409 634
/// `message`, followed by what to do about it when that is known.
410 635
pub fn with_hint(message: String) -> String {
411 636
    match hint_for(&message) {
…
15 unchanged lines
427 652
    let agent = ureq::AgentBuilder::new()
428 653
        .tls_config(tls_client_config(options.ca.as_deref())?)
429 654
        .timeout(std::time::Duration::from_secs(60))
655 +
        .user_agent(&user_agent())
430 656
        .build();
431 657
    let mut req = agent.request(method, url).set("Accept", "application/json");
432 658
    if let Some(t) = &options.token {
…
19 unchanged lines
452 678
    refuse_plain_token(url, options)?;
453 679
    let agent = ureq::AgentBuilder::new()
454 680
        .tls_config(tls_client_config(options.ca.as_deref())?)
681 +
        .user_agent(&user_agent())
455 682
        .build();
456 683
    let mut req = agent.get(url).set("Accept", "application/json");
457 684
    if let Some(t) = &options.token {
…
47 unchanged lines
505 732
        )
506 733
    }
507 734
    fn checkpoint(&self, heads: &[ChangeHash]) -> Result<Bundle> {
508 -
        wire::decode_bundle(&self.call(true, "checkpoint", Some(&wire::encode_hashes(heads)))?)
735 +
        let body = {
736 +
            let _t = crate::timing::phase("remote: checkpoint answer");
737 +
            self.call(true, "checkpoint", Some(&wire::encode_hashes(heads)))?
738 +
        };
739 +
        let _t = crate::timing::phase("remote: decode the checkpoint");
740 +
        wire::decode_bundle(&body)
741 +
    }
742 +
    fn checkpoint_index(&self, heads: &[ChangeHash]) -> Result<Bundle> {
743 +
        let body = {
744 +
            let _t = crate::timing::phase("remote: checkpoint index");
745 +
            match self.call(true, "checkpoint-index", Some(&wire::encode_hashes(heads))) {
746 +
                Ok(b) => b,
747 +
                // A server from before checkpoint-index: the whole answer.
748 +
                Err(RepoError::Msg(m)) if m.contains("HTTP 404") => return self.checkpoint(heads),
749 +
                Err(e) => return Err(e),
750 +
            }
751 +
        };
752 +
        wire::decode_bundle(&body)
509 753
    }
510 754
    fn shallow(&self, heads: &[ChangeHash], depth: u32) -> Result<Vec<ChangeHash>> {
511 755
        wire::decode_hashes(&self.call(
…
176 unchanged lines
688 932
            }
689 933
            changes.push(Value::Array(vec![Value::Bytes(bytes), sigs_value(s)]));
690 934
        }
691 -
        Value::map([
935 +
        let mut v = Value::map([
692 936
            (0, Value::Array(changes)),
693 937
            (
694 938
                1,
…
18 unchanged lines
713 957
                3,
714 958
                Value::Array(b.policies.iter().map(policy_value).collect()),
715 959
            ),
716 -
        ])
960 +
        ]);
961 +
        // Only when there are any; older peers ignore the field.
962 +
        if !b.certificates.is_empty() {
963 +
            if let Value::Map(m) = &mut v {
964 +
                m.insert(
965 +
                    4,
966 +
                    Value::Array(
967 +
                        b.certificates
968 +
                            .iter()
969 +
                            .map(|c| Value::Bytes(c.encode()))
970 +
                            .collect(),
971 +
                    ),
972 +
                );
973 +
            }
974 +
        }
975 +
        v
717 976
    }
718 977
719 978
    pub fn bundle_from_value(v: &Value) -> Result<Bundle> {
…
53 unchanged lines
773 1032
                b.policies.push(policy_from(p)?);
774 1033
            }
775 1034
        }
1035 +
        if let Some(cs) = m.get(&4) {
1036 +
            for c in o::array(cs, "certificates")? {
1037 +
                b.certificates.push(
1038 +
                    crate::cert::SignedCertificate::decode(o::bytes(c, "certificate")?)
1039 +
                        .map_err(|e| schema(&format!("certificate: {e}")))?,
1040 +
                );
1041 +
            }
1042 +
        }
776 1043
        Ok(b)
777 1044
    }
778 1045
…
32 unchanged lines
811 1078
        Value::Array(
812 1079
            tags.iter()
813 1080
                .map(|t| {
814 -
                    Value::map([
1081 +
                    let mut v = Value::map([
815 1082
                        (0, Value::Text(t.name.clone())),
816 1083
                        (1, Value::Bytes(t.state.0.to_vec())),
817 1084
                        (2, hashes_value(&t.heads)),
818 1085
                        (3, opt_attest_value(&t.attestation)),
819 -
                    ])
1086 +
                    ]);
1087 +
                    // Only when there is one; older peers ignore them
1088 +
                    // (oxygen#129).
1089 +
                    if let Value::Map(m) = &mut v {
1090 +
                        if t.accepted.is_some() {
1091 +
                            m.insert(4, opt_attest_value(&t.accepted));
1092 +
                        }
1093 +
                        if let Some(l) = &t.logged {
1094 +
                            m.insert(5, logged_value(l));
1095 +
                        }
1096 +
                        if !t.integration.is_empty() {
1097 +
                            m.insert(6, integration_value(&t.integration));
1098 +
                        }
1099 +
                    }
1100 +
                    v
820 1101
                })
821 1102
                .collect(),
822 1103
        )
…
7 unchanged lines
830 1111
                state: o::oref(field(tm, 1, "state")?, "state")?,
831 1112
                heads: hashes_from(field(tm, 2, "heads")?)?,
832 1113
                attestation: opt_attest_from(field(tm, 3, "attestation")?)?,
1114 +
                accepted: match tm.get(&4) {
1115 +
                    Some(a) => opt_attest_from(a)?,
1116 +
                    None => None,
1117 +
                },
1118 +
                logged: tm.get(&5).map(logged_from).transpose()?,
1119 +
                integration: match tm.get(&6) {
1120 +
                    Some(l) => integration_from(l)?,
1121 +
                    None => Vec::new(),
1122 +
                },
1123 +
            });
1124 +
        }
1125 +
        Ok(out)
1126 +
    }
1127 +
1128 +
    /// The tags of a refs answer as oxy 0.5.0 read them (and the runners
1129 +
    /// built with it), keys 0 to 3 only: what an older reader makes of an
1130 +
    /// answer with keys 4 to 6.
1131 +
    #[cfg(test)]
1132 +
    pub(super) fn tags_as_0_5_read_them(bytes: &[u8]) -> Result<Vec<TagRef>> {
1133 +
        let v = cbor::decode(bytes).map_err(ObjectError::from)?;
1134 +
        let m = map(&v, "refs")?;
1135 +
        let mut out = Vec::new();
1136 +
        for t in o::array(field(m, 3, "tags")?, "tags")? {
1137 +
            let tm = map(t, "tag")?;
1138 +
            out.push(TagRef::new(
1139 +
                o::text(field(tm, 0, "name")?, "name")?.to_owned(),
1140 +
                o::oref(field(tm, 1, "state")?, "state")?,
1141 +
                hashes_from(field(tm, 2, "heads")?)?,
1142 +
                opt_attest_from(field(tm, 3, "attestation")?)?,
1143 +
            ));
1144 +
        }
1145 +
        Ok(out)
1146 +
    }
1147 +
1148 +
    /// A log place, `[entry number, entry bytes, [proof hash]]` (spec §14.8).
1149 +
    fn logged_value(l: &Logged) -> Value {
1150 +
        Value::Array(vec![
1151 +
            Value::Uint(l.index),
1152 +
            Value::Bytes(l.entry.clone()),
1153 +
            proof_value(&l.proof),
1154 +
        ])
1155 +
    }
1156 +
    fn logged_from(v: &Value) -> Result<Logged> {
1157 +
        let a = o::array(v, "logged")?;
1158 +
        if a.len() != 3 {
1159 +
            return Err(schema("logged: expected [index, entry, proof]"));
1160 +
        }
1161 +
        Ok(Logged {
1162 +
            index: o::uint(&a[0], "index")?,
1163 +
            entry: o::bytes(&a[1], "entry")?.to_vec(),
1164 +
            proof: proof_from(&a[2])?,
1165 +
        })
1166 +
    }
1167 +
1168 +
    /// Cosigned checkpoints holding an entry, `[[note, [proof hash]]]`.
1169 +
    fn integration_value(integration: &[Integration]) -> Value {
1170 +
        Value::Array(
1171 +
            integration
1172 +
                .iter()
1173 +
                .map(|i| Value::Array(vec![Value::Text(i.note.clone()), proof_value(&i.proof)]))
1174 +
                .collect(),
1175 +
        )
1176 +
    }
1177 +
    fn integration_from(v: &Value) -> Result<Vec<Integration>> {
1178 +
        let mut out = Vec::new();
1179 +
        for i in o::array(v, "integration")? {
1180 +
            let a = o::array(i, "integration")?;
1181 +
            if a.len() != 2 {
1182 +
                return Err(schema("integration: expected [note, proof]"));
1183 +
            }
1184 +
            out.push(Integration {
1185 +
                note: o::text(&a[0], "note")?.to_owned(),
1186 +
                proof: proof_from(&a[1])?,
833 1187
            });
834 1188
        }
835 1189
        Ok(out)
836 1190
    }
837 1191
838 1192
    pub fn encode_refs(id: [u8; 16], resp: &RefsResponse) -> Vec<u8> {
839 -
        cbor::encode(&Value::map([
1193 +
        let mut v = Value::map([
840 1194
            (
841 1195
                0,
842 1196
                Value::Array(
843 1197
                    resp.refs
844 1198
                        .iter()
845 1199
                        .map(|r| {
846 -
                            Value::map([
1200 +
                            let mut v = Value::map([
847 1201
                                (0, Value::Text(r.name.clone())),
848 1202
                                (1, Value::Bytes(r.state.0.to_vec())),
849 1203
                                (2, hashes_value(&r.heads)),
850 1204
                                (3, opt_attest_value(&r.attestation)),
851 1205
                                (4, r.last_serial.map(Value::Uint).unwrap_or(Value::Null)),
852 -
                            ])
1206 +
                            ]);
1207 +
                            // Only when there is one; older peers ignore it.
1208 +
                            if let Value::Map(m) = &mut v {
1209 +
                                if r.accepted.is_some() {
1210 +
                                    m.insert(5, opt_attest_value(&r.accepted));
1211 +
                                }
1212 +
                                if let Some(l) = &r.logged {
1213 +
                                    m.insert(6, logged_value(l));
1214 +
                                }
1215 +
                                if !r.integration.is_empty() {
1216 +
                                    m.insert(7, integration_value(&r.integration));
1217 +
                                }
1218 +
                            }
1219 +
                            v
853 1220
                        })
854 1221
                        .collect(),
855 1222
                ),
…
12 unchanged lines
868 1235
                    .map(Value::Text)
869 1236
                    .unwrap_or(Value::Null),
870 1237
            ),
871 -
        ]))
1238 +
        ]);
1239 +
        if let Value::Map(m) = &mut v {
1240 +
            if let Some(log) = &resp.log {
1241 +
                m.insert(6, Value::Text(log.clone()));
1242 +
            }
1243 +
            if !resp.withdrawn.is_empty() {
1244 +
                m.insert(8, tags_value(&resp.withdrawn));
1245 +
            }
1246 +
            if !resp.witnesses.is_empty() {
1247 +
                m.insert(
1248 +
                    7,
1249 +
                    Value::Array(
1250 +
                        resp.witnesses
1251 +
                            .iter()
1252 +
                            .map(|(name, id)| {
1253 +
                                Value::Array(vec![
1254 +
                                    Value::Text(name.clone()),
1255 +
                                    Value::Bytes(id.clone()),
1256 +
                                ])
1257 +
                            })
1258 +
                            .collect(),
1259 +
                    ),
1260 +
                );
1261 +
            }
1262 +
        }
1263 +
        cbor::encode(&v)
872 1264
    }
873 1265
    pub fn decode_refs(bytes: &[u8]) -> Result<([u8; 16], RefsResponse)> {
874 1266
        let v = cbor::decode(bytes).map_err(ObjectError::from)?;
…
15 unchanged lines
890 1282
                } else {
891 1283
                    Some(o::uint(ls, "last serial")?)
892 1284
                },
1285 +
                accepted: match rm.get(&5) {
1286 +
                    Some(a) => opt_attest_from(a)?,
1287 +
                    None => None,
1288 +
                },
1289 +
                logged: rm.get(&6).map(logged_from).transpose()?,
1290 +
                integration: match rm.get(&7) {
1291 +
                    Some(l) => integration_from(l)?,
1292 +
                    None => Vec::new(),
1293 +
                },
893 1294
            });
894 1295
        }
895 1296
        let policy = match m.get(&2) {
…
14 unchanged lines
910 1311
            Some(Value::Text(t)) => Some(t.clone()),
911 1312
            _ => None,
912 1313
        };
1314 +
        let log = match m.get(&6) {
1315 +
            Some(Value::Text(t)) => Some(t.clone()),
1316 +
            _ => None,
1317 +
        };
1318 +
        let mut witnesses = Vec::new();
1319 +
        if let Some(w) = m.get(&7) {
1320 +
            for x in o::array(w, "witnesses")? {
1321 +
                let a = o::array(x, "witness")?;
1322 +
                if a.len() != 2 {
1323 +
                    return Err(schema("witness: expected [name, identity]"));
1324 +
                }
1325 +
                witnesses.push((
1326 +
                    o::text(&a[0], "witness name")?.to_owned(),
1327 +
                    o::bytes(&a[1], "witness identity")?.to_vec(),
1328 +
                ));
1329 +
            }
1330 +
        }
1331 +
        let withdrawn = match m.get(&8) {
1332 +
            Some(t) => tags_from(t)?,
1333 +
            None => Vec::new(),
1334 +
        };
913 1335
        Ok((
914 1336
            id,
915 1337
            RefsResponse {
…
2 unchanged lines
918 1340
                tags,
919 1341
                history,
920 1342
                default_bookmark,
1343 +
                log,
1344 +
                witnesses,
1345 +
                withdrawn,
921 1346
            },
922 1347
        ))
923 1348
    }
924 1349
1350 +
    /// A proof's hashes, from the leaf upwards (spec §14.8).
1351 +
    fn proof_value(proof: &[[u8; 32]]) -> Value {
1352 +
        Value::Array(proof.iter().map(|h| Value::Bytes(h.to_vec())).collect())
1353 +
    }
1354 +
1355 +
    fn proof_from(v: &Value) -> Result<Vec<[u8; 32]>> {
1356 +
        let mut proof = Vec::new();
1357 +
        for h in o::array(v, "proof")? {
1358 +
            proof.push(
1359 +
                o::bytes(h, "proof hash")?
1360 +
                    .try_into()
1361 +
                    .map_err(|_| schema("proof hash: expected 32 bytes"))?,
1362 +
            );
1363 +
        }
1364 +
        Ok(proof)
1365 +
    }
1366 +
925 1367
    pub fn encode_negotiate(have: &[ChangeHash], want: &[ChangeHash]) -> Vec<u8> {
926 1368
        cbor::encode(&Value::map([
927 1369
            (0, hashes_value(have)),
…
151 unchanged lines
1079 1521
    use crate::attest::AttestBody;
1080 1522
    use crate::objects::Hlc;
1081 1523
1524 +
    /// oxygen#133: a proxy's HTML error page is shown by its title, and a
1525 +
    /// server's own text as it is; a 413 says to push in smaller parts.
1526 +
    #[test]
1527 +
    fn html_error_pages_are_shown_by_their_title() {
1528 +
        let cloudflare = "<html>\r\n<head><title>413 Request Entity Too Large</title></head>\r\n<body><center><h1>413</h1></center><hr><center>cloudflare</center></body></html>";
1529 +
        assert_eq!(
1530 +
            plain_error_text(cloudflare),
1531 +
            "413 Request Entity Too Large (an HTML error page, from a proxy in front of the server perhaps)"
1532 +
        );
1533 +
        assert_eq!(
1534 +
            plain_error_text("  refused: a secret\nat a.txt:3 "),
1535 +
            "refused: a secret\nat a.txt:3"
1536 +
        );
1537 +
        assert!(hint_for("https://hub/x: HTTP 413: too large")
1538 +
            .unwrap()
1539 +
            .contains("push.batch-bytes"));
1540 +
    }
1541 +
1542 +
    #[test]
1543 +
    fn client_features_are_declared_and_read() {
1544 +
        assert_eq!(CLIENT_FEATURES.join(","), "certified-tags");
1545 +
        assert!(user_agent().starts_with("oxy/"));
1546 +
        assert!(declares_feature(
1547 +
            Some("certified-tags"),
1548 +
            FEATURE_CERTIFIED_TAGS
1549 +
        ));
1550 +
        assert!(declares_feature(
1551 +
            Some("later-thing, Certified-Tags ,x"),
1552 +
            FEATURE_CERTIFIED_TAGS
1553 +
        ));
1554 +
        assert!(!declares_feature(
1555 +
            Some("certified-tags-v2"),
1556 +
            FEATURE_CERTIFIED_TAGS
1557 +
        ));
1558 +
        assert!(!declares_feature(Some(""), FEATURE_CERTIFIED_TAGS));
1559 +
        assert!(!declares_feature(None, FEATURE_CERTIFIED_TAGS));
1560 +
    }
1561 +
1082 1562
    #[test]
1083 1563
    fn wire_round_trips() {
1084 1564
        let c = crate::objects::tests::sample_change();
…
24 unchanged lines
1109 1589
            }],
1110 1590
            hashes: vec![],
1111 1591
            raw_changes: vec![],
1592 +
            certificates: vec![],
1112 1593
        };
1113 1594
        let d = wire::decode_bundle(&wire::encode_bundle(&bundle)).unwrap();
1114 1595
        assert_eq!(
…
5 unchanged lines
1120 1601
        assert_eq!(d.attests, bundle.attests);
1121 1602
        assert_eq!(d.objects, bundle.objects);
1122 1603
        assert_eq!(d.policies, bundle.policies);
1604 +
        assert!(d.certificates.is_empty());
1605 +
        // Certificates (keyless, oxygenhub#241) travel as field 4, only
1606 +
        // when there are any: a bundle without them encodes as before.
1607 +
        let plain = wire::encode_bundle(&bundle);
1608 +
        let mut with = bundle.clone();
1609 +
        let issuer_sk = crate::identity::SecretKey::generate();
1610 +
        let issuer = issuer_sk.identity("hub", "h@x", 1);
1611 +
        with.certificates.push(crate::cert::SignedCertificate::sign(
1612 +
            crate::cert::Certificate {
1613 +
                key: id.clone(),
1614 +
                issuer: issuer.hash(),
1615 +
                subject: "agent:a@b".into(),
1616 +
                claims: vec![],
1617 +
                scope: crate::identity::Scope::CHANGES,
1618 +
                not_before: 1,
1619 +
                expires: 2,
1620 +
                grant: None,
1621 +
            },
1622 +
            &issuer,
1623 +
            &issuer_sk,
1624 +
        ));
1625 +
        let encoded = wire::encode_bundle(&with);
1626 +
        assert_ne!(encoded, plain);
1627 +
        assert_eq!(
1628 +
            wire::decode_bundle(&encoded).unwrap().certificates,
1629 +
            with.certificates
1630 +
        );
1631 +
        with.certificates.clear();
1632 +
        assert_eq!(wire::encode_bundle(&with), plain);
1123 1633
1124 1634
        let att = Attest {
1125 1635
            repo: [3; 16],
…
15 unchanged lines
1141 1651
                    sigs: vec![sig.clone()],
1142 1652
                }),
1143 1653
                last_serial: Some(4),
1654 +
                accepted: None,
1655 +
                logged: None,
1656 +
                integration: vec![],
1144 1657
            },
1145 1658
            RemoteRef {
1146 1659
                name: "dev".into(),
…
1 unchanged lines
1148 1661
                heads: vec![],
1149 1662
                attestation: None,
1150 1663
                last_serial: None,
1664 +
                accepted: None,
1665 +
                logged: None,
1666 +
                integration: vec![],
1151 1667
            },
1152 1668
        ];
1153 -
        let tag = TagRef {
1154 -
            name: "v1".into(),
1155 -
            state: ObjectRef([2; 32]),
1156 -
            heads: vec![c.hash()],
1157 -
            attestation: None,
1158 -
        };
1159 -
        let resp = RefsResponse {
1669 +
        let tag = TagRef::new("v1".into(), ObjectRef([2; 32]), vec![c.hash()], None);
1670 +
        let mut resp = RefsResponse {
1160 1671
            refs,
1161 1672
            policy: Some(SignedPolicy {
1162 1673
                policy: policy.clone(),
…
5 unchanged lines
1168 1679
                sigs: vec![],
1169 1680
            }],
1170 1681
            default_bookmark: Some("develop".into()),
1682 +
            log: Some("hub.example/log\n2\nroot\n\n- sig\n".into()),
1683 +
            witnesses: vec![],
1684 +
            withdrawn: vec![],
1685 +
        };
1686 +
        // Without integrated times (spec §14.8) nothing is added: an old
1687 +
        // peer's answer, and what it reads.
1688 +
        let old = wire::encode_refs([3; 16], &resp);
1689 +
        let back = wire::decode_refs(&old).unwrap().1;
1690 +
        assert!(back.witnesses.is_empty() && back.refs.iter().all(|r| r.integration.is_empty()));
1691 +
        resp.refs[1].logged = Some(Logged {
1692 +
            index: 41,
1693 +
            entry: vec![1, 2, 3],
1694 +
            proof: vec![[9; 32], [8; 32]],
1695 +
        });
1696 +
        resp.refs[1].integration = vec![Integration {
1697 +
            note: "hub.example/log\n42\nroot\n\n- sig\n".into(),
1698 +
            proof: vec![[7; 32]],
1699 +
        }];
1700 +
        resp.witnesses = vec![("w.example".into(), vec![4, 5, 6])];
1701 +
        let new = wire::encode_refs([3; 16], &resp);
1702 +
        let (id_back, back) = wire::decode_refs(&new).unwrap();
1703 +
        assert_eq!(back.log, resp.log);
1704 +
        assert_eq!(back.refs[1].logged, resp.refs[1].logged);
1705 +
        assert_eq!(back.refs[1].integration, resp.refs[1].integration);
1706 +
        assert_eq!(back.witnesses, resp.witnesses);
1707 +
        assert_eq!(back.refs[0].logged, None);
1708 +
        // The new keys are 7s an old reader never asks for.
1709 +
        let crate::cbor::Value::Map(m) = crate::cbor::decode(&new).unwrap() else {
1710 +
            panic!()
1171 1711
        };
1172 -
        let (id_back, back) = wire::decode_refs(&wire::encode_refs([3; 16], &resp)).unwrap();
1712 +
        assert!(m.contains_key(&7));
1173 1713
        assert_eq!(id_back, [3; 16]);
1174 1714
        assert_eq!(back.refs.len(), 2);
1175 1715
        assert_eq!(back.refs[0].attestation.as_ref().unwrap().attest, att);
…
2 unchanged lines
1178 1718
        assert_eq!(back.history, resp.history);
1179 1719
        assert_eq!(back.tags, vec![tag.clone()]);
1180 1720
1721 +
        // A tag's acceptance and log place (oxygen#129): keys 4 to 6, only
1722 +
        // when there are any, so a tag without them encodes as before.
1723 +
        let crate::cbor::Value::Map(m) = crate::cbor::decode(&new).unwrap() else {
1724 +
            panic!()
1725 +
        };
1726 +
        let crate::cbor::Value::Array(tags) = &m[&3] else {
1727 +
            panic!()
1728 +
        };
1729 +
        let crate::cbor::Value::Map(plain) = &tags[0] else {
1730 +
            panic!()
1731 +
        };
1732 +
        assert_eq!(plain.keys().copied().collect::<Vec<_>>(), vec![0, 1, 2, 3]);
1733 +
        let mut full = tag.clone();
1734 +
        full.name = "v2".into();
1735 +
        full.attestation = Some(SignedAttest {
1736 +
            attest: att.clone(),
1737 +
            sigs: vec![sig.clone()],
1738 +
        });
1739 +
        full.accepted = full.attestation.clone();
1740 +
        full.logged = Some(Logged {
1741 +
            index: 7,
1742 +
            entry: vec![4],
1743 +
            proof: vec![[3; 32]],
1744 +
        });
1745 +
        full.integration = vec![Integration {
1746 +
            note: "hub.example/log\n8\nroot\n\n- sig\n".into(),
1747 +
            proof: vec![],
1748 +
        }];
1749 +
        resp.tags = vec![full.clone(), tag.clone()];
1750 +
        let bytes = wire::encode_refs([3; 16], &resp);
1751 +
        assert_eq!(wire::decode_refs(&bytes).unwrap().1.tags, resp.tags);
1752 +
        // In a push too.
1753 +
        let pushed = wire::encode_push(&Bundle::default(), &[], None, &resp.tags);
1754 +
        assert_eq!(wire::decode_push(&pushed).unwrap().3, resp.tags);
1755 +
        // A reader that knows only keys 0 to 3 (oxy 0.5.0 and older, and
1756 +
        // the runners built with it) reads the same tags from the new answer.
1757 +
        let stripped: Vec<TagRef> = resp
1758 +
            .tags
1759 +
            .iter()
1760 +
            .map(|t| {
1761 +
                TagRef::new(
1762 +
                    t.name.clone(),
1763 +
                    t.state,
1764 +
                    t.heads.clone(),
1765 +
                    t.attestation.clone(),
1766 +
                )
1767 +
            })
1768 +
            .collect();
1769 +
        assert_eq!(wire::tags_as_0_5_read_them(&bytes).unwrap(), stripped);
1770 +
1181 1771
        let (h, w) = wire::decode_negotiate(&wire::encode_negotiate(
1182 1772
            &[ObjectRef([1; 32])],
1183 1773
            &[ObjectRef([2; 32])],
…
67 unchanged lines
modified crates/oxy-core/src/receipt.rs +163 −6
…
90 unchanged lines
91 91
    Unsigned(String),
92 92
    #[error("the runner's signature does not verify: {0}")]
93 93
    BadSignature(String),
94 +
    #[error("its certificate does not hold: {0}")]
95 +
    Certificate(String),
94 96
    #[error("{0}")]
95 97
    Schema(String),
96 98
}
…
6 unchanged lines
103 105
104 106
/// A receipt as a runner hands it over: the receipt, the attestation
105 107
/// binding it to a state, the runner's signature over that attestation, and
106 -
/// the runner's identity, so it verifies with nothing else.
108 +
/// the runner's identity, so it verifies with nothing else. A runner that
109 +
/// signs with a short-lived key (keyless, oxygenhub#241) adds that key's
110 +
/// certificate and its issuer's identity.
107 111
#[derive(Clone, Debug, PartialEq, Eq)]
108 112
pub struct SignedReceipt {
109 113
    pub receipt: Receipt,
110 114
    pub attest: Attest,
111 115
    pub sigs: Vec<Signature>,
112 116
    pub runner: Identity,
117 +
    pub certificate: Option<CertifiedBy>,
118 +
}
119 +
120 +
/// A short-lived signing key's certificate, with the identity of the key
121 +
/// that issued it (spec §14.7).
122 +
#[derive(Clone, Debug, PartialEq, Eq)]
123 +
pub struct CertifiedBy {
124 +
    pub cert: crate::cert::SignedCertificate,
125 +
    pub issuer: Identity,
113 126
}
114 127
115 128
impl SignedReceipt {
…
22 unchanged lines
138 151
            attest,
139 152
            sigs: vec![sig],
140 153
            runner: runner.clone(),
154 +
            certificate: None,
141 155
        }
142 156
    }
143 157
158 +
    /// The same, signed by a short-lived key whose certificate `cert`
159 +
    /// `issuer` issued: the certificate rides along.
160 +
    pub fn with_certificate(
161 +
        mut self,
162 +
        cert: crate::cert::SignedCertificate,
163 +
        issuer: Identity,
164 +
    ) -> SignedReceipt {
165 +
        self.certificate = Some(CertifiedBy { cert, issuer });
166 +
        self
167 +
    }
168 +
144 169
    /// The state the receipt was made on.
145 170
    pub fn state(&self) -> ObjectRef {
146 171
        match &self.attest.body {
…
26 unchanged lines
173 198
            .find(|s| s.key == key)
174 199
            .ok_or_else(|| ReceiptError::Unsigned(key.short(12)))?;
175 200
        verify(&self.runner, sig, Domain::Attest, &target)
176 -
            .map_err(|e| ReceiptError::BadSignature(e.to_string()))
201 +
            .map_err(|e| ReceiptError::BadSignature(e.to_string()))?;
202 +
        if let Some(c) = &self.certificate {
203 +
            c.cert
204 +
                .verify(&c.issuer)
205 +
                .map_err(|e| ReceiptError::Certificate(e.to_string()))?;
206 +
            if c.cert.cert.key_id() != key {
207 +
                return Err(ReceiptError::Certificate(
208 +
                    "it certifies another key than the one that signed".into(),
209 +
                ));
210 +
            }
211 +
            // The attestation's own time is the signer's claim: a host
212 +
            // checks the time it accepted the receipt as well.
213 +
            if !c
214 +
                .cert
215 +
                .cert
216 +
                .covers(crate::identity::Scope::SEAL, self.attest.hlc.wall_ms)
217 +
            {
218 +
                return Err(ReceiptError::Certificate(
219 +
                    "the receipt was signed outside the certificate's window, or it gives no seal scope".into(),
220 +
                ));
221 +
            }
222 +
        }
223 +
        Ok(())
177 224
    }
178 225
179 226
    /// `[receipt, attest, [Signature], identity]`, each object as its bytes.
227 +
    /// `[receipt, attest, [Signature], identity]`, and
228 +
    /// `[certificate, issuer identity]` after them when a short-lived key
229 +
    /// signed.
180 230
    pub fn encode(&self) -> Vec<u8> {
181 -
        cbor::encode(&Value::Array(vec![
231 +
        let mut parts = vec![
182 232
            Value::Bytes(self.receipt.encode()),
183 233
            Value::Bytes(self.attest.encode()),
184 234
            Value::Array(self.sigs.iter().map(|s| s.to_value()).collect()),
185 235
            Value::Bytes(self.runner.encode()),
186 -
        ]))
236 +
        ];
237 +
        if let Some(c) = &self.certificate {
238 +
            parts.push(Value::Array(vec![
239 +
                Value::Bytes(c.cert.encode()),
240 +
                Value::Bytes(c.issuer.encode()),
241 +
            ]));
242 +
        }
243 +
        cbor::encode(&Value::Array(parts))
187 244
    }
188 245
189 246
    pub fn decode(b: &[u8]) -> Result<SignedReceipt, ReceiptError> {
190 247
        let v = cbor::decode(b).map_err(ObjectError::from)?;
191 248
        let a = array(&v, "signed receipt")?;
192 -
        if a.len() != 4 {
249 +
        if a.len() != 4 && a.len() != 5 {
193 250
            return Err(ReceiptError::Schema(
194 -
                "signed receipt: expected [receipt, attest, sigs, identity]".into(),
251 +
                "signed receipt: expected [receipt, attest, sigs, identity, (certificate, issuer)]"
252 +
                    .into(),
195 253
            ));
196 254
        }
255 +
        let certificate = match a.get(4) {
256 +
            None => None,
257 +
            Some(c) => {
258 +
                let c = array(c, "certificate")?;
259 +
                if c.len() != 2 {
260 +
                    return Err(ReceiptError::Schema(
261 +
                        "signed receipt: expected [certificate, issuer identity]".into(),
262 +
                    ));
263 +
                }
264 +
                Some(CertifiedBy {
265 +
                    cert: crate::cert::SignedCertificate::decode(bytes(&c[0], "certificate")?)
266 +
                        .map_err(|e| ReceiptError::Certificate(e.to_string()))?,
267 +
                    issuer: Identity::decode(bytes(&c[1], "issuer")?)?,
268 +
                })
269 +
            }
270 +
        };
197 271
        Ok(SignedReceipt {
198 272
            receipt: Receipt::decode(bytes(&a[0], "receipt")?)?,
199 273
            attest: Attest::decode(bytes(&a[1], "attest")?)?,
…
2 unchanged lines
202 276
                .map(Signature::from_value)
203 277
                .collect::<Result<_, _>>()?,
204 278
            runner: Identity::decode(bytes(&a[3], "identity")?)?,
279 +
            certificate,
205 280
        })
206 281
    }
207 282
}
…
64 unchanged lines
272 347
        swapped.runner = other;
273 348
        assert!(matches!(swapped.verify(), Err(ReceiptError::Unsigned(_))));
274 349
    }
350 +
351 +
    /// A receipt signed with a short-lived key carries its certificate
352 +
    /// (oxygenhub#241): it verifies, round-trips, and is refused when the
353 +
    /// certificate is another key's, gives no seal scope, or does not cover
354 +
    /// when it was signed.
355 +
    #[test]
356 +
    fn a_receipt_by_a_certified_key_carries_and_checks_its_certificate() {
357 +
        use crate::cert::{Certificate, SignedCertificate};
358 +
        use crate::identity::Scope;
359 +
        let hub_sk = SecretKey::generate();
360 +
        let hub = hub_sk.identity("oxygenhub.io", "[email protected]", 1);
361 +
        let job_sk = SecretKey::generate();
362 +
        let job = job_sk.identity("job 7 test", "[email protected]", 1);
363 +
        let certify = |key: &Identity, scope: Scope, from: u64, to: u64| {
364 +
            SignedCertificate::sign(
365 +
                Certificate {
366 +
                    key: key.clone(),
367 +
                    issuer: hub.hash(),
368 +
                    subject: "repo:andy/demo:bookmark:main".into(),
369 +
                    claims: vec![("job".into(), "test".into())],
370 +
                    scope,
371 +
                    not_before: from,
372 +
                    expires: to,
373 +
                    grant: None,
374 +
                },
375 +
                &hub,
376 +
                &hub_sk,
377 +
            )
378 +
        };
379 +
        let sr = SignedReceipt::sign(
380 +
            sample(),
381 +
            [7; 16],
382 +
            ObjectRef([9; 32]),
383 +
            "andy/demo#7",
384 +
            &job,
385 +
            &job_sk,
386 +
        );
387 +
        let now = sr.attest.hlc.wall_ms;
388 +
        let good = sr.clone().with_certificate(
389 +
            certify(&job, Scope::SEAL, now - 1000, now + 600_000),
390 +
            hub.clone(),
391 +
        );
392 +
        good.verify().unwrap();
393 +
        let back = SignedReceipt::decode(&good.encode()).unwrap();
394 +
        assert_eq!(back, good);
395 +
        assert_eq!(
396 +
            SignedReceipt::decode(&sr.encode()).unwrap().certificate,
397 +
            None,
398 +
            "a plain receipt reads as before"
399 +
        );
400 +
401 +
        let other = SecretKey::generate().identity("other", "o@x", 1);
402 +
        let wrong_key = sr.clone().with_certificate(
403 +
            certify(&other, Scope::SEAL, now - 1000, now + 600_000),
404 +
            hub.clone(),
405 +
        );
406 +
        assert!(
407 +
            matches!(wrong_key.verify(), Err(ReceiptError::Certificate(m)) if m.contains("another key"))
408 +
        );
409 +
        let no_seal = sr.clone().with_certificate(
410 +
            certify(&job, Scope::CHANGES, now - 1000, now + 600_000),
411 +
            hub.clone(),
412 +
        );
413 +
        assert!(matches!(
414 +
            no_seal.verify(),
415 +
            Err(ReceiptError::Certificate(_))
416 +
        ));
417 +
        let too_late = sr.clone().with_certificate(
418 +
            certify(&job, Scope::SEAL, now - 700_000, now - 100_000),
419 +
            hub.clone(),
420 +
        );
421 +
        assert!(
422 +
            matches!(too_late.verify(), Err(ReceiptError::Certificate(m)) if m.contains("window"))
423 +
        );
424 +
        let wrong_issuer = sr
425 +
            .clone()
426 +
            .with_certificate(certify(&job, Scope::SEAL, now - 1000, now + 600_000), other);
427 +
        assert!(matches!(
428 +
            wrong_issuer.verify(),
429 +
            Err(ReceiptError::Certificate(_))
430 +
        ));
431 +
    }
275 432
}
modified crates/oxy-core/src/repo/bookmarks.rs +149 −7
…
13 unchanged lines
14 14
    /// Changes ahead of / behind the default remote's copy, when tracked.
15 15
    pub ahead: Option<usize>,
16 16
    pub behind: Option<usize>,
17 +
    /// The remote `ahead` and `behind` are measured against.
18 +
    pub remote: Option<String>,
17 19
}
18 20
19 21
#[derive(Clone, Debug)]
…
64 unchanged lines
84 86
    /// only remote there is (finding 29 of the metalgear feedback: a
85 87
    /// checkout whose one remote is `earth`).
86 88
    pub fn default_remote(&self) -> String {
89 +
        if let Some(r) = &self.config.default_remote {
90 +
            return r.clone();
91 +
        }
87 92
        match self.config.remotes.keys().collect::<Vec<_>>().as_slice() {
88 93
            [only] if *only != "origin" => (*only).clone(),
89 94
            _ => "origin".into(),
90 95
        }
91 96
    }
92 97
98 +
    /// The remote a bookmark is compared with: the default when it has
99 +
    /// the bookmark, else the first remote by name that does (a repository
100 +
    /// with `earth` and `gcs` and no `origin`, oxygen#83).
101 +
    fn tracking_remote(&self, view: &View, name: &str) -> Option<String> {
102 +
        // Only remotes still configured: a removed one's copies linger.
103 +
        let configured = |r: &str| self.config.remotes.contains_key(r);
104 +
        let default = self.default_remote();
105 +
        if configured(&default)
106 +
            && view
107 +
                .remotes
108 +
                .get(&default)
109 +
                .is_some_and(|b| b.contains_key(name))
110 +
        {
111 +
            return Some(default);
112 +
        }
113 +
        view.remotes
114 +
            .iter()
115 +
            .find(|(r, b)| configured(r) && b.contains_key(name))
116 +
            .map(|(r, _)| r.clone())
117 +
    }
118 +
93 119
    /// How bookmark `name` stands against `remote`'s copy of it: the
94 120
    /// changes only here and only there. `None` when either side has no
95 121
    /// such bookmark.
…
17 unchanged lines
113 139
    /// remote's copy.
114 140
    pub fn bookmarks(&self) -> Result<Vec<BookmarkInfo>> {
115 141
        let view = self.view()?;
116 -
        let remote = self.default_remote();
117 142
        let mut out = Vec::new();
118 143
        for (name, st) in &view.bookmarks {
119 144
            let heads = self.store.get_state(st)?.heads;
120 -
            let (ahead, behind) = match view.remotes.get(&remote).and_then(|b| b.get(name)) {
145 +
            let remote = self.tracking_remote(&view, name);
146 +
            let (ahead, behind) = match remote
147 +
                .as_ref()
148 +
                .and_then(|r| view.remotes.get(r))
149 +
                .and_then(|b| b.get(name))
150 +
            {
121 151
                Some(rs) => {
122 152
                    let remote_heads = self.store.get_state(rs)?.heads;
123 153
                    (
…
9 unchanged lines
133 163
                current: *name == self.config.bookmark,
134 164
                ahead,
135 165
                behind,
166 +
                remote,
136 167
            });
137 168
        }
138 169
        Ok(out)
139 170
    }
140 171
172 +
    /// Move `name` to its default remote's state when it is only behind
173 +
    /// it: no local change the remote lacks, so nothing is lost
174 +
    /// (oxygen#74). Returns how many changes it caught up, if any.
175 +
    pub fn catch_up_bookmark(&self, name: &str) -> Result<Option<usize>> {
176 +
        let (op_ref, op) = self.head_op()?;
177 +
        let remote = self.default_remote();
178 +
        let (Some(local), Some(rs)) = (
179 +
            op.view.bookmarks.get(name).copied(),
180 +
            op.view
181 +
                .remotes
182 +
                .get(&remote)
183 +
                .and_then(|b| b.get(name))
184 +
                .copied(),
185 +
        ) else {
186 +
            return Ok(None);
187 +
        };
188 +
        if local == rs {
189 +
            return Ok(None);
190 +
        }
191 +
        let heads = self.store.get_state(&local)?.heads;
192 +
        let remote_heads = self.store.get_state(&rs)?.heads;
193 +
        if !self.negotiate(&remote_heads, &heads)?.is_empty() {
194 +
            return Ok(None);
195 +
        }
196 +
        let behind = self.negotiate(&heads, &remote_heads)?.len();
197 +
        if behind == 0 {
198 +
            return Ok(None);
199 +
        }
200 +
        let mut view = op.view.clone();
201 +
        view.bookmarks.insert(name.to_string(), rs);
202 +
        self.record_op(
203 +
            &[op_ref],
204 +
            &format!("oxy bookmark switch {name} (catch up with {remote})"),
205 +
            view,
206 +
        )?;
207 +
        Ok(Some(behind))
208 +
    }
209 +
141 210
    /// Bookmarks of remotes with no local bookmark of the same name, by
142 211
    /// remote then name.
143 212
    pub fn remote_only_bookmarks(&self) -> Result<Vec<RemoteOnlyBookmark>> {
…
259 unchanged lines
403 472
            }
404 473
        }
405 474
        let target_state = self.load_state(heads)?.materialize();
475 +
        // A subrepo whose nested repository is already at the target's pin
476 +
        // is no carried work: the target records exactly what is on disk
477 +
        // (oxygen#74).
478 +
        let at_target_pin = |p: &String| -> bool {
479 +
            let Some(f) = target_state.files.get(p) else {
480 +
                return false;
481 +
            };
482 +
            if f.kind != NodeKind::Subrepo {
483 +
                return false;
484 +
            }
485 +
            let FileContent::Opaque { blobs } = &f.content else {
486 +
                return false;
487 +
            };
488 +
            let Some(want) = blobs
489 +
                .first()
490 +
                .and_then(|b| self.read_blob(b).ok())
491 +
                .and_then(|b| Pin::decode(&b).ok())
492 +
            else {
493 +
                return false;
494 +
            };
495 +
            let hint = self
496 +
                .modules_on_disk()
497 +
                .entries
498 +
                .get(p)
499 +
                .and_then(|m| m.bookmark.clone());
500 +
            super::subrepos::read_pin(&self.root.join(p), hint.as_deref())
501 +
                .is_ok_and(|have| have.same_state(&want))
502 +
        };
503 +
        touched.retain(|p| !at_target_pin(p));
406 504
        let same = |p: &String| {
407 505
            let a = snap.base_snapshot.files.get(p);
408 506
            let b = target_state.files.get(p);
…
5 unchanged lines
414 512
                _ => false,
415 513
            }
416 514
        };
515 +
        // Work already the target's: the file on disk is exactly the
516 +
        // target's version (or both are absent), so nothing needs carrying
517 +
        // and it is no conflict (oxygen#100), as when a proposal's work
518 +
        // has landed on the bookmark switched to.
519 +
        let already = |p: &String| -> bool {
520 +
            let full = self.root.join(p);
521 +
            match (fs::symlink_metadata(&full), target_state.files.get(p)) {
522 +
                (Err(_), None) => true,
523 +
                (Ok(m), Some(f)) if m.is_file() && f.kind == NodeKind::File => {
524 +
                    if is_executable(&m) != f.mode.exec {
525 +
                        return false;
526 +
                    }
527 +
                    let want = match &f.content {
528 +
                        FileContent::Text { bytes, .. } => Some(bytes.clone()),
529 +
                        FileContent::Opaque { blobs } if blobs.len() == 1 => {
530 +
                            self.read_blob(&blobs[0]).ok()
531 +
                        }
532 +
                        _ => None,
533 +
                    };
534 +
                    want.is_some_and(|w| fs::read(&full).is_ok_and(|have| have == w))
535 +
                }
536 +
                _ => false,
537 +
            }
538 +
        };
539 +
        touched.retain(|p| same(p) || !already(p));
417 540
        let differ: Vec<&String> = touched.iter().filter(|p| !same(p)).collect();
418 541
        if !differ.is_empty() {
419 542
            let list: Vec<&str> = differ.iter().take(10).map(|p| p.as_str()).collect();
…
35 unchanged lines
455 578
            };
456 579
            carried.push((p.clone(), item));
457 580
        }
458 -
        let report = self.checkout_keeping(heads, true, command, &keep)?;
581 +
        let mut report = self.checkout_keeping(heads, true, command, &keep)?;
459 582
        for (p, item) in &carried {
460 583
            match item {
461 584
                Carried::Symlink(target) => self.write_symlink(p, target)?,
…
4 unchanged lines
466 589
                Carried::Subrepo => {}
467 590
            }
468 591
        }
592 +
        // What was carried is on disk, not removed (oxygen#120): a new file
593 +
        // the target lacks is taken off by the checkout and written back, so
594 +
        // the report names only what is really gone.
595 +
        let back: BTreeSet<&str> = carried
596 +
            .iter()
597 +
            .filter(|(p, item)| {
598 +
                matches!(item, Carried::File(..) | Carried::Symlink(_))
599 +
                    && !target_state.files.contains_key(p)
600 +
            })
601 +
            .map(|(p, _)| p.as_str())
602 +
            .collect();
603 +
        report.removed = report.removed.saturating_sub(back.len());
604 +
        report.removed_paths.retain(|p| !back.contains(p.as_str()));
469 605
        self.drop_manifest();
470 606
        self.snapshot(command)?;
471 607
        Ok((report, carried.into_iter().map(|(p, _)| p).collect()))
…
7 unchanged lines
479 615
    }
480 616
481 617
    /// Union the heads of bookmark `name` into the current bookmark (spec
482 -
    /// §10.1). A working copy without changes follows.
618 +
    /// §10.1): a bookmark here, or a remote's as its tracking bookmark
619 +
    /// holds it (`origin/develop`, oxygen#117), so merging what a remote
620 +
    /// has needs no switch. A working copy without changes follows.
483 621
    pub fn merge(&self, name: &str) -> Result<MergeReport> {
484 622
        let _wc = self.lock_working_copy()?;
485 623
        let (_, op) = self.head_op()?;
486 624
        if name == self.config.bookmark {
487 625
            return fail(format!("{name} is the current bookmark"));
488 626
        }
489 -
        let Some(st) = op.view.bookmarks.get(name) else {
490 -
            return fail(format!("no bookmark {name}"));
627 +
        let here = op.view.bookmarks.get(name).copied();
628 +
        let Some(state) = here.or_else(|| remote_bookmark(&op.view, name).map(|(_, _, st)| st))
629 +
        else {
630 +
            return fail(format!(
631 +
                "no bookmark {name}, here or at a remote (oxy bookmark list --all lists them)"
632 +
            ));
491 633
        };
492 -
        let other = self.store.get_state(st)?.heads;
634 +
        let other = self.store.get_state(&state)?.heads;
493 635
        self.merge_heads(other, &format!("oxy merge {name}"))
494 636
    }
495 637
…
83 unchanged lines
modified crates/oxy-core/src/repo/checkout.rs +52 −2
…
5 unchanged lines
6 6
pub struct CheckoutReport {
7 7
    pub written: usize,
8 8
    pub removed: usize,
9 +
    /// The first files removed, to name them (oxygen#102).
10 +
    pub removed_paths: Vec<String>,
9 11
    pub conflicted: bool,
10 12
    /// Nested repositories that could not be moved, and other remarks.
11 13
    pub notes: Vec<String>,
12 14
    /// How many changes of the state left behind the new one lacks.
13 15
    pub lacks: usize,
16 +
    /// Union files whose adjacent insertions were settled by keeping both
17 +
    /// sides (oxygen#145).
18 +
    pub kept_both: Vec<KeptBoth>,
14 19
}
15 20
16 21
/// Text files rendered at a time by a checkout: bounds its memory.
…
44 unchanged lines
61 66
        let mut target = self.load_state_indexed(heads)?;
62 67
        let heads = target.maximal(heads);
63 68
        let tv = crate::timing::phase("checkout: target view");
64 -
        let tview = target.tree().view();
69 +
        let tview = self.tree_view(&mut target)?;
65 70
        drop(tv);
66 71
        let mut report = CheckoutReport::default();
67 72
        // Paths this file system cannot hold side by side (spec §7.7).
…
28 unchanged lines
96 101
            }
97 102
            fs::remove_file(&full)?;
98 103
            report.removed += 1;
104 +
            if report.removed_paths.len() < 5 {
105 +
                report.removed_paths.push(path.to_string());
106 +
            }
99 107
            if let Some(parent) = full.parent() {
100 108
                dirs_to_prune.insert(parent.to_path_buf());
101 109
            }
…
301 unchanged lines
403 411
        }
404 412
405 413
        drop(_render);
414 +
        // How the files rendered: which way each graph was ordered, summed
415 +
        // over every core (`OXY_TIMINGS`).
416 +
        crate::timing::report_totals();
406 417
        let recording = crate::timing::phase("checkout: record");
407 418
        let (op_ref, op) = self.head_op()?;
408 419
        let wc = self.new_working_copy(heads.clone(), "");
…
35 unchanged lines
444 455
            *self.walked_dirs.borrow_mut() = Some((dirs, old_manifest.nested.clone()));
445 456
        }
446 457
        self.write_manifest_entries(report.conflicted, false, &entries, &HashMap::new())?;
458 +
        if report.conflicted && !self.bare {
459 +
            report.kept_both = self.settle_conflicts(command)?;
460 +
        }
447 461
        let _ = self.run_hook(
448 462
            "post-checkout",
449 463
            &format!(
…
62 unchanged lines
512 526
            if i == 0 && *comp == OXY_DIR {
513 527
                return fail(format!("refusing to write inside {OXY_DIR}: {rel:?}"));
514 528
            }
529 +
            // A `.git` path could plant hooks a side-by-side Git would run;
530 +
            // Git refuses them at any depth and in any case, and so does oxy.
531 +
            if comp.eq_ignore_ascii_case(".git") {
532 +
                return fail(format!("refusing to write a .git path: {rel:?}"));
533 +
            }
515 534
            if i + 1 < comps.len() {
516 535
                full.push(comp);
517 536
                if let Ok(m) = fs::symlink_metadata(&full) {
…
64 unchanged lines
582 601
}
583 602
584 603
/// Refuse a path that could escape the workspace (spec §7.7): an empty,
585 -
/// `.` or `..` component, a backslash, or a first component `.oxy`.
604 +
/// `.` or `..` component, a backslash, a first component `.oxy`, or a
605 +
/// `.git` component anywhere (it could plant Git hooks).
586 606
fn check_path_name(rel: &str) -> Result<()> {
587 607
    for (i, comp) in rel.split('/').enumerate() {
588 608
        if comp.is_empty() || comp == "." || comp == ".." || comp.contains('\\') {
…
2 unchanged lines
591 611
        if i == 0 && comp == OXY_DIR {
592 612
            return fail(format!("refusing to write inside {OXY_DIR}: {rel:?}"));
593 613
        }
614 +
        if comp.eq_ignore_ascii_case(".git") {
615 +
            return fail(format!("refusing to write a .git path: {rel:?}"));
616 +
        }
594 617
    }
595 618
    Ok(())
596 619
}
…
37 unchanged lines
634 657
    fs::write(full, bytes)?;
635 658
    set_executable(full, exec)
636 659
}
660 +
661 +
#[cfg(test)]
662 +
mod path_name_tests {
663 +
    use super::check_path_name;
664 +
665 +
    #[test]
666 +
    fn git_paths_are_refused_at_any_depth_and_case() {
667 +
        for bad in [
668 +
            ".git/hooks/post-checkout",
669 +
            "sub/.git/config",
670 +
            ".GIT/hooks/x",
671 +
            "a/.Git",
672 +
            ".oxy/x",
673 +
            "../x",
674 +
        ] {
675 +
            assert!(check_path_name(bad).is_err(), "{bad}");
676 +
        }
677 +
        for good in [
678 +
            "src/main.rs",
679 +
            ".gitignore",
680 +
            ".github/workflows/ci.yml",
681 +
            "a/.gitmodules",
682 +
        ] {
683 +
            assert!(check_path_name(good).is_ok(), "{good}");
684 +
        }
685 +
    }
686 +
}
modified crates/oxy-core/src/repo/checkpoint.rs +68 −19
…
112 unchanged lines
113 113
        write_atomic(&self.checkpoint_pointer(bookmark), r.to_hex().as_bytes())
114 114
    }
115 115
116 -
    /// A bundle holding a checkpoint of exactly `heads` with its payload
117 -
    /// (spec §16.1 `checkpoint`); the current bookmark checkpoint is reused
118 -
    /// when it matches, otherwise one is made and, when `heads` are the
119 -
    /// bookmark's, kept.
120 -
    pub fn checkpoint_bundle_for(&self, heads: &[ChangeHash]) -> Result<Bundle> {
116 +
    /// The checkpoint of exactly `heads`: the bookmark's latest when it
117 +
    /// is that state, else one made now (kept as the bookmark's when the
118 +
    /// bookmark is at `heads`).
119 +
    fn checkpoint_ref_for(&self, heads: &[ChangeHash]) -> Result<ObjectRef> {
120 +
        let _t = crate::timing::phase("checkpoint: find or make");
121 121
        let wanted = StateObject::new(heads.to_vec()).heads;
122 122
        let bookmark = self.config.bookmark.clone();
123 123
        let current = self.bookmark_heads()?;
124 -
        let r = match self.latest_checkpoint(&bookmark)? {
124 +
        Ok(match self.latest_checkpoint(&bookmark)? {
125 125
            Some((r, cp)) if cp.heads == wanted => r,
126 126
            _ => {
127 127
                let state = self.load_state(&wanted)?;
…
9 unchanged lines
137 137
                    self.store.put(ObjectType::Checkpoint, &cp.encode())?
138 138
                }
139 139
            }
140 -
        };
140 +
        })
141 +
    }
142 +
143 +
    /// A checkpoint of exactly `heads` without its graph blobs (spec
144 +
    /// §16.1 `checkpoint-index`, oxygen#137): the checkpoint object, its
145 +
    /// payload blob and the blob's chunks. The client asks for the graph
146 +
    /// blobs the payload names in parts, with `objects`, so neither side
147 +
    /// holds a big state's whole answer at once.
148 +
    pub fn checkpoint_index_for(&self, heads: &[ChangeHash]) -> Result<Bundle> {
149 +
        let r = self.checkpoint_ref_for(heads)?;
141 150
        let cp = Checkpoint::decode(&self.store.get_typed(&r, ObjectType::Checkpoint)?)?;
142 151
        let mut bundle = Bundle::default();
143 -
        for c in Blob::decode(&self.store.get_typed(&cp.payload, ObjectType::Blob)?)?.chunks {
144 -
            bundle.objects.push(self.store.get(&c)?);
152 +
        let chunks = Blob::decode(&self.store.get_typed(&cp.payload, ObjectType::Blob)?)?.chunks;
153 +
        for got in self.store.get_many(&chunks) {
154 +
            bundle.objects.push(got?);
145 155
        }
146 156
        bundle.objects.push(self.store.get(&cp.payload)?);
147 -
        for g in self.checkpoint_graph_blobs(&cp)? {
148 -
            for c in Blob::decode(&self.store.get_typed(&g, ObjectType::Blob)?)?.chunks {
149 -
                bundle.objects.push(self.store.get(&c)?);
157 +
        bundle.objects.push((ObjectType::Checkpoint, cp.encode()));
158 +
        Ok(bundle)
159 +
    }
160 +
161 +
    /// A checkpoint of exactly `heads` with its graph blobs and their
162 +
    /// chunks, in one answer (spec §16.1 `checkpoint`): what a client
163 +
    /// older than `checkpoint-index` asks for.
164 +
    pub fn checkpoint_bundle_for(&self, heads: &[ChangeHash]) -> Result<Bundle> {
165 +
        let mut bundle = self.checkpoint_index_for(heads)?;
166 +
        let _t = crate::timing::phase("checkpoint: gather its objects");
167 +
        let cp = match bundle.objects.last() {
168 +
            Some((ObjectType::Checkpoint, body)) => Checkpoint::decode(body)?,
169 +
            _ => return fail("a checkpoint index without its checkpoint"),
170 +
        };
171 +
        // The graph blobs and their chunks in two parallel reads (#137): a
172 +
        // big state's are tens of thousands, and one by one they took six
173 +
        // minutes for VS Code.
174 +
        let graphs = self.checkpoint_graph_blobs(&cp)?;
175 +
        let mut blobs = Vec::with_capacity(graphs.len());
176 +
        for got in self.store.get_many(&graphs) {
177 +
            blobs.push(got?);
178 +
        }
179 +
        let mut chunks = Vec::new();
180 +
        for (ty, body) in &blobs {
181 +
            if *ty != ObjectType::Blob {
182 +
                return fail("a checkpoint graph is not a blob");
150 183
            }
151 -
            bundle.objects.push(self.store.get(&g)?);
184 +
            chunks.extend(Blob::decode(body)?.chunks);
152 185
        }
153 -
        bundle.objects.push((ObjectType::Checkpoint, cp.encode()));
186 +
        let last = bundle.objects.pop();
187 +
        for got in self.store.get_many(&chunks) {
188 +
            bundle.objects.push(got?);
189 +
        }
190 +
        bundle.objects.extend(blobs);
191 +
        bundle.objects.extend(last);
154 192
        Ok(bundle)
155 193
    }
156 194
…
35 unchanged lines
192 230
            };
193 231
            let cp_heads: BTreeSet<ChangeHash> = cp.heads.iter().copied().collect();
194 232
            let mut found: BTreeSet<ChangeHash> = BTreeSet::new();
195 -
            let mut delta: Vec<Change> = Vec::new();
233 +
            let mut delta: Vec<(ChangeHash, Change, Vec<crate::objects::TextRef>)> = Vec::new();
196 234
            let mut seen: BTreeSet<ChangeHash> = BTreeSet::new();
197 235
            let mut stack: Vec<ChangeHash> = heads.to_vec();
198 236
            while let Some(h) = stack.pop() {
…
6 unchanged lines
205 243
                    }
206 244
                    continue;
207 245
                }
208 -
                let c = self.store.get_change(&h)?;
246 +
                // For history, the skeleton: text is read only for the
247 +
                // graphs built later, and a state holding none can be
248 +
                // indexed (oxygen#93).
249 +
                let (c, texts) = if history {
250 +
                    self.store.get_change_lazy(&h)?
251 +
                } else {
252 +
                    (self.store.get_change(&h)?, Vec::new())
253 +
                };
209 254
                stack.extend(c.deps.iter().copied());
210 -
                delta.push(c);
255 +
                delta.push((h, c, texts));
211 256
            }
212 257
            if found.len() != cp_heads.len() {
213 258
                continue;
214 259
            }
215 260
            let _t = crate::timing::phase("load: apply delta and validate");
216 261
            let mut state = base;
217 -
            for c in delta {
218 -
                state.add_unchecked(c);
262 +
            for (h, c, texts) in delta {
263 +
                if history {
264 +
                    state.add_known(h, c, texts);
265 +
                } else {
266 +
                    state.add_unchecked(c);
267 +
                }
219 268
            }
220 269
            state.validate()?;
221 270
            return Ok(Some(state));
…
4 unchanged lines
modified crates/oxy-core/src/repo/config.rs +109 −3
…
17 unchanged lines
18 18
    /// agent sets it to its own key, so its changes carry the human as
19 19
    /// author and the agent's signature (spec §14.2, agent workflows §2).
20 20
    pub signer_key: Option<KeyId>,
21 +
    /// `author.signer = "keyless"`: sign with fresh keys the hub certifies
22 +
    /// under the agent's grant, keeping no key on disk (spec §14.7,
23 +
    /// oxygenhub#241).
24 +
    pub signer_keyless: bool,
25 +
    /// `keyless.hub`: the hub that certifies them (default: the default
26 +
    /// remote's hub).
27 +
    pub keyless_hub: Option<String>,
28 +
    /// `author.for` (oxygen#82): in a keyless agent's home, the account it
29 +
    /// acts for; the home then authors as that person's newest identity,
30 +
    /// which the hub names with each certificate, rather than a pinned
31 +
    /// `author.key` that goes stale with every rotation.
32 +
    pub author_for: Option<String>,
21 33
    /// `core.sign`: sign every change at commit (`true`) or leave signing to
22 34
    /// push-time seals (`false`, the spec default).
23 35
    pub sign_changes: bool,
…
4 unchanged lines
28 40
    pub remote_tokens: BTreeMap<String, String>,
29 41
    /// `remote.<name>.ca`: PEM file of extra roots to trust for that remote.
30 42
    pub remote_cas: BTreeMap<String, String>,
43 +
    /// `core.remote`: the remote `pull`, `fetch` and `push` use when none is
44 +
    /// named, for a repository whose remotes include no `origin`.
45 +
    pub default_remote: Option<String>,
46 +
    /// `git.claim`: a Git import authors the commits whose Git author email
47 +
    /// is this repository's identity's with its key (oxygen#97).
48 +
    pub git_claim: bool,
49 +
    /// `git.colocate`: keep a `.git` beside `.oxy` with the current
50 +
    /// bookmark exported after every operation, so editors' Git features
51 +
    /// work on an Oxygen checkout (one way: oxy to Git).
52 +
    pub git_colocate: bool,
31 53
    /// `core.op-retention`: days for which operations keep their states reachable (spec §12.1, §21).
32 54
    pub op_retention_days: u64,
33 55
    /// `core.gc-grace`: days an unreachable object survives, protecting concurrent writers (spec §21).
34 56
    pub gc_grace_days: u64,
57 +
    /// `core.snapshot-retention`: days for which past operations keep their
58 +
    /// working copies reachable (oxygen#98); each workspace's current one is
59 +
    /// always kept. A file only ever in a working copy goes with it.
60 +
    pub snapshot_retention_days: u64,
61 +
    /// `snapshot.max-new-file-size`: a new, untracked file larger than this
62 +
    /// is left out of working-copy snapshots and named instead (oxygen#98);
63 +
    /// 0 takes in files of any size.
64 +
    pub max_new_file: u64,
35 65
    /// `core.checkpoint-interval`: changes between automatic checkpoints (spec §11).
36 66
    pub checkpoint_interval: u64,
67 +
    /// `push.batch-bytes`: about how many bytes of changes and their texts
68 +
    /// one request of a long push carries (spec §16.3).
69 +
    pub push_batch_bytes: u64,
37 70
    /// `filter.max-blob`: blobs larger than this many bytes are fetched lazily (spec §17).
38 71
    pub max_blob: Option<u64>,
39 72
    /// `filter.paths`: only these subtrees are materialized (spec §17); empty means everything.
…
2 unchanged lines
42 75
    pub semantic_deps: bool,
43 76
    /// `revset.<name>`: a saved revset, used as `name()`.
44 77
    pub revsets: BTreeMap<String, String>,
78 +
    /// `storage.url`: the store the repository's objects and operation
79 +
    /// head are kept on (`dir://`, `s3://`, …) instead of `.oxy/objects`
80 +
    /// (oxygen#68).
81 +
    pub storage_url: Option<String>,
82 +
    /// `storage.cache`: the most the local cache in front of that store
83 +
    /// holds, in bytes.
84 +
    pub storage_cache: u64,
45 85
}
46 86
47 87
impl Config {
88 +
    /// Whether the repository's own file sets `key` (as against the user
89 +
    /// config or a default).
90 +
    pub fn is_repo_set(&self, key: &str) -> bool {
91 +
        self.repo_layer.contains_key(key)
92 +
    }
93 +
48 94
    pub(super) fn default_for_user() -> Config {
49 95
        let user = std::env::var("USER").unwrap_or_else(|_| "unknown".into());
50 96
        Config {
…
5 unchanged lines
56 102
            signer_key: std::env::var("OXY_SIGNER")
57 103
                .ok()
58 104
                .and_then(|v| KeyId::from_hex(&v)),
105 +
            signer_keyless: std::env::var("OXY_SIGNER").is_ok_and(|v| v == "keyless"),
106 +
            keyless_hub: None,
107 +
            author_for: None,
59 108
            sign_changes: false,
60 109
            bookmark: "main".into(),
61 110
            remotes: BTreeMap::new(),
62 111
            remote_tokens: BTreeMap::new(),
63 112
            remote_cas: BTreeMap::new(),
113 +
            default_remote: None,
114 +
            git_claim: false,
115 +
            git_colocate: false,
64 116
            op_retention_days: 90,
65 117
            gc_grace_days: 14,
118 +
            max_new_file: 100 << 20,
119 +
            snapshot_retention_days: 7,
66 120
            checkpoint_interval: 256,
121 +
            // Well under the 100 MB a proxy in front of a hub may take
122 +
            // (oxygen#133).
123 +
            push_batch_bytes: 64 << 20,
67 124
            max_blob: None,
68 125
            sparse_paths: Vec::new(),
69 126
            semantic_deps: false,
70 127
            revsets: BTreeMap::new(),
128 +
            storage_url: None,
129 +
            storage_cache: DEFAULT_STORAGE_CACHE,
71 130
        }
72 131
    }
73 132
…
38 unchanged lines
112 171
                }
113 172
            }
114 173
            "author.signer" => {
115 -
                self.signer_key = if value.is_empty() {
174 +
                self.signer_keyless = value == "keyless";
175 +
                self.signer_key = if value.is_empty() || self.signer_keyless {
116 176
                    None
117 177
                } else {
118 178
                    Some(KeyId::from_hex(value).ok_or_else(|| {
119 -
                        RepoError::Msg("author.signer must be a 64-character hex KeyId".into())
179 +
                        RepoError::Msg(
180 +
                            "author.signer must be a 64-character hex KeyId, or \"keyless\"".into(),
181 +
                        )
120 182
                    })?)
121 183
                }
122 184
            }
123 185
            "core.sign" => {
124 186
                self.sign_changes = match value {
125 187
                    "change" => true,
126 -
                    "seal" => false,
188 +
                    // Empty: unset, the default.
189 +
                    "seal" | "" => false,
127 190
                    _ => return fail("core.sign must be \"change\" or \"seal\""),
128 191
                }
129 192
            }
130 193
            "workspace.bookmark" => self.bookmark = value.into(),
194 +
            "keyless.hub" => self.keyless_hub = (!value.is_empty()).then(|| value.to_string()),
195 +
            "author.for" => self.author_for = (!value.is_empty()).then(|| value.to_string()),
131 196
            // Read by the credential store from the user file.
132 197
            "auth.helper" => {}
133 198
            "core.op-retention" => self.op_retention_days = parse_days(value)?,
134 199
            "core.gc-grace" => self.gc_grace_days = parse_days(value)?,
200 +
            "snapshot.max-new-file-size" => self.max_new_file = parse_size(value)?,
201 +
            "core.snapshot-retention" => self.snapshot_retention_days = parse_days(value)?,
135 202
            "filter.max-blob" => {
136 203
                self.max_blob = if value.is_empty() {
137 204
                    None
…
1 unchanged lines
139 206
                    Some(parse_size(value)?)
140 207
                }
141 208
            }
209 +
            "storage.url" => {
210 +
                self.storage_url = (!value.is_empty()).then(|| value.to_string());
211 +
            }
212 +
            "storage.cache" => {
213 +
                self.storage_cache = if value.is_empty() {
214 +
                    DEFAULT_STORAGE_CACHE
215 +
                } else {
216 +
                    parse_size(value)?
217 +
                }
218 +
            }
219 +
            "core.remote" => {
220 +
                self.default_remote = (!value.is_empty()).then(|| value.to_string());
221 +
            }
142 222
            "hooks.semantic-deps" => self.semantic_deps = matches!(value, "true" | "1" | "yes"),
223 +
            "git.claim" => self.git_claim = matches!(value, "true" | "1" | "yes"),
224 +
            "git.colocate" => self.git_colocate = matches!(value, "true" | "1" | "yes"),
143 225
            "filter.paths" => {
144 226
                self.sparse_paths = value
145 227
                    .split(',')
…
11 unchanged lines
157 239
                    .parse()
158 240
                    .map_err(|_| RepoError::Msg(format!("expected a number, got {value:?}")))?;
159 241
            }
242 +
            "push.batch-bytes" => {
243 +
                self.push_batch_bytes = value.parse().ok().filter(|n| *n > 0).ok_or_else(|| {
244 +
                    RepoError::Msg(format!("expected a number of bytes, got {value:?}"))
245 +
                })?;
246 +
            }
160 247
            k if k.starts_with("remote.") => {
161 248
                let rest = &k["remote.".len()..];
162 249
                let Some((name, field)) = rest.rsplit_once('.') else {
…
55 unchanged lines
218 305
                c.repo_layer.insert(k, v);
219 306
            }
220 307
        }
308 +
        // author.for (oxygen#82): the identity the hub last named as the
309 +
        // person's newest, kept in the home; a repository's own
310 +
        // author.key pin still wins.
311 +
        if c.author_for.is_some() && !c.repo_layer.contains_key("author.key") {
312 +
            if let Some(k) = user_path
313 +
                .and_then(Path::parent)
314 +
                .and_then(|home| fs::read_to_string(home.join("keys").join(AUTHOR_FOR_FILE)).ok())
315 +
                .and_then(|s| KeyId::from_hex(s.trim()))
316 +
            {
317 +
                c.author_key = Some(k);
318 +
            }
319 +
        }
221 320
        Ok(c)
222 321
    }
223 322
…
7 unchanged lines
231 330
    }
232 331
}
233 332
333 +
/// In a home's `keys/`: the key of the identity `author.for` follows,
334 +
/// as the hub last named it (oxygen#82).
335 +
pub const AUTHOR_FOR_FILE: &str = "author.for";
336 +
234 337
/// A byte count with an optional `K`, `M`, or `G` suffix (binary units).
338 +
/// `storage.cache` when unset: 2 GiB.
339 +
pub const DEFAULT_STORAGE_CACHE: u64 = 2 << 30;
340 +
235 341
pub fn parse_size(v: &str) -> Result<u64> {
236 342
    let v = v.trim();
237 343
    let (num, mult) = match v.chars().last() {
…
47 unchanged lines
modified crates/oxy-core/src/repo/doctor.rs +68 −2
…
61 unchanged lines
62 62
        self.doctor_identity(&mut rep);
63 63
        self.doctor_author(&mut rep);
64 64
        self.doctor_portability(&mut rep);
65 +
        self.doctor_trust(&mut rep);
65 66
        for name in self.config.remotes.keys() {
66 67
            self.doctor_remote(&mut rep, name, online);
67 68
        }
…
141 unchanged lines
209 210
        let Ok(heads) = self.bookmark_heads() else {
210 211
            return;
211 212
        };
212 -
        let Ok(state) = self.load_state_history(&heads) else {
213 +
        let Ok(mut state) = self.load_state_history(&heads) else {
213 214
            return;
214 215
        };
215 -
        let view = state.tree().view();
216 +
        let Ok(view) = self.tree_view(&mut state) else {
217 +
            return;
218 +
        };
216 219
        let paths: Vec<&str> = view.entries.keys().map(String::as_str).collect();
217 220
        let mut problems: Vec<String> =
218 221
            crate::portable::Platform::windows().problems(paths.iter().copied());
…
46 unchanged lines
265 268
        }
266 269
    }
267 270
271 +
    /// The trust policy (spec §14.5): without one, nothing fetched is
272 +
    /// verified, which fetch says each time; doctor says it too, with who
273 +
    /// fixes it (oxygen#127).
274 +
    fn doctor_trust(&self, rep: &mut DoctorReport) {
275 +
        let policy = match self.current_policy() {
276 +
            Ok(p) => p,
277 +
            Err(e) => {
278 +
                rep.push(
279 +
                    "trust",
280 +
                    Health::Fail,
281 +
                    format!("the trust policy does not load: {e}"),
282 +
                    Some("run `oxy fsck` for details".into()),
283 +
                );
284 +
                return;
285 +
            }
286 +
        };
287 +
        let Some((r, sp)) = policy else {
288 +
            rep.push(
289 +
                "trust",
290 +
                Health::Warn,
291 +
                "no trust policy: changes and bookmark moves fetched here are not verified",
292 +
                Some(
293 +
                    "the repository's owner sets one: `oxy policy init`, then `oxy policy grant FILE` for each \
294 +
                     contributor and `oxy policy grant-subject --issuer HUB_KEY agent:NAME@ACCOUNT` for keyless \
295 +
                     agents, and `oxy push`"
296 +
                        .into(),
297 +
                ),
298 +
            );
299 +
            return;
300 +
        };
301 +
        let p = &sp.policy;
302 +
        let now = Hlc::now().wall_ms;
303 +
        let what = format!(
304 +
            "policy #{} serial {}: {} root{}, {} rule{}",
305 +
            &r.to_hex()[..12],
306 +
            p.serial,
307 +
            p.roots.len(),
308 +
            if p.roots.len() == 1 { "" } else { "s" },
309 +
            p.rules.len(),
310 +
            if p.rules.len() == 1 { "" } else { "s" },
311 +
        );
312 +
        let renew = Some(
313 +
            "a root of the policy publishes a new serial (any `oxy policy` change does)".into(),
314 +
        );
315 +
        if p.is_expired(now) {
316 +
            rep.push("trust", Health::Fail, format!("{what}, expired"), renew);
317 +
        } else if p.expires - now < 30 * 86_400_000 {
318 +
            rep.push(
319 +
                "trust",
320 +
                Health::Warn,
321 +
                format!("{what}, expires in {} days", (p.expires - now) / 86_400_000),
322 +
                renew,
323 +
            );
324 +
        } else {
325 +
            rep.push(
326 +
                "trust",
327 +
                Health::Ok,
328 +
                format!("{what}, expires in {} days", (p.expires - now) / 86_400_000),
329 +
                None,
330 +
            );
331 +
        }
332 +
    }
333 +
268 334
    fn doctor_remote(&self, rep: &mut DoctorReport, name: &str, online: bool) {
269 335
        let area = format!("remote {name}");
270 336
        let url = &self.config.remotes[name];
…
73 unchanged lines
modified crates/oxy-core/src/repo/fsck.rs +11 −6
…
100 unchanged lines
101 101
        ObjectType::Identity => Identity::decode(body).map(|_| ()),
102 102
        ObjectType::Policy => crate::policy::Policy::decode(body).map(|_| ()),
103 103
        ObjectType::Receipt => crate::receipt::Receipt::decode(body).map(|_| ()),
104 +
        ObjectType::Certificate => crate::cert::Certificate::decode(body).map(|_| ()),
105 +
        ObjectType::Grant => crate::cert::Grant::decode(body).map(|_| ()),
106 +
        ObjectType::Artifact => crate::artifact::Artifact::decode(body).map(|_| ()),
104 107
        ObjectType::State => crate::objects::StateObject::decode(body).map(|_| ()),
105 108
    };
106 109
    r.map_err(|e| e.to_string())
…
2 unchanged lines
109 112
impl Repo {
110 113
    /// Check the whole repository; see the module documentation.
111 114
    pub fn fsck(&self) -> Result<FsckReport> {
115 +
        // The head as it is now, as the operations pass reads it.
116 +
        self.forget_head();
112 117
        let mut rep = FsckReport::default();
113 118
        let bad = {
114 119
            let _t = crate::timing::phase("fsck: objects");
…
45 unchanged lines
160 165
161 166
    /// Pass 2.
162 167
    fn fsck_ops(&self, rep: &mut FsckReport, bad: &HashSet<ObjectRef>) {
163 -
        let head = match fs::read_to_string(self.oxy.join("ops").join("head")) {
164 -
            Ok(h) => h,
168 +
        let head = match self.read_head() {
169 +
            Ok((h, _)) => h,
170 +
            Err(RepoError::Msg(m)) if m.starts_with("corrupt") => {
171 +
                rep.error(None, ".oxy/ops/head does not hold an object reference");
172 +
                return;
173 +
            }
165 174
            Err(e) => {
166 175
                rep.error(None, format!("cannot read .oxy/ops/head: {e}"));
167 176
                return;
168 177
            }
169 -
        };
170 -
        let Some(head) = ObjectRef::from_hex(head.trim()) else {
171 -
            rep.error(None, ".oxy/ops/head does not hold an object reference");
172 -
            return;
173 178
        };
174 179
        let mut seen = HashSet::new();
175 180
        let mut queue: VecDeque<(ObjectRef, Option<(ObjectRef, u64)>)> =
…
337 unchanged lines
modified crates/oxy-core/src/repo/gc.rs +82 −12
…
1 unchanged lines
2 2
//! nothing reachable refers to.
3 3
//!
4 4
//! Roots: the current view, the views of every operation newer than
5 -
//! `core.op-retention`, reference attestations, and the policy; receipts
5 +
//! `core.op-retention` (their working copies only within the shorter
6 +
//! `core.snapshot-retention`), reference attestations, and the policy; receipts
6 7
//! live as long as the changes of their state. Operation
7 8
//! objects are always kept so the log stays walkable, but an operation older
8 9
//! than the retention window no longer keeps its states alive, so undoing
9 10
//! that far back may fail once its objects are gone.
10 11
11 12
use super::*;
12 -
use crate::store::{PackReport, PruneReport};
13 +
use crate::store::{PackReport, PruneReport, RetiredReport};
13 14
use std::collections::HashSet;
14 15
15 16
#[derive(Debug, Clone, Default, PartialEq, Eq)]
…
3 unchanged lines
19 20
    pub reachable: usize,
20 21
    /// Packs folded into the one a full repack writes.
21 22
    pub merged_packs: usize,
23 +
    /// Packs and loose objects earlier repacks retired, deleted now that
24 +
    /// [`crate::store::RETIRED_FOR`] has passed (oxygenhub#249).
25 +
    pub retired: RetiredReport,
22 26
}
23 27
24 28
impl Repo {
25 -
    /// Prune unreachable objects older than the grace period (`now`
26 -
    /// ignores it), then repack everything into one pack (spec §13.4).
29 +
    /// Delete what earlier repacks retired more than
30 +
    /// [`crate::store::RETIRED_FOR`] ago, prune unreachable objects older
31 +
    /// than the grace period (`now` ignores it, but not the retired
32 +
    /// files' hour), then repack everything into one pack (spec §13.4).
33 +
    /// The packs and loose objects this repack replaces are retired in
34 +
    /// turn, for the next collection. A store that is not this disk's
35 +
    /// directory then folds its signatures and index entries (§13.7).
27 36
    pub fn gc(&self, now: bool) -> Result<GcReport> {
28 37
        self.gc_with(now, &mut |_| {})
29 38
    }
…
1 unchanged lines
31 40
    /// [`Repo::gc`], reporting each phase and the packing's progress
32 41
    /// (oxygen#66), for a spinner or a log.
33 42
    pub fn gc_with(&self, now: bool, progress: crate::store::Progress) -> Result<GcReport> {
43 +
        self.gc_keeping(now, &[], progress)
44 +
    }
45 +
46 +
    /// [`Repo::gc_with`], keeping `extra` blobs and their chunks as well:
47 +
    /// objects a host stored for itself and names from outside the
48 +
    /// repository's history, such as a forge's job artifacts
49 +
    /// (oxygenhub#126).
50 +
    pub fn gc_keeping(
51 +
        &self,
52 +
        now: bool,
53 +
        extra: &[ObjectRef],
54 +
        progress: crate::store::Progress,
55 +
    ) -> Result<GcReport> {
56 +
        // Retired files first: the readers that used them have had their
57 +
        // hour, and what replaced them is in the packs read from here on.
58 +
        let retired = self
59 +
            .store
60 +
            .expire_retired(crate::store::RETIRED_FOR, progress)?;
34 61
        progress("finding what is reachable");
35 -
        let keep = self.reachable()?;
62 +
        let mut keep = self.reachable()?;
63 +
        for r in extra {
64 +
            if !self.store.has(r) || !keep.insert(*r) {
65 +
                continue;
66 +
            }
67 +
            if let Ok(body) = self.store.get_typed(r, ObjectType::Blob) {
68 +
                if let Ok(blob) = Blob::decode(&body) {
69 +
                    keep.extend(blob.chunks);
70 +
                }
71 +
            }
72 +
        }
36 73
        let grace = if now {
37 74
            std::time::Duration::ZERO
38 75
        } else {
…
5 unchanged lines
44 81
        let (packed, merged_packs) = self
45 82
            .store
46 83
            .repack_all_with(crate::store::REPACK_ZSTD_LEVEL, progress)?;
84 +
        // Last, on a store that is not this disk's directory, every
85 +
        // signature and index entry into one list (spec §13.7, #116).
86 +
        self.store.fold_entries(progress)?;
47 87
        Ok(GcReport {
48 88
            packed,
49 89
            pruned,
50 90
            reachable: keep.len(),
51 91
            merged_packs,
92 +
            retired,
52 93
        })
53 94
    }
54 95
…
5 unchanged lines
60 101
        let mut attests: Vec<ObjectRef> = Vec::new();
61 102
        let mut policies: Vec<ObjectRef> = Vec::new();
62 103
63 -
        // Operations: always kept; views within retention are roots.
64 -
        let cutoff = Hlc::now()
65 -
            .wall_ms
66 -
            .saturating_sub(self.config.op_retention_days * 86_400_000);
104 +
        // Operations: always kept; views within retention are roots. Past
105 +
        // working copies only within the shorter `core.snapshot-retention`
106 +
        // (oxygen#98): a file only ever snapshotted, such as a build output
107 +
        // a status took in, goes once they expire.
108 +
        let now_ms = Hlc::now().wall_ms;
109 +
        let cutoff = now_ms.saturating_sub(self.config.op_retention_days * 86_400_000);
110 +
        let wc_cutoff = now_ms
111 +
            .saturating_sub(self.config.snapshot_retention_days * 86_400_000)
112 +
            .max(cutoff);
67 113
        for (i, (r, op)) in self.op_log()?.into_iter().enumerate() {
68 114
            keep.insert(r);
69 115
            if i == 0 || op.hlc.wall_ms >= cutoff {
70 116
                states.extend(op.view.bookmarks.values().copied());
71 117
                states.extend(op.view.remotes.values().flat_map(|b| b.values().copied()));
118 +
                policies.extend(op.view.policy);
119 +
            }
120 +
            if i == 0 || (self.config.snapshot_retention_days > 0 && op.hlc.wall_ms >= wc_cutoff) {
72 121
                changes.extend(op.view.workspaces.values().copied());
73 -
                policies.extend(op.view.policy);
74 122
            }
75 123
        }
76 124
        // Checkpoints: the object, its payload blob, and the blob's chunks.
…
19 unchanged lines
96 144
        states.extend(view_now.tags.values().copied());
97 145
        for name in view_now.tags.keys() {
98 146
            if let Some((r, _)) = self.tag_attest(name)? {
147 +
                // With its issuer's acceptance (spec §14.7).
148 +
                if let Some(acc) = self.acceptance_of(&r)? {
149 +
                    attests.push(acc.attest.hash());
150 +
                }
99 151
                attests.push(r);
100 152
            }
101 153
        }
102 154
        // Reference attestations.
103 155
        for name in self.view()?.bookmarks.keys() {
104 156
            if let Some((r, _)) = self.latest_ref_attest(name)? {
157 +
                // With a host's acceptance of it (spec §14.7).
158 +
                if let Some(acc) = self.acceptance_of(&r)? {
159 +
                    attests.push(acc.attest.hash());
160 +
                }
105 161
                attests.push(r);
106 162
            }
107 163
        }
…
45 unchanged lines
153 209
                keep.insert(c.author);
154 210
            }
155 211
            for sig in self.store.signatures(&h)? {
156 -
                keep.insert(sig.key);
212 +
                self.keep_signer(&mut keep, sig.key)?;
157 213
            }
158 214
            for e in &c.edits {
159 215
                if let Edit::Replace { old, new, .. } = e {
…
9 unchanged lines
169 225
            attests.extend(self.store.seals_for(&h)?);
170 226
        }
171 227
        self.reachable_receipts(&mut keep)?;
228 +
        self.reachable_approvals(&mut keep)?;
172 229
        for a in attests {
173 230
            if keep.insert(a) {
174 231
                for sig in self.store.signatures(&a)? {
175 -
                    keep.insert(sig.key);
232 +
                    self.keep_signer(&mut keep, sig.key)?;
176 233
                }
177 234
            }
178 235
        }
179 236
        Ok(keep)
180 237
    }
238 +
239 +
    /// Keep a signer's identity and, for a short-lived key, its
240 +
    /// certificate and the certificate's issuer (spec §14.7).
241 +
    pub(crate) fn keep_signer(&self, keep: &mut HashSet<ObjectRef>, key: ObjectRef) -> Result<()> {
242 +
        keep.insert(key);
243 +
        if let Some(c) = self.store.certificate_of(&key)? {
244 +
            keep.insert(c);
245 +
            if let Ok(cert) = self.store.get_certificate(&c) {
246 +
                keep.insert(cert.cert.issuer);
247 +
            }
248 +
        }
249 +
        Ok(())
250 +
    }
181 251
}
modified crates/oxy-core/src/repo/git.rs +667 −28
…
112 unchanged lines
113 113
        let stdout = std::io::BufReader::new(child.stdout.take().unwrap());
114 114
        // A thread writes, so git never waits on a full pipe for us to read.
115 115
        let feeder = std::thread::spawn(move || {
116 +
            // An import that stops before reading every diff kills git
117 +
            // while this thread may still be writing. oxy restores
118 +
            // SIGPIPE's default (to end quietly under `| head`), so that
119 +
            // write would kill the whole process (signal 13, oxygen#81);
120 +
            // blocked here, it fails with EPIPE and the thread ends.
121 +
            #[cfg(unix)]
122 +
            unsafe {
123 +
                let mut set: libc::sigset_t = std::mem::zeroed();
124 +
                libc::sigemptyset(&mut set);
125 +
                libc::sigaddset(&mut set, libc::SIGPIPE);
126 +
                libc::pthread_sigmask(libc::SIG_BLOCK, &set, std::ptr::null_mut());
127 +
            }
116 128
            let mut w = std::io::BufWriter::new(&mut stdin);
117 129
            for (c, p) in pairs {
118 130
                if writeln!(w, "{c} {p}").is_err() {
…
66 unchanged lines
185 197
    }
186 198
}
187 199
200 +
/// The email of a Git author `Name <email>`, lowercased.
201 +
fn git_email(author: &str) -> Option<String> {
202 +
    let (_, rest) = author.rsplit_once('<')?;
203 +
    let email = rest.strip_suffix('>')?.trim();
204 +
    (!email.is_empty()).then(|| email.to_lowercase())
205 +
}
206 +
188 207
/// One commit as `git log` describes it.
189 208
struct GitCommit {
190 209
    sha: String,
…
97 unchanged lines
288 307
289 308
/// A merge bringing more commits than this sheds its base's graphs first.
290 309
const MERGE_SHED_COMMITS: usize = 100;
310 +
311 +
/// The atoms a kept state's built graphs may hold before they go back into
312 +
/// its history (#25): about a few hundred megabytes.
313 +
const KEPT_ATOMS: usize = 1_500_000;
291 314
/// A kept state with more built graphs than this sheds them.
292 315
const KEPT_GRAPHS: usize = 2000;
316 +
/// An import moves its bookmark to the commits done so far every this
317 +
/// many commits, so one that stops part way resumes from there: imported
318 +
/// commits are found from the bookmark's history (`imported_commits`).
319 +
const CHECKPOINT_COMMITS: usize = 2000;
293 320
294 321
/// Text files of a merge rendered this many at a time (oxygen#25): a
295 322
/// batch's graphs are built beside the state and dropped once rendered,
…
16 unchanged lines
312 339
    MERGE_BATCH_OVERRIDE
313 340
        .with(|c| c.get())
314 341
        .filter(|n| *n > 0)
342 +
        .or_else(|| {
343 +
            std::env::var("OXY_MERGE_BATCH")
344 +
                .ok()
345 +
                .and_then(|v| v.parse().ok())
346 +
        })
315 347
        .unwrap_or(MERGE_BATCH)
316 348
}
317 349
…
106 unchanged lines
424 456
425 457
impl Repo {
426 458
    fn git_marks_dir(&self, git_dir: &Path) -> Result<PathBuf> {
427 -
        let canonical = git_dir
428 -
            .canonicalize()
429 -
            .unwrap_or_else(|_| git_dir.to_path_buf());
459 +
        let canonical =
460 +
            crate::portable::canonical(git_dir).unwrap_or_else(|_| git_dir.to_path_buf());
430 461
        let key = hash_object(ObjectType::Chunk, canonical.to_string_lossy().as_bytes()).short(16);
431 462
        let dir = self.oxy.join("git").join(key);
432 463
        fs::create_dir_all(&dir)?;
…
80 unchanged lines
513 544
    /// (spec §23). Works on a repository that already has history: the
514 545
    /// new changes join the bookmark like a pull.
515 546
    pub fn git_fetch(&self, url: &str, branch: &str) -> Result<GitImportReport> {
547 +
        // A local path, made absolute: git runs it from inside the mirror.
548 +
        let local = crate::portable::canonical(Path::new(url))
549 +
            .ok()
550 +
            .filter(|_| !url.contains("://"));
551 +
        let absolute = local.as_ref().map(|p| p.to_string_lossy().into_owned());
552 +
        let url = absolute.as_deref().unwrap_or(url);
516 553
        let mirror = self.git_mirror(url)?;
517 554
        self.git_mirror_fetch(&mirror, url, branch)?;
518 555
        if git(
…
374 unchanged lines
893 930
                .unwrap_or(0)
894 931
        };
895 932
        let mut done = 0usize;
933 +
        // Atoms a kept state's graphs may hold before they are shed;
934 +
        // OXY_IMPORT_KEPT_ATOMS overrides it, for measuring.
935 +
        let kept_atoms: usize = std::env::var("OXY_IMPORT_KEPT_ATOMS")
936 +
            .ok()
937 +
            .and_then(|v| v.parse().ok())
938 +
            .unwrap_or(KEPT_ATOMS);
896 939
        // Graphs turned back into history (see `State::shed_graphs`).
897 940
        let mut shed = 0usize;
898 941
        let mut lap = |i: usize, clock: &mut std::time::Instant| {
…
2 unchanged lines
901 944
            *clock = now;
902 945
        };
903 946
947 +
        // OXY_IMPORT_CHECKPOINT overrides it, for testing.
948 +
        let checkpoint_every: usize = std::env::var("OXY_IMPORT_CHECKPOINT")
949 +
            .ok()
950 +
            .and_then(|v| v.parse().ok())
951 +
            .filter(|n| *n > 0)
952 +
            .unwrap_or(CHECKPOINT_COMMITS);
953 +
        // OXY_IMPORT_STOP_AFTER ends the import cleanly after that many
954 +
        // commits, its timings reported, for measuring at depth.
955 +
        let stop_after: Option<usize> = std::env::var("OXY_IMPORT_STOP_AFTER")
956 +
            .ok()
957 +
            .and_then(|v| v.parse().ok());
958 +
        // Merges built from the paths they can change (oxygen#80): on VS
959 +
        // Code at 139k commits the same 400 commits came out identical to
960 +
        // whole-tree merges, with merges 3.2x faster. OXY_IMPORT_WHOLE_MERGES=1
961 +
        // builds them from the whole tree, for comparing.
962 +
        let partial_merges = std::env::var_os("OXY_IMPORT_WHOLE_MERGES").is_none_or(|v| v == "0");
904 963
        let total = commits.len();
964 +
        // The importing identity's own commits, claimed (`git.claim`,
965 +
        // oxygen#97): a commit whose Git author email is its email gets
966 +
        // its key as author; anyone else's keeps the zero key.
967 +
        let claim: Option<(KeyId, String)> = if self.config.git_claim {
968 +
            let Some(key) = self.config.author_key else {
969 +
                return fail("git.claim authors your own commits with your key, and there is none: run `oxy identity create` first, or `oxy config git.claim false`");
970 +
            };
971 +
            Some((key, self.identity(&key)?.email.to_lowercase()))
972 +
        } else {
973 +
            None
974 +
        };
905 975
        for (index, gc) in commits.iter().enumerate() {
906 976
            if known.imported.contains_key(&gc.sha) || known.exported.contains_key(&gc.sha) {
907 977
                continue;
908 978
            }
979 +
            if stop_after.is_some_and(|n| done >= n) {
980 +
                break;
981 +
            }
982 +
            let commit_started = std::time::Instant::now();
909 983
            if index % 500 == 0 {
910 984
                progress(&format!(
911 985
                    "{index} of {total} commits, {} merges; at {} {}",
…
53 unchanged lines
965 1039
            // tree cannot vouch for them.
966 1040
            let mut touched: HashSet<NodeId> = HashSet::new();
967 1041
            let mut merged = 0usize;
1042 +
            // Where the first parent has each node, read before the merged
1043 +
            // changes move anything: a merge's partial build names the paths
1044 +
            // they touch in the first parent's tree as well as in the merged
1045 +
            // state (oxygen#80).
1046 +
            let mut first_parent_view: Option<crate::tree::TreeView> = None;
968 1047
            if gc.parents.len() > 1 {
969 1048
                report.merges += 1;
1049 +
                if prev_tree.is_some() && partial_merges {
1050 +
                    first_parent_view = Some(base.tree().view());
1051 +
                }
970 1052
                let mut args: Vec<String> = gc.parents[1..].to_vec();
971 1053
                args.push(format!("^{}", gc.parents[0]));
972 1054
                let refs: Vec<&str> = args.iter().map(String::as_str).collect();
…
7 unchanged lines
980 1062
                    let mut count_args = vec!["rev-list", "--count"];
981 1063
                    count_args.extend(refs.iter().copied());
982 1064
                    let others: usize = git(git_dir, &count_args)?.trim().parse().unwrap_or(0);
983 -
                    if others > MERGE_SHED_COMMITS {
1065 +
                    let shed_at = std::env::var("OXY_MERGE_SHED")
1066 +
                        .ok()
1067 +
                        .and_then(|v| v.parse().ok())
1068 +
                        .unwrap_or(MERGE_SHED_COMMITS);
1069 +
                    if others > shed_at {
984 1070
                        shed += base.shed_graphs(|b| self.store_blob(b))?;
985 1071
                    }
986 1072
                }
…
40 unchanged lines
1027 1113
            // files that decide how they are stored, and the directories its
1028 1114
            // removals may leave empty. A merge, or a commit whose parent's
1029 1115
            // tree is not at hand, is built from its whole tree.
1116 +
            // A merge can differ from its first parent only at the paths
1117 +
            // Git's diff names and where the merged changes touched: those
1118 +
            // paths in the first parent's tree and in the merged state. With
1119 +
            // them added it is built like an ordinary commit, instead of
1120 +
            // from its whole tree (18 s for any merge at 118k commits of
1121 +
            // VS Code, oxygen#80). OXY_IMPORT_WHOLE_MERGES=1 takes the
1122 +
            // whole-tree path.
1123 +
            let mut changed = changed;
1124 +
            // A tree conflict in the merged state (both sides placed a file
1125 +
            // at one path) is resolved by the whole-tree path.
1126 +
            let mut merge_how = if gc.parents.len() > 1 {
1127 +
                if first_parent_view.is_some() {
1128 +
                    "partial"
1129 +
                } else {
1130 +
                    "whole"
1131 +
                }
1132 +
            } else {
1133 +
                "-"
1134 +
            };
1135 +
            // Only a conflict among what the merged changes touched; one
1136 +
            // elsewhere stays as it was in the first parent either way.
1137 +
            if first_parent_view.is_some()
1138 +
                && base.tree().view().conflicts.iter().any(|c| {
1139 +
                    use crate::tree::TreeConflict::*;
1140 +
                    match c {
1141 +
                        Name { nodes, .. } => nodes.iter().any(|n| touched.contains(n)),
1142 +
                        MultiPlacement { node, .. } | Cycle { node } | Mode { node, .. } => {
1143 +
                            touched.contains(node)
1144 +
                        }
1145 +
                        Orphan { node, parent } => {
1146 +
                            touched.contains(node) || touched.contains(parent)
1147 +
                        }
1148 +
                    }
1149 +
                })
1150 +
            {
1151 +
                first_parent_view = None;
1152 +
                merge_how = "conflicted";
1153 +
            }
1154 +
            if let (Some(ch), Some(fp)) = (changed.as_mut(), &first_parent_view) {
1155 +
                let union = base.tree().view();
1156 +
                for v in [fp, &union] {
1157 +
                    for (path, e) in v.entries.iter() {
1158 +
                        if touched.contains(&e.node) {
1159 +
                            ch.insert(path.clone());
1160 +
                        }
1161 +
                    }
1162 +
                }
1163 +
            }
1030 1164
            let partial = changed.is_some()
1031 -
                && gc.parents.len() == 1
1165 +
                && (gc.parents.len() == 1 || first_parent_view.is_some())
1032 1166
                && !tree.values().any(|(mode, _)| mode == "160000");
1033 1167
            let mut rel: BTreeSet<&str> = BTreeSet::new();
1168 +
            // Text files whose graphs this commit builds, for the trace.
1169 +
            let need_n: usize;
1034 1170
            let base_snapshot = if let (true, Some(changed)) = (partial, &changed) {
1035 1171
                let view = base.tree().view();
1036 1172
                rel.extend(changed.iter().map(String::as_str));
…
19 unchanged lines
1056 1192
                    .map(|e| e.node)
1057 1193
                    .collect();
1058 1194
                crate::timing::add("render: same and need", clock.elapsed());
1195 +
                need_n = need.len();
1059 1196
                let t = std::time::Instant::now();
1060 1197
                self.load_graphs_of(&mut base, &need)?;
1061 1198
                crate::timing::add("render: load graphs", t.elapsed());
…
37 unchanged lines
1099 1236
                    .map(|e| e.node)
1100 1237
                    .collect();
1101 1238
                crate::timing::add("render: same and need", clock.elapsed());
1239 +
                need_n = need.len();
1102 1240
                let t = std::time::Instant::now();
1103 1241
                let snap = if need.len() > merge_batch() {
1104 1242
                    // A merge onto a base with no built graphs: thousands
1105 1243
                    // of files, whose graphs would not fit at once.
1106 1244
                    self.materialize_batched(&mut base, &need, &skip)?
1107 1245
                } else {
1246 +
                    let g = std::time::Instant::now();
1108 1247
                    self.load_graphs_of(&mut base, &need)?;
1109 -
                    base.materialize_except(&skip)
1248 +
                    crate::timing::add("render: whole, load graphs", g.elapsed());
1249 +
                    let m = std::time::Instant::now();
1250 +
                    let snap = base.materialize_except(&skip);
1251 +
                    crate::timing::add("render: whole, materialize", m.elapsed());
1252 +
                    snap
1110 1253
                };
1111 1254
                crate::timing::add("render: whole tree", t.elapsed());
1112 1255
                snap
…
151 unchanged lines
1264 1407
                version: crate::objects::CHANGE_VERSION,
1265 1408
                change_id: change_id_for(&gc.sha, &gc.message),
1266 1409
                deps: vec![],
1267 -
                author: ObjectRef::ZERO,
1410 +
                author: claim
1411 +
                    .as_ref()
1412 +
                    .filter(|(_, email)| git_email(&gc.author).is_some_and(|e| e == *email))
1413 +
                    .map_or(ObjectRef::ZERO, |(key, _)| *key),
1268 1414
                hlc: Hlc {
1269 1415
                    wall_ms: gc.author_secs * 1000,
1270 1416
                    logical: 0,
…
40 unchanged lines
1311 1457
            if waiting.get(gc.sha.as_str()).is_some_and(|n| *n > 0) {
1312 1458
                // A kept state holds its graphs until its children come:
1313 1459
                // past a few thousand, they go back into its history.
1314 -
                if base.built_count() > KEPT_GRAPHS {
1460 +
                // Past a few thousand graphs, or past a budget of atoms: a
1461 +
                // long-lived file's graph keeps every line it ever had, so a
1462 +
                // handful of them can hold gigabytes (#25).
1463 +
                if base.built_count() > KEPT_GRAPHS || base.built_atoms() > kept_atoms {
1315 1464
                    shed += base.shed_graphs(|b| self.store_blob(b))?;
1316 1465
                }
1317 1466
                kept.insert(gc.sha.clone(), (base, tree));
…
8 unchanged lines
1326 1475
            report.commits += 1;
1327 1476
            lap(5, &mut clock);
1328 1477
            done += 1;
1478 +
            if done.is_multiple_of(checkpoint_every) {
1479 +
                self.checkpoint_import(&heads_now, done)?;
1480 +
            }
1481 +
            // A merge renders thousands of files on many threads and frees
1482 +
            // it all; the allocator keeps those pages unless asked to give
1483 +
            // them back, and the next merge piles on top (#25).
1484 +
            let before_trim = if trace { rss_mb() } else { 0 };
1485 +
            if gc.parents.len() > 1 {
1486 +
                crate::alloc_stats::release_freed_memory();
1487 +
            }
1329 1488
            if trace {
1330 1489
                let built = kept
1331 1490
                    .get(gc.sha.as_str())
1332 1491
                    .map(|(st, _)| st.built_count())
1333 1492
                    .unwrap_or(0);
1493 +
                let (edits, most) = kept
1494 +
                    .get(gc.sha.as_str())
1495 +
                    .map(|(st, _)| st.deferred_edit_count())
1496 +
                    .unwrap_or((0, 0));
1497 +
                let atoms = kept
1498 +
                    .get(gc.sha.as_str())
1499 +
                    .map(|(st, _)| st.built_atoms())
1500 +
                    .unwrap_or(0);
1334 1501
                eprintln!(
1335 -
                    "trace {done} {} parents={} base={base_how} merged={merged} built={built} kept={} shed={shed} rss_mb={}",
1502 +
                    "trace {done} {} parents={} base={base_how} merged={merged} built={built} atoms={atoms} kept={} shed={shed} rss_mb={} before_trim_mb={before_trim} deferred_edits={edits} most={most} need={need_n} merge={merge_how} ms={}",
1336 1503
                    &gc.sha[..12.min(gc.sha.len())],
1337 1504
                    gc.parents.len(),
1338 1505
                    kept.len(),
1339 -
                    rss_mb()
1506 +
                    rss_mb(),
1507 +
                    commit_started.elapsed().as_millis()
1340 1508
                );
1341 1509
            }
1342 1510
        }
…
48 unchanged lines
1391 1559
            base.sort();
1392 1560
            let mut sorted_old = old_heads.clone();
1393 1561
            sorted_old.sort();
1394 -
            if clean && base == sorted_old {
1562 +
            // An empty base too: an import that stopped after a checkpoint
1563 +
            // never checked anything out, and resuming it should.
1564 +
            if clean && (base == sorted_old || base.is_empty()) {
1395 1565
                self.checkout(&heads, false, &format!("{command} (checkout)"))?;
1396 1566
            }
1397 1567
        }
…
67 unchanged lines
1465 1635
        Ok(())
1466 1636
    }
1467 1637
1638 +
    /// Recover an import that stopped before it moved the bookmark: its
1639 +
    /// unfinished pack indexed (`Store::adopt_unfinished_packs`), then every
1640 +
    /// change in the store that carries a `Git-Commit:` trailer, its heads
1641 +
    /// (those no other such change depends on) made the bookmark's heads,
1642 +
    /// so `oxy git fetch` carries on from there. Reads every object once.
1643 +
    /// Returns how many imported changes it found and the heads.
1644 +
    pub fn recover_git_import(&self, progress: &dyn Fn(&str)) -> Result<(usize, usize)> {
1645 +
        // A stopped import's objects are in a pack it never indexed.
1646 +
        // OXY_RECOVER_MIN_AGE (seconds) overrides the five minutes it must
1647 +
        // have been left alone, for testing.
1648 +
        let min_age = std::env::var("OXY_RECOVER_MIN_AGE")
1649 +
            .ok()
1650 +
            .and_then(|v| v.parse().ok())
1651 +
            .unwrap_or(300);
1652 +
        self.store
1653 +
            .adopt_unfinished_packs(std::time::Duration::from_secs(min_age), progress)?;
1654 +
        let all = self
1655 +
            .store
1656 +
            .list()
1657 +
            .map_err(|e| RepoError::Msg(format!("listing the store: {e}")))?;
1658 +
        let total = all.len();
1659 +
        let mut found: HashMap<ChangeHash, Vec<ChangeHash>> = HashMap::new();
1660 +
        for (i, r) in all.iter().enumerate() {
1661 +
            if i.is_multiple_of(100_000) {
1662 +
                progress(&format!(
1663 +
                    "{i} of {total} objects read, {} imported changes",
1664 +
                    found.len()
1665 +
                ));
1666 +
            }
1667 +
            let Ok(meta) = self.store.get_change_meta(r) else {
1668 +
                continue;
1669 +
            };
1670 +
            if meta.message.lines().any(|l| l.starts_with("Git-Commit: ")) {
1671 +
                found.insert(*r, meta.deps.clone());
1672 +
            }
1673 +
        }
1674 +
        if found.is_empty() {
1675 +
            return fail("no imported Git commit's change in this store");
1676 +
        }
1677 +
        let depended: HashSet<ChangeHash> = found.values().flatten().copied().collect();
1678 +
        let mut heads: Vec<ChangeHash> = found
1679 +
            .keys()
1680 +
            .filter(|h| !depended.contains(h))
1681 +
            .copied()
1682 +
            .collect();
1683 +
        // The bookmark's present heads stay: a recovery only adds.
1684 +
        heads.extend(self.bookmark_heads().unwrap_or_default());
1685 +
        let st = self.load_state_history(&heads)?;
1686 +
        let mut heads = st.maximal(&heads);
1687 +
        heads.sort();
1688 +
        heads.dedup();
1689 +
        let (op_ref, op) = self.head_op()?;
1690 +
        let mut view = op.view.clone();
1691 +
        let state_ref = self.store.put_state(&StateObject::new(heads.clone()))?;
1692 +
        view.bookmarks
1693 +
            .insert(self.config.bookmark.clone(), state_ref);
1694 +
        self.record_op(&[op_ref], "oxy git recover", view)?;
1695 +
        Ok((found.len(), heads.len()))
1696 +
    }
1697 +
1698 +
    /// Point the bookmark at an import's heads so far (see
1699 +
    /// `CHECKPOINT_COMMITS`); the working copy is left alone until the end.
1700 +
    fn checkpoint_import(&self, heads: &BTreeSet<ChangeHash>, done: usize) -> Result<()> {
1701 +
        let heads: Vec<ChangeHash> = heads.iter().copied().collect();
1702 +
        let (op_ref, op) = self.head_op()?;
1703 +
        let mut view = op.view.clone();
1704 +
        let state_ref = self.store.put_state(&StateObject::new(heads))?;
1705 +
        view.bookmarks
1706 +
            .insert(self.config.bookmark.clone(), state_ref);
1707 +
        self.record_op(
1708 +
            &[op_ref],
1709 +
            &format!("oxy git import (checkpoint after {done} commits)"),
1710 +
            view,
1711 +
        )?;
1712 +
        Ok(())
1713 +
    }
1714 +
1468 1715
    /// Commits this repository can relate to Oxygen history: imported
1469 1716
    /// ones by change, exported ones (through `git_dir`'s marks) by state.
1470 1717
    fn git_known(&self, git_dir: &Path) -> Result<GitKnown> {
…
23 unchanged lines
1494 1741
    }
1495 1742
1496 1743
    fn git_author(&self, c: &Change) -> String {
1497 -
        match self.store.get_identity(&c.author) {
1498 -
            Ok(id) if !id.name.is_empty() => format!(
1499 -
                "{} <{}>",
1500 -
                id.name.replace(['<', '>', '\n'], " "),
1501 -
                id.email.replace(['<', '>', '\n', ' '], "")
1502 -
            ),
1503 -
            _ => "oxy <oxy@localhost>".to_string(),
1504 -
        }
1744 +
        self.git_author_key(&c.author)
1505 1745
    }
1506 1746
1507 1747
    #[allow(clippy::too_many_arguments)]
…
93 unchanged lines
1601 1841
    pub files_compared: usize,
1602 1842
    /// `(commit, what differs)`; empty when every commit's tree is reproduced.
1603 1843
    pub mismatches: Vec<(String, String)>,
1844 +
    /// Commits of the branch after the last imported one: an import not
1845 +
    /// finished yet. Not checked.
1846 +
    pub not_yet: usize,
1604 1847
}
1605 1848
1606 1849
impl Repo {
…
3 unchanged lines
1610 1853
    /// for byte, with the same executable bits, symlink targets and
1611 1854
    /// submodule pins. Reads only; stops listing after 50 differences.
1612 1855
    pub fn verify_git_import(&self, git_dir: &Path, branch: &str) -> Result<GitVerifyReport> {
1613 -
        self.verify_git_import_last(git_dir, branch, None)
1856 +
        self.verify_git_import_last(git_dir, branch, None, 50)
1614 1857
    }
1615 1858
1616 1859
    /// [`Repo::verify_git_import`] of the last `last` commits only (all
1617 1860
    /// when `None`): earlier ones are only followed, not rebuilt, for
1618 1861
    /// looking again at where an import went wrong.
1862 +
    /// It stops after `max` differences (0: never), so one that repeats
1863 +
    /// through history does not hide the commits after it.
1619 1864
    pub fn verify_git_import_last(
1620 1865
        &self,
1621 1866
        git_dir: &Path,
1622 1867
        branch: &str,
1623 1868
        last: Option<usize>,
1869 +
        max: usize,
1624 1870
    ) -> Result<GitVerifyReport> {
1625 -
        let commits = list_commits(git_dir, branch)?;
1871 +
        let mut rep = GitVerifyReport::default();
1872 +
        let mut commits = list_commits(git_dir, branch)?;
1626 1873
        let imported = self.imported_commits()?;
1874 +
        // An import still going (or stopped) has reached part of the
1875 +
        // branch: what follows its last imported commit is not checked,
1876 +
        // only counted, and --last counts back from there (oxygen#81).
1877 +
        if let Some(end) = commits.iter().rposition(|c| imported.contains_key(&c.sha)) {
1878 +
            let rest = commits.len() - end - 1;
1879 +
            if rest > 0 {
1880 +
                commits.truncate(end + 1);
1881 +
            }
1882 +
            rep.not_yet = rest;
1883 +
        }
1627 1884
        let mut cat = CatFile::start(git_dir)?;
1628 -
        let mut rep = GitVerifyReport::default();
1629 1885
        // Changes of each commit and all its ancestors, built up in topological order.
1630 1886
        let mut closure: HashMap<String, Arc<BTreeSet<ChangeHash>>> = HashMap::new();
1631 1887
        // Graphs (by key) and blobs already found equal to a Git blob.
…
3 unchanged lines
1635 1891
        // come, so a child adds its own changes instead of loading all its
1636 1892
        // history again (as the import does).
1637 1893
        let mut waiting: HashMap<&str, usize> = HashMap::new();
1894 +
        // Children still to come per commit, through any parent: a commit's
1895 +
        // closure is dropped after its last child, so only the open
1896 +
        // frontier is held. Kept for every commit, the closures grew with
1897 +
        // the square of the history and VS Code's verify ran out of memory
1898 +
        // (oxygen#81).
1899 +
        let mut children: HashMap<&str, usize> = HashMap::new();
1638 1900
        for c in &commits {
1639 1901
            if let Some(p1) = c.parents.first() {
1640 1902
                *waiting.entry(p1.as_str()).or_default() += 1;
1641 1903
            }
1904 +
            for p in &c.parents {
1905 +
                *children.entry(p.as_str()).or_default() += 1;
1906 +
            }
1642 1907
        }
1643 1908
        let mut kept: HashMap<String, State> = HashMap::new();
1644 1909
        let mut kept_order: std::collections::VecDeque<String> = std::collections::VecDeque::new();
1645 1910
        for c in &commits {
1646 1911
            rep.commits += 1;
1912 +
            // The first parent's closure, taken over when this is its last
1913 +
            // child (no copy), then the other parents' added.
1647 1914
            let mut set: BTreeSet<ChangeHash> = BTreeSet::new();
1648 -
            for p in &c.parents {
1649 -
                if let Some(ps) = closure.get(p) {
1650 -
                    set.extend(ps.iter().copied());
1915 +
            for (i, p) in c.parents.iter().enumerate() {
1916 +
                let left = children.get_mut(p.as_str()).map(|n| {
1917 +
                    *n -= 1;
1918 +
                    *n
1919 +
                });
1920 +
                let ps = if left == Some(0) {
1921 +
                    closure.remove(p)
1922 +
                } else {
1923 +
                    closure.get(p).cloned()
1924 +
                };
1925 +
                if let Some(ps) = ps {
1926 +
                    if i == 0 && set.is_empty() {
1927 +
                        set = Arc::try_unwrap(ps).unwrap_or_else(|a| (*a).clone());
1928 +
                    } else {
1929 +
                        set.extend(ps.iter().copied());
1930 +
                    }
1651 1931
                }
1652 1932
            }
1653 1933
            match imported.get(&c.sha) {
…
8 unchanged lines
1662 1942
                }
1663 1943
            }
1664 1944
            let set = Arc::new(set);
1665 -
            closure.insert(c.sha.clone(), set.clone());
1666 -
            if rep.mismatches.len() >= 50 {
1945 +
            if children.get(c.sha.as_str()).is_some_and(|n| *n > 0) {
1946 +
                closure.insert(c.sha.clone(), set.clone());
1947 +
            }
1948 +
            if max > 0 && rep.mismatches.len() >= max {
1667 1949
                break;
1668 1950
            }
1669 1951
            if last.is_some_and(|n| rep.commits + n <= commits.len()) {
…
186 unchanged lines
1856 2138
                    }
1857 2139
                }
1858 2140
            }
1859 -
            if rep.mismatches.len() >= 50 {
2141 +
            if max > 0 && rep.mismatches.len() >= max {
1860 2142
                break;
1861 2143
            }
1862 2144
        }
…
25 unchanged lines
1888 2170
        w.len()
1889 2171
    )
1890 2172
}
2173 +
2174 +
/// What `git_export_graph` wrote.
2175 +
#[derive(Debug, Clone, Default, PartialEq, Eq)]
2176 +
pub struct GitGraphReport {
2177 +
    /// Changes written as commits this time (the rest were exported before).
2178 +
    pub commits: usize,
2179 +
    /// Branches and tags pointed at their commits.
2180 +
    pub branches: usize,
2181 +
    pub tags: usize,
2182 +
    /// Bookmarks or tags whose names Git does not take as refs.
2183 +
    pub skipped_refs: Vec<String>,
2184 +
    /// Subrepo nodes with Oxygen pins, which Git cannot represent.
2185 +
    pub skipped_subrepos: usize,
2186 +
}
2187 +
2188 +
/// A bookmark or tag name Git takes as a ref without escaping.
2189 +
fn git_ref_ok(name: &str) -> bool {
2190 +
    !name.is_empty()
2191 +
        && !name.starts_with(['-', '/', '.'])
2192 +
        && !name.ends_with(['/', '.'])
2193 +
        && !name.contains("..")
2194 +
        && !name.contains("//")
2195 +
        && !name.ends_with(".lock")
2196 +
        && name
2197 +
            .bytes()
2198 +
            .all(|b| b.is_ascii_alphanumeric() || b"-_./+".contains(&b))
2199 +
}
2200 +
2201 +
impl Repo {
2202 +
    /// Export every bookmark and tag to the bare Git repository at
2203 +
    /// `git_dir` from the change graph (spec §23, oxygenhub D1), for a hub
2204 +
    /// to serve to Git clients. Unlike `git_export`, which follows one
2205 +
    /// checkout's operation log, this depends only on the changes, so the
2206 +
    /// same changes give the same Git commits wherever it runs:
2207 +
    ///
2208 +
    /// - each change is a commit whose tree is the state of the change and
2209 +
    ///   everything it depends on, whose parents are its dependencies'
2210 +
    ///   commits, with the change's author, time and message and its
2211 +
    ///   `Change-Id:` and `Oxy-Hash:` trailers (a change that depends on
2212 +
    ///   nothing is a root, so Git sees Oxygen's dependencies, not the order
2213 +
    ///   changes were made in);
2214 +
    /// - a bookmark with one head is a branch at that head's commit; with
2215 +
    ///   several, a merge of their commits whose tree is the bookmark's
2216 +
    ///   state, at the newest head's time;
2217 +
    /// - a tag points at its heads' commit, or such a merge.
2218 +
    ///
2219 +
    /// Commits already exported are remembered (marks), so a later run
2220 +
    /// writes only new changes. Branches for bookmarks that are gone are
2221 +
    /// deleted.
2222 +
    pub fn git_export_graph(&self, git_dir: &Path) -> Result<GitGraphReport> {
2223 +
        if !git_dir.join("HEAD").exists() {
2224 +
            fs::create_dir_all(git_dir)?;
2225 +
            git(git_dir, &["init", "--bare", "-q"])?;
2226 +
        }
2227 +
        let marks_dir = self.git_marks_dir(git_dir)?;
2228 +
        let git_marks = marks_dir.join("git-marks");
2229 +
        let graph_marks = marks_dir.join("graph-marks");
2230 +
        let mut marks = Self::read_oxy_marks(&graph_marks)?;
2231 +
        let mut next = marks.values().copied().max().unwrap_or(0) + 1;
2232 +
        let mut report = GitGraphReport::default();
2233 +
        let mut stream: Vec<u8> = Vec::new();
2234 +
2235 +
        let mut refs: Vec<(String, Vec<ChangeHash>)> = Vec::new();
2236 +
        for b in self.bookmarks()? {
2237 +
            if git_ref_ok(&b.name) {
2238 +
                refs.push((format!("refs/heads/{}", b.name), b.heads));
2239 +
            } else {
2240 +
                report.skipped_refs.push(b.name);
2241 +
            }
2242 +
        }
2243 +
        let tags = self.tags()?;
2244 +
        for t in &tags {
2245 +
            if git_ref_ok(&t.name) {
2246 +
                refs.push((format!("refs/tags/{}", t.name), t.heads.clone()));
2247 +
            } else {
2248 +
                report.skipped_refs.push(t.name.clone());
2249 +
            }
2250 +
        }
2251 +
2252 +
        // Every change the refs reach, dependencies before dependents: a
2253 +
        // depth-first walk in hash order, so the order is the same everywhere.
2254 +
        let mut order: Vec<ChangeHash> = Vec::new();
2255 +
        let mut seen: HashSet<ChangeHash> = HashSet::new();
2256 +
        let mut metas: HashMap<ChangeHash, crate::state::ChangeMeta> = HashMap::new();
2257 +
        let mut roots: Vec<ChangeHash> = refs.iter().flat_map(|(_, h)| h.iter().copied()).collect();
2258 +
        roots.sort();
2259 +
        roots.dedup();
2260 +
        for root in roots {
2261 +
            if !seen.insert(root) {
2262 +
                continue;
2263 +
            }
2264 +
            let mut stack: Vec<(ChangeHash, bool)> = vec![(root, false)];
2265 +
            while let Some((h, done)) = stack.pop() {
2266 +
                if done {
2267 +
                    order.push(h);
2268 +
                    continue;
2269 +
                }
2270 +
                stack.push((h, true));
2271 +
                let meta = self.store.get_change_meta(&h)?;
2272 +
                let mut deps = meta.deps.clone();
2273 +
                deps.sort();
2274 +
                for d in deps.into_iter().rev() {
2275 +
                    if seen.insert(d) {
2276 +
                        stack.push((d, false));
2277 +
                    }
2278 +
                }
2279 +
                metas.insert(h, meta);
2280 +
            }
2281 +
        }
2282 +
2283 +
        for h in &order {
2284 +
            if marks.contains_key(h) {
2285 +
                continue;
2286 +
            }
2287 +
            let meta = &metas[h];
2288 +
            let snapshot = self.load_state(&[*h])?.materialize();
2289 +
            let mut message = meta.message.clone();
2290 +
            if !message.ends_with('\n') {
2291 +
                message.push('\n');
2292 +
            }
2293 +
            message.push_str(&format!(
2294 +
                "\nChange-Id: {}\nOxy-Hash: {}\n",
2295 +
                meta.change_id.to_hex(),
2296 +
                h.to_hex()
2297 +
            ));
2298 +
            if snapshot.is_conflicted() {
2299 +
                message.push_str("Oxy-Conflicted: true\n");
2300 +
            }
2301 +
            let mut deps = meta.deps.clone();
2302 +
            deps.sort();
2303 +
            let parents: Vec<String> = deps
2304 +
                .iter()
2305 +
                .filter_map(|d| marks.get(d))
2306 +
                .map(|m| format!(":{m}"))
2307 +
                .collect();
2308 +
            let mark = next;
2309 +
            next += 1;
2310 +
            if parents.is_empty() {
2311 +
                // Without `from`, fast-import would make the scratch ref's
2312 +
                // tip its parent; a change that depends on nothing is a root.
2313 +
                stream.extend_from_slice(b"reset refs/oxy/export\n\n");
2314 +
            }
2315 +
            self.write_commit(
2316 +
                &mut stream,
2317 +
                "refs/oxy/export",
2318 +
                mark,
2319 +
                &self.git_author_key(&meta.author),
2320 +
                meta.hlc.wall_ms / 1000,
2321 +
                &message,
2322 +
                &parents,
2323 +
                &snapshot,
2324 +
            )?;
2325 +
            marks.insert(*h, mark);
2326 +
            report.commits += 1;
2327 +
        }
2328 +
2329 +
        let mut wanted: HashSet<String> = HashSet::new();
2330 +
        for (name, heads) in &refs {
2331 +
            let mut heads = heads.clone();
2332 +
            heads.sort();
2333 +
            heads.dedup();
2334 +
            let target = match heads.as_slice() {
2335 +
                [] => continue,
2336 +
                [one] => format!(":{}", marks[one]),
2337 +
                many => {
2338 +
                    // A merge of the heads, named by them so it is made once.
2339 +
                    let key = hash_object(
2340 +
                        ObjectType::Chunk,
2341 +
                        many.iter()
2342 +
                            .map(|h| h.to_hex())
2343 +
                            .collect::<Vec<_>>()
2344 +
                            .join(" ")
2345 +
                            .as_bytes(),
2346 +
                    );
2347 +
                    if let Some(m) = marks.get(&key) {
2348 +
                        format!(":{m}")
2349 +
                    } else {
2350 +
                        let snapshot = self.load_state(many)?.materialize();
2351 +
                        let newest = many.iter().map(|h| metas[h].hlc.wall_ms).max().unwrap_or(0);
2352 +
                        let mut message = format!(
2353 +
                            "Merge of {} heads\n\nOxy-Heads: {}\n",
2354 +
                            many.len(),
2355 +
                            many.iter()
2356 +
                                .map(|h| h.to_hex())
2357 +
                                .collect::<Vec<_>>()
2358 +
                                .join(" ")
2359 +
                        );
2360 +
                        if snapshot.is_conflicted() {
2361 +
                            message.push_str("Oxy-Conflicted: true\n");
2362 +
                        }
2363 +
                        let parents: Vec<String> =
2364 +
                            many.iter().map(|h| format!(":{}", marks[h])).collect();
2365 +
                        let mark = next;
2366 +
                        next += 1;
2367 +
                        self.write_commit(
2368 +
                            &mut stream,
2369 +
                            "refs/oxy/export",
2370 +
                            mark,
2371 +
                            "oxy <oxy@localhost>",
2372 +
                            newest / 1000,
2373 +
                            &message,
2374 +
                            &parents,
2375 +
                            &snapshot,
2376 +
                        )?;
2377 +
                        marks.insert(key, mark);
2378 +
                        format!(":{mark}")
2379 +
                    }
2380 +
                }
2381 +
            };
2382 +
            stream.extend_from_slice(format!("reset {name}\nfrom {target}\n\n").as_bytes());
2383 +
            wanted.insert(name.clone());
2384 +
            if name.starts_with("refs/tags/") {
2385 +
                report.tags += 1;
2386 +
            } else {
2387 +
                report.branches += 1;
2388 +
            }
2389 +
        }
2390 +
        stream.extend_from_slice(b"done\n");
2391 +
        run_fast_import(git_dir, &stream, &git_marks)?;
2392 +
        let text: String = marks
2393 +
            .iter()
2394 +
            .map(|(h, m)| format!("{} {m}\n", h.to_hex()))
2395 +
            .collect();
2396 +
        write_atomic(&graph_marks, text.as_bytes())?;
2397 +
        let _ = git(git_dir, &["update-ref", "-d", "refs/oxy/export"]);
2398 +
        // Branches and tags of bookmarks and tags that are gone.
2399 +
        let listed = git(
2400 +
            git_dir,
2401 +
            &[
2402 +
                "for-each-ref",
2403 +
                "--format=%(refname)",
2404 +
                "refs/heads",
2405 +
                "refs/tags",
2406 +
            ],
2407 +
        )?;
2408 +
        for r in listed
2409 +
            .lines()
2410 +
            .map(str::trim)
2411 +
            .filter(|r| !r.is_empty() && !wanted.contains(*r))
2412 +
        {
2413 +
            git(git_dir, &["update-ref", "-d", r])?;
2414 +
        }
2415 +
        // HEAD names the main bookmark when there is one, so a clone checks it out.
2416 +
        let main = ["main", "master", "trunk", "develop"]
2417 +
            .into_iter()
2418 +
            .find(|b| wanted.contains(&format!("refs/heads/{b}")));
2419 +
        if let Some(b) = main {
2420 +
            git(
2421 +
                git_dir,
2422 +
                &["symbolic-ref", "HEAD", &format!("refs/heads/{b}")],
2423 +
            )?;
2424 +
        }
2425 +
        report.skipped_subrepos = self.git_export_skipped.replace(0);
2426 +
        Ok(report)
2427 +
    }
2428 +
2429 +
    /// `Name <email>` for a Git commit, from the author's identity.
2430 +
    fn git_author_key(&self, key: &KeyId) -> String {
2431 +
        match self.store.get_identity(key) {
2432 +
            Ok(id) if !id.name.is_empty() => format!(
2433 +
                "{} <{}>",
2434 +
                id.name.replace(['<', '>', '\n'], " "),
2435 +
                id.email.replace(['<', '>', '\n', ' '], "")
2436 +
            ),
2437 +
            _ => "oxy <oxy@localhost>".to_string(),
2438 +
        }
2439 +
    }
2440 +
}
2441 +
2442 +
/// Feed a fast-import stream to `git fast-import` in `git_dir`, keeping its
2443 +
/// marks in `git_marks` so a later run can refer to commits it wrote.
2444 +
fn run_fast_import(git_dir: &Path, stream: &[u8], git_marks: &Path) -> Result<()> {
2445 +
    let mut cmd = Command::new("git");
2446 +
    // `--force`: a ref may move to an unrelated commit (a bookmark rewritten).
2447 +
    cmd.arg("-C")
2448 +
        .arg(git_dir)
2449 +
        .args(["fast-import", "--quiet", "--done", "--force"])
2450 +
        .arg(format!("--export-marks={}", git_marks.display()));
2451 +
    if git_marks.exists() {
2452 +
        cmd.arg(format!("--import-marks={}", git_marks.display()));
2453 +
    }
2454 +
    let mut child = cmd
2455 +
        .stdin(Stdio::piped())
2456 +
        .stdout(Stdio::null())
2457 +
        .stderr(Stdio::piped())
2458 +
        .spawn()
2459 +
        .map_err(|e| RepoError::Msg(format!("running git fast-import: {e}")))?;
2460 +
    child.stdin.take().unwrap().write_all(stream)?;
2461 +
    let out = child.wait_with_output()?;
2462 +
    if !out.status.success() {
2463 +
        return fail(format!(
2464 +
            "git fast-import failed: {}",
2465 +
            String::from_utf8_lossy(&out.stderr).trim()
2466 +
        ));
2467 +
    }
2468 +
    Ok(())
2469 +
}
2470 +
2471 +
impl Repo {
2472 +
    /// Bring the side-by-side Git repository up to date (`git.colocate`):
2473 +
    /// a `.git` beside `.oxy`, made with `git init` when missing and told to
2474 +
    /// leave `.oxy/` alone, holding the current bookmark as exported by
2475 +
    /// `git_export_as` (one commit per state the bookmark passed through),
2476 +
    /// with HEAD on that branch and Git's index at its tree, so `git status`
2477 +
    /// shows only the working copy's uncommitted work. Files on disk are
2478 +
    /// never touched. Does nothing when the bookmark has not moved since the
2479 +
    /// last time; `Ok(true)` when it exported.
2480 +
    pub fn git_colocate_sync(&self) -> Result<bool> {
2481 +
        if self.bare {
2482 +
            return fail("a bare repository has no working copy for a .git beside it");
2483 +
        }
2484 +
        let git_dir = self.root.join(".git");
2485 +
        if git_dir.is_file() {
2486 +
            return fail(".git is a file (a Git worktree or submodule link); remove it to keep a side-by-side Git repository here");
2487 +
        }
2488 +
        if !git_dir.join("HEAD").exists() {
2489 +
            git(&self.root, &["init", "-q"])?;
2490 +
        }
2491 +
        let exclude = git_dir.join("info").join("exclude");
2492 +
        let have = fs::read_to_string(&exclude).unwrap_or_default();
2493 +
        if !have.lines().any(|l| l.trim() == "/.oxy/") {
2494 +
            fs::create_dir_all(exclude.parent().expect("info/"))?;
2495 +
            let sep = if have.is_empty() || have.ends_with('\n') {
2496 +
                ""
2497 +
            } else {
2498 +
                "\n"
2499 +
            };
2500 +
            fs::write(
2501 +
                &exclude,
2502 +
                format!("{have}{sep}# Oxygen's own repository (oxy git colocate)\n/.oxy/\n"),
2503 +
            )?;
2504 +
        }
2505 +
        let bookmark = self.config.bookmark.clone();
2506 +
        let state = self
2507 +
            .view()?
2508 +
            .bookmarks
2509 +
            .get(&bookmark)
2510 +
            .map(|s| s.to_hex())
2511 +
            .unwrap_or_default();
2512 +
        let stamp_file = self.oxy.join("git").join("colocate-stamp");
2513 +
        let stamp = format!("{bookmark} {state}");
2514 +
        if fs::read_to_string(&stamp_file).is_ok_and(|s| s.trim() == stamp) {
2515 +
            return Ok(false);
2516 +
        }
2517 +
        if !state.is_empty() {
2518 +
            self.git_export_as(&git_dir, &bookmark)?;
2519 +
            git(
2520 +
                &self.root,
2521 +
                &["symbolic-ref", "HEAD", &format!("refs/heads/{bookmark}")],
2522 +
            )?;
2523 +
            git(&self.root, &["read-tree", "HEAD"])?;
2524 +
        }
2525 +
        fs::create_dir_all(stamp_file.parent().expect("git/"))?;
2526 +
        fs::write(&stamp_file, stamp)?;
2527 +
        Ok(!state.is_empty())
2528 +
    }
2529 +
}
modified crates/oxy-core/src/repo/history.rs +28 −0
…
199 unchanged lines
200 200
            .unwrap())
201 201
    }
202 202
203 +
    /// Fold only `paths` of the working copy into the head it sits on
204 +
    /// (oxygen#118): they are committed as a change of their own, on the
205 +
    /// head, which is then squashed into it; the rest stays uncommitted
206 +
    /// work. A new `message` replaces the head's.
207 +
    pub fn amend_paths(&self, message: Option<&str>, paths: &[String]) -> Result<ChangeHash> {
208 +
        let snap = self.snapshot("oxy amend")?;
209 +
        let base_heads = snap.base.maximal(&snap.wc.deps);
210 +
        let [head] = base_heads.as_slice() else {
211 +
            return fail(format!(
212 +
                "the working copy sits on {} heads; check out the single change to amend",
213 +
                base_heads.len()
214 +
            ));
215 +
        };
216 +
        let head = *head;
217 +
        self.rewritable_head(&head)?;
218 +
        drop(snap);
219 +
        let (part, _) = self.commit_selection_partial(
220 +
            "oxy amend: the paths to fold in",
221 +
            &[head],
222 +
            &Selection::Paths(paths.to_vec()),
223 +
        )?;
224 +
        let folded = self.squash(&part, &head)?;
225 +
        match message {
226 +
            Some(m) => self.describe(&folded, m),
227 +
            None => Ok(folded),
228 +
        }
229 +
    }
230 +
203 231
    /// Fold the working copy into the head it sits on: the head is rebuilt
204 232
    /// as the difference between the state beneath it and the files on disk.
205 233
    pub fn amend(&self, message: Option<&str>) -> Result<ChangeHash> {
…
1006 unchanged lines
modified crates/oxy-core/src/repo/hosted.rs +324 −2
…
25 unchanged lines
26 26
        message: &str,
27 27
        author: KeyId,
28 28
    ) -> Result<Change> {
29 +
        self.build_change_with(heads, edits, &[], message, author)
30 +
    }
31 +
32 +
    /// Build (but do not store or sign) a change that moves the subrepo at
33 +
    /// `path` in the state with `heads` to `pin`: a hub moving a consumer's
34 +
    /// pin for Diffusion (oxygenhub#148). The path must already be a
35 +
    /// subrepo; nothing else changes.
36 +
    pub fn build_pin_change(
37 +
        &self,
38 +
        heads: &[ChangeHash],
39 +
        path: &str,
40 +
        pin: &crate::subrepo::Pin,
41 +
        message: &str,
42 +
        author: KeyId,
43 +
    ) -> Result<Change> {
44 +
        self.build_change_with(
45 +
            heads,
46 +
            &[],
47 +
            &[(path.to_string(), pin.clone())],
48 +
            message,
49 +
            author,
50 +
        )
51 +
    }
52 +
53 +
    fn build_change_with(
54 +
        &self,
55 +
        heads: &[ChangeHash],
56 +
        edits: &[FileEdit],
57 +
        pins: &[(String, crate::subrepo::Pin)],
58 +
        message: &str,
59 +
        author: KeyId,
60 +
    ) -> Result<Change> {
29 61
        if message.trim().is_empty() {
30 62
            return fail("a change message is required");
31 63
        }
32 -
        if edits.is_empty() {
64 +
        if edits.is_empty() && pins.is_empty() {
33 65
            return fail("nothing to change");
34 66
        }
35 67
        let base = self.load_state(heads)?;
…
61 unchanged lines
97 129
                }
98 130
            }
99 131
        }
132 +
        for (path, pin) in pins {
133 +
            let path = path.trim_matches('/');
134 +
            match base_snapshot.files.get(path) {
135 +
                Some(f) if f.kind == NodeKind::Subrepo => {}
136 +
                Some(_) => return fail(format!("{path} is not a subrepo")),
137 +
                None => return fail(format!("{path} does not exist")),
138 +
            }
139 +
            entries.insert(
140 +
                path.to_string(),
141 +
                FsEntry::new(NodeKind::Subrepo, false, pin.encode()),
142 +
            );
143 +
        }
100 144
        // A path the user writes settles its replace conflict, if any: they
101 145
        // chose that content (spec §9.3).
102 146
        let written: std::collections::HashSet<String> = edits
103 147
            .iter()
104 148
            .filter(|e| e.content.is_some())
105 149
            .map(|e| e.path.trim_matches('/').to_string())
150 +
            .chain(pins.iter().map(|(p, _)| p.trim_matches('/').to_string()))
106 151
            .collect();
107 152
        let built = self.build_from_entries_resolving(
108 153
            &base,
…
84 unchanged lines
193 238
        })
194 239
    }
195 240
241 +
    /// Create tag `name` for the state of `heads` as a host does for a
242 +
    /// signed-in user (oxygenhub#295): attested by `signer` (a delegate of
243 +
    /// the user, or the host's own key), with `identities` it needs, and
244 +
    /// received as a pushed tag is, so the same policy checks apply.
245 +
    pub fn host_tag_create(
246 +
        &self,
247 +
        name: &str,
248 +
        heads: &[ChangeHash],
249 +
        message: &str,
250 +
        signer: (&KeyId, &SecretKey),
251 +
        identities: &[Identity],
252 +
    ) -> Result<PushReport> {
253 +
        crate::objects::validate_name(name)
254 +
            .map_err(|e| RepoError::Msg(format!("tag name: {e}")))?;
255 +
        let st = self.load_state_history(heads)?;
256 +
        let heads = st.maximal(heads);
257 +
        if heads.is_empty() {
258 +
            return fail("a tag needs at least one change");
259 +
        }
260 +
        let state_object = StateObject::new(heads.clone());
261 +
        let state = state_object.hash();
262 +
        let body = AttestBody::Tag {
263 +
            name: name.into(),
264 +
            state,
265 +
            message: message.into(),
266 +
        };
267 +
        let mut bundle = self.hosted_bundle(identities);
268 +
        bundle
269 +
            .objects
270 +
            .push((ObjectType::State, state_object.encode()));
271 +
        let t = crate::protocol::TagRef::new(
272 +
            name.into(),
273 +
            state,
274 +
            heads,
275 +
            Some(self.host_attest(body, signer)?),
276 +
        );
277 +
        self.receive_push_report(bundle, vec![], None, vec![t])
278 +
    }
279 +
280 +
    /// Withdraw tag `name` as a host does for a signed-in user
281 +
    /// (oxygenhub#301): an `untag` attestation by `signer`, received as a
282 +
    /// pushed withdrawal is.
283 +
    pub fn host_tag_withdrawal(
284 +
        &self,
285 +
        name: &str,
286 +
        reason: &str,
287 +
        signer: (&KeyId, &SecretKey),
288 +
        identities: &[Identity],
289 +
    ) -> Result<PushReport> {
290 +
        let Some(state) = self.view()?.tags.get(name).copied() else {
291 +
            return fail(format!("no tag {name:?}"));
292 +
        };
293 +
        let body = AttestBody::Untag {
294 +
            name: name.into(),
295 +
            state,
296 +
            reason: reason.into(),
297 +
        };
298 +
        let t = crate::protocol::TagRef::new(
299 +
            name.into(),
300 +
            state,
301 +
            vec![],
302 +
            Some(self.host_attest(body, signer)?),
303 +
        );
304 +
        self.receive_push_report(self.hosted_bundle(identities), vec![], None, vec![t])
305 +
    }
306 +
307 +
    fn host_attest(
308 +
        &self,
309 +
        body: AttestBody,
310 +
        (key, sk): (&KeyId, &SecretKey),
311 +
    ) -> Result<SignedAttest> {
312 +
        let attest = Attest {
313 +
            repo: self.id()?,
314 +
            hlc: Hlc::now(),
315 +
            body,
316 +
        };
317 +
        let sig = sk.sign(Domain::Attest, key, &attest.hash());
318 +
        Ok(SignedAttest {
319 +
            attest,
320 +
            sigs: vec![sig],
321 +
        })
322 +
    }
323 +
324 +
    fn hosted_bundle(&self, identities: &[Identity]) -> Bundle {
325 +
        let mut bundle = Bundle::default();
326 +
        for id in identities {
327 +
            bundle.objects.push((ObjectType::Identity, id.encode()));
328 +
        }
329 +
        bundle
330 +
    }
331 +
196 332
    /// Apply a host-made update as if it had been pushed: the new change
197 333
    /// (with its signatures) if any, the identities it needs, and the
198 334
    /// reference update, all through `receive_push_report`, so the same
…
388 unchanged lines
587 723
        user.delegate_add(agent_id.clone(), Scope::CHANGES, None)
588 724
            .unwrap();
589 725
        assert!(user.push("origin", false).unwrap().iter().all(|x| x.ok));
726 +
        host.refresh().unwrap();
590 727
        let owner = {
591 728
            let _ = owner;
592 729
            user.config.author_key.unwrap()
…
23 unchanged lines
616 753
        // The agent cannot push: its attestation of the reference is
617 754
        // refused under the policy. The owner can.
618 755
        let r = as_agent.push("origin", false).unwrap();
756 +
        host.refresh().unwrap();
619 757
        assert!(r.iter().all(|x| !x.ok), "{r:?}");
620 758
        assert!(!host.bookmark_heads().unwrap().contains(&h));
621 759
        // The owner's own handle (its key directory was fixed at `setup`;
622 -
        // the environment's `OXY_HOME` may have moved since).
760 +
        // the environment's `OXY_HOME` may have moved since), which reads
761 +
        // the head again to see the agent's commit.
762 +
        user.refresh().unwrap();
623 763
        let r = user.push("origin", false).unwrap();
764 +
        host.refresh().unwrap();
624 765
        assert!(r.iter().all(|x| x.ok), "{r:?}");
625 766
        assert!(host.bookmark_heads().unwrap().contains(&h));
626 767
        assert!(matches!(
…
2 unchanged lines
629 770
        ));
630 771
    }
631 772
773 +
    /// oxygenhub#129: a delegate that may move references makes a bookmark
774 +
    /// at its owner's change, signing no change of its own. The push
775 +
    /// carries its identity, so the host can check the move (it was
776 +
    /// refused as "unknown key" until the delegate had pushed a change).
777 +
    #[test]
778 +
    fn a_delegate_s_first_push_of_a_bookmark_carries_its_identity() {
779 +
        let dir = tempfile::tempdir().unwrap();
780 +
        let (host, mut user, _, _, _) = setup(dir.path(), true, false);
781 +
        let agent_home = dir.path().join("agent-home");
782 +
        let mut agent_repo = Repo::init(&dir.path().join("agent-work"))
783 +
            .unwrap()
784 +
            .with_home(agent_home.clone());
785 +
        let (agent, agent_id) = agent_repo
786 +
            .create_identity("agent", "agent@example", IdentityScope::User)
787 +
            .unwrap();
788 +
        user.delegate_add(agent_id, Scope::CHANGES | Scope::REFS, None)
789 +
            .unwrap();
790 +
        assert!(user.push("origin", false).unwrap().iter().all(|x| x.ok));
791 +
        let work = dir.path().join("work");
792 +
        let mut as_agent = Repo::open(&work).unwrap().with_home(agent_home);
793 +
        as_agent
794 +
            .set_user_config("author.signer", &agent.to_hex())
795 +
            .unwrap();
796 +
        assert!(
797 +
            host.store().get_identity(&agent).is_err(),
798 +
            "the host has never seen the agent"
799 +
        );
800 +
        let heads = as_agent.bookmark_heads().unwrap();
801 +
        as_agent.bookmark_create("feature", Some(&heads)).unwrap();
802 +
        let r = as_agent.push_bookmark("origin", "feature", false).unwrap();
803 +
        assert!(r.iter().all(|x| x.ok), "{r:?}");
804 +
        assert!(
805 +
            host.store().get_identity(&agent).is_ok(),
806 +
            "its identity came with the move"
807 +
        );
808 +
    }
809 +
810 +
    /// oxygenhub#266: a bookmark whose latest attestation a delegate made,
811 +
    /// signing no change, is fetched by a third party. The fetch bundle
812 +
    /// carries the delegate's identity even when it asks for no change,
813 +
    /// so a client as it was (import the bundle, then check each
814 +
    /// bookmark's attestation) verifies the move instead of failing the
815 +
    /// whole fetch with "unknown key".
816 +
    #[test]
817 +
    fn a_bookmark_attested_by_a_key_that_signed_no_change_is_fetched() {
818 +
        let dir = tempfile::tempdir().unwrap();
819 +
        let (host, mut user, _, _, _) = setup(dir.path(), true, false);
820 +
        let agent_home = dir.path().join("agent-home");
821 +
        let mut agent_repo = Repo::init(&dir.path().join("agent-work"))
822 +
            .unwrap()
823 +
            .with_home(agent_home.clone());
824 +
        let (agent, agent_id) = agent_repo
825 +
            .create_identity("agent", "agent@example", IdentityScope::User)
826 +
            .unwrap();
827 +
        user.delegate_add(agent_id, Scope::CHANGES | Scope::REFS, None)
828 +
            .unwrap();
829 +
        assert!(user.push("origin", false).unwrap().iter().all(|x| x.ok));
830 +
        let mut as_agent = Repo::open(&dir.path().join("work"))
831 +
            .unwrap()
832 +
            .with_home(agent_home);
833 +
        as_agent
834 +
            .set_user_config("author.signer", &agent.to_hex())
835 +
            .unwrap();
836 +
        let heads = as_agent.bookmark_heads().unwrap();
837 +
        as_agent.bookmark_create("feature", Some(&heads)).unwrap();
838 +
        let r = as_agent.push_bookmark("origin", "feature", false).unwrap();
839 +
        assert!(r.iter().all(|x| x.ok), "{r:?}");
840 +
        host.refresh().unwrap();
841 +
        let (_, sa) = host.latest_ref_attest("feature").unwrap().unwrap();
842 +
        assert_eq!(sa.sigs[0].key, agent, "the agent attested the move");
843 +
844 +
        // What a fetch that names no change brings: the agent's identity.
845 +
        let remote = crate::protocol::LocalRemote::open(&dir.path().join("host")).unwrap();
846 +
        let carries = |b: &Bundle| {
847 +
            b.objects
848 +
                .iter()
849 +
                .any(|(t, body)| *t == ObjectType::Identity && hash_object(*t, body) == agent)
850 +
        };
851 +
        assert!(carries(
852 +
            &crate::protocol::Remote::fetch(&remote, &[], None).unwrap()
853 +
        ));
854 +
        assert!(carries(&host.bundle_for_filtered(&[], None).unwrap()));
855 +
856 +
        // A third party's clone takes both bookmarks, verified.
857 +
        let theirs = Repo::clone(
858 +
            dir.path().join("host").to_str().unwrap(),
859 +
            &dir.path().join("theirs"),
860 +
        )
861 +
        .unwrap();
862 +
        let tracked = theirs.view().unwrap().remotes["origin"].clone();
863 +
        assert_eq!(
864 +
            tracked.get("feature"),
865 +
            host.view().unwrap().bookmarks.get("feature")
866 +
        );
867 +
        assert!(theirs.store().get_identity(&agent).is_ok());
868 +
        let report = theirs.fetch("origin").unwrap();
869 +
        assert!(report.skipped.is_empty(), "{:?}", report.skipped);
870 +
    }
871 +
872 +
    /// Make `bookmark`'s latest attestation on `host` one signed as `key`
873 +
    /// with `sk`: by a key the host never saw (as a host that did not send
874 +
    /// the signer's identity looked), or a bad signature.
875 +
    fn reattest(host: &Repo, bookmark: &str, key: KeyId, sk: &SecretKey) {
876 +
        let state = host.view().unwrap().bookmarks[bookmark];
877 +
        let (_, last) = host.latest_ref_attest(bookmark).unwrap().unwrap();
878 +
        let AttestBody::Ref {
879 +
            old_state, serial, ..
880 +
        } = last.attest.body
881 +
        else {
882 +
            panic!("not a ref attestation")
883 +
        };
884 +
        let attest = Attest {
885 +
            repo: host.id().unwrap(),
886 +
            hlc: Hlc::now(),
887 +
            body: AttestBody::Ref {
888 +
                name: bookmark.into(),
889 +
                old_state,
890 +
                new_state: state,
891 +
                serial: serial + 1,
892 +
            },
893 +
        };
894 +
        let r = host.store().put_attest(&attest).unwrap();
895 +
        host.store()
896 +
            .put_signature(&r, &sk.sign(Domain::Attest, &key, &attest.hash()))
897 +
            .unwrap();
898 +
        host.set_ref_attest(bookmark, &r).unwrap();
899 +
    }
900 +
901 +
    /// oxygenhub#266: a bookmark whose attestation a client cannot verify
902 +
    /// (its signer unknown, or its signature bad) is left as it was, named
903 +
    /// with why, and the rest are fetched. The current bookmark is not
904 +
    /// skipped: a clone, fetch or pull that would take it fails.
905 +
    #[test]
906 +
    fn an_unverifiable_bookmark_is_skipped_but_never_the_current_one() {
907 +
        let dir = tempfile::tempdir().unwrap();
908 +
        let (host, user, _, _, _) = setup(dir.path(), false, false);
909 +
        let author = user.config.author_key.unwrap();
910 +
        let heads = user.bookmark_heads().unwrap();
911 +
        for b in ["unknown", "forged", "fine"] {
912 +
            user.bookmark_create(b, Some(&heads)).unwrap();
913 +
            let r = user.push_bookmark("origin", b, false).unwrap();
914 +
            assert!(r.iter().all(|x| x.ok), "{r:?}");
915 +
        }
916 +
        host.refresh().unwrap();
917 +
        let stranger_sk = SecretKey::generate();
918 +
        let stranger = stranger_sk.identity("s", "s@example", 1).hash();
919 +
        reattest(&host, "unknown", stranger, &stranger_sk);
920 +
        reattest(&host, "forged", author, &stranger_sk);
921 +
922 +
        let url = dir.path().join("host").to_string_lossy().to_string();
923 +
        let (theirs, skipped) =
924 +
            Repo::clone_reporting(&url, &dir.path().join("theirs"), None, &[], None).unwrap();
925 +
        let names: Vec<&str> = skipped.iter().map(|(n, _)| n.as_str()).collect();
926 +
        assert_eq!(names, ["forged", "unknown"]);
927 +
        assert!(skipped[1].1.contains("unknown key"), "{skipped:?}");
928 +
        assert!(skipped[1].1.contains("origin/unknown"), "{skipped:?}");
929 +
        let tracked = theirs.view().unwrap().remotes["origin"].clone();
930 +
        assert!(tracked.contains_key("main") && tracked.contains_key("fine"));
931 +
        assert!(!tracked.contains_key("unknown") && !tracked.contains_key("forged"));
932 +
        assert_eq!(theirs.config.bookmark, "main");
933 +
        assert_eq!(theirs.fetch("origin").unwrap().skipped.len(), 2);
934 +
        assert_eq!(theirs.pull("origin").unwrap().skipped.len(), 2);
935 +
936 +
        // The current bookmark's attestation unverifiable: a fetch and a
937 +
        // pull fail, and so does a fresh clone.
938 +
        reattest(&host, "main", stranger, &stranger_sk);
939 +
        for err in [
940 +
            theirs.fetch("origin").unwrap_err(),
941 +
            theirs.pull("origin").unwrap_err(),
942 +
            Repo::clone(&url, &dir.path().join("again"))
943 +
                .err()
944 +
                .expect("the clone fails"),
945 +
        ] {
946 +
            let err = err.to_string();
947 +
            assert!(
948 +
                err.contains("attestation for origin/main: unknown key"),
949 +
                "{err}"
950 +
            );
951 +
        }
952 +
    }
953 +
632 954
    /// A delegation past its expiry authenticates nothing made after it,
633 955
    /// and the delegator may revoke the key; a stranger's key, they may not.
634 956
    #[test]
…
125 unchanged lines
modified crates/oxy-core/src/repo/identity.rs +238 −8
…
17 unchanged lines
18 18
    Invalid(String),
19 19
}
20 20
21 +
/// Make a user identity with no repository (oxygen#144), so a first
22 +
/// identity comes before any repository: its secret and a copy of it go to
23 +
/// this home's keys, and the user file names it as the author, as `oxy
24 +
/// identity create` does inside one. A repository reads it from the keys
25 +
/// when it first signs with it. `name` and `email` default to the user
26 +
/// file's author, then to `$USER` and `OXY_AUTHOR_*`.
27 +
pub fn create_user_identity(
28 +
    name: Option<&str>,
29 +
    email: Option<&str>,
30 +
    expires: Option<u64>,
31 +
) -> Result<(KeyId, Identity)> {
32 +
    let user = default_user_config();
33 +
    let defaults = Config::load(Some(&user), Path::new(""))?;
34 +
    let name = name.map(str::to_string).unwrap_or(defaults.author_name);
35 +
    let email = email.map(str::to_string).unwrap_or(defaults.author_email);
36 +
    let sk = SecretKey::generate();
37 +
    let mut id = sk.identity(&name, &email, Hlc::now().wall_ms);
38 +
    id.expires = expires;
39 +
    let key = id.hash();
40 +
    let keys = default_keys_dir();
41 +
    fs::create_dir_all(&keys)?;
42 +
    crate::durable::write_atomic_private(
43 +
        &keys.join(format!("{}.secret", key.to_hex())),
44 +
        &sk.encode(&key),
45 +
    )?;
46 +
    write_atomic(
47 +
        &keys.join(format!("{}.identity", key.to_hex())),
48 +
        &id.encode(),
49 +
    )?;
50 +
    Config::set_user(&user, "author.name", &id.name)?;
51 +
    Config::set_user(&user, "author.email", &id.email)?;
52 +
    Config::set_user(&user, "author.key", &key.to_hex())?;
53 +
    Ok((key, id))
54 +
}
55 +
21 56
/// The outcome of replacing the configured identity (spec §14.2).
22 57
#[derive(Clone, Debug)]
23 58
pub struct Rotation {
…
113 unchanged lines
137 172
        if new.previous != Some(old) {
138 173
            return fail("the successor identity must name the current key as previous");
139 174
        }
175 +
        // A key that already has a successor is not the identity's newest:
176 +
        // building another from it forks the identity into two branches
177 +
        // (oxygen#163), as a checkout whose stale `author.key` pin outlived
178 +
        // a rotation made elsewhere once did.
179 +
        if let Some(newer) = self.successors(&old)?.into_iter().next() {
180 +
            let newest = self.newest(&newer)?;
181 +
            let fix = if self.config.is_repo_set("author.key") {
182 +
                format!(
183 +
                    "This repository's config pins author.key to the older key: `oxy config author.key {}` here \
184 +
                     (or `oxy config --unset author.key` to follow the user config), then run it again.",
185 +
                    newest.to_hex()
186 +
                )
187 +
            } else {
188 +
                format!(
189 +
                    "The user config names the older key: `oxy config --user author.key {}`, then run it again.",
190 +
                    newest.to_hex()
191 +
                )
192 +
            };
193 +
            return fail(format!(
194 +
                "`{command}` would start from key {}, which is not your identity's newest ({} succeeds it, and {} is the newest). \
195 +
                 Another successor would fork the identity into two branches.\n{fix}",
196 +
                old.short(12),
197 +
                newer.short(12),
198 +
                newest.short(12),
199 +
            ));
200 +
        }
140 201
        let new_key = self.install_key(&new, &sk)?;
141 202
        let attest = Attest {
142 203
            repo: self.id()?,
…
20 unchanged lines
163 224
        })
164 225
    }
165 226
227 +
    /// Keys in this home whose identity names `key` as `previous`: the
228 +
    /// successors already made from it. More than one means a fork.
229 +
    pub fn successors(&self, key: &KeyId) -> Result<Vec<KeyId>> {
230 +
        let mut found = Vec::new();
231 +
        let entries = match fs::read_dir(&self.keys_dir) {
232 +
            Ok(e) => e,
233 +
            Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(found),
234 +
            Err(e) => return Err(e.into()),
235 +
        };
236 +
        for entry in entries {
237 +
            let path = entry?.path();
238 +
            if path.extension().and_then(|e| e.to_str()) != Some("identity") {
239 +
                continue;
240 +
            }
241 +
            let Ok(bytes) = fs::read(&path) else { continue };
242 +
            let Ok(id) = Identity::decode(&bytes) else {
243 +
                continue;
244 +
            };
245 +
            if id.previous == Some(*key) {
246 +
                found.push(id.hash());
247 +
            }
248 +
        }
249 +
        found.sort();
250 +
        Ok(found)
251 +
    }
252 +
253 +
    /// The newest key of the line through `key` in this home: its
254 +
    /// successors followed to the end (the first, where there is a fork).
255 +
    pub fn newest(&self, key: &KeyId) -> Result<KeyId> {
256 +
        let mut cur = *key;
257 +
        for _ in 0..256 {
258 +
            match self.successors(&cur)?.into_iter().next() {
259 +
                Some(next) => cur = next,
260 +
                None => break,
261 +
            }
262 +
        }
263 +
        Ok(cur)
264 +
    }
265 +
166 266
    /// Rotate to a fresh key pair; the new identity names the old one as
167 267
    /// `previous` and keeps its name, email, delegates, and expiry.
168 268
    pub fn rotate_identity(&mut self) -> Result<Rotation> {
…
159 unchanged lines
328 428
    /// The key this repository signs with: `author.signer` when set (an
329 429
    /// agent's own key, acting for the author), else `author.key`.
330 430
    pub fn signing_key(&self) -> Option<KeyId> {
431 +
        if self.config.signer_keyless {
432 +
            // The key certified last, once this command has signed.
433 +
            return self.keyless_key.borrow().as_ref().map(|(k, _, _)| *k);
434 +
        }
331 435
        self.config.signer_key.or(self.config.author_key)
332 436
    }
333 437
…
2 unchanged lines
336 440
    /// so a repository that has never seen it (a fresh clone) can store it
337 441
    /// when it first signs.
338 442
    pub fn secret_key(&self) -> Result<Option<(KeyId, SecretKey)>> {
443 +
        if self.config.signer_keyless {
444 +
            return self.keyless_secret().map(Some);
445 +
        }
339 446
        let Some(key) = self.signing_key() else {
340 447
            return Ok(None);
341 448
        };
…
43 unchanged lines
385 492
                        }
386 493
                        Ok(id)
387 494
                    }
495 +
                    // The configured author's identity is neither in the
496 +
                    // repository nor in this home's keys: say why and what
497 +
                    // to do (oxygen#82), rather than "object … not found".
498 +
                    Err(_) if self.config.author_key == Some(*key) => fail(format!(
499 +
                        "this repository has no identity {} (author.key, set in {}): it is not in the repository, nor in {}.\n\
500 +
                         An agent home names its human's identity, and each delegation rotates that identity, so the key goes stale. \
501 +
                         Pin the identity this repository uses, as its owner sees it: `oxy identity show` in the owner's shell gives the key, \
502 +
                         then `oxy config author.key <KEY>` here; or ask the owner.",
503 +
                        key.short(12),
504 +
                        if self.config.is_repo_set("author.key") {
505 +
                            "this repository's config"
506 +
                        } else {
507 +
                            "the user config"
508 +
                        },
509 +
                        self.keys_dir.display()
510 +
                    )),
388 511
                    Err(_) => Err(e.into()),
389 512
                }
390 513
            }
…
41 unchanged lines
432 555
                Ok(id) => {
433 556
                    return Ok(SignatureStatus::Valid {
434 557
                        key: sig.key,
435 -
                        name: id.name,
558 +
                        name: self.certified_subject(&sig.key).unwrap_or(id.name),
436 559
                        email: id.email,
437 560
                        sealed: false,
438 561
                        revoked: self.is_revoked(&sig.key)?,
…
5 unchanged lines
444 567
        Ok(SignatureStatus::Invalid(last_err))
445 568
    }
446 569
570 +
    /// Whether `signer` is a short-lived key acting for `author` under a
571 +
    /// grant (spec §14.7): its certificate verifies with its issuer and
572 +
    /// carries a grant `author` signed, for the certificate's subject and
573 +
    /// issuer, giving `needed` at `at_ms` (the change's time, as for a
574 +
    /// delegation). Whether the certificate covered the time it signed is
575 +
    /// the policy's check, at a time the signer did not choose. `None`
576 +
    /// when it does; otherwise why not.
577 +
    fn granted(
578 +
        &self,
579 +
        author: &KeyId,
580 +
        signer: &KeyId,
581 +
        needed: Scope,
582 +
        at_ms: u64,
583 +
    ) -> std::result::Result<Option<String>, String> {
584 +
        let neither = || {
585 +
            Some(format!(
586 +
                "key {} is neither the author nor a delegate",
587 +
                signer.short(12)
588 +
            ))
589 +
        };
590 +
        let Ok(Some(c)) = self.store.certificate_of(signer) else {
591 +
            return Ok(neither());
592 +
        };
593 +
        let Ok(sc) = self.store.get_certificate(&c) else {
594 +
            return Ok(neither());
595 +
        };
596 +
        let Some(g) = &sc.cert.grant else {
597 +
            return Ok(neither());
598 +
        };
599 +
        let who = signer.short(12);
600 +
        let issuer = self
601 +
            .store
602 +
            .get_identity(&sc.cert.issuer)
603 +
            .map_err(|_| format!("key {who}'s certificate names an unknown issuer"))?;
604 +
        sc.verify(&issuer)
605 +
            .map_err(|e| format!("key {who}'s certificate: {e}"))?;
606 +
        // A grant by an earlier key of the author's identity counts while
607 +
        // it was made before that key was rotated away (oxygen#135).
608 +
        if g.grant.grantor != *author && !self.granted_before_rotation(author, &g.grant) {
609 +
            return Ok(neither());
610 +
        }
611 +
        let grantor = self
612 +
            .identity(&g.grant.grantor)
613 +
            .map_err(|_| format!("key {who}'s grant is by an unknown author"))?;
614 +
        g.verify(&grantor)
615 +
            .map_err(|e| format!("key {who}'s grant: {e}"))?;
616 +
        if g.grant.subject != sc.cert.subject || g.grant.issuer != sc.cert.issuer {
617 +
            return Err(format!(
618 +
                "key {who}'s grant is for {} from issuer {}, not its certificate's",
619 +
                g.grant.subject,
620 +
                g.grant.issuer.short(12)
621 +
            ));
622 +
        }
623 +
        if !sc.cert.scope.has(needed) || !g.grant.scope.has(needed) {
624 +
            return Err(format!("key {who} is not granted {needed}"));
625 +
        }
626 +
        if !g.grant.covers(needed, at_ms) {
627 +
            return Err(format!(
628 +
                "key {who}'s grant did not cover the time the change was made"
629 +
            ));
630 +
        }
631 +
        Ok(None)
632 +
    }
633 +
634 +
    /// Whether `grant` is by a key `author`'s identity was rotated from,
635 +
    /// made before that rotation (oxygen#135). An owner rotates with every
636 +
    /// delegation, and a repository may name the newest key as the author
637 +
    /// while the agent's grant is by an older one; the grant stays good, as
638 +
    /// a policy keeps a predecessor's signatures from before the rotation
639 +
    /// (spec §14.5), but a key rotated away grants nothing new.
640 +
    fn granted_before_rotation(&self, author: &KeyId, grant: &crate::cert::Grant) -> bool {
641 +
        let mut cur = self.identity(author).ok();
642 +
        for _ in 0..32 {
643 +
            let Some(id) = cur else {
644 +
                return false;
645 +
            };
646 +
            match id.previous {
647 +
                Some(prev) if prev == grant.grantor => return grant.not_before < id.created,
648 +
                Some(prev) => cur = self.identity(&prev).ok(),
649 +
                None => return false,
650 +
            }
651 +
        }
652 +
        false
653 +
    }
654 +
655 +
    /// The subject a short-lived key was certified for (spec §14.7), such
656 +
    /// as `agent:claude@andy`: what names it, since its identity is only
657 +
    /// a fresh key's.
658 +
    pub fn certified_subject(&self, key: &KeyId) -> Option<String> {
659 +
        let c = self.store.certificate_of(key).ok()??;
660 +
        Some(self.store.get_certificate(&c).ok()?.cert.subject)
661 +
    }
662 +
447 663
    /// Verify `sig` over `target` and check the author-or-delegate rule for `change`.
448 664
    fn check_signer(
449 665
        &self,
…
23 unchanged lines
473 689
                .ok()
474 690
                .and_then(|a| a.delegate(&sig.key).copied());
475 691
            match delegation {
692 +
                // A certified key acting under the author's grant (§14.7).
476 693
                None => {
477 -
                    return Err(format!(
478 -
                        "key {} is neither the author nor a delegate",
479 -
                        sig.key.short(12)
480 -
                    ))
694 +
                    if let Some(why) =
695 +
                        self.granted(&change.author, &sig.key, needed, change.hlc.wall_ms)?
696 +
                    {
697 +
                        return Err(why);
698 +
                    }
481 699
                }
482 700
                Some(d) if !d.scope.has(needed) => {
483 701
                    return Err(format!(
…
75 unchanged lines
559 777
                    Ok(id) => {
560 778
                        return Ok(SignatureStatus::Valid {
561 779
                            key: sig.key,
562 -
                            name: id.name,
780 +
                            name: self.certified_subject(&sig.key).unwrap_or(id.name),
563 781
                            email: id.email,
564 782
                            sealed: true,
565 783
                            revoked: self.is_revoked(&sig.key)?,
…
43 unchanged lines
609 827
        Ok(r)
610 828
    }
611 829
830 +
    /// Whether `signer` seals changes by `author`: its own, anonymous ones
831 +
    /// (§14.4), and, when it signs for the configured author (a delegate,
832 +
    /// or a short-lived key under the author's grant, §14.7), the
833 +
    /// author's. A receiver still checks it may.
834 +
    pub(crate) fn seals_for_author(&self, author: &KeyId, signer: &KeyId) -> bool {
835 +
        author == signer
836 +
            || author.is_zero()
837 +
            || ((self.config.signer_key.is_some() || self.config.signer_keyless)
838 +
                && self.config.author_key == Some(*author))
839 +
    }
840 +
612 841
    /// The changes of the current bookmark this identity can authenticate
613 -
    /// but has not: its own unsigned ones and anonymous ones.
842 +
    /// but has not: its own unsigned ones, anonymous ones, and the
843 +
    /// author's when it signs for them.
614 844
    pub fn sealable(&self) -> Result<Vec<ChangeHash>> {
615 845
        let Some((key, _)) = self.secret_key()? else {
616 846
            return fail("no identity: run oxy identity create");
…
2 unchanged lines
619 849
        let state = self.load_state_history(&heads)?;
620 850
        let mut out = Vec::new();
621 851
        for (h, meta) in state.changes() {
622 -
            if (meta.author == key || meta.author.is_zero())
852 +
            if self.seals_for_author(&meta.author, &key)
623 853
                && !matches!(self.authenticate_change(h)?, SignatureStatus::Valid { .. })
624 854
            {
625 855
                out.push(*h);
…
24 unchanged lines
modified crates/oxy-core/src/repo/inspect.rs +2 −2
…
61 unchanged lines
62 62
        let (_, wc) = self.working_copy()?;
63 63
        // The working copy's base is what the state index holds.
64 64
        let mut state = self.load_state_indexed(&wc.deps)?;
65 -
        let tree = state.tree().view();
65 +
        let tree = self.tree_view(&mut state)?;
66 66
        if let Some(e) = tree.entries.get(path) {
67 67
            self.load_graphs_of(&mut state, &[e.node])?;
68 68
        }
…
8 unchanged lines
77 77
    /// Blame `path` in the state with these heads, building only its graph.
78 78
    fn blame_heads(&self, path: &str, heads: &[ChangeHash], what: &str) -> Result<Vec<BlameLine>> {
79 79
        let mut state = self.load_state_history(heads)?;
80 -
        let tree = state.tree().view();
80 +
        let tree = self.tree_view(&mut state)?;
81 81
        if let Some(e) = tree.entries.get(path) {
82 82
            self.load_graphs_of(&mut state, &[e.node])?;
83 83
        }
…
397 unchanged lines
added crates/oxy-core/src/repo/keyless.rs +280 −0
1 +
//! Signing with short-lived keys (spec §14.7, oxygenhub#241): with
2 +
//! `author.signer = keyless`, oxy asks the hub to certify a fresh key
3 +
//! whenever it must sign, with the agent's grant token as the credential,
4 +
//! and keeps the key in memory for its certificate's minutes. Nothing that
5 +
//! signs is ever written to disk: the home holds only the grant token,
6 +
//! which its owner revokes on the hub. The certificate carries the owner's
7 +
//! grant, so what the key signs counts in the owner's name.
8 +
9 +
use super::*;
10 +
use crate::cert::{certify_challenge, SignedCertificate};
11 +
use crate::credentials::{hub_of, Credentials};
12 +
use crate::protocol::{http_get, http_json, RemoteOptions};
13 +
14 +
/// A certified key is used until this long before its certificate ends,
15 +
/// so a push started near the end does not outlive it.
16 +
const MARGIN_MS: u64 = 2 * 60 * 1000;
17 +
18 +
impl Repo {
19 +
    /// The hub that certifies keys: `keyless.hub`, else the default
20 +
    /// remote's.
21 +
    fn keyless_hub(&self) -> Result<String> {
22 +
        let url = match &self.config.keyless_hub {
23 +
            Some(h) => h.clone(),
24 +
            None => self
25 +
                .config
26 +
                .remotes
27 +
                .get(&self.default_remote())
28 +
                .cloned()
29 +
                .ok_or_else(|| {
30 +
                    RepoError::Msg(
31 +
                        "author.signer is keyless, but neither keyless.hub nor a default remote names the hub that certifies keys"
32 +
                            .into(),
33 +
                    )
34 +
                })?,
35 +
        };
36 +
        hub_of(&url).ok_or_else(|| RepoError::Msg(format!("{url} is not a hub URL")))
37 +
    }
38 +
39 +
    /// The key to sign with now: the one certified last while it has more
40 +
    /// than a couple of minutes left, else a fresh one the hub certifies.
41 +
    pub(crate) fn keyless_secret(&self) -> Result<(KeyId, SecretKey)> {
42 +
        let now = Hlc::now().wall_ms;
43 +
        if let Some((k, sk, until)) = self.keyless_key.borrow().as_ref() {
44 +
            if now + MARGIN_MS < *until {
45 +
                return Ok((*k, sk.clone()));
46 +
            }
47 +
        }
48 +
        let hub = self.keyless_hub()?;
49 +
        let home = self
50 +
            .user_config
51 +
            .parent()
52 +
            .map(Path::to_path_buf)
53 +
            .unwrap_or_else(default_home);
54 +
        let creds = Credentials::at(home.clone());
55 +
        let token = creds.grant(&hub).map_err(RepoError::Msg)?.ok_or_else(|| {
56 +
            RepoError::Msg(format!(
57 +
                "author.signer is keyless, but there is no agent grant for {hub}: its owner makes one with `oh agent grant NAME`, and `oxy auth grant TOKEN` keeps it"
58 +
            ))
59 +
        })?;
60 +
        let options = RemoteOptions {
61 +
            token: Some(token.clone()),
62 +
            ca: creds.get(&hub).map_err(RepoError::Msg)?.and_then(|e| e.ca),
63 +
            ..RemoteOptions::default()
64 +
        };
65 +
        let sk = SecretKey::generate();
66 +
        let id = sk.identity(&self.config.author_name, &self.config.author_email, now);
67 +
        let key = id.hash();
68 +
        let proof = sk
69 +
            .sign_challenge(&key, &certify_challenge(&token))
70 +
            .map_err(|e| RepoError::Msg(e.to_string()))?;
71 +
        let body = serde_json::json!({
72 +
            "identity": crate::dsse::base64(&id.encode()),
73 +
            "proof": crate::dsse::base64(&crate::cbor::encode(&proof.to_value())),
74 +
            "scope": "changes,refs,seal",
75 +
        });
76 +
        let (code, answer) = http_json(
77 +
            "POST",
78 +
            &format!("{hub}/api/v1/keyless/certify"),
79 +
            Some(&body.to_string()),
80 +
            &options,
81 +
        )?;
82 +
        let reply: serde_json::Value = serde_json::from_str(&answer).unwrap_or_default();
83 +
        if !(200..300).contains(&code) {
84 +
            let why = reply["error"].as_str().unwrap_or(answer.trim());
85 +
            return fail(format!(
86 +
                "{hub} would not certify a key (HTTP {code}): {why}"
87 +
            ));
88 +
        }
89 +
        let cert = reply["certificate"]
90 +
            .as_str()
91 +
            .and_then(crate::dsse::base64_decode)
92 +
            .ok_or_else(|| RepoError::Msg(format!("{hub} answered with no certificate")))?;
93 +
        let cert = SignedCertificate::decode(&cert)
94 +
            .map_err(|e| RepoError::Msg(format!("{hub}'s certificate: {e}")))?;
95 +
        if cert.cert.key_id() != key {
96 +
            return fail(format!(
97 +
                "{hub} certified another key than the one it was asked for"
98 +
            ));
99 +
        }
100 +
        // The issuer's identity, so this repository (and whoever it pushes
101 +
        // to) can check the certificate: the hub's own, as it publishes it.
102 +
        let issuer = match self.store.get_identity(&cert.cert.issuer) {
103 +
            Ok(i) => i,
104 +
            Err(_) => {
105 +
                let bytes = http_get(
106 +
                    &format!("{hub}/.well-known/oxyhub-key"),
107 +
                    &RemoteOptions {
108 +
                        token: None,
109 +
                        ..options.clone()
110 +
                    },
111 +
                )?;
112 +
                let i = Identity::decode(&bytes)?;
113 +
                if i.hash() != cert.cert.issuer {
114 +
                    return fail(format!(
115 +
                        "{hub}'s published key is not the one that signed its certificate"
116 +
                    ));
117 +
                }
118 +
                i
119 +
            }
120 +
        };
121 +
        cert.verify(&issuer)
122 +
            .map_err(|e| RepoError::Msg(format!("{hub}'s certificate: {e}")))?;
123 +
        // The grant's author, so the grant can be checked here and wherever
124 +
        // this pushes (oxygen#134): from the hub's answer, else this home's
125 +
        // copy of it. Without it, a repository that never saw the owner's
126 +
        // current identity reads the change as signed by an unknown author.
127 +
        if let Some(g) = &cert.cert.grant {
128 +
            let grantor = g.grant.grantor;
129 +
            if self.store.get_identity(&grantor).is_err() {
130 +
                let sent = reply["grantor_identity"]
131 +
                    .as_str()
132 +
                    .and_then(crate::dsse::base64_decode)
133 +
                    .and_then(|b| Identity::decode(&b).ok());
134 +
                let kept = || {
135 +
                    std::fs::read(
136 +
                        home.join("keys")
137 +
                            .join(format!("{}.identity", grantor.to_hex())),
138 +
                    )
139 +
                    .ok()
140 +
                    .and_then(|b| Identity::decode(&b).ok())
141 +
                };
142 +
                match sent.filter(|i| i.hash() == grantor).or_else(|| kept().filter(|i| i.hash() == grantor)) {
143 +
                    Some(i) => {
144 +
                        self.store.put_identity(&i)?;
145 +
                    }
146 +
                    None => {
147 +
                        return fail(format!(
148 +
                            "the grant is signed by {}, whose identity neither {hub} nor this home has: export it there (oxy identity export) into this home's keys/",
149 +
                            grantor.short(12)
150 +
                        ))
151 +
                    }
152 +
                }
153 +
            }
154 +
        }
155 +
        if let (Some(owner), Some(g)) = (&self.config.author_for, &cert.cert.grant) {
156 +
            self.follow_author(owner, &cert.cert.subject, g.grant.grantor, &reply, &home)?;
157 +
        }
158 +
        self.store.put_identity(&issuer)?;
159 +
        self.store.put_identity(&id)?;
160 +
        self.store.put_certificate(&cert)?;
161 +
        *self.keyless_key.borrow_mut() = Some((key, sk.clone(), cert.cert.expires));
162 +
        Ok((key, sk))
163 +
    }
164 +
}
165 +
166 +
impl Repo {
167 +
    /// `author.for` (oxygen#82): keep the identity the hub names as the
168 +
    /// grant's owner's newest (`author_chain`, newest first, each the
169 +
    /// successor of the next, the last the grant's key's), store the chain
170 +
    /// here so the grant checks against it, and note its key in the home,
171 +
    /// so this and the home's next commands author as it.
172 +
    fn follow_author(
173 +
        &self,
174 +
        owner: &str,
175 +
        subject: &str,
176 +
        grantor: KeyId,
177 +
        reply: &serde_json::Value,
178 +
        home: &Path,
179 +
    ) -> Result<()> {
180 +
        let granted_by = subject.rsplit_once('@').map(|(_, o)| o).unwrap_or(subject);
181 +
        if !granted_by.eq_ignore_ascii_case(owner) {
182 +
            return fail(format!(
183 +
                "author.for is {owner}, but this home's grant is from {granted_by}: set author.for to the account that granted it"
184 +
            ));
185 +
        }
186 +
        let chain: Vec<Identity> = reply["author_chain"]
187 +
            .as_array()
188 +
            .map(|a| {
189 +
                a.iter()
190 +
                    .filter_map(|v| v.as_str().and_then(crate::dsse::base64_decode))
191 +
                    .filter_map(|b| Identity::decode(&b).ok())
192 +
                    .collect()
193 +
            })
194 +
            .unwrap_or_default();
195 +
        let linked = chain.windows(2).all(|w| w[0].previous == Some(w[1].hash()))
196 +
            && chain.last().is_none_or(|l| l.previous == Some(grantor));
197 +
        if !linked {
198 +
            return fail(
199 +
                "the hub's author chain does not lead back to the grant's key".to_string(),
200 +
            );
201 +
        }
202 +
        let keys = home.join("keys");
203 +
        std::fs::create_dir_all(&keys)?;
204 +
        for id in &chain {
205 +
            self.store.put_identity(id)?;
206 +
            let path = keys.join(format!("{}.identity", id.hash().to_hex()));
207 +
            if !path.exists() {
208 +
                crate::durable::write_atomic(&path, &id.encode())?;
209 +
            }
210 +
        }
211 +
        let newest = chain.first().map(Identity::hash).unwrap_or(grantor);
212 +
        crate::durable::write_atomic(
213 +
            &keys.join(super::config::AUTHOR_FOR_FILE),
214 +
            format!("{}\n", newest.to_hex()).as_bytes(),
215 +
        )?;
216 +
        Ok(())
217 +
    }
218 +
}
219 +
220 +
/// The key a home names as its author (`author.key` in its
221 +
/// `config.toml`), with its secret from the home's `keys/`: what signs
222 +
/// outside any repository, such as a grant to an agent (`oh agent grant`).
223 +
/// `None` when the home names no key or does not hold its secret.
224 +
pub fn home_key(home: &Path) -> Result<Option<(Identity, SecretKey)>> {
225 +
    let config = read_config_file(&home.join("config.toml"))?;
226 +
    let Some(hex) = config.get("author.key").filter(|v| !v.is_empty()) else {
227 +
        return Ok(None);
228 +
    };
229 +
    let key = KeyId::from_hex(hex)
230 +
        .ok_or_else(|| RepoError::Msg("author.key must be a 64-character hex KeyId".into()))?;
231 +
    let keys = home.join("keys");
232 +
    let secret = match fs::read(keys.join(format!("{hex}.secret"))) {
233 +
        Ok(b) => b,
234 +
        Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(None),
235 +
        Err(e) => return Err(e.into()),
236 +
    };
237 +
    let (k, sk) = SecretKey::decode(&secret)?;
238 +
    let id = Identity::decode(&fs::read(keys.join(format!("{hex}.identity")))?)?;
239 +
    if k != key || id.hash() != key {
240 +
        return fail(format!(
241 +
            "{} holds another key than author.key",
242 +
            keys.display()
243 +
        ));
244 +
    }
245 +
    Ok(Some((id, sk)))
246 +
}
247 +
248 +
#[cfg(test)]
249 +
mod tests {
250 +
    use super::*;
251 +
252 +
    #[test]
253 +
    fn a_home_s_key_signs_outside_any_repository() {
254 +
        static HOME: std::sync::OnceLock<tempfile::TempDir> = std::sync::OnceLock::new();
255 +
        HOME.get_or_init(|| {
256 +
            let d = tempfile::tempdir().unwrap();
257 +
            std::env::set_var("OXY_HOME", d.path());
258 +
            d
259 +
        });
260 +
        let dir = tempfile::tempdir().unwrap();
261 +
        let home = dir.path().join("home");
262 +
        assert!(home_key(&home).unwrap().is_none(), "no config, no key");
263 +
        let mut r = Repo::init(&dir.path().join("work"))
264 +
            .unwrap()
265 +
            .with_home(home.clone());
266 +
        let (key, id) = r
267 +
            .create_identity("Andy", "[email protected]", IdentityScope::User)
268 +
            .unwrap();
269 +
        let (got, sk) = home_key(&home).unwrap().expect("the home's author key");
270 +
        assert_eq!(got, id);
271 +
        let sig = sk.sign(Domain::Grant, &key, &ObjectRef([1; 32]));
272 +
        assert!(verify_signature(&id, &sig, Domain::Grant, &ObjectRef([1; 32])).is_ok());
273 +
        // An agent whose signer is keyless signs nothing until it asks.
274 +
        r.set_user_config("author.signer", "keyless").unwrap();
275 +
        assert!(r.config.signer_keyless && r.config.signer_key.is_none());
276 +
        assert_eq!(r.signing_key(), None);
277 +
        let err = r.secret_key().unwrap_err().to_string();
278 +
        assert!(err.contains("keyless.hub nor a default remote"), "{err}");
279 +
    }
280 +
}
modified crates/oxy-core/src/repo/mod.rs +875 −184
…
59 unchanged lines
60 60
61 61
pub type Result<T> = std::result::Result<T, RepoError>;
62 62
63 +
/// The operation head's reference record on a store that is not this
64 +
/// disk's directory (oxygen#68).
65 +
pub const HEAD_RECORD: &str = "ops/head";
66 +
67 +
/// How often recording an operation reads the head again when other
68 +
/// processes keep moving it.
69 +
const HEAD_TRIES: usize = 8;
70 +
71 +
/// The object store of the repository directory `oxy`: `.oxy/objects`,
72 +
/// or the store `storage.url` names behind a cache of `storage.cache`
73 +
/// bytes in `.oxy/cache` (oxygen#68). `.oxy/objects` is then only
74 +
/// scratch space: packs being written, and derived caches.
75 +
fn open_objects(oxy: &Path, config: &Config) -> Result<Store> {
76 +
    let Some(url) = &config.storage_url else {
77 +
        return Ok(Store::open(oxy.join("objects"))?);
78 +
    };
79 +
    let options = crate::protocol::RemoteOptions {
80 +
        cache_dir: Some(oxy.join("cache")),
81 +
        ..Default::default()
82 +
    };
83 +
    let remote = crate::protocol::open_store(url, &options)
84 +
        .ok_or_else(|| {
85 +
            RepoError::Msg(format!(
86 +
                "storage.url {url}: this oxy cannot open that kind of store (dir://, or s3://, gs:// and az:// in builds with cloud stores)"
87 +
            ))
88 +
        })?
89 +
        .map_err(|e| RepoError::Msg(format!("open the repository's store {url}: {e}")))?;
90 +
    let cached = crate::strata::CachingStore::open(
91 +
        std::sync::Arc::from(remote),
92 +
        oxy.join("cache"),
93 +
        config.storage_cache,
94 +
    )
95 +
    .map_err(|e| RepoError::Msg(format!("open the cache for {url}: {e}")))?;
96 +
    Ok(Store::open_on(
97 +
        std::sync::Arc::new(cached),
98 +
        oxy.join("objects"),
99 +
    )?)
100 +
}
101 +
63 102
fn fail<T>(s: impl Into<String>) -> Result<T> {
64 103
    Err(RepoError::Msg(s.into()))
65 104
}
…
11 unchanged lines
77 116
mod hosted;
78 117
mod identity;
79 118
mod inspect;
119 +
mod keyless;
80 120
mod ops;
81 121
mod policy;
82 122
mod prompt;
…
6 unchanged lines
89 129
mod tags;
90 130
#[cfg(test)]
91 131
mod tests;
132 +
mod touched;
133 +
mod transparency;
92 134
mod walk;
93 135
mod workspace;
94 136
…
6 unchanged lines
101 143
pub use git::{GitExportReport, GitImportReport, GitVerifyReport};
102 144
pub use hooks::{json_array, json_str, HookOutcome};
103 145
pub use hosted::FileEdit;
104 -
pub use identity::{Rotation, SignatureStatus};
146 +
pub use identity::{create_user_identity, Rotation, SignatureStatus};
105 147
pub use inspect::{BisectStep, BlameLine, Description};
106 -
pub use ops::{RestoreReport, RevertReport, ViewChange};
148 +
pub use keyless::home_key;
149 +
pub use ops::{RestoreReport, RevertReport, Undone, ViewChange};
107 150
pub use policy::PolicyUpdate;
108 151
pub use prompt::{PromptInfo, DEFAULT_PROMPT_FORMAT, GLYPH_PROMPT_FORMAT};
109 152
pub use receipts::ReceiptInfo;
110 153
pub use revset::{Revset, REVSET_BUDGET};
111 154
pub use snapshot::{
112 155
    added_node_paths, diff_files_of, has_conflict_markers, is_binary, referenced_changes,
113 -
    render_change, render_hunks, similarity, ChangeStatus, DiffHunk, FileDiff, Hunk, PathStatus,
114 -
    Selection, SnapshotResult,
156 +
    render_change, render_hunks, similarity, ChangeStatus, DiffHunk, FileDiff, Hunk, KeptBoth,
157 +
    PathStatus, Selection, SnapshotResult,
115 158
};
116 159
pub use subrepos::{SubrepoInfo, SubrepoSync};
117 -
pub use sync::{FetchReport, PullReport, PushReport, RefUpdated};
160 +
pub use sync::{FetchReport, PullReport, PushReport, RefUpdated, OBJECTS_RESULT};
118 161
pub use tags::TagInfo;
162 +
pub use transparency::LogAudit;
119 163
pub use workspace::{FileLock, SnapshotLock, WorkingCopyLock, WorkspaceInfo};
120 164
121 165
/// The file name under which per-bookmark or per-tag data is kept: `/`
…
86 unchanged lines
208 252
}
209 253
210 254
pub struct Repo {
255 +
    /// The key a host accepts certified updates with (spec §14.7): when
256 +
    /// set, accepting a reference update signed by a short-lived key also
257 +
    /// records the host's acceptance and its time.
258 +
    pub(crate) acceptor: Option<std::sync::Arc<(Identity, crate::identity::SecretKey)>>,
259 +
    /// With `author.signer = keyless`, the fresh key the hub last
260 +
    /// certified and when its certificate ends (spec §14.7): kept in memory
261 +
    /// only, for one command.
262 +
    pub(crate) keyless_key: std::cell::RefCell<Option<(KeyId, crate::identity::SecretKey, u64)>>,
211 263
    /// Oxygen-pinned subrepos skipped by the running Git export.
212 264
    pub(crate) git_export_skipped: std::cell::Cell<usize>,
213 265
    /// The manifest as last read, with the file identity it was read from,
…
6 unchanged lines
220 272
    /// The nested repositories the last walk left out, being neither
221 273
    /// registered nor tracked subrepos (oxygen#43).
222 274
    pub(crate) untracked_nested: std::cell::RefCell<Vec<String>>,
275 +
    /// New untracked files the last walk left out for their size, with
276 +
    /// their sizes (oxygen#98).
277 +
    pub(crate) skipped_large: std::cell::RefCell<Vec<(String, u64)>>,
223 278
    root: PathBuf,
224 279
    oxy: PathBuf,
225 280
    store: Store,
…
11 unchanged lines
237 292
    /// `--at-op`: read the repository as it was after this operation;
238 293
    /// recording an operation is refused.
239 294
    at_op: Option<ObjectRef>,
295 +
    /// The operation head this handle reads (oxygenhub#249): read once, on
296 +
    /// first use, and moved only by this handle's own operations or by
297 +
    /// [`Repo::refresh`], so one command or request never mixes two heads.
298 +
    head_seen: std::cell::Cell<Option<ObjectRef>>,
240 299
    /// The directories (with their times) and nested repositories of the
241 300
    /// last walk, for the manifest written from it.
242 301
    pub(crate) walked_dirs: std::cell::RefCell<Option<WalkedDirs>>,
…
58 unchanged lines
301 360
    /// Initialize a repository with a given repository ID (clones share the
302 361
    /// ID of their origin, spec §14.4).
303 362
    pub fn init_with_id(root: &Path, id: [u8; 16]) -> Result<Repo> {
363 +
        Self::init_on(root, id, None)
364 +
    }
365 +
366 +
    /// A new repository with a fresh id, bare or not, on `storage` or in
367 +
    /// `.oxy/objects` (`oxy init [--bare] [--store URL]`).
368 +
    pub fn init_new(root: &Path, bare: bool, storage: Option<&str>) -> Result<Repo> {
369 +
        let mut id = [0u8; 16];
370 +
        rand::rngs::OsRng.fill_bytes(&mut id);
371 +
        if bare {
372 +
            Self::init_bare_on(root, id, storage)
373 +
        } else {
374 +
            Self::init_on(root, id, storage)
375 +
        }
376 +
    }
377 +
378 +
    /// Initialize a repository whose objects and operation head are kept
379 +
    /// on the store at `storage` (`storage.url`), or in `.oxy/objects`.
380 +
    pub fn init_on(root: &Path, id: [u8; 16], storage: Option<&str>) -> Result<Repo> {
304 381
        fs::create_dir_all(root)?;
305 -
        let root = root.canonicalize().unwrap_or_else(|_| root.to_path_buf());
382 +
        let root = crate::portable::canonical(root).unwrap_or_else(|_| root.to_path_buf());
306 383
        let oxy = root.join(OXY_DIR);
307 384
        if oxy.exists() {
308 385
            return fail(format!("{} is already an oxy repository", root.display()));
…
1 unchanged lines
310 387
        fs::create_dir_all(oxy.join("ops"))?;
311 388
        write_atomic(&oxy.join("format"), FORMAT_LINE.as_bytes())?;
312 389
        write_atomic(&oxy.join("id"), hex::encode(id).as_bytes())?;
313 -
        let config = Config::load(Some(&default_user_config()), &oxy.join("config"))?;
390 +
        let mut config = Config::load(Some(&default_user_config()), &oxy.join("config"))?;
391 +
        if let Some(url) = storage {
392 +
            config.set("storage.url", url)?;
393 +
        }
314 394
        config.save(&oxy.join("config"))?;
315 395
        let workspace = WorkspaceId::random();
316 396
        write_atomic(&oxy.join("workspace"), workspace.to_hex().as_bytes())?;
317 -
        let store = Store::open(oxy.join("objects"))?;
397 +
        let store = open_objects(&oxy, &config)?;
318 398
        let repo = Repo {
399 +
            acceptor: None,
400 +
            keyless_key: Default::default(),
319 401
            last_change: Default::default(),
320 402
            at_op: None,
403 +
            head_seen: Default::default(),
321 404
            walked_dirs: Default::default(),
322 405
            listed_dirs: Default::default(),
323 406
            git_export_skipped: Default::default(),
324 407
            manifest_cache: Default::default(),
325 408
            listing_cache: Default::default(),
326 409
            untracked_nested: Default::default(),
410 +
            skipped_large: Default::default(),
327 411
            root,
328 412
            oxy,
329 413
            store,
…
28 unchanged lines
358 442
    }
359 443
360 444
    pub fn init_bare_with_id(root: &Path, id: [u8; 16]) -> Result<Repo> {
445 +
        Self::init_bare_on(root, id, None)
446 +
    }
447 +
448 +
    /// [`Repo::init_on`] for a bare repository.
449 +
    pub fn init_bare_on(root: &Path, id: [u8; 16], storage: Option<&str>) -> Result<Repo> {
361 450
        fs::create_dir_all(root)?;
362 -
        let root = root.canonicalize().unwrap_or_else(|_| root.to_path_buf());
451 +
        let root = crate::portable::canonical(root).unwrap_or_else(|_| root.to_path_buf());
363 452
        let oxy = root.join(OXY_DIR);
364 453
        if oxy.exists() {
365 454
            return fail(format!("{} is already an oxy repository", root.display()));
…
2 unchanged lines
368 457
        write_atomic(&oxy.join("format"), FORMAT_LINE.as_bytes())?;
369 458
        write_atomic(&oxy.join("id"), hex::encode(id).as_bytes())?;
370 459
        write_atomic(&oxy.join(BARE_MARKER), b"")?;
371 -
        let config = Config::load(Some(&default_user_config()), &oxy.join("config"))?;
460 +
        let mut config = Config::load(Some(&default_user_config()), &oxy.join("config"))?;
461 +
        if let Some(url) = storage {
462 +
            config.set("storage.url", url)?;
463 +
        }
372 464
        config.save(&oxy.join("config"))?;
373 -
        let store = Store::open(oxy.join("objects"))?;
465 +
        let store = open_objects(&oxy, &config)?;
374 466
        let repo = Repo {
467 +
            acceptor: None,
468 +
            keyless_key: Default::default(),
375 469
            last_change: Default::default(),
376 470
            at_op: None,
471 +
            head_seen: Default::default(),
377 472
            walked_dirs: Default::default(),
378 473
            listed_dirs: Default::default(),
379 474
            git_export_skipped: Default::default(),
380 475
            manifest_cache: Default::default(),
381 476
            listing_cache: Default::default(),
382 477
            untracked_nested: Default::default(),
478 +
            skipped_large: Default::default(),
383 479
            root,
384 480
            oxy,
385 481
            store,
…
47 unchanged lines
433 529
434 530
    /// Open the repository containing `start` (searching upwards).
435 531
    pub fn open(start: &Path) -> Result<Repo> {
436 -
        let start = start.canonicalize()?;
532 +
        let start = crate::portable::canonical(start)?;
437 533
        let mut dir: Option<&Path> = Some(&start);
438 534
        while let Some(d) = dir {
439 535
            let oxy = d.join(OXY_DIR);
…
8 unchanged lines
448 544
                    return fail(format!("unsupported repository format: {}", format.trim()));
449 545
                }
450 546
                let config = Config::load(Some(&default_user_config()), &shared.join("config"))?;
451 -
                let store = Store::open(shared.join("objects"))?;
547 +
                let store = open_objects(&shared, &config)?;
452 548
                let mut repo = Repo {
549 +
                    acceptor: None,
550 +
                    keyless_key: Default::default(),
453 551
                    last_change: Default::default(),
454 552
                    at_op: None,
553 +
                    head_seen: Default::default(),
455 554
                    walked_dirs: Default::default(),
456 555
                    listed_dirs: Default::default(),
457 556
                    git_export_skipped: Default::default(),
458 557
                    manifest_cache: Default::default(),
459 558
                    listing_cache: Default::default(),
460 559
                    untracked_nested: Default::default(),
560 +
                    skipped_large: Default::default(),
461 561
                    root: d.to_path_buf(),
462 562
                    oxy: shared,
463 563
                    store,
…
17 unchanged lines
481 581
                    WorkspaceId::from_hex(fs::read_to_string(oxy.join("workspace"))?.trim())
482 582
                        .ok_or_else(|| RepoError::Msg("corrupt .oxy/workspace".into()))?
483 583
                };
484 -
                let store = Store::open(oxy.join("objects"))?;
584 +
                let store = open_objects(&oxy, &config)?;
485 585
                let mut repo = Repo {
586 +
                    acceptor: None,
587 +
                    keyless_key: Default::default(),
486 588
                    last_change: Default::default(),
487 589
                    at_op: None,
590 +
                    head_seen: Default::default(),
488 591
                    walked_dirs: Default::default(),
489 592
                    listed_dirs: Default::default(),
490 593
                    git_export_skipped: Default::default(),
491 594
                    manifest_cache: Default::default(),
492 595
                    listing_cache: Default::default(),
493 596
                    untracked_nested: Default::default(),
597 +
                    skipped_large: Default::default(),
494 598
                    root: d.to_path_buf(),
495 599
                    oxy,
496 600
                    store,
…
25 unchanged lines
522 626
    /// Rewrite a format-1 operation log, whose bookmarks were head lists,
523 627
    /// into format 2, where each bookmark names a `state` object.
524 628
    fn migrate_format1(&self) -> Result<()> {
525 -
        let hex = fs::read_to_string(self.oxy.join("ops").join("head"))?;
526 -
        let head = ObjectRef::from_hex(hex.trim())
527 -
            .ok_or_else(|| RepoError::Msg("corrupt .oxy/ops/head".into()))?;
629 +
        let (head, version) = self.read_head()?;
528 630
        // Collect the chain root-first.
529 631
        let mut chain: Vec<Format1Op> = Vec::new();
530 632
        let mut cur = Some(head);
…
18 unchanged lines
549 651
                hlc: old.hlc,
550 652
                command: old.command,
551 653
                view,
654 +
                workspace: None,
655 +
                undoes: None,
552 656
            };
553 657
            new_parent = Some(self.store.put(ObjectType::Op, &op.encode())?);
554 658
        }
555 -
        write_atomic(
556 -
            &self.oxy.join("ops").join("head"),
557 -
            new_parent.unwrap().to_hex().as_bytes(),
558 -
        )?;
659 +
        if !self.swap_head(&new_parent.unwrap(), version.as_ref())? {
660 +
            return fail("the operation log changed while it was rewritten; run the command again");
661 +
        }
559 662
        write_atomic(&self.oxy.join("format"), FORMAT_LINE.as_bytes())?;
560 663
        Ok(())
561 664
    }
562 665
563 -
    /// Pack automatically once many loose objects have accumulated.
666 +
    /// Pack automatically once many loose objects have accumulated, and
667 +
    /// delete what packing retired over [`crate::store::RETIRED_FOR`] ago
668 +
    /// (a store nobody runs `oxy gc` on would keep both copies forever).
564 669
    pub(crate) fn maybe_autopack(&self) -> Result<()> {
670 +
        self.store
671 +
            .expire_retired(crate::store::RETIRED_FOR, &mut |_| {})?;
565 672
        if self.store.loose_count()? > AUTOPACK_THRESHOLD {
566 673
            self.store.pack_loose()?;
567 674
            self.store.merge_packs()?;
…
25 unchanged lines
593 700
    pub fn root(&self) -> &Path {
594 701
        &self.root
595 702
    }
703 +
    /// The repository's `.oxy` directory (shared by its workspaces), for
704 +
    /// what other crates keep beside the store (Cylinder's artifacts).
705 +
    pub fn oxy_dir(&self) -> &Path {
706 +
        &self.oxy
707 +
    }
596 708
    pub fn store(&self) -> &Store {
597 709
        &self.store
598 710
    }
…
5 unchanged lines
604 716
    /// (tests, or `$OXY_HOME`). Reloads the effective configuration.
605 717
    /// Act as if the working copy were on another platform's file system
606 718
    /// (tests of Windows and macOS behaviour on any host).
719 +
    /// Accept certified updates as `identity` (a host's key), recording when
720 +
    /// (spec §14.7).
721 +
    pub fn with_acceptor(mut self, identity: Identity, sk: crate::identity::SecretKey) -> Repo {
722 +
        self.acceptor = Some(std::sync::Arc::new((identity, sk)));
723 +
        self
724 +
    }
725 +
607 726
    pub fn with_platform(mut self, platform: crate::portable::Platform) -> Repo {
608 727
        self.platform = platform;
609 728
        self
…
73 unchanged lines
683 802
        if let Some(r) = self.at_op {
684 803
            return Ok((r, Op::decode(&self.store.get_typed(&r, ObjectType::Op)?)?));
685 804
        }
686 -
        let hex = fs::read_to_string(self.oxy.join("ops").join("head"))?;
687 -
        let r = ObjectRef::from_hex(hex.trim())
688 -
            .ok_or_else(|| RepoError::Msg("corrupt .oxy/ops/head".into()))?;
805 +
        let r = self.head_ref()?;
689 806
        let op = Op::decode(&self.store.get_typed(&r, ObjectType::Op)?)?;
690 807
        Ok((r, op))
691 808
    }
…
3 unchanged lines
695 812
    }
696 813
697 814
    fn record_op(&self, parents: &[ObjectRef], command: &str, view: View) -> Result<ObjectRef> {
815 +
        self.record_op_undoing(parents, command, view, None)
816 +
    }
817 +
818 +
    /// [`Repo::record_op`], for an undo that reverts operation `undoes`.
819 +
    fn record_op_undoing(
820 +
        &self,
821 +
        parents: &[ObjectRef],
822 +
        command: &str,
823 +
        view: View,
824 +
        undoes: Option<ObjectRef>,
825 +
    ) -> Result<ObjectRef> {
698 826
        if self.at_op.is_some() {
699 827
            return fail(
700 828
                "--at-op shows the repository as it was; this command would change it (run it without --at-op)",
…
4 unchanged lines
705 833
            hlc: Hlc::now(),
706 834
            command: command.into(),
707 835
            view,
836 +
            workspace: Some(self.workspace),
837 +
            undoes,
708 838
        };
709 839
        let r = self.store.put(ObjectType::Op, &op.encode())?;
710 840
        // The head is the commit point: everything it names must be on disk first.
711 841
        let flushing = crate::timing::phase("op: flush the store");
712 842
        self.store.flush()?;
713 843
        drop(flushing);
714 -
        // Compare and swap under a short lock (spec §12.1): when another
715 -
        // process recorded an operation since this one read the head, the
716 -
        // two are merged instead of one overwriting the other.
717 -
        let _lock = self.lock_ops()?;
718 -
        let current = match parents.first() {
844 +
        // Compare and swap (spec §12.1): when another process recorded an
845 +
        // operation since this one read the head, the two are merged
846 +
        // instead of one overwriting the other. On this disk the head moves
847 +
        // under a short lock; on any other store by its record's version,
848 +
        // and a record that moved meanwhile is read again and merged again.
849 +
        let _lock = if self.store.is_local() {
850 +
            Some(self.lock_ops()?)
851 +
        } else {
852 +
            None
853 +
        };
854 +
        let Some(parent) = parents.first() else {
719 855
            // The first operation (init): there is no head yet.
720 -
            None => None,
721 -
            Some(_) => Some(self.head_ref()?),
722 -
        };
723 -
        let Some(current) = current.filter(|c| Some(c) != parents.first()) else {
724 -
            write_atomic(&self.oxy.join("ops").join("head"), r.to_hex().as_bytes())?;
856 +
            if !self.swap_head(&r, None)? {
857 +
                return fail(format!(
858 +
                    "{} already holds a repository's operation log",
859 +
                    self.store.backend().describe()
860 +
                ));
861 +
            }
725 862
            return Ok(r);
726 863
        };
727 -
        let base = match parents.first() {
728 -
            Some(p) => Op::decode(&self.store.get_typed(p, ObjectType::Op)?)?.view,
729 -
            None => View::default(),
730 -
        };
731 -
        let theirs = Op::decode(&self.store.get_typed(&current, ObjectType::Op)?)?.view;
732 -
        let merged = Op {
733 -
            parents: vec![current, r],
734 -
            hlc: Hlc::now(),
735 -
            command: format!("{command} (merged with a concurrent operation)"),
736 -
            view: self.merge_views(&base, &op.view, &theirs)?,
737 -
        };
738 -
        let m = self.store.put(ObjectType::Op, &merged.encode())?;
739 -
        self.store.flush()?;
740 -
        write_atomic(&self.oxy.join("ops").join("head"), m.to_hex().as_bytes())?;
741 -
        Ok(m)
864 +
        let base = Op::decode(&self.store.get_typed(parent, ObjectType::Op)?)?.view;
865 +
        for _ in 0..HEAD_TRIES {
866 +
            let (current, version) = self.read_head()?;
867 +
            let to = if current == *parent {
868 +
                r
869 +
            } else {
870 +
                let theirs = Op::decode(&self.store.get_typed(&current, ObjectType::Op)?)?.view;
871 +
                let merged = Op {
872 +
                    parents: vec![current, r],
873 +
                    hlc: Hlc::now(),
874 +
                    command: format!("{command} (merged with a concurrent operation)"),
875 +
                    view: self.merge_views(&base, &op.view, &theirs)?,
876 +
                    workspace: Some(self.workspace),
877 +
                    undoes: None,
878 +
                };
879 +
                let m = self.store.put(ObjectType::Op, &merged.encode())?;
880 +
                self.store.flush()?;
881 +
                m
882 +
            };
883 +
            if self.swap_head(&to, version.as_ref())? {
884 +
                return Ok(to);
885 +
            }
886 +
        }
887 +
        fail("other oxy processes kept changing the operation log while this one recorded; run the command again")
742 888
    }
743 889
744 890
    /// The operation head's lock, held only while the head pointer moves.
…
5 unchanged lines
750 896
        )
751 897
    }
752 898
899 +
    /// The operation head this handle reads: the one it first read, or
900 +
    /// the last one it moved the head to (see [`Repo::refresh`]).
753 901
    fn head_ref(&self) -> Result<ObjectRef> {
754 -
        let hex = fs::read_to_string(self.oxy.join("ops").join("head"))?;
755 -
        ObjectRef::from_hex(hex.trim())
756 -
            .ok_or_else(|| RepoError::Msg("corrupt .oxy/ops/head".into()))
902 +
        if let Some(r) = self.head_seen.get() {
903 +
            return Ok(r);
904 +
        }
905 +
        let r = self.read_head()?.0;
906 +
        self.head_seen.set(Some(r));
907 +
        Ok(r)
908 +
    }
909 +
910 +
    /// Read the operation head again: what another process or handle
911 +
    /// recorded since this handle first read it becomes visible. A handle
912 +
    /// keeps the head it first read for the rest of its life otherwise,
913 +
    /// so whatever it reads comes from one view (oxygenhub#249); a loop
914 +
    /// that waits for others' operations, or a step that must see what
915 +
    /// another handle just recorded, refreshes. Packs written since are
916 +
    /// picked up too.
917 +
    pub fn refresh(&self) -> Result<()> {
918 +
        self.forget_head();
919 +
        self.store.refresh()?;
920 +
        Ok(())
757 921
    }
758 922
923 +
    /// Read the operation head again on next use, as [`Repo::refresh`]
924 +
    /// does, leaving the packs as they are.
925 +
    pub(crate) fn forget_head(&self) {
926 +
        self.head_seen.set(None);
927 +
    }
928 +
929 +
    /// The operation head, and on a store its record's version for
930 +
    /// [`Repo::swap_head`]. On this disk the head is `.oxy/ops/head`; on
931 +
    /// any other store (a bucket, or memory in tests) it is the store's
932 +
    /// [`HEAD_RECORD`] reference record (oxygen#68).
933 +
    fn read_head(&self) -> Result<(ObjectRef, Option<crate::strata::Version>)> {
934 +
        if self.store.is_local() {
935 +
            let hex = fs::read_to_string(self.oxy.join("ops").join("head"))?;
936 +
            let r = ObjectRef::from_hex(hex.trim())
937 +
                .ok_or_else(|| RepoError::Msg("corrupt .oxy/ops/head".into()))?;
938 +
            return Ok((r, None));
939 +
        }
940 +
        let backend = self.store.backend();
941 +
        match backend.read_ref(HEAD_RECORD) {
942 +
            Ok(Some((bytes, version))) => {
943 +
                let r = ObjectRef::from_hex(String::from_utf8_lossy(&bytes).trim()).ok_or_else(
944 +
                    || {
945 +
                        RepoError::Msg(format!(
946 +
                            "the operation head in {} is corrupt",
947 +
                            backend.describe()
948 +
                        ))
949 +
                    },
950 +
                )?;
951 +
                Ok((r, Some(version)))
952 +
            }
953 +
            Ok(None) => fail(format!(
954 +
                "{} holds no operation log (no {HEAD_RECORD} record)",
955 +
                backend.describe()
956 +
            )),
957 +
            Err(e) => fail(format!("read the operation head: {e}")),
958 +
        }
959 +
    }
960 +
961 +
    /// Move the operation head to `to` from the `expected` version
962 +
    /// [`Repo::read_head`] gave (`None`: there is no head yet). On this
963 +
    /// disk the caller holds the operations lock and the head always
964 +
    /// moves; on a store it moves only if the record is unchanged, and
965 +
    /// `false` says another process moved it first.
966 +
    fn swap_head(&self, to: &ObjectRef, expected: Option<&crate::strata::Version>) -> Result<bool> {
967 +
        if self.store.is_local() {
968 +
            write_atomic(&self.oxy.join("ops").join("head"), to.to_hex().as_bytes())?;
969 +
            self.head_seen.set(Some(*to));
970 +
            return Ok(true);
971 +
        }
972 +
        match self
973 +
            .store
974 +
            .backend()
975 +
            .write_ref(HEAD_RECORD, to.to_hex().as_bytes(), expected)
976 +
        {
977 +
            Ok(_) => {
978 +
                self.head_seen.set(Some(*to));
979 +
                Ok(true)
980 +
            }
981 +
            Err(crate::strata::Error::Conflict(_)) => Ok(false),
982 +
            Err(e) => fail(format!("move the operation head: {e}")),
983 +
        }
984 +
    }
985 +
759 986
    /// Three-way merge of two views that both started from `base` (spec
760 987
    /// §12.1). A reference only one side changed takes that side's value.
761 988
    /// When both changed it differently: a bookmark keeps both sides'
…
110 unchanged lines
872 1099
        Ok(out)
873 1100
    }
874 1101
875 -
    /// Undo the last user command (spec §12.1). Operations that only recorded
876 -
    /// a working-copy snapshot are skipped, since the files are still on disk
877 -
    /// and the next snapshot recreates them, and a command's own checkout
878 -
    /// (`… (checkout)`) goes with the command. When the working copy is
879 -
    /// clean and was clean after the operation undone to, its files follow
880 -
    /// too, since nothing on disk can be lost; otherwise they stay as they
881 -
    /// are (an undone commit's edits come back as uncommitted work).
882 -
    pub fn undo(&self) -> Result<Op> {
1102 +
    /// Undo this workspace's last command (spec §12.1, oxygen#99).
1103 +
    ///
1104 +
    /// The command undone is the newest one this workspace ran: operations
1105 +
    /// that only recorded a working-copy snapshot are skipped (the files are
1106 +
    /// still on disk and the next snapshot recreates them), as are pushes
1107 +
    /// (the remote keeps what it was sent; see [`Undone::pushed`]), a command's own
1108 +
    /// checkout (`… (checkout)`) goes with the command, other workspaces'
1109 +
    /// operations are never undone, and a command an earlier undo already
1110 +
    /// reverted is passed over, so undoing again goes further back.
1111 +
    ///
1112 +
    /// When nothing newer came from another workspace, the operation head
1113 +
    /// moves back to before the command. Otherwise the command is reverted
1114 +
    /// in place: every reference it changed goes back to its value before
1115 +
    /// it, recorded as a new operation that names it, and anything another
1116 +
    /// workspace changed since is kept. A reference both the command and a
1117 +
    /// later operation of another workspace changed is not guessed at: the
1118 +
    /// undo is refused, naming it.
1119 +
    ///
1120 +
    /// When the working copy is clean and was clean after the command's
1121 +
    /// predecessor, its files follow too, since nothing on disk can be
1122 +
    /// lost; otherwise they stay as they are (an undone commit's edits come
1123 +
    /// back as uncommitted work).
1124 +
    pub fn undo(&self) -> Result<Undone> {
883 1125
        let _wc = self.lock_working_copy()?;
884 1126
        let clean_now =
885 1127
            self.clean_by_manifest()? || self.snapshot_status("oxy undo")?.statuses.is_empty();
886 -
        let (read_head, mut op) = self.head_op()?;
887 -
        let wc_now = op.view.workspaces.get(&self.workspace).copied();
888 -
        let up = |op: Op| -> Result<Option<Op>> {
889 -
            match op.parents.first() {
890 -
                Some(p) => Ok(Some(Op::decode(&self.store.get_typed(p, ObjectType::Op)?)?)),
891 -
                None => Ok(None),
892 -
            }
1128 +
        let (head_ref, head) = self.head_op()?;
1129 +
        let wc_now = head.view.workspaces.get(&self.workspace).copied();
1130 +
        let load = |r: &ObjectRef| -> Result<Op> {
1131 +
            Ok(Op::decode(&self.store.get_typed(r, ObjectType::Op)?)?)
893 1132
        };
894 -
        loop {
895 -
            while op.is_snapshot() {
896 -
                match up(op.clone())? {
897 -
                    Some(p) => op = p,
898 -
                    None => break,
1133 +
        let mut skip: BTreeSet<ObjectRef> = BTreeSet::new();
1134 +
        let mut pending_checkout: Option<Op> = None;
1135 +
        let mut others = 0usize;
1136 +
        let (mut r, mut op) = (head_ref, head.clone());
1137 +
        let (target_ref, target, parent) = loop {
1138 +
            let parent = match op.parents.first() {
1139 +
                Some(p) => Some((*p, load(p)?)),
1140 +
                None => None,
1141 +
            };
1142 +
            if skip.contains(&r) {
1143 +
                pending_checkout = None;
1144 +
            } else if op.is_snapshot() || Self::is_push(&op, parent.as_ref().map(|(_, p)| p)) {
1145 +
                // A push cannot be taken back: undoing it would only make
1146 +
                // this repository forget what the remote holds.
1147 +
            } else if !self.op_is_ours(&op, parent.as_ref().map(|(_, p)| p)) {
1148 +
                others += 1;
1149 +
            } else if let Some(u) = op.undoes {
1150 +
                skip.insert(u);
1151 +
                pending_checkout = None;
1152 +
            } else if op.command.ends_with(" (checkout)") {
1153 +
                pending_checkout = Some(op.clone());
1154 +
            } else {
1155 +
                match parent {
1156 +
                    Some((_, p)) => break (r, op, p),
1157 +
                    None => return fail("nothing to undo in this workspace"),
899 1158
                }
900 1159
            }
901 -
            if !op.command.ends_with(" (checkout)") {
902 -
                break;
903 -
            }
904 -
            match up(op.clone())? {
905 -
                Some(p) => op = p,
906 -
                None => break,
1160 +
            match parent {
1161 +
                Some((pr, p)) => (r, op) = (pr, p),
1162 +
                None => return fail("nothing to undo in this workspace"),
907 1163
            }
908 -
        }
909 -
        let Some(parent) = op.parents.first() else {
910 -
            return fail("nothing to undo");
911 1164
        };
912 -
        let parent_op = Op::decode(&self.store.get_typed(parent, ObjectType::Op)?)?;
913 -
        {
914 -
            let _lock = self.lock_ops()?;
915 -
            if self.head_ref()? != read_head {
1165 +
        let before = parent.view;
1166 +
        let after = pending_checkout
1167 +
            .map(|o| o.view)
1168 +
            .unwrap_or_else(|| target.view.clone());
1169 +
        let changes = self.view_changes(&before, &after)?;
1170 +
        let pushed = self.pushed_by(&before, &after, &head.view)?;
1171 +
        let restored_wc = before.workspaces.get(&self.workspace).copied();
1172 +
        if others == 0 {
1173 +
            // Only this workspace's operations since: move the head back.
1174 +
            let _lock = if self.store.is_local() {
1175 +
                Some(self.lock_ops()?)
1176 +
            } else {
1177 +
                None
1178 +
            };
1179 +
            let parent_ref = *target.parents.first().expect("the target has a parent");
1180 +
            let (now, version) = self.read_head()?;
1181 +
            if now != head_ref || !self.swap_head(&parent_ref, version.as_ref())? {
916 1182
                return fail("another oxy command ran while undoing; run undo again");
917 1183
            }
918 -
            write_atomic(
919 -
                &self.oxy.join("ops").join("head"),
920 -
                parent.to_hex().as_bytes(),
921 -
            )?;
1184 +
        } else {
1185 +
            let view = self.revert_view(&head.view, &before, &after)?;
1186 +
            self.record_op_undoing(&[head_ref], "oxy undo", view, Some(target_ref))?;
922 1187
        }
923 1188
        self.drop_manifest();
924 -
        if let Some(then) = parent_op.view.workspaces.get(&self.workspace) {
925 -
            if clean_now && Some(*then) != wc_now {
926 -
                let wc = self.store.get_change(then)?;
1189 +
        if let Some(then) = restored_wc {
1190 +
            if clean_now && Some(then) != wc_now {
1191 +
                // Past `core.snapshot-retention` the working copy may be
1192 +
                // gone: the files then stay as they are.
1193 +
                let Ok(wc) = self.store.get_change(&then) else {
1194 +
                    return Ok(Undone {
1195 +
                        command: target.command,
1196 +
                        changes,
1197 +
                        others_kept: others,
1198 +
                        pushed,
1199 +
                    });
1200 +
                };
927 1201
                if wc.edits.is_empty() && wc.tree_ops.is_empty() {
928 1202
                    let base = self.load_state_history(&wc.deps)?.maximal(&wc.deps);
929 1203
                    // The files differ from the restored working copy by
…
3 unchanged lines
933 1207
                }
934 1208
            }
935 1209
        }
936 -
        Ok(parent_op)
1210 +
        Ok(Undone {
1211 +
            command: target.command,
1212 +
            changes,
1213 +
            others_kept: others,
1214 +
            pushed,
1215 +
        })
1216 +
    }
1217 +
1218 +
    /// Whether operation `op` was a push: it changed only what this
1219 +
    /// repository knows of remotes.
1220 +
    fn is_push(op: &Op, parent: Option<&Op>) -> bool {
1221 +
        let Some(p) = parent else {
1222 +
            return false;
1223 +
        };
1224 +
        op.command.starts_with("oxy push")
1225 +
            && op.view.bookmarks == p.view.bookmarks
1226 +
            && op.view.tags == p.view.tags
1227 +
            && op.view.workspaces == p.view.workspaces
1228 +
            && op.view.policy == p.view.policy
1229 +
    }
1230 +
1231 +
    /// Whether operation `op` (whose predecessor is `parent`) was this
1232 +
    /// workspace's. Operations record their workspace (oxygen#99); for older
1233 +
    /// ones it is read from the view: one that changed only other
1234 +
    /// workspaces' working copies was theirs, anything else counts as ours.
1235 +
    fn op_is_ours(&self, op: &Op, parent: Option<&Op>) -> bool {
1236 +
        if let Some(w) = op.workspace {
1237 +
            return w == self.workspace;
1238 +
        }
1239 +
        let Some(p) = parent else {
1240 +
            return true;
1241 +
        };
1242 +
        let ids: BTreeSet<&WorkspaceId> = op
1243 +
            .view
1244 +
            .workspaces
1245 +
            .keys()
1246 +
            .chain(p.view.workspaces.keys())
1247 +
            .collect();
1248 +
        let moved: Vec<&WorkspaceId> = ids
1249 +
            .into_iter()
1250 +
            .filter(|id| op.view.workspaces.get(id) != p.view.workspaces.get(id))
1251 +
            .collect();
1252 +
        moved.is_empty() || moved.contains(&&self.workspace)
937 1253
    }
938 1254
1255 +
    /// `current` with what changed from `before` to `after` put back, this
1256 +
    /// workspace's working copy included; refused when a later operation
1257 +
    /// changed one of the same references again.
1258 +
    fn revert_view(&self, current: &View, before: &View, after: &View) -> Result<View> {
1259 +
        fn back<K: Ord + Clone, V: Clone + PartialEq>(
1260 +
            cur: &mut BTreeMap<K, V>,
1261 +
            before: &BTreeMap<K, V>,
1262 +
            after: &BTreeMap<K, V>,
1263 +
            name: impl Fn(&K) -> String,
1264 +
            clashes: &mut Vec<String>,
1265 +
        ) {
1266 +
            let keys: BTreeSet<&K> = before.keys().chain(after.keys()).collect();
1267 +
            for k in keys {
1268 +
                let (b, a) = (before.get(k), after.get(k));
1269 +
                if a == b {
1270 +
                    continue;
1271 +
                }
1272 +
                if cur.get(k) != a {
1273 +
                    clashes.push(name(k));
1274 +
                    continue;
1275 +
                }
1276 +
                match b {
1277 +
                    Some(v) => cur.insert(k.clone(), v.clone()),
1278 +
                    None => cur.remove(k),
1279 +
                };
1280 +
            }
1281 +
        }
1282 +
        let mut out = current.clone();
1283 +
        let mut clashes = Vec::new();
1284 +
        // A bookmark another workspace moved since is put back head by
1285 +
        // head: the heads the command added go, the heads it replaced
1286 +
        // return, and the other workspace's heads stay. Only when a later
1287 +
        // change was built on one the command added is that not possible.
1288 +
        let names: BTreeSet<&String> = before
1289 +
            .bookmarks
1290 +
            .keys()
1291 +
            .chain(after.bookmarks.keys())
1292 +
            .collect();
1293 +
        for name in names {
1294 +
            let (b, a) = (before.bookmarks.get(name), after.bookmarks.get(name));
1295 +
            let cur = current.bookmarks.get(name);
1296 +
            if a == b {
1297 +
                continue;
1298 +
            }
1299 +
            if cur == a {
1300 +
                match b {
1301 +
                    Some(st) => out.bookmarks.insert(name.clone(), *st),
1302 +
                    None => out.bookmarks.remove(name),
1303 +
                };
1304 +
                continue;
1305 +
            }
1306 +
            let (Some(b), Some(a), Some(cur)) = (b, a, cur) else {
1307 +
                clashes.push(format!("bookmark {name}"));
1308 +
                continue;
1309 +
            };
1310 +
            let was: BTreeSet<ChangeHash> = self.store.get_state(b)?.heads.into_iter().collect();
1311 +
            let became: BTreeSet<ChangeHash> = self.store.get_state(a)?.heads.into_iter().collect();
1312 +
            let now: BTreeSet<ChangeHash> = self.store.get_state(cur)?.heads.into_iter().collect();
1313 +
            let added: BTreeSet<&ChangeHash> = became.difference(&was).collect();
1314 +
            if !added.iter().all(|h| now.contains(h)) {
1315 +
                clashes.push(format!(
1316 +
                    "bookmark {name} (a later change builds on this one)"
1317 +
                ));
1318 +
                continue;
1319 +
            }
1320 +
            let mut heads: Vec<ChangeHash> = now
1321 +
                .iter()
1322 +
                .filter(|h| !added.contains(h))
1323 +
                .chain(was.difference(&became))
1324 +
                .copied()
1325 +
                .collect();
1326 +
            heads.sort();
1327 +
            heads.dedup();
1328 +
            let heads = self.load_state(&heads)?.maximal(&heads);
1329 +
            out.bookmarks.insert(
1330 +
                name.clone(),
1331 +
                self.store.put_state(&StateObject::new(heads))?,
1332 +
            );
1333 +
        }
1334 +
1335 +
        back(
1336 +
            &mut out.tags,
1337 +
            &before.tags,
1338 +
            &after.tags,
1339 +
            |k| format!("tag {k}"),
1340 +
            &mut clashes,
1341 +
        );
1342 +
        let mine = self.workspace;
1343 +
        let others = |v: &View| -> BTreeMap<WorkspaceId, ChangeHash> {
1344 +
            v.workspaces
1345 +
                .iter()
1346 +
                .filter(|(k, _)| **k != mine)
1347 +
                .map(|(k, v)| (*k, *v))
1348 +
                .collect()
1349 +
        };
1350 +
        let mut theirs = others(&out);
1351 +
        let workspace_name = |id: &WorkspaceId| match self.workspace_path(id) {
1352 +
            Some(p) => format!("the workspace at {}", p.display()),
1353 +
            None => format!("workspace {}", id.to_hex()),
1354 +
        };
1355 +
        back(
1356 +
            &mut theirs,
1357 +
            &others(before),
1358 +
            &others(after),
1359 +
            workspace_name,
1360 +
            &mut clashes,
1361 +
        );
1362 +
        let empty = BTreeMap::new();
1363 +
        let names: BTreeSet<&String> = before.remotes.keys().chain(after.remotes.keys()).collect();
1364 +
        for n in names {
1365 +
            let mut cur = out.remotes.get(n).cloned().unwrap_or_default();
1366 +
            back(
1367 +
                &mut cur,
1368 +
                before.remotes.get(n).unwrap_or(&empty),
1369 +
                after.remotes.get(n).unwrap_or(&empty),
1370 +
                |k| format!("{n}/{k}"),
1371 +
                &mut clashes,
1372 +
            );
1373 +
            if cur.is_empty() {
1374 +
                out.remotes.remove(n);
1375 +
            } else {
1376 +
                out.remotes.insert(n.clone(), cur);
1377 +
            }
1378 +
        }
1379 +
        if before.policy != after.policy {
1380 +
            if out.policy == after.policy {
1381 +
                out.policy = before.policy;
1382 +
            } else {
1383 +
                clashes.push("the policy".into());
1384 +
            }
1385 +
        }
1386 +
        if !clashes.is_empty() {
1387 +
            return fail(format!(
1388 +
                "undo would also take back what another workspace has done since to {}; \
1389 +
                 see `oxy op log`, and restore a whole earlier state with `oxy op restore OP` if that is what you want",
1390 +
                clashes.join(", ")
1391 +
            ));
1392 +
        }
1393 +
        out.workspaces = theirs;
1394 +
        match before.workspaces.get(&mine) {
1395 +
            Some(wc) => out.workspaces.insert(mine, *wc),
1396 +
            None => out.workspaces.remove(&mine),
1397 +
        };
1398 +
        Ok(out)
1399 +
    }
1400 +
1401 +
    /// Bookmarks whose heads, added between `before` and `after`, a remote
1402 +
    /// in `now` already holds: undoing them locally leaves them there.
1403 +
    fn pushed_by(&self, before: &View, after: &View, now: &View) -> Result<Vec<(String, String)>> {
1404 +
        let mut out = Vec::new();
1405 +
        for (name, st) in &after.bookmarks {
1406 +
            if before.bookmarks.get(name) == Some(st) {
1407 +
                continue;
1408 +
            }
1409 +
            let had: BTreeSet<ChangeHash> = match before.bookmarks.get(name) {
1410 +
                Some(b) => self.store.get_state(b)?.heads.into_iter().collect(),
1411 +
                None => BTreeSet::new(),
1412 +
            };
1413 +
            let added: Vec<ChangeHash> = self
1414 +
                .store
1415 +
                .get_state(st)?
1416 +
                .heads
1417 +
                .into_iter()
1418 +
                .filter(|h| !had.contains(h))
1419 +
                .collect();
1420 +
            if added.is_empty() {
1421 +
                continue;
1422 +
            }
1423 +
            for (remote, bookmarks) in &now.remotes {
1424 +
                if let Some(rs) = bookmarks.get(name) {
1425 +
                    let theirs = self.store.get_state(rs)?.heads;
1426 +
                    if added.iter().any(|h| theirs.contains(h)) {
1427 +
                        out.push((name.clone(), remote.clone()));
1428 +
                    }
1429 +
                }
1430 +
            }
1431 +
        }
1432 +
        Ok(out)
1433 +
    }
1434 +
939 1435
    /// The last operation that was not a snapshot.
940 1436
    pub fn last_command(&self) -> Result<Op> {
941 1437
        let (_, mut op) = self.head_op()?;
…
29 unchanged lines
971 1467
972 1468
    /// Build these file graphs in a history-only state.
973 1469
    pub fn load_graphs_of(&self, state: &mut State, nodes: &[crate::ids::NodeId]) -> Result<()> {
974 -
        Ok(state.load_graphs(
975 -
            nodes,
976 -
            |refs| {
977 -
                self.read_blobs(refs).map_err(|e| {
978 -
                    crate::objects::ObjectError::Schema(format!("graph and text blobs: {e}"))
979 -
                })
980 -
            },
981 -
            |h| self.change_bytes(h),
982 -
        )?)
1470 +
        state
1471 +
            .load_graphs(
1472 +
                nodes,
1473 +
                |refs| {
1474 +
                    self.read_blobs(refs).map_err(|e| {
1475 +
                        crate::objects::ObjectError::Schema(format!("graph and text blobs: {e}"))
1476 +
                    })
1477 +
                },
1478 +
                |h| self.change_bytes(h),
1479 +
            )
1480 +
            .map_err(|e| self.index_damage(e))
1481 +
    }
1482 +
1483 +
    /// The tree of `state` as every reader sees it ([`State::view`]), with
1484 +
    /// the graphs built that it compares: same-path additions with
1485 +
    /// identical content are one entry, not a name conflict.
1486 +
    pub fn tree_view(&self, state: &mut State) -> Result<crate::tree::TreeView> {
1487 +
        let need = state.unify_candidates();
1488 +
        if !need.is_empty() {
1489 +
            self.load_graphs_of(state, &need)?;
1490 +
        }
1491 +
        Ok(state.view())
1492 +
    }
1493 +
1494 +
    /// An error building graphs, and when it is a state index found
1495 +
    /// damaged (a file's history that does not match its hash), the
1496 +
    /// workspace's state indexes removed: they are rebuilt from history by
1497 +
    /// the next command.
1498 +
    fn index_damage(&self, e: crate::objects::ObjectError) -> RepoError {
1499 +
        let crate::objects::ObjectError::DamagedIndex(why) = &e else {
1500 +
            return e.into();
1501 +
        };
1502 +
        for path in self.state_indexes() {
1503 +
            let _ = fs::remove_file(path);
1504 +
        }
1505 +
        RepoError::Msg(format!(
1506 +
            "this workspace's state index was damaged ({why}) and has been removed; run the command again to rebuild it"
1507 +
        ))
983 1508
    }
984 1509
985 1510
    /// The graphs of `nodes` of a history-only state, built beside it (see
…
12 unchanged lines
998 1523
                    })
999 1524
                },
1000 1525
                |h| self.change_bytes(h),
1001 -
            )?
1526 +
            )
1527 +
            .map_err(|e| self.index_damage(e))?
1002 1528
            .into_iter()
1003 1529
            .collect())
1004 1530
    }
…
34 unchanged lines
1039 1565
1040 1566
    /// Build every graph a history-only state still lacks, making it full.
1041 1567
    pub fn load_all_graphs(&self, state: &mut State) -> Result<()> {
1042 -
        Ok(state.load_all_graphs(
1043 -
            |refs| {
1044 -
                self.read_blobs(refs).map_err(|e| {
1045 -
                    crate::objects::ObjectError::Schema(format!("graph and text blobs: {e}"))
1046 -
                })
1047 -
            },
1048 -
            |h| self.change_bytes(h),
1049 -
        )?)
1568 +
        state
1569 +
            .load_all_graphs(
1570 +
                |refs| {
1571 +
                    self.read_blobs(refs).map_err(|e| {
1572 +
                        crate::objects::ObjectError::Schema(format!("graph and text blobs: {e}"))
1573 +
                    })
1574 +
                },
1575 +
                |h| self.change_bytes(h),
1576 +
            )
1577 +
            .map_err(|e| self.index_damage(e))
1050 1578
    }
1051 1579
1052 1580
    /// A change's encoding, for building graphs of its inline inserts. The
…
119 unchanged lines
1172 1700
    /// through [`Repo::read_blob`], which may fetch it lazily.
1173 1701
    pub fn read_blobs(&self, refs: &[ObjectRef]) -> Result<Vec<Vec<u8>>> {
1174 1702
        let mut blobs: Vec<Option<Blob>> = Vec::with_capacity(refs.len());
1175 -
        for (r, got) in refs.iter().zip(self.store.get_many(refs)) {
1703 +
        let t = crate::timing::phase("blobs: get_many blobs");
1704 +
        let got_all = self.store.get_many(refs);
1705 +
        drop(t);
1706 +
        for (r, got) in refs.iter().zip(got_all) {
1176 1707
            blobs.push(match got {
1177 1708
                Ok((ObjectType::Blob, body)) => Some(Blob::decode(&body)?),
1178 1709
                Ok((ty, _)) => return fail(format!("{} is a {ty:?}, not a blob", r.short(12))),
…
6 unchanged lines
1185 1716
            .flatten()
1186 1717
            .flat_map(|b| b.chunks.iter().copied())
1187 1718
            .collect();
1719 +
        let t = crate::timing::phase("blobs: get_many chunks");
1188 1720
        let mut chunks = self.store.get_many(&chunk_refs).into_iter();
1721 +
        drop(t);
1722 +
        if crate::timing::enabled() {
1723 +
            eprintln!(
1724 +
                "timing blobs: {} blobs, {} chunks, {} one by one",
1725 +
                refs.len(),
1726 +
                chunk_refs.len(),
1727 +
                blobs.iter().filter(|b| b.is_none()).count()
1728 +
            );
1729 +
        }
1189 1730
        let mut out = Vec::with_capacity(refs.len());
1190 1731
        for (r, blob) in refs.iter().zip(blobs) {
1191 1732
            match blob {
…
51 unchanged lines
1243 1784
        self.untracked_nested.borrow().clone()
1244 1785
    }
1245 1786
1787 +
    /// New untracked files the last snapshot left out for being larger than
1788 +
    /// `snapshot.max-new-file-size`, with their sizes (oxygen#98).
1789 +
    pub fn skipped_large(&self) -> Vec<(String, u64)> {
1790 +
        self.skipped_large.borrow().clone()
1791 +
    }
1792 +
1246 1793
    /// Attributes from a materialized state's `.oxyattributes` files.
1247 1794
    pub fn state_attributes(snapshot: &Snapshot) -> crate::attributes::Attributes {
1248 1795
        crate::attributes::Attributes::from_files(snapshot.files.iter().filter_map(|(p, f)| {
…
67 unchanged lines
1316 1863
            let mut have = st.maximal(deps);
1317 1864
            have.sort();
1318 1865
            if deps.iter().all(|d| st.contains(d)) && have == heads {
1866 +
                // Re-encoding reads every history: one found damaged is
1867 +
                // left to the build that reads it, which reports it.
1319 1868
                #[cfg(test)]
1320 -
                assert_eq!(
1321 -
                    st.write_index(),
1322 -
                    self.load_state_history(deps)?.write_index(),
1323 -
                    "a state index holds the state history gives"
1324 -
                );
1869 +
                if let Some(mine) = st.write_index() {
1870 +
                    assert_eq!(
1871 +
                        Some(mine),
1872 +
                        self.load_state_history(deps)?.write_index(),
1873 +
                        "a state index holds the state history gives"
1874 +
                    );
1875 +
                }
1325 1876
                return Ok(st);
1326 1877
            }
1327 1878
            // An index a few changes behind (a commit, a pull since): add
…
49 unchanged lines
1377 1928
    /// The index of `deps` if there is one, else the newest, which the
1378 1929
    /// caller may bring up to date.
1379 1930
    fn read_state_index(&self, deps: &[ChangeHash]) -> Option<(State, Vec<ChangeHash>)> {
1931 +
        use std::io::Read;
1380 1932
        let _t = crate::timing::phase("load: state index");
1381 1933
        let exact = self.state_index_path(deps);
1382 -
        let bytes = match fs::read(&exact) {
1383 -
            Ok(b) => {
1934 +
        let reading = crate::timing::phase("load:   of which reading the file");
1935 +
        let mut f = match fs::File::open(&exact) {
1936 +
            Ok(f) => {
1384 1937
                // Used now: the newest, so the last to be dropped.
1385 1938
                let _ = fs::File::options()
1386 1939
                    .write(true)
1387 1940
                    .open(&exact)
1388 1941
                    .and_then(|f| f.set_modified(std::time::SystemTime::now()));
1389 -
                b
1942 +
                f
1390 1943
            }
1391 -
            Err(_) => fs::read(self.state_indexes().first()?).ok()?,
1944 +
            Err(_) => fs::File::open(self.state_indexes().first()?).ok()?,
1392 1945
        };
1393 -
        let mut r = crate::flat::Reader::new(&bytes);
1394 -
        let mut heads = Vec::new();
1395 -
        for _ in 0..r.count().ok()? {
1396 -
            heads.push(r.object().ok()?);
1946 +
        let total = f.metadata().ok()?.len();
1947 +
        let mut word = [0u8; 4];
1948 +
        f.read_exact(&mut word).ok()?;
1949 +
        let n = u32::from_le_bytes(word) as usize;
1950 +
        let mut head = vec![0u8; 32 * n + 64 + 8];
1951 +
        f.read_exact(&mut head).ok()?;
1952 +
        let heads: Vec<ChangeHash> = head[..32 * n]
1953 +
            .chunks_exact(32)
1954 +
            .map(|c| ObjectRef(c.try_into().unwrap()))
1955 +
            .collect();
1956 +
        let (changes_sum, rest_sum) = (&head[32 * n..32 * n + 32], &head[32 * n + 32..32 * n + 64]);
1957 +
        let front_len = u64::from_le_bytes(head[32 * n + 64..].try_into().unwrap());
1958 +
        let at = (4 + head.len()) as u64;
1959 +
        if at.checked_add(front_len).is_none_or(|end| end > total) {
1960 +
            return None;
1397 1961
        }
1398 -
        let (changes_sum, rest_sum) = (r.object().ok()?, r.object().ok()?);
1399 -
        let body = &bytes[4 + 32 * heads.len() + 64..];
1400 -
        let split = 4 + r.count().ok()?;
1401 -
        // Checked whole, so a part decoded later (the tree's maps, a
1402 -
        // file's history) can trust its bytes.
1403 -
        let (changes, rest) = (body.get(..split)?, body.get(split..)?);
1404 -
        let mut h = blake3::Hasher::new();
1405 -
        h.update_rayon(rest);
1406 -
        if *blake3::hash(changes).as_bytes() != changes_sum.0
1407 -
            || *h.finalize().as_bytes() != rest_sum.0
1408 -
        {
1962 +
        // The front only: the files' histories after it are read, each
1963 +
        // checked by its own hash, when a command builds that file.
1964 +
        let mut front = vec![0u8; front_len as usize];
1965 +
        f.read_exact(&mut front).ok()?;
1966 +
        drop(reading);
1967 +
        let checking = crate::timing::phase("load:   of which checking the file");
1968 +
        let split = 4 + u32::from_le_bytes(front.get(..4)?.try_into().unwrap()) as usize;
1969 +
        let (changes, rest) = (front.get(..split)?, front.get(split..)?);
1970 +
        let sum = |b: &[u8]| *blake3::Hasher::new().update_rayon(b).finalize().as_bytes();
1971 +
        if sum(changes) != changes_sum || sum(rest) != rest_sum {
1409 1972
            return None;
1410 1973
        }
1411 -
        Some((State::read_index(body).ok()?, heads))
1974 +
        drop(checking);
1975 +
        let histories = crate::state::Histories::File(
1976 +
            std::sync::Arc::new(f),
1977 +
            at + front_len,
1978 +
            total - at - front_len,
1979 +
        );
1980 +
        Some((State::read_index_front(&front, histories).ok()?, heads))
1412 1981
    }
1413 1982
1414 1983
    /// The changes of this workspace's state index (the closure of its
1415 1984
    /// heads), read without the rest of the index.
1985 +
    #[allow(clippy::type_complexity)]
1416 1986
    pub(crate) fn read_state_index_changes(
1417 1987
        &self,
1418 -
    ) -> Option<std::collections::HashMap<ChangeHash, std::sync::Arc<crate::state::ChangeMeta>>>
1419 -
    {
1988 +
    ) -> Option<(Vec<ChangeHash>, Vec<(ChangeHash, crate::state::ChangeMeta)>)> {
1420 1989
        use std::io::Read;
1421 1990
        let _t = crate::timing::phase("load: state index changes");
1422 1991
        let mut f = fs::File::open(self.state_indexes().first()?).ok()?;
1423 1992
        let mut word = [0u8; 4];
1424 1993
        f.read_exact(&mut word).ok()?;
1425 1994
        let n = u32::from_le_bytes(word) as usize;
1426 -
        let mut head = vec![0u8; 32 * n + 64 + 4];
1995 +
        // Heads, the two sums, the front's length, then the changes block.
1996 +
        let mut head = vec![0u8; 32 * n + 64 + 8 + 4];
1427 1997
        f.read_exact(&mut head).ok()?;
1998 +
        let heads: Vec<ChangeHash> = head[..32 * n]
1999 +
            .chunks_exact(32)
2000 +
            .map(|c| ObjectRef(c.try_into().unwrap()))
2001 +
            .collect();
1428 2002
        let sum = &head[32 * n..32 * n + 32];
1429 -
        let len = u32::from_le_bytes(head[32 * n + 64..].try_into().unwrap()) as usize;
2003 +
        let len = u32::from_le_bytes(head[32 * n + 72..].try_into().unwrap()) as usize;
1430 2004
        let mut changes = vec![0u8; 4 + len];
1431 -
        changes[..4].copy_from_slice(&head[32 * n + 64..]);
2005 +
        changes[..4].copy_from_slice(&head[32 * n + 72..]);
1432 2006
        f.read_exact(&mut changes[4..]).ok()?;
1433 -
        if blake3::hash(&changes).as_bytes() != sum {
2007 +
        let checking = crate::timing::phase("load:   of which checking");
2008 +
        let mut h = blake3::Hasher::new();
2009 +
        h.update_rayon(&changes);
2010 +
        if h.finalize().as_bytes() != sum {
1434 2011
            return None;
1435 2012
        }
1436 -
        State::read_index_changes(&changes[4..]).ok()
2013 +
        drop(checking);
2014 +
        let _decoding = crate::timing::phase("load:   of which decoding");
2015 +
        Some((heads, State::read_index_changes_listed(&changes[4..]).ok()?))
1437 2016
    }
1438 2017
1439 2018
    /// Save `st`, the history-only state of `deps`, as this workspace's
…
18 unchanged lines
1458 2037
        for h in &heads {
1459 2038
            w.object(h);
1460 2039
        }
1461 -
        let split = 4 + u32::from_le_bytes(body[..4].try_into().unwrap()) as usize;
1462 -
        w.fixed(blake3::hash(&body[..split]).as_bytes());
1463 -
        let mut sum = blake3::Hasher::new();
1464 -
        sum.update_rayon(&body[split..]);
1465 -
        w.fixed(sum.finalize().as_bytes());
2040 +
        // The body is the front's length, the front, then the files'
2041 +
        // histories, which carry their own hashes: the sums cover the
2042 +
        // front's changes block and the rest of the front.
2043 +
        let front_len = u64::from_le_bytes(body[..8].try_into().unwrap()) as usize;
2044 +
        let front = &body[8..8 + front_len];
2045 +
        let split = 4 + u32::from_le_bytes(front[..4].try_into().unwrap()) as usize;
2046 +
        let sum = |b: &[u8]| *blake3::Hasher::new().update_rayon(b).finalize().as_bytes();
2047 +
        w.fixed(&sum(&front[..split]));
2048 +
        w.fixed(&sum(&front[split..]));
1466 2049
        w.fixed(&body);
1467 2050
        let path = self.state_index_path(&heads);
1468 2051
        let _ = fs::create_dir_all(self.oxy.join("index"));
…
99 unchanged lines
1568 2151
            // written after it was snapshotted leaves it out again).
1569 2152
            None => walk::list_knowing(&self.root, &in_scope, &|p: &str| tracked.contains_key(p))?,
1570 2153
        };
2154 +
        // A new file larger than the limit is left out and named, not taken
2155 +
        // in unseen (oxygen#98): a build's output or a downloaded image
2156 +
        // would otherwise be stored by the next status, kept by the
2157 +
        // operation log. Files tracked, or already in the working copy, stay.
2158 +
        let limit = self.config.max_new_file;
2159 +
        let mut skipped = Vec::new();
2160 +
        let files: Vec<_> = files
2161 +
            .into_iter()
2162 +
            .filter(|(rel, _, meta)| {
2163 +
                let new = !tracked.contains_key(rel)
2164 +
                    && !cache.is_some_and(|m| m.entries.contains_key(rel));
2165 +
                if limit > 0 && new && meta.size > limit {
2166 +
                    skipped.push((rel.clone(), meta.size));
2167 +
                    return false;
2168 +
                }
2169 +
                true
2170 +
            })
2171 +
            .collect();
2172 +
        *self.skipped_large.borrow_mut() = skipped;
1571 2173
        let keys = keys.filter(|k| {
1572 2174
            let on_disk: HashSet<&str> = files.iter().map(|(r, _, _)| r.as_str()).collect();
1573 2175
            let attrs_unchanged =
…
51 unchanged lines
1625 2227
        // already tracked as subrepo nodes, they contribute their pin;
1626 2228
        // unregistered ones are left alone.
1627 2229
        let modules = self.modules_on_disk();
1628 -
        let removed = self.removed_subrepos();
1629 -
        for rel in &removed {
1630 -
            if tracked.get(rel).map(|(k, _)| *k) != Some(NodeKind::Subrepo) {
2230 +
        // A subrepo at the state its recorded pin names is unmodified,
2231 +
        // whatever bookmark it sits on (#147): the recorded pin stands.
2232 +
        let subrepo_entry = |rel: &str, pin: crate::subrepo::Pin| -> FsEntry {
2233 +
            if let Some((NodeKind::Subrepo, Some(b))) = tracked.get(rel) {
2234 +
                let recorded = self
2235 +
                    .read_blob(b)
2236 +
                    .ok()
2237 +
                    .and_then(|x| crate::subrepo::Pin::decode(&x).ok());
2238 +
                if recorded.is_some_and(|r| r.same_state(&pin)) {
2239 +
                    return FsEntry {
2240 +
                        blob: Some(*b),
2241 +
                        ..FsEntry::new(NodeKind::Subrepo, false, vec![])
2242 +
                    };
2243 +
                }
2244 +
            }
2245 +
            FsEntry::new(NodeKind::Subrepo, false, pin.encode())
2246 +
        };
2247 +
        // A removal's marker is spent once the removal is recorded (no
2248 +
        // subrepo node is tracked there) or the path is registered again:
2249 +
        // dropped before it is read, or the change registering it again
2250 +
        // would leave it out (oxygen#152).
2251 +
        let mut removed = self.removed_subrepos();
2252 +
        removed.retain(|rel| {
2253 +
            let spent = tracked.get(rel).map(|(k, _)| *k) != Some(NodeKind::Subrepo)
2254 +
                || modules.entries.contains_key(rel);
2255 +
            if spent {
1631 2256
                self.clear_removed_subrepo(rel);
1632 2257
            }
1633 -
        }
2258 +
            !spent
2259 +
        });
1634 2260
        let mut untracked_nested = Vec::new();
1635 2261
        for dir in nested {
1636 2262
            let rel = self.rel_path(&dir);
…
7 unchanged lines
1644 2270
                continue;
1645 2271
            }
1646 2272
            let pin = subrepos::read_pin(&dir, registered.and_then(|m| m.bookmark.as_deref()))?;
1647 -
            out.insert(rel, FsEntry::new(NodeKind::Subrepo, false, pin.encode()));
2273 +
            let entry = subrepo_entry(&rel, pin);
2274 +
            out.insert(rel, entry);
1648 2275
        }
1649 2276
        untracked_nested.sort();
1650 2277
        *self.untracked_nested.borrow_mut() = untracked_nested;
…
14 unchanged lines
1665 2292
                        &path,
1666 2293
                        modules.entries.get(rel).and_then(|m| m.bookmark.as_deref()),
1667 2294
                    )?;
1668 -
                    out.insert(
1669 -
                        rel.clone(),
1670 -
                        FsEntry::new(NodeKind::Subrepo, false, pin.encode()),
1671 -
                    );
2295 +
                    out.insert(rel.clone(), subrepo_entry(rel, pin));
1672 2296
                } else if let Some(b) = blob {
1673 2297
                    // Not cloned (or emptied): the pin stays as recorded, as a
1674 2298
                    // Git gitlink does when its directory is absent.
…
82 unchanged lines
1757 2381
        if self.bare {
1758 2382
            return self.log_state(&heads);
1759 2383
        }
1760 -
        let (wc_hash, _) = self.working_copy()?;
2384 +
        let (wc_hash, wc) = self.working_copy()?;
1761 2385
        let mut all = heads.clone();
1762 2386
        all.push(wc_hash);
1763 2387
        let head_set: BTreeSet<ChangeHash> = heads.into_iter().collect();
…
6 unchanged lines
1770 2394
                .map(|(h, m)| (*h, m.clone()))
1771 2395
                .collect()
1772 2396
        } else {
1773 -
            self.metas_of(&all)?
2397 +
            // The working copy's change is in hand, and is never in the
2398 +
            // state index: its metadata is not looked up again.
2399 +
            let mut below: Vec<ChangeHash> = head_set.iter().copied().collect();
2400 +
            below.extend(wc.deps.iter().copied());
2401 +
            let _t = crate::timing::phase("log: metadata");
2402 +
            let mut metas = self.metas_of(&below)?;
2403 +
            if !metas.iter().any(|(h, _)| *h == wc_hash) {
2404 +
                // First, where the log puts it, so the order may hold.
2405 +
                metas.insert(0, (wc_hash, crate::state::ChangeMeta::of(&wc)));
2406 +
            }
2407 +
            metas
1774 2408
        };
1775 -
        for (h, meta) in metas {
1776 -
            entries.push(LogEntry {
1777 -
                hash: h,
1778 -
                change: meta,
1779 -
                is_working_copy: h == wc_hash,
1780 -
                is_head: head_set.contains(&h),
1781 -
            });
1782 -
        }
1783 -
        entries.sort_by(|a, b| {
2409 +
        entries.extend(metas.into_iter().map(|(h, meta)| LogEntry {
2410 +
            hash: h,
2411 +
            change: meta,
2412 +
            is_working_copy: h == wc_hash,
2413 +
            is_head: head_set.contains(&h),
2414 +
        }));
2415 +
        let _sorting = crate::timing::phase("log: sort");
2416 +
        // The hash makes the order total, so an unstable sort gives the same.
2417 +
        // From the state index the order already holds (oxygen#94).
2418 +
        let order = |a: &LogEntry, b: &LogEntry| {
1784 2419
            b.is_working_copy
1785 2420
                .cmp(&a.is_working_copy)
1786 -
                .then(b.change.hlc.cmp(&a.change.hlc))
1787 -
                .then(b.hash.cmp(&a.hash))
1788 -
        });
2421 +
                .then(crate::state::log_order(
2422 +
                    (&a.hash, &a.change.hlc),
2423 +
                    (&b.hash, &b.change.hlc),
2424 +
                ))
2425 +
        };
2426 +
        if !entries.is_sorted_by(|a, b| order(a, b).is_le()) {
2427 +
            entries.sort_unstable_by(order);
2428 +
        }
1789 2429
        Ok(entries)
1790 2430
    }
1791 2431
…
31 unchanged lines
1823 2463
                is_working_copy: false,
1824 2464
            })
1825 2465
            .collect();
1826 -
        entries.sort_by(|a, b| b.change.hlc.cmp(&a.change.hlc).then(b.hash.cmp(&a.hash)));
2466 +
        let order = |a: &LogEntry, b: &LogEntry| {
2467 +
            crate::state::log_order((&a.hash, &a.change.hlc), (&b.hash, &b.change.hlc))
2468 +
        };
2469 +
        if !entries.is_sorted_by(|a, b| order(a, b).is_le()) {
2470 +
            entries.sort_unstable_by(order);
2471 +
        }
1827 2472
        Ok(entries)
1828 2473
    }
1829 2474
…
8 unchanged lines
1838 2483
        // The state index holds the metadata of its heads' closure: a
1839 2484
        // change found there is not read from the store (a change object
1840 2485
        // can be as large as the tree it adds).
1841 -
        let indexed = self.read_state_index_changes();
2486 +
        let read = self.read_state_index_changes();
2487 +
        // Every one of `heads` in the index, and every head of the index
2488 +
        // among them: their closure is the index's changes, all of them
2489 +
        // (it is closed), and no walk is needed to find it.
2490 +
        if let Some((index_heads, all)) = &read {
2491 +
            let asked: crate::fasthash::FastSet<&ChangeHash> = heads.iter().collect();
2492 +
            let in_index = || {
2493 +
                let index_heads: crate::fasthash::FastSet<&ChangeHash> =
2494 +
                    index_heads.iter().collect();
2495 +
                // Usually the heads asked are the index's: no set of all.
2496 +
                heads.iter().all(|h| index_heads.contains(h)) || {
2497 +
                    let all: crate::fasthash::FastSet<&ChangeHash> =
2498 +
                        all.iter().map(|(h, _)| h).collect();
2499 +
                    heads.iter().all(|h| all.contains(h))
2500 +
                }
2501 +
            };
2502 +
            if index_heads.iter().all(|h| asked.contains(h)) && in_index() {
2503 +
                let (_, all) = read.expect("matched above");
2504 +
                #[cfg(test)]
2505 +
                {
2506 +
                    // The walk the shortcut saves finds the same changes.
2507 +
                    let mut seen: HashSet<ChangeHash> = HashSet::new();
2508 +
                    let mut stack = heads.to_vec();
2509 +
                    while let Some(h) = stack.pop() {
2510 +
                        if seen.insert(h) {
2511 +
                            stack.extend(self.store.get_change_meta(&h)?.deps);
2512 +
                        }
2513 +
                    }
2514 +
                    assert_eq!(seen, all.iter().map(|(h, _)| *h).collect::<HashSet<_>>());
2515 +
                }
2516 +
                return Ok(all);
2517 +
            }
2518 +
        }
2519 +
        let mut indexed: Option<crate::fasthash::FastMap<ChangeHash, crate::state::ChangeMeta>> =
2520 +
            read.map(|(_, all)| all.into_iter().collect());
2521 +
        let _walk = crate::timing::phase("metadata: walk");
1842 2522
        let (mut stored, mut from_store) = (0usize, std::time::Duration::ZERO);
1843 -
        let mut seen: HashSet<ChangeHash> = HashSet::new();
2523 +
        let mut seen: crate::fasthash::FastSet<ChangeHash> = Default::default();
1844 2524
        let mut stack: Vec<ChangeHash> = heads.to_vec();
1845 -
        let mut out = Vec::new();
2525 +
        let mut out = Vec::with_capacity(indexed.as_ref().map_or(0, |m| m.len()));
1846 2526
        while let Some(h) = stack.pop() {
1847 2527
            if !seen.insert(h) {
1848 2528
                continue;
1849 2529
            }
1850 -
            if let Some(meta) = indexed.as_ref().and_then(|m| m.get(&h)) {
2530 +
            // Taken out of the index's map, each once: not copied.
2531 +
            if let Some(meta) = indexed.as_mut().and_then(|m| m.remove(&h)) {
1851 2532
                #[cfg(test)]
1852 2533
                assert_eq!(
1853 -
                    **meta,
2534 +
                    meta,
1854 2535
                    self.store.get_change_meta(&h)?,
1855 2536
                    "the state index holds a change's metadata"
1856 2537
                );
1857 2538
                stack.extend(meta.deps.iter().copied());
1858 -
                out.push((h, (**meta).clone()));
2539 +
                out.push((h, meta));
1859 2540
                continue;
1860 2541
            }
1861 2542
            let started = std::time::Instant::now();
…
31 unchanged lines
1893 2574
            ));
1894 2575
        }
1895 2576
        let mut state = self.load_state_history(heads)?;
1896 -
        let Some(node) = state.tree().view().entries.get(path).map(|e| e.node) else {
2577 +
        let Some(node) = self
2578 +
            .tree_view(&mut state)?
2579 +
            .entries
2580 +
            .get(path)
2581 +
            .map(|e| e.node)
2582 +
        else {
1897 2583
            return fail(format!("{path} is not in that state"));
1898 2584
        };
1899 2585
        self.load_graphs_of(&mut state, &[node])?;
…
27 unchanged lines
1927 2613
                is_head: head_set.contains(h),
1928 2614
            })
1929 2615
            .collect();
1930 -
        entries.sort_by(|a, b| b.change.hlc.cmp(&a.change.hlc).then(b.hash.cmp(&a.hash)));
2616 +
        entries.sort_unstable_by(|a, b| b.change.hlc.cmp(&a.change.hlc).then(b.hash.cmp(&a.hash)));
1931 2617
        entries
1932 2618
    }
1933 2619
…
38 unchanged lines
1972 2658
            return fail("empty change specifier");
1973 2659
        }
1974 2660
        let entries = self.log()?;
1975 -
        let matches: Vec<&LogEntry> = if let Some(hp) = spec.strip_prefix('#') {
2661 +
        // A full hash as `--json` prints it, without its `#`: 64 hex digits
2662 +
        // cannot be a ChangeId, which has 32 (oxygen#103).
2663 +
        let full_hash =
2664 +
            (spec.len() == 64 && spec.bytes().all(|b| b.is_ascii_hexdigit())).then_some(spec);
2665 +
        let matches: Vec<&LogEntry> = if let Some(hp) = spec.strip_prefix('#').or(full_hash) {
1976 2666
            entries
1977 2667
                .iter()
1978 2668
                .filter(|e| e.hash.starts_with_hex(hp))
…
97 unchanged lines
2076 2766
        let nodes: Vec<NodeId> = edited.into_iter().collect();
2077 2767
        self.load_graphs_of(&mut base, &nodes)?;
2078 2768
        let snapshot = Snapshot {
2079 -
            tree: base.tree().view(),
2769 +
            tree: self.tree_view(&mut base)?,
2080 2770
            files: BTreeMap::new(),
2081 2771
            unrendered: BTreeSet::new(),
2082 2772
        };
…
12 unchanged lines
2095 2785
            change.edits.iter().map(|e| e.node()).collect();
2096 2786
        let nodes: Vec<NodeId> = edited.iter().copied().collect();
2097 2787
        self.load_graphs_of(&mut base, &nodes)?;
2788 +
        self.tree_view(&mut base)?;
2098 2789
        let snapshot = {
2099 2790
            let _t = crate::timing::phase("diff:   of which materialize");
2100 2791
            base.materialize_nodes(&edited)
…
157 unchanged lines
modified crates/oxy-core/src/repo/ops.rs +29 −5
…
36 unchanged lines
37 37
    },
38 38
}
39 39
40 +
/// What `undo` did (oxygen#99).
41 +
#[derive(Debug)]
42 +
pub struct Undone {
43 +
    /// The command undone.
44 +
    pub command: String,
45 +
    /// What that command had changed, now put back.
46 +
    pub changes: Vec<ViewChange>,
47 +
    /// Newer operations from other workspaces, left as they are.
48 +
    pub others_kept: usize,
49 +
    /// `(bookmark, remote)`: changes the command added to a bookmark that
50 +
    /// the remote already holds; undoing here does not take them back there.
51 +
    pub pushed: Vec<(String, String)>,
52 +
}
53 +
40 54
/// What `op restore` did.
41 55
#[derive(Debug)]
42 56
pub struct RestoreReport {
…
137 unchanged lines
180 194
        let mut view = head.view.clone();
181 195
        view.bookmarks = target.view.bookmarks.clone();
182 196
        self.record_op(&[head_ref], &command, view)?;
183 -
        let base = match target.view.workspaces.get(&self.workspace) {
184 -
            Some(w) => {
185 -
                let wc = self.store.get_change(w)?;
186 -
                self.load_state_history(&wc.deps)?.maximal(&wc.deps)
197 +
        // A working copy past `core.snapshot-retention` may be gone: then
198 +
        // the bookmark as it was.
199 +
        let base = match target
200 +
            .view
201 +
            .workspaces
202 +
            .get(&self.workspace)
203 +
            .map(|w| self.store.get_change(w))
204 +
        {
205 +
            Some(Ok(wc)) => self.load_state_history(&wc.deps)?.maximal(&wc.deps),
206 +
            Some(Err(StoreError::NotFound(_))) | None => {
207 +
                match target.view.bookmarks.get(&self.config.bookmark) {
208 +
                    Some(st) => self.store.get_state(st)?.heads,
209 +
                    None => self.bookmark_heads()?,
210 +
                }
187 211
            }
188 -
            None => self.bookmark_heads()?,
212 +
            Some(Err(e)) => return Err(e.into()),
189 213
        };
190 214
        let checkout = self.checkout(&base, discard, &format!("{command} (checkout)"))?;
191 215
        Ok(RestoreReport {
…
85 unchanged lines
modified crates/oxy-core/src/repo/policy.rs +57 −2
…
335 unchanged lines
336 336
                threshold: 1,
337 337
                require: Require::Any,
338 338
                allow_conflicts: true,
339 +
                subjects: vec![],
339 340
            }),
340 341
        }
341 342
        if p.identity(&key).is_none() {
…
2 unchanged lines
344 345
        self.sign_policy(p, &format!("oxy policy grant {}", key.short(12)))
345 346
    }
346 347
348 +
    /// Let holders of certificates `issuer` gives, for subjects matching
349 +
    /// `subject`, sign within `scope` on `pattern` (default every
350 +
    /// bookmark; §14.7). The issuer's identity is embedded so verifiers need
351 +
    /// no fetch.
352 +
    pub fn policy_grant_subject(
353 +
        &self,
354 +
        subject: &str,
355 +
        issuer: Identity,
356 +
        scope: crate::identity::Scope,
357 +
        pattern: Option<&str>,
358 +
    ) -> Result<PolicyUpdate> {
359 +
        if subject.trim().is_empty() {
360 +
            return fail("a subject pattern is needed, such as repo:OWNER/NAME:bookmark:main");
361 +
        }
362 +
        let mut p = self.next_policy()?;
363 +
        let key = issuer.hash();
364 +
        let pattern = pattern.unwrap_or("*");
365 +
        let grant = crate::policy::SubjectGrant {
366 +
            pattern: subject.into(),
367 +
            issuer: key,
368 +
            scope,
369 +
        };
370 +
        match p.rules.iter_mut().find(|r| r.pattern == pattern) {
371 +
            Some(rule) => {
372 +
                rule.subjects
373 +
                    .retain(|g| !(g.pattern == grant.pattern && g.issuer == grant.issuer));
374 +
                rule.subjects.push(grant);
375 +
            }
376 +
            None => p.rules.push(Rule {
377 +
                pattern: pattern.into(),
378 +
                keys: vec![],
379 +
                threshold: 1,
380 +
                require: Require::Any,
381 +
                allow_conflicts: true,
382 +
                subjects: vec![grant],
383 +
            }),
384 +
        }
385 +
        if p.identity(&key).is_none() {
386 +
            p.identities.push(issuer);
387 +
        }
388 +
        self.sign_policy(p, &format!("oxy policy grant-subject {subject}"))
389 +
    }
390 +
347 391
    /// Revoke `key`: it may no longer update anything, and existing rules
348 392
    /// drop it. A root key can be revoked by another root as long as the
349 393
    /// remaining roots still meet the threshold.
…
16 unchanged lines
366 410
        for r in &mut p.rules {
367 411
            r.keys.retain(|k| k != key);
368 412
        }
369 -
        p.rules.retain(|r| !r.keys.is_empty());
413 +
        // A rule of certified subjects alone stays.
414 +
        p.rules
415 +
            .retain(|r| !r.keys.is_empty() || !r.subjects.is_empty());
370 416
        if !p.revoked.contains(key) {
371 417
            p.revoked.push(*key);
372 418
            p.revoked.sort();
…
30 unchanged lines
403 449
                    .map(|c| c.author)
404 450
                    .map_err(|e| e.to_string())?;
405 451
                let principal = if author.is_zero() { key } else { author };
452 +
                // A change is checked when it is received: now is a time
453 +
                // its signer did not choose.
406 454
                let covered = rule
407 455
                    .keys
408 456
                    .iter()
409 -
                    .any(|k| policy.succeeds(k, &key) || policy.succeeds(k, &principal));
457 +
                    .any(|k| policy.succeeds(k, &key) || policy.succeeds(k, &principal))
458 +
                    || self.certified_authorized(
459 +
                        policy,
460 +
                        &key,
461 +
                        reference,
462 +
                        crate::identity::Scope::CHANGES,
463 +
                        crate::objects::Hlc::now().wall_ms,
464 +
                    );
410 465
                if !covered {
411 466
                    return Err(format!(
412 467
                        "change {} is signed by {}, which may not update {reference:?}",
…
48 unchanged lines
modified crates/oxy-core/src/repo/prompt.rs +66 −3
…
724 unchanged lines
725 725
                        md.len() == e.size
726 726
                            && fs::read(&path).is_ok_and(|b| *blake3::hash(&b).as_bytes() == e.hash)
727 727
                    }
728 +
                    // A subrepo: its pin is compared on its own (`subrepo_moved`),
729 +
                    // its directory's time says nothing (oxygen#75).
730 +
                    Ok(md) if md.is_dir() => {
731 +
                        path.join(OXY_DIR).exists() || path.join(".git").exists()
732 +
                    }
728 733
                    _ => false,
729 734
                };
730 735
                !same
…
117 unchanged lines
848 853
        self.disk_differs_listing(manifest, keep)
849 854
    }
850 855
856 +
    /// The manifest's entries that are subrepos (nested repositories it
857 +
    /// tracks): a directory, compared by its pin, not its stat.
858 +
    /// Known from the walk's nested repositories, and from `.oxymodules`:
859 +
    /// a commit's manifest does not list the nested repositories.
860 +
    fn subrepo_entries<'m>(&self, manifest: &'m Manifest) -> Vec<(&'m String, &'m ManifestEntry)> {
861 +
        let modules = self.modules_on_disk();
862 +
        let mut out: Vec<(&'m String, &'m ManifestEntry)> = manifest
863 +
            .nested
864 +
            .iter()
865 +
            .map(String::as_str)
866 +
            .chain(modules.entries.keys().map(String::as_str))
867 +
            .filter_map(|n| manifest.entries.get_key_value(n))
868 +
            .collect();
869 +
        out.sort_by(|a, b| a.0.cmp(b.0));
870 +
        out.dedup_by(|a, b| a.0 == b.0);
871 +
        out
872 +
    }
873 +
874 +
    /// Whether a subrepo is at another pin than the manifest recorded: its
875 +
    /// pin now (the nested repository's base, or Git's `HEAD`), encoded as
876 +
    /// a snapshot records it, against the recorded hash. Absent, it keeps
877 +
    /// its recorded pin, as a snapshot does; unreadable, it counts as moved.
878 +
    fn subrepo_moved(&self, rel: &str, me: &ManifestEntry) -> bool {
879 +
        let dir = self.root.join(rel);
880 +
        if !(dir.join(OXY_DIR).exists() || dir.join(".git").exists()) {
881 +
            return false;
882 +
        }
883 +
        let modules = self.modules_on_disk();
884 +
        let hint = modules.entries.get(rel).and_then(|m| m.bookmark.clone());
885 +
        match super::subrepos::read_pin(&dir, hint.as_deref()) {
886 +
            // A commit's manifest names the pin by its blob (its hash is of
887 +
            // no bytes); a snapshot's by the bytes' hash.
888 +
            Ok(pin) => match me.blob {
889 +
                Some(b) => Repo::blob_ref(&pin.encode()) != b,
890 +
                None => *blake3::hash(&pin.encode()).as_bytes() != me.hash,
891 +
            },
892 +
            Err(_) => true,
893 +
        }
894 +
    }
895 +
851 896
    /// [`Repo::disk_differs`] without listing the tree, when the manifest
852 897
    /// knows its directories: a directory's time moves when an entry is
853 898
    /// added, removed or renamed in it, so with every directory's time as
…
28 unchanged lines
882 927
        if !dirs_same {
883 928
            return None;
884 929
        }
885 -
        let rels: Vec<&String> = manifest.entries.keys().collect();
930 +
        // Subrepos are directories, compared by their pins below.
931 +
        let subrepos = self.subrepo_entries(manifest);
932 +
        let is_subrepo = |rel: &str| subrepos.iter().any(|(r, _)| r.as_str() == rel);
933 +
        let tracked: Vec<(&String, &ManifestEntry)> = manifest
934 +
            .entries
935 +
            .iter()
936 +
            .filter(|(rel, _)| !is_subrepo(rel))
937 +
            .collect();
938 +
        let rels: Vec<&String> = tracked.iter().map(|(r, _)| *r).collect();
886 939
        let now = stat_tracked(root, &rels);
887 940
        let differs_from = |me: &ManifestEntry, rel: &str, now: &Tracked| match now {
888 941
            Tracked::Link(t) => *blake3::hash(t).as_bytes() != me.hash,
…
11 unchanged lines
900 953
            return None;
901 954
        }
902 955
        // The entries in the order `rels` was taken in: no lookup by path.
903 -
        let entries: Vec<&ManifestEntry> = manifest.entries.values().collect();
956 +
        let entries: Vec<&ManifestEntry> = tracked.iter().map(|(_, me)| *me).collect();
904 957
        let differs = rels
905 958
            .par_iter()
906 959
            .zip(entries.par_iter())
907 960
            .zip(now.par_iter())
908 -
            .any(|((rel, me), n)| differs_from(me, rel, n.as_ref().unwrap()));
961 +
            .any(|((rel, me), n)| differs_from(me, rel, n.as_ref().unwrap()))
962 +
            || subrepos.iter().any(|(rel, me)| self.subrepo_moved(rel, me));
909 963
        if keep && (differs || racy) {
910 964
            let mut listing = super::walk::Listing {
911 965
                dirs: manifest.dirs.clone(),
…
88 unchanged lines
1000 1054
                })
1001 1055
                .collect();
1002 1056
            let root = &self.root;
1057 +
            // Subrepos are not listed as files: compared by their pins.
1058 +
            let subrepos = self.subrepo_entries(manifest);
1059 +
            if subrepos.iter().any(|(rel, me)| self.subrepo_moved(rel, me)) {
1060 +
                return Ok(true);
1061 +
            }
1062 +
            let skipped: Vec<_> = skipped
1063 +
                .into_iter()
1064 +
                .filter(|(rel, _)| !subrepos.iter().any(|(r, _)| r == rel))
1065 +
                .collect();
1003 1066
            Ok(skipped.par_iter().any(|(rel, me)| {
1004 1067
                let path = root.join(rel.as_str());
1005 1068
                let Ok(m) = fs::symlink_metadata(&path) else {
…
114 unchanged lines
modified crates/oxy-core/src/repo/receipts.rs +63 −1
…
26 unchanged lines
27 27
            return fail("that receipt is for another repository");
28 28
        }
29 29
        self.store.put_identity(&sr.runner)?;
30 +
        if let Some(c) = &sr.certificate {
31 +
            self.store.put_identity(&c.issuer)?;
32 +
            self.store.put_certificate(&c.cert)?;
33 +
        }
30 34
        self.store.put_receipt(&sr.receipt)?;
31 35
        let r = self.store.put_attest(&sr.attest)?;
32 36
        for sig in &sr.sigs {
…
15 unchanged lines
48 52
            return fail(format!("receipt #{} has no signature", r.short(12)));
49 53
        };
50 54
        let runner = self.store.get_identity(&key)?;
55 +
        // Signed by a short-lived key: its certificate and issuer too.
56 +
        let certificate = match self.store.certificate_of(&key)? {
57 +
            Some(c) => {
58 +
                let cert = self.store.get_certificate(&c)?;
59 +
                let issuer = self.store.get_identity(&cert.cert.issuer)?;
60 +
                Some(crate::receipt::CertifiedBy { cert, issuer })
61 +
            }
62 +
            None => None,
63 +
        };
51 64
        let signed = SignedReceipt {
52 65
            receipt,
53 66
            attest,
54 67
            sigs,
55 68
            runner,
69 +
            certificate,
56 70
        };
57 71
        let problem = signed.verify().err().map(|e| e.to_string());
58 72
        Ok(ReceiptInfo {
…
22 unchanged lines
81 95
82 96
    /// Receipts to keep through garbage collection: those whose state's
83 97
    /// heads `keep` holds, with their receipts, runners and outputs.
98 +
    /// Keep a signed approval (oxygenhub#235): verified, for this repository,
99 +
    /// with the signer's identity and indexed by the state it approves, so
100 +
    /// garbage collection keeps it while that state lives. Who may approve is
101 +
    /// the caller's rule.
102 +
    pub fn add_approval(&self, sa: &crate::approval::SignedApproval) -> Result<ObjectRef> {
103 +
        sa.verify()
104 +
            .map_err(|e| RepoError::Msg(format!("approval does not verify: {e}")))?;
105 +
        if sa.attest.repo != self.id()? {
106 +
            return fail("that approval is for another repository");
107 +
        }
108 +
        let state = sa
109 +
            .statement()
110 +
            .map_err(|e| RepoError::Msg(e.to_string()))?
111 +
            .state;
112 +
        self.store.put_identity(&sa.signer)?;
113 +
        let r = self.store.put_attest(&sa.attest)?;
114 +
        for sig in &sa.sigs {
115 +
            self.store.put_signature(&r, sig)?;
116 +
        }
117 +
        self.store.index_approval(&state, &r)?;
118 +
        Ok(r)
119 +
    }
120 +
121 +
    /// Approvals of states garbage collection keeps are kept, with their
122 +
    /// signatures and signers.
123 +
    pub(crate) fn reachable_approvals(&self, keep: &mut HashSet<ObjectRef>) -> Result<()> {
124 +
        for state in self.store.approval_states()? {
125 +
            if !keep.contains(&state) {
126 +
                let heads = match self.store.get_state(&state) {
127 +
                    Ok(s) => s.heads,
128 +
                    Err(StoreError::NotFound(_)) => vec![],
129 +
                    Err(e) => return Err(e.into()),
130 +
                };
131 +
                if !heads.iter().all(|h| keep.contains(h)) {
132 +
                    continue;
133 +
                }
134 +
                keep.insert(state);
135 +
            }
136 +
            for a in self.store.approvals_for(&state)? {
137 +
                keep.insert(a);
138 +
                for sig in self.store.signatures(&a)? {
139 +
                    keep.insert(sig.key);
140 +
                }
141 +
            }
142 +
        }
143 +
        Ok(())
144 +
    }
145 +
84 146
    pub(crate) fn reachable_receipts(&self, keep: &mut HashSet<ObjectRef>) -> Result<()> {
85 147
        for state in self.store.receipt_states()? {
86 148
            let heads = match self.store.get_state(&state) {
…
10 unchanged lines
97 159
            for a in self.store.receipts_for(&state)? {
98 160
                keep.insert(a);
99 161
                for sig in self.store.signatures(&a)? {
100 -
                    keep.insert(sig.key);
162 +
                    self.keep_signer(keep, sig.key)?;
101 163
                }
102 164
                let Ok(attest) = self.store.get_attest(&a) else {
103 165
                    continue;
…
20 unchanged lines
modified crates/oxy-core/src/repo/revset.rs +43 −18
…
267 unchanged lines
268 268
            .build()
269 269
            .map_err(|e| err(format!("touches({glob:?}): {e}")))?
270 270
            .compile_matcher();
271 -
        let heads = match self.working_copy() {
272 -
            Some(wc) => self
273 -
                .graph
274 -
                .get(&wc)
275 -
                .map(|m| m.deps.clone())
276 -
                .or_else(|| self.repo.working_copy().ok().map(|(_, c)| c.deps))
277 -
                .unwrap_or_default(),
278 -
            None => self.repo.bookmark_heads()?,
271 +
        // The tree alone names the nodes: no file graph is built (but for
272 +
        // same-path additions it compares). The working copy's base comes
273 +
        // from the workspace's state index.
274 +
        let mut state = match self.working_copy() {
275 +
            Some(wc) => {
276 +
                let deps = self
277 +
                    .graph
278 +
                    .get(&wc)
279 +
                    .map(|m| m.deps.clone())
280 +
                    .or_else(|| self.repo.working_copy().ok().map(|(_, c)| c.deps))
281 +
                    .unwrap_or_default();
282 +
                self.repo.load_state_indexed(&deps)?
283 +
            }
284 +
            None => self.repo.load_state_history(&self.repo.bookmark_heads()?)?,
279 285
        };
280 -
        let state = self.repo.load_state(&heads)?;
281 -
        let nodes: HashSet<NodeId> = state
282 -
            .tree()
283 -
            .view()
286 +
        let view = self.repo.tree_view(&mut state)?;
287 +
        let mut nodes: HashSet<NodeId> = view
284 288
            .entries
285 289
            .iter()
286 290
            .filter(|(path, _)| matcher.is_match(path))
287 291
            .map(|(_, e)| e.node)
288 292
            .collect();
293 +
        // A file shown as one of identical additions is all of them.
294 +
        for (dup, shown) in &view.unified {
295 +
            if nodes.contains(shown) {
296 +
                nodes.insert(*dup);
297 +
            }
298 +
        }
289 299
        if nodes.is_empty() {
290 300
            return Ok(Set::new());
291 301
        }
292 302
        let candidates = self.universe(within)?;
303 +
        let mut touched = self.repo.touched();
304 +
        touched.fill(self.repo, &candidates.iter().copied().collect::<Vec<_>>());
293 305
        let mut out = Set::new();
294 306
        for (i, h) in candidates.iter().enumerate() {
295 -
            let Ok(c) = self.repo.store().get_change(h) else {
296 -
                continue;
297 -
            };
298 -
            if c.tree_ops.iter().any(|o| nodes.contains(&o.node()))
299 -
                || c.edits.iter().any(|e| nodes.contains(&e.node()))
300 -
            {
307 +
            if touched.touches(self.repo, h, &nodes) {
301 308
                out.insert(*h);
302 309
            }
303 310
            if i % 256 == 255 {
304 311
                self.tick()?;
305 312
            }
306 313
        }
314 +
        touched.save();
307 315
        Ok(out)
308 316
    }
309 317
…
185 unchanged lines
495 503
                let kind = self.text_arg(name, args, 0)?.to_string();
496 504
                self.attested(within, &kind)
497 505
            }
506 +
            // A bookmark's heads by name: what a bare bookmark symbol means,
507 +
            // without falling through to tags and change prefixes.
508 +
            "bookmark" => {
509 +
                self.arity(name, args, 1)?;
510 +
                let b = self.text_arg(name, args, 0)?;
511 +
                let view = self.repo.view()?;
512 +
                let st = match view.bookmarks.get(b) {
513 +
                    Some(st) => *st,
514 +
                    None => match remote_bookmark(&view, b) {
515 +
                        Some((_, _, st)) => st,
516 +
                        None => return Err(err(format!("bookmark({b}): no such bookmark"))),
517 +
                    },
518 +
                };
519 +
                let heads = self.repo.store().get_state(&st)?.heads;
520 +
                self.ensure(&heads)?;
521 +
                Ok(heads.into_iter().collect())
522 +
            }
498 523
            "built" => {
499 524
                self.arity(name, args, 1)?;
500 525
                let job = self.text_arg(name, args, 0)?.to_string();
…
306 unchanged lines
modified crates/oxy-core/src/repo/snapshot.rs +659 −49
…
25 unchanged lines
26 26
    })
27 27
}
28 28
29 +
/// Whether `content` still holds a conflict marker oxy wrote for this file
30 +
/// (oxygen#126): a marker-like line of the materialized conflicted file
31 +
/// that is not one of its unconflicted lines (`plain`). Text that merely
32 +
/// looks like a marker, as oxy's own documentation or a test fixture has,
33 +
/// is in the unconflicted lines, so it is not taken for one.
34 +
pub fn has_own_conflict_markers(content: &[u8], materialized: &[u8], plain: &[&[u8]]) -> bool {
35 +
    let line = |l: &[u8]| -> Vec<u8> {
36 +
        let l = l.strip_suffix(b"\n").unwrap_or(l);
37 +
        l.strip_suffix(b"\r").unwrap_or(l).to_vec()
38 +
    };
39 +
    let marker = |l: &[u8]| {
40 +
        l.starts_with(b"<<<<<<< oxy conflict")
41 +
            || l.starts_with(b"======= oxy conflict")
42 +
            || l.starts_with(b">>>>>>> oxy conflict")
43 +
    };
44 +
    let unconflicted: HashSet<Vec<u8>> = plain.iter().map(|l| line(l)).collect();
45 +
    let own: HashSet<Vec<u8>> = split_lines(materialized)
46 +
        .into_iter()
47 +
        .map(line)
48 +
        .filter(|l| marker(l) && !unconflicted.contains(l))
49 +
        .collect();
50 +
    !own.is_empty()
51 +
        && split_lines(content)
52 +
            .into_iter()
53 +
            .any(|l| own.contains(&line(l)))
54 +
}
55 +
29 56
#[derive(Clone, Debug, PartialEq, Eq)]
30 57
pub struct PathStatus {
31 58
    pub path: String,
…
46 unchanged lines
78 105
    written: u128,
79 106
}
80 107
108 +
/// Whether `base` has two or more live versions of the opaque file or
109 +
/// subrepo pin at `path` (a replace conflict, spec §9.3).
110 +
fn has_replace_conflict(base: &Snapshot, path: &str) -> bool {
111 +
    matches!(base.files.get(path).map(|f| &f.content), Some(FileContent::Opaque { blobs }) if blobs.len() > 1)
112 +
}
113 +
114 +
/// What `oxy resolve --both` did to one file (oxygen#121).
115 +
#[derive(Debug, Default, Clone, PartialEq, Eq)]
116 +
pub struct KeptBoth {
117 +
    pub path: String,
118 +
    /// The conflicts settled by keeping every side, by number.
119 +
    pub settled: Vec<usize>,
120 +
    /// Of those, the ones whose sides were identical, settled by keeping
121 +
    /// one (oxygen#146).
122 +
    pub identical: Vec<usize>,
123 +
    /// The conflicts that keep their markers, by number, and why.
124 +
    pub left: Vec<(usize, &'static str)>,
125 +
}
126 +
127 +
/// `text` (a working-copy file) with the marker block of each conflict that
128 +
/// `sides` (from [`State::both_sides`]) says keeping both settles replaced
129 +
/// by its sides' lines; the conflicts settled, and those left and why.
130 +
pub(crate) fn keep_both_text(
131 +
    text: &[u8],
132 +
    sides: &[Option<Vec<Vec<u8>>>],
133 +
    reverse: bool,
134 +
) -> (Vec<u8>, Vec<usize>, Vec<(usize, &'static str)>) {
135 +
    let plans: Vec<Option<crate::state::Settle>> = sides
136 +
        .iter()
137 +
        .map(|s| {
138 +
            s.as_ref().map(|w| crate::state::Settle {
139 +
                sides: w.clone(),
140 +
                kept: w.clone(),
141 +
                identical: false,
142 +
            })
143 +
        })
144 +
        .collect();
145 +
    settle_text(text, &plans, reverse)
146 +
}
147 +
148 +
/// `text` (a working-copy file) with the marker block of each conflict
149 +
/// that `plans` (from [`State::settle_plans`]) settle replaced by its
150 +
/// kept text, the sides in order (`reverse`d on request); the conflicts
151 +
/// settled, and those left and why. A block whose text between the
152 +
/// markers is not its sides' keeps its markers.
153 +
pub(crate) fn settle_text(
154 +
    text: &[u8],
155 +
    plans: &[Option<crate::state::Settle>],
156 +
    reverse: bool,
157 +
) -> (Vec<u8>, Vec<usize>, Vec<(usize, &'static str)>) {
158 +
    // A side as the markers hold it: the renderer ends each with a newline.
159 +
    let with_newline = |s: &[u8]| {
160 +
        let mut s = s.to_vec();
161 +
        if !s.is_empty() && !s.ends_with(b"\n") {
162 +
            s.push(b'\n');
163 +
        }
164 +
        s
165 +
    };
166 +
    let lines: Vec<&[u8]> = text.split_inclusive(|&b| b == b'\n').collect();
167 +
    let (mut out, mut settled, mut left) = (Vec::new(), Vec::new(), Vec::new());
168 +
    let mut i = 0;
169 +
    while i < lines.len() {
170 +
        let Some((n, total)) = conflict_header(lines[i]) else {
171 +
            out.extend_from_slice(lines[i]);
172 +
            i += 1;
173 +
            continue;
174 +
        };
175 +
        let end = format!(">>>>>>> oxy conflict {n} of {total} end");
176 +
        let next_side = format!("======= oxy conflict {n} of {total}, side ");
177 +
        let Some(stop) =
178 +
            (i + 1..lines.len()).find(|&j| lines[j].trim_ascii_end() == end.as_bytes())
179 +
        else {
180 +
            left.push((n, "its end marker is missing; edit it by hand"));
181 +
            out.extend_from_slice(lines[i]);
182 +
            i += 1;
183 +
            continue;
184 +
        };
185 +
        let mut found: Vec<Vec<u8>> = vec![Vec::new()];
186 +
        for line in &lines[i + 1..stop] {
187 +
            if line.starts_with(next_side.as_bytes()) {
188 +
                found.push(Vec::new());
189 +
            } else {
190 +
                found.last_mut().unwrap().extend_from_slice(line);
191 +
            }
192 +
        }
193 +
        let wanted = match plans.get(n.wrapping_sub(1)) {
194 +
            _ if total != plans.len() => {
195 +
                Err("the markers are not the file's conflicts now; see `oxy status`")
196 +
            }
197 +
            Some(Some(w)) => Ok(w),
198 +
            _ => Err("both sides changed these lines; edit the markers"),
199 +
        };
200 +
        let keep = wanted.and_then(|w| {
201 +
            let mut a: Vec<Vec<u8>> = w.sides.iter().map(|s| with_newline(s)).collect();
202 +
            let mut b = found.clone();
203 +
            a.sort();
204 +
            b.sort();
205 +
            if a == b {
206 +
                Ok(&w.kept)
207 +
            } else {
208 +
                Err("the lines between its markers were edited; finish it by hand")
209 +
            }
210 +
        });
211 +
        match keep {
212 +
            Ok(w) => {
213 +
                let order: Vec<&Vec<u8>> = if reverse {
214 +
                    w.iter().rev().collect()
215 +
                } else {
216 +
                    w.iter().collect()
217 +
                };
218 +
                for s in order {
219 +
                    out.extend_from_slice(&with_newline(s));
220 +
                }
221 +
                settled.push(n);
222 +
            }
223 +
            Err(why) => {
224 +
                left.push((n, why));
225 +
                for line in &lines[i..=stop] {
226 +
                    out.extend_from_slice(line);
227 +
                }
228 +
            }
229 +
        }
230 +
        i = stop + 1;
231 +
    }
232 +
    (out, settled, left)
233 +
}
234 +
235 +
/// `(n, total)` from a conflict's opening marker (spec §9.5).
236 +
fn conflict_header(line: &[u8]) -> Option<(usize, usize)> {
237 +
    let rest = line.strip_prefix(b"<<<<<<< oxy conflict ")?;
238 +
    let rest = std::str::from_utf8(rest).ok()?;
239 +
    let (n, rest) = rest.split_once(" of ")?;
240 +
    let total = rest.split(|c: char| !c.is_ascii_digit()).next()?;
241 +
    Some((n.parse().ok()?, total.parse().ok()?))
242 +
}
243 +
81 244
pub fn is_binary(content: &[u8]) -> bool {
82 245
    content[..content.len().min(BINARY_SNIFF)].contains(&0)
83 246
}
…
266 unchanged lines
350 513
        if seed.is_some() {
351 514
            b.retire_stray_placements();
352 515
        }
516 +
        b.retire_unified();
353 517
        laps.lap("build: stray placements");
354 518
        let mut statuses = b.statuses;
355 519
        statuses.sort_by(|a, b| a.path.cmp(&b.path));
…
25 unchanged lines
381 545
        Ok(hash)
382 546
    }
383 547
548 +
    fn resolutions_path(&self) -> PathBuf {
549 +
        self.oxy
550 +
            .join("workspaces")
551 +
            .join(self.workspace.to_hex())
552 +
            .join("resolve")
553 +
    }
554 +
555 +
    /// The paths whose replace conflicts the user chose to settle with what
556 +
    /// the working copy holds there (`oxy resolve PATH`, spec §9.3).
557 +
    pub fn resolutions(&self) -> Result<BTreeSet<String>> {
558 +
        match fs::read_to_string(self.resolutions_path()) {
559 +
            Ok(text) => Ok(text
560 +
                .lines()
561 +
                .filter(|l| !l.is_empty())
562 +
                .map(String::from)
563 +
                .collect()),
564 +
            Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(BTreeSet::new()),
565 +
            Err(e) => Err(e.into()),
566 +
        }
567 +
    }
568 +
569 +
    /// Settle the replace conflicts at `paths` (a file's or a subrepo pin's,
570 +
    /// spec §9.3) with what the working copy holds there: the next snapshot
571 +
    /// replaces every live candidate with it. Refused for a path whose base
572 +
    /// has no replace conflict.
573 +
    pub fn choose_resolutions(&self, base: &Snapshot, paths: &[String]) -> Result<()> {
574 +
        for p in paths {
575 +
            if !has_replace_conflict(base, p) {
576 +
                return Err(RepoError::Msg(format!(
577 +
                    "{p} has no replace conflict to settle (a text file's conflict is settled by editing out its markers, or `oxy resolve --both {p}`)"
578 +
                )));
579 +
            }
580 +
        }
581 +
        let mut all = self.resolutions()?;
582 +
        all.extend(paths.iter().cloned());
583 +
        let text: String = all.iter().map(|p| format!("{p}\n")).collect();
584 +
        crate::durable::write_atomic(&self.resolutions_path(), text.as_bytes())?;
585 +
        Ok(())
586 +
    }
587 +
588 +
    /// Settle the adjacent-insertion conflicts of the text files at `paths`
589 +
    /// (oxygen#121) by keeping every side, earliest change first (latest
590 +
    /// first with `reverse`): each one's markers in the working copy give
591 +
    /// way to its sides' lines, and the next snapshot records the result.
592 +
    /// A conflict whose sides changed the same lines, or whose text between
593 +
    /// the markers was edited, keeps its markers. With no paths, every
594 +
    /// conflicted text file.
595 +
    pub fn keep_both(
596 +
        &self,
597 +
        snap: &SnapshotResult,
598 +
        paths: &[String],
599 +
        reverse: bool,
600 +
    ) -> Result<Vec<KeptBoth>> {
601 +
        let paths: Vec<String> = if paths.is_empty() {
602 +
            snap.base_snapshot
603 +
                .files
604 +
                .iter()
605 +
                .filter(|(_, f)| matches!(&f.content, FileContent::Text { materialized, .. } if materialized.is_conflicted()))
606 +
                .map(|(p, _)| p.clone())
607 +
                .collect()
608 +
        } else {
609 +
            paths.to_vec()
610 +
        };
611 +
        let mut out = Vec::new();
612 +
        for p in paths {
613 +
            let conflicted = match snap.base_snapshot.files.get(&p).map(|f| (f, &f.content)) {
614 +
                Some((f, FileContent::Text { materialized, .. }))
615 +
                    if materialized.is_conflicted() =>
616 +
                {
617 +
                    f
618 +
                }
619 +
                Some((_, FileContent::Opaque { blobs })) if blobs.len() > 1 => {
620 +
                    return Err(RepoError::Msg(format!(
621 +
                        "{p} has a replace conflict: keep one version with `oxy resolve {p}`"
622 +
                    )))
623 +
                }
624 +
                _ => {
625 +
                    return Err(RepoError::Msg(format!(
626 +
                        "{p} has no text conflict to settle"
627 +
                    )))
628 +
                }
629 +
            };
630 +
            let sides = snap.base.both_sides(&conflicted.node);
631 +
            let disk = self.root().join(&p);
632 +
            let text = fs::read(&disk)?;
633 +
            let (bytes, settled, left) = keep_both_text(&text, &sides, reverse);
634 +
            if !settled.is_empty() {
635 +
                fs::write(&disk, bytes)?;
636 +
            }
637 +
            out.push(KeptBoth {
638 +
                path: p,
639 +
                settled,
640 +
                identical: Vec::new(),
641 +
                left,
642 +
            });
643 +
        }
644 +
        Ok(out)
645 +
    }
646 +
647 +
    /// Settle what a checkout may settle by itself in each conflicted text
648 +
    /// file ([`State::settle_plans`]): conflicts whose sides are identical,
649 +
    /// keeping one (oxygen#146), and in files whose `.oxyattributes` say
650 +
    /// `merge=union`, adjacent insertions, keeping both sides earliest
651 +
    /// change first (oxygen#145). The next commit records the resolution.
652 +
    /// The files it settled something in.
653 +
    pub(crate) fn settle_conflicts(&self, command: &str) -> Result<Vec<KeptBoth>> {
654 +
        let snap = self.snapshot(command)?;
655 +
        let mut out = Vec::new();
656 +
        for (p, f) in &snap.base_snapshot.files {
657 +
            if !matches!(&f.content, FileContent::Text { materialized, .. } if materialized.is_conflicted())
658 +
            {
659 +
                continue;
660 +
            }
661 +
            let plans = snap
662 +
                .base
663 +
                .settle_plans(&f.node, snap.attributes.lookup(p).union);
664 +
            if plans.iter().all(Option::is_none) {
665 +
                continue;
666 +
            }
667 +
            let disk = self.root().join(p);
668 +
            let Ok(text) = fs::read(&disk) else { continue };
669 +
            let (bytes, settled, left) = settle_text(&text, &plans, false);
670 +
            if settled.is_empty() {
671 +
                continue;
672 +
            }
673 +
            fs::write(&disk, bytes)?;
674 +
            let identical = settled
675 +
                .iter()
676 +
                .copied()
677 +
                .filter(|n| plans[n - 1].as_ref().is_some_and(|w| w.identical))
678 +
                .collect();
679 +
            out.push(KeptBoth {
680 +
                path: p.clone(),
681 +
                settled,
682 +
                identical,
683 +
                left,
684 +
            });
685 +
        }
686 +
        Ok(out)
687 +
    }
688 +
689 +
    /// Every conflict of `snap`, `state`'s materialization, settled as a
690 +
    /// checkout settles them ([`State::settle_plans`]): the new text of
691 +
    /// each conflicted file, when each is a text file whose every conflict
692 +
    /// has identical sides (one kept, oxygen#146) or, in a file its
693 +
    /// `.oxyattributes` mark `merge=union`, is an adjacent insertion (both
694 +
    /// kept, oxygen#145); `None` when any conflict needs a person, or there
695 +
    /// is none. A host lands a proposal with these as a resolution, as a
696 +
    /// checkout settles them in a working copy.
697 +
    pub fn union_resolution(state: &State, snap: &Snapshot) -> Option<Vec<(String, Vec<u8>)>> {
698 +
        if snap.tree.is_conflicted() {
699 +
            return None;
700 +
        }
701 +
        let attrs = Repo::state_attributes(snap);
702 +
        let mut out = Vec::new();
703 +
        for (p, f) in &snap.files {
704 +
            if !f.is_conflicted() {
705 +
                continue;
706 +
            }
707 +
            let FileContent::Text { bytes, .. } = &f.content else {
708 +
                return None;
709 +
            };
710 +
            let plans = state.settle_plans(&f.node, attrs.lookup(p).union);
711 +
            let (text, _, left) = settle_text(bytes, &plans, false);
712 +
            if !left.is_empty() || plans.iter().any(Option::is_none) {
713 +
                return None;
714 +
            }
715 +
            out.push((p.clone(), text));
716 +
        }
717 +
        (!out.is_empty()).then_some(out)
718 +
    }
719 +
720 +
    /// The chosen resolutions `base` still needs; the rest are forgotten.
721 +
    fn resolutions_for(&self, base: &Snapshot) -> Result<HashSet<String>> {
722 +
        let all = self.resolutions()?;
723 +
        if all.is_empty() {
724 +
            return Ok(HashSet::new());
725 +
        }
726 +
        let live: HashSet<String> = all
727 +
            .iter()
728 +
            .filter(|p| has_replace_conflict(base, p))
729 +
            .cloned()
730 +
            .collect();
731 +
        if live.is_empty() {
732 +
            match fs::remove_file(self.resolutions_path()) {
733 +
                Ok(()) => {}
734 +
                Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
735 +
                Err(e) => return Err(e.into()),
736 +
            }
737 +
        }
738 +
        Ok(live)
739 +
    }
740 +
384 741
    /// Snapshot the working copy into the working-copy change (spec §12.3).
385 742
    /// The result carries the full base state and its whole rendering.
386 743
    pub fn snapshot(&self, command: &str) -> Result<SnapshotResult> {
…
49 unchanged lines
436 793
        };
437 794
        let view = {
438 795
            let _t = crate::timing::phase("snapshot: tree view");
439 -
            base.tree().view()
796 +
            self.tree_view(&mut base)?
440 797
        };
441 798
        let manifest = {
442 799
            let _t = crate::timing::phase("snapshot: read manifest");
…
66 unchanged lines
509 866
        // paths, so an unchanged working copy hashes identically (spec §12.3).
510 867
        let reuse = added_node_paths(&old_wc, &base_snapshot);
511 868
        self.platform_normalize(&mut fs_entries, &base_snapshot);
869 +
        // Replace conflicts the user settled with `oxy resolve PATH`
870 +
        // (oxygen#140): those still in the base; a choice the base no longer
871 +
        // needs (once committed) is forgotten.
872 +
        let resolve = self.resolutions_for(&base_snapshot)?;
512 873
        let built = {
513 874
            let _t = crate::timing::phase("snapshot: diff against base");
514 -
            if partial && self.config.sparse_paths.is_empty() {
875 +
            if !resolve.is_empty() {
876 +
                self.build_from_entries_resolving(
877 +
                    &base,
878 +
                    &base_snapshot,
879 +
                    reuse,
880 +
                    &fs_entries,
881 +
                    &resolve,
882 +
                )?
883 +
            } else if partial && self.config.sparse_paths.is_empty() {
515 884
                #[cfg(test)]
516 885
                let full =
517 886
                    self.build_from_entries(&base, &base_snapshot, reuse.clone(), &fs_entries)?;
…
267 unchanged lines
785 1154
            ));
786 1155
        }
787 1156
        Ok(format!(
788 -
            "{{\"bookmark\":{},\"working_copy\":{},\"heads\":{},\"clean\":{},\"conflicted\":{},\"paths\":{paths},\"conflicts\":{},\"untracked_nested\":{}}}",
1157 +
            "{{\"bookmark\":{},\"working_copy\":{},\"heads\":{},\"clean\":{},\"conflicted\":{},\"paths\":{paths},\"conflicts\":{},\"untracked_nested\":{},\"skipped_large\":{}}}",
789 1158
            json_str(&self.config.bookmark),
790 1159
            json_str(&snap.wc.change_id.to_hex()),
791 1160
            heads.len(),
792 1161
            snap.statuses.is_empty(),
793 1162
            base_snapshot.is_conflicted(),
794 1163
            json_array(conflicts),
795 -
            json_array(snap.untracked_nested.iter().map(|p| json_str(p)))
1164 +
            json_array(snap.untracked_nested.iter().map(|p| json_str(p))),
1165 +
            json_array(
1166 +
                self.skipped_large
1167 +
                    .borrow()
1168 +
                    .iter()
1169 +
                    .map(|(p, n)| format!("{{\"path\":{},\"size\":{n}}}", json_str(p)))
1170 +
            )
796 1171
        ))
797 1172
    }
798 1173
…
27 unchanged lines
826 1201
        attrs: &crate::attributes::Attributes,
827 1202
    ) -> HashMap<String, [u8; 32]> {
828 1203
        let _t = crate::timing::phase("commit: graph keys");
829 -
        let view = st.tree().view();
1204 +
        let view = st.view();
830 1205
        paths
831 1206
            .iter()
832 1207
            .filter_map(|p| {
…
11 unchanged lines
844 1219
        extra_deps: &[ChangeHash],
845 1220
        selection: &Selection,
846 1221
    ) -> Result<ChangeHash> {
1222 +
        Ok(self
1223 +
            .commit_selection_partial(message, extra_deps, selection)?
1224 +
            .0)
1225 +
    }
1226 +
1227 +
    /// As [`Repo::commit_selection`], and whether the commit left part of
1228 +
    /// the working copy uncommitted: when it did not, nothing is left, and
1229 +
    /// no snapshot is needed to say so.
1230 +
    pub fn commit_selection_partial(
1231 +
        &self,
1232 +
        message: &str,
1233 +
        extra_deps: &[ChangeHash],
1234 +
        selection: &Selection,
1235 +
    ) -> Result<(ChangeHash, bool)> {
847 1236
        // The commit's objects are written together, made durable at once
848 1237
        // before the operation that names them (spec §13.6).
849 1238
        let batch = self.store.begin_batch()?;
…
30 unchanged lines
880 1269
        message: &str,
881 1270
        extra_deps: &[ChangeHash],
882 1271
        selection: &Selection,
883 -
    ) -> Result<ChangeHash> {
1272 +
    ) -> Result<(ChangeHash, bool)> {
884 1273
        let _wc = self.lock_working_copy()?;
885 1274
        let mut snap = match self.snapshot_from_cache()? {
886 1275
            Some(snap) => snap,
…
34 unchanged lines
921 1310
                (o, e, partial)
922 1311
            }
923 1312
            Selection::Hunks(indices) => {
924 -
                let (o, e) = select_hunks(&snap.wc, indices);
1313 +
                let (o, e) = select_hunks(&snap.wc, indices, &snap.base_snapshot.tree.unified);
925 1314
                let partial = o.len() != snap.wc.tree_ops.len() || e.len() != snap.wc.edits.len();
926 1315
                (o, e, partial)
927 1316
            }
…
1 unchanged lines
929 1318
        if tree_ops.is_empty() && edits.is_empty() {
930 1319
            return fail("nothing selected to commit");
931 1320
        }
1321 +
        // A resolution may leave other work uncommitted (oxygen#118), but
1322 +
        // takes every file it resolves, whole.
932 1323
        if partial && snap.resolves {
933 -
            return fail("a conflict resolution must be committed whole");
1324 +
            let resolved = snap
1325 +
                .statuses
1326 +
                .iter()
1327 +
                .filter(|s| s.status == ChangeStatus::Resolved)
1328 +
                .map(|s| s.path.as_str());
1329 +
            let missing: Vec<&str> = match selection {
1330 +
                Selection::Paths(paths) => resolved
1331 +
                    .filter(|p| {
1332 +
                        !paths.iter().any(|x| {
1333 +
                            p == x || p.starts_with(&format!("{}/", x.trim_end_matches('/')))
1334 +
                        })
1335 +
                    })
1336 +
                    .collect(),
1337 +
                _ => resolved.collect(),
1338 +
            };
1339 +
            if !missing.is_empty() {
1340 +
                return fail(format!(
1341 +
                    "a conflict resolution takes every file it resolves, whole: commit by paths, including {}",
1342 +
                    missing.join(", ")
1343 +
                ));
1344 +
            }
934 1345
        }
935 1346
        // pre-commit hook: may veto.
936 1347
        self.run_hook(
…
174 unchanged lines
1111 1522
                json_str(message)
1112 1523
            ),
1113 1524
        );
1114 -
        Ok(hash)
1525 +
        Ok((hash, partial))
1115 1526
    }
1116 1527
}
1117 1528
…
53 unchanged lines
1171 1582
            nodes.insert(n);
1172 1583
        }
1173 1584
    }
1585 +
    // The nodes shown as one with a selected node are retired with it.
1586 +
    for (dup, shown) in &base_snapshot.tree.unified {
1587 +
        if nodes.contains(shown) {
1588 +
            nodes.insert(*dup);
1589 +
        }
1590 +
    }
1174 1591
    // New parents of selected new nodes come along.
1175 1592
    let mut changed = true;
1176 1593
    while changed {
…
22 unchanged lines
1199 1616
/// The hunks `indices` name plus whatever they refer to within the working
1200 1617
/// copy (the parent directory of an added node, the insert that introduced
1201 1618
/// an anchor line), renumbered so the result is a well-formed change.
1202 -
fn select_hunks(wc: &Change, indices: &[usize]) -> (Vec<TreeOp>, Vec<Edit>) {
1619 +
/// The nodes `unified` shows as one with another (a base's
1620 +
/// [`TreeView::unified`](crate::tree::TreeView::unified)) are retired with
1621 +
/// any hunk of the node shown.
1622 +
fn select_hunks(
1623 +
    wc: &Change,
1624 +
    indices: &[usize],
1625 +
    unified: &BTreeMap<NodeId, NodeId>,
1626 +
) -> (Vec<TreeOp>, Vec<Edit>) {
1203 1627
    let n_ops = wc.tree_ops.len();
1204 1628
    let mut ops_sel = vec![false; n_ops];
1205 1629
    let mut edits_sel = vec![false; wc.edits.len()];
…
93 unchanged lines
1299 1723
            }
1300 1724
        }
1301 1725
    }
1726 +
    // The retirement of a node shown as one with a selected node.
1727 +
    if !unified.is_empty() {
1728 +
        let chosen: HashSet<NodeId> = (0..n_ops)
1729 +
            .filter(|i| ops_sel[*i])
1730 +
            .map(|i| wc.tree_ops[i].node())
1731 +
            .chain(
1732 +
                (0..wc.edits.len())
1733 +
                    .filter(|j| edits_sel[*j])
1734 +
                    .map(|j| wc.edits[j].node()),
1735 +
            )
1736 +
            .collect();
1737 +
        for (i, o) in wc.tree_ops.iter().enumerate() {
1738 +
            if let TreeOp::RemoveNode { node, .. } = o {
1739 +
                if unified.get(node).is_some_and(|s| chosen.contains(s)) {
1740 +
                    ops_sel[i] = true;
1741 +
                }
1742 +
            }
1743 +
        }
1744 +
    }
1302 1745
    // Renumber self-references: old atom index → new index among selected inserts.
1303 1746
    let mut remap: HashMap<(NodeId, u32), u32> = HashMap::new();
1304 1747
    let mut new_counts: HashMap<NodeId, u32> = HashMap::new();
…
63 unchanged lines
1368 1811
    }
1369 1812
    let mut out = Vec::new();
1370 1813
    for (i, op) in change.tree_ops.iter().enumerate() {
1814 +
        // A retired duplicate goes with the node shown: no hunk of its own.
1815 +
        if is_unified_retirement(op, base_snapshot) {
1816 +
            continue;
1817 +
        }
1371 1818
        let one = Change {
1372 1819
            tree_ops: vec![op.clone()],
1373 1820
            edits: vec![],
…
45 unchanged lines
1419 1866
impl Repo {
1420 1867
    /// The content of `path` in a state: `heads`, or the working copy's base.
1421 1868
    pub fn file_at(&self, path: &str, heads: Option<&[ChangeHash]>) -> Result<Vec<u8>> {
1422 -
        let snapshot = match heads {
1423 -
            Some(h) => self.load_state(h)?.materialize(),
1424 -
            None => {
1425 -
                let (_, wc) = self.working_copy()?;
1426 -
                self.load_state(&wc.deps)?.materialize()
1427 -
            }
1869 +
        // The state's history and tree, then the one file's graph: nothing
1870 +
        // else is built or rendered (oxygen#92).
1871 +
        // The working copy's base, and a state named by its heads (`-r
1872 +
        // main` on main), come from the workspace's state index.
1873 +
        let base = if self.is_bare() {
1874 +
            None
1875 +
        } else {
1876 +
            Some(self.working_copy()?.1.deps)
1877 +
        };
1878 +
        let same = |a: &[ChangeHash], b: &[ChangeHash]| {
1879 +
            let (mut a, mut b) = (a.to_vec(), b.to_vec());
1880 +
            a.sort();
1881 +
            b.sort();
1882 +
            a == b
1883 +
        };
1884 +
        let mut state = match (heads, base) {
1885 +
            (Some(h), Some(b)) if same(h, &b) => self.load_state_indexed(&b)?,
1886 +
            (Some(h), _) => self.load_state_history(h)?,
1887 +
            (None, Some(b)) => self.load_state_indexed(&b)?,
1888 +
            (None, None) => return fail("this is a bare repository: it has no working copy"),
1428 1889
        };
1890 +
        let view = self.tree_view(&mut state)?;
1891 +
        let Some(node) = view.entries.get(path).map(|e| e.node) else {
1892 +
            return fail(format!("{path} is not in that state"));
1893 +
        };
1894 +
        self.load_graphs_of(&mut state, &[node])?;
1895 +
        let snapshot = state.materialize_paths(view, [path]);
1429 1896
        let Some(f) = snapshot.files.get(path) else {
1430 1897
            return fail(format!("{path} is not in that state"));
1431 1898
        };
…
104 unchanged lines
1536 2003
            .map(|h| h.index)
1537 2004
            .filter(|i| !indices.contains(i))
1538 2005
            .collect();
1539 -
        let (ops, edits) = select_hunks(&snap.wc, &keep);
2006 +
        let (ops, edits) = select_hunks(&snap.wc, &keep, &snap.base_snapshot.tree.unified);
1540 2007
        let dropped_nodes: BTreeSet<NodeId> = hunks
1541 2008
            .iter()
1542 2009
            .filter(|h| indices.contains(&h.index))
…
92 unchanged lines
1635 2102
                let (base, snapshot) = self.change_base(&c)?;
1636 2103
                (c, base, snapshot)
1637 2104
            }
1638 -
            None => self.working_copy_with_base()?,
2105 +
            None => match self.working_copy_with_base_unless_unchanged()? {
2106 +
                Some(all) => all,
2107 +
                None => return Ok(Vec::new()),
2108 +
            },
1639 2109
        };
1640 2110
        Ok(diff_files_of(&c, &base, &base_snapshot))
1641 2111
    }
…
52 unchanged lines
1694 2164
    /// answer, no snapshot is taken, and the base's snapshot holds its tree
1695 2165
    /// but no file; otherwise a snapshot makes all three.
1696 2166
    pub fn working_copy_with_base(&self) -> Result<(Change, State, Snapshot)> {
2167 +
        match self.working_copy_with_base_unless_unchanged()? {
2168 +
            Some(all) => Ok(all),
2169 +
            None => {
2170 +
                let (_, wc) = self.working_copy()?;
2171 +
                let base = self.load_state_indexed(&wc.deps)?;
2172 +
                let (base, snapshot) = self.tree_base_for(&wc, base)?;
2173 +
                Ok((wc, base, snapshot))
2174 +
            }
2175 +
        }
2176 +
    }
2177 +
2178 +
    /// [`Repo::working_copy_with_base`], or `None` when the status cache
2179 +
    /// holds and the working-copy change changes nothing: its diff is
2180 +
    /// empty, and the base is not loaded for it (oxygen#94).
2181 +
    pub fn working_copy_with_base_unless_unchanged(
2182 +
        &self,
2183 +
    ) -> Result<Option<(Change, State, Snapshot)>> {
1697 2184
        if self.status_by_manifest()?.is_some() {
1698 2185
            let (_, wc) = self.working_copy()?;
2186 +
            if wc.tree_ops.is_empty() && wc.edits.is_empty() {
2187 +
                return Ok(None);
2188 +
            }
1699 2189
            let base = self.load_state_indexed(&wc.deps)?;
1700 2190
            let (base, snapshot) = self.tree_base_for(&wc, base)?;
1701 -
            return Ok((wc, base, snapshot));
2191 +
            return Ok(Some((wc, base, snapshot)));
1702 2192
        }
1703 2193
        let snap = self.snapshot_status("oxy diff")?;
1704 -
        Ok((snap.wc, snap.base, snap.base_snapshot))
2194 +
        Ok(Some((snap.wc, snap.base, snap.base_snapshot)))
1705 2195
    }
1706 2196
1707 2197
    /// The log restricted to changes touching `path` (its node, so renames
…
5 unchanged lines
1713 2203
        }
1714 2204
        let entries = self.log()?;
1715 2205
        let (_, wc) = self.working_copy()?;
1716 -
        let base_snapshot = self.load_state(&wc.deps)?.materialize();
1717 -
        let state_snapshot = self.load_state(&heads)?.materialize();
1718 -
        let node = base_snapshot
1719 -
            .files
1720 -
            .get(path)
1721 -
            .or_else(|| state_snapshot.files.get(path))
1722 -
            .map(|f| f.node);
1723 -
        let Some(node) = node else {
2206 +
        // The path's node from the trees alone: the base's (from the
2207 +
        // workspace's state index), else the bookmark's. Nothing is
2208 +
        // rendered.
2209 +
        let node_in = |mut state: State| -> Result<Option<HashSet<NodeId>>> {
2210 +
            Ok(nodes_at(&self.tree_view(&mut state)?, path))
2211 +
        };
2212 +
        let nodes = match node_in(self.load_state_indexed(&wc.deps)?)? {
2213 +
            Some(n) => Some(n),
2214 +
            None => node_in(self.load_state_history(&heads)?)?,
2215 +
        };
2216 +
        let Some(nodes) = nodes else {
1724 2217
            return fail(format!("{path} is not tracked"));
1725 2218
        };
1726 -
        Ok(self.entries_touching(entries, node))
2219 +
        Ok(self.entries_touching_up_to(entries, nodes, None))
1727 2220
    }
1728 2221
1729 2222
    /// The log of the state with these heads restricted to changes touching
…
11 unchanged lines
1741 2234
        path: &str,
1742 2235
        limit: Option<usize>,
1743 2236
    ) -> Result<Vec<LogEntry>> {
1744 -
        let Some(node) = state.tree().view().entries.get(path).map(|e| e.node) else {
2237 +
        let Some(nodes) = nodes_at(&state.view(), path) else {
1745 2238
            return fail(format!("{path} is not in that state"));
1746 2239
        };
1747 -
        Ok(self.entries_touching_up_to(self.log_of(state), node, limit))
1748 -
    }
1749 -
1750 -
    fn entries_touching(&self, entries: Vec<LogEntry>, node: NodeId) -> Vec<LogEntry> {
1751 -
        self.entries_touching_up_to(entries, node, None)
2240 +
        Ok(self.entries_touching_up_to(self.log_of(state), nodes, limit))
1752 2241
    }
1753 2242
1754 2243
    fn entries_touching_up_to(
1755 2244
        &self,
1756 2245
        entries: Vec<LogEntry>,
1757 -
        node: NodeId,
2246 +
        wanted: HashSet<NodeId>,
1758 2247
        limit: Option<usize>,
1759 2248
    ) -> Vec<LogEntry> {
2249 +
        let mut touched = self.touched();
2250 +
        if limit.is_none() {
2251 +
            let hashes: Vec<ChangeHash> = entries.iter().map(|e| e.hash).collect();
2252 +
            touched.fill(self, &hashes);
2253 +
        }
1760 2254
        let mut out = Vec::new();
1761 2255
        for e in entries {
1762 2256
            if limit.is_some_and(|n| out.len() >= n) {
1763 2257
                break;
1764 2258
            }
1765 -
            let Ok(c) = self.store.get_change(&e.hash) else {
1766 -
                continue;
1767 -
            };
1768 -
            if c.tree_ops.iter().any(|o| o.node() == node)
1769 -
                || c.edits.iter().any(|x| x.node() == node)
1770 -
            {
2259 +
            if touched.touches(self, &e.hash, &wanted) {
1771 2260
                out.push(e);
1772 2261
            }
1773 2262
        }
2263 +
        touched.save();
1774 2264
        out
1775 2265
    }
1776 2266
}
1777 2267
2268 +
/// The node at `path` in `view`, with the nodes it shows as one with it
2269 +
/// (identical additions, spec §9.2): the file's history is all of theirs.
2270 +
fn nodes_at(view: &crate::tree::TreeView, path: &str) -> Option<HashSet<NodeId>> {
2271 +
    let node = view.entries.get(path)?.node;
2272 +
    let mut out = HashSet::from([node]);
2273 +
    out.extend(
2274 +
        view.unified
2275 +
            .iter()
2276 +
            .filter(|(_, shown)| **shown == node)
2277 +
            .map(|(dup, _)| *dup),
2278 +
    );
2279 +
    Some(out)
2280 +
}
2281 +
1778 2282
/// Per-file hunks of `change` against `base`.
1779 2283
pub fn diff_files_of(change: &Change, base: &State, base_snapshot: &Snapshot) -> Vec<FileDiff> {
1780 2284
    let by_node = base_snapshot.tree.by_node();
…
26 unchanged lines
1807 2311
            });
1808 2312
        }
1809 2313
    };
1810 -
    for op in &change.tree_ops {
2314 +
    for op in change
2315 +
        .tree_ops
2316 +
        .iter()
2317 +
        .filter(|o| !is_unified_retirement(o, base_snapshot))
2318 +
    {
1811 2319
        match op {
1812 2320
            TreeOp::AddNode { node, kind, .. } => {
1813 2321
                if *kind != NodeKind::Dir {
…
411 unchanged lines
2225 2733
        e: &FsEntry,
2226 2734
        expected: &crate::state::FileState,
2227 2735
    ) -> Result<()> {
2228 -
        let entry = &self.base_snapshot.tree.entries[old];
2736 +
        let placement = self.base_snapshot.tree.entries[old].placement;
2229 2737
        let (dir, name) = split_path(new);
2230 2738
        let parent = self.ensure_dir(dir);
2231 2739
        self.tree_ops.push(TreeOp::Move {
2232 2740
            node: expected.node,
2233 -
            old: op_ref(entry.placement),
2741 +
            old: op_ref(placement),
2234 2742
            parent,
2235 2743
            name: name.to_string(),
2236 2744
        });
2745 +
        for k in self.same_place(&expected.node, placement) {
2746 +
            self.tree_ops.push(TreeOp::RemoveNode {
2747 +
                node: expected.node,
2748 +
                placement: op_ref(k),
2749 +
            });
2750 +
        }
2237 2751
        self.update_path(repo, old, e, expected)?;
2238 2752
        // update_path reports under the old path; the rename is what the user sees.
2239 2753
        self.statuses.retain(|s| s.path != old);
…
85 unchanged lines
2325 2839
        }
2326 2840
    }
2327 2841
2842 +
    /// Nodes the base shows as one with another (identical additions at one
2843 +
    /// path, spec §9.2) are retired by the change that first touches the
2844 +
    /// node shown: edited, moved, removed or given a mode alone, it would
2845 +
    /// differ from them and the name conflict would come back. Nothing
2846 +
    /// else changes for the user, so no status reports it.
2847 +
    fn retire_unified(&mut self) {
2848 +
        let unified = &self.base_snapshot.tree.unified;
2849 +
        if unified.is_empty() {
2850 +
            return;
2851 +
        }
2852 +
        let touched: HashSet<NodeId> = self
2853 +
            .tree_ops
2854 +
            .iter()
2855 +
            .map(|o| o.node())
2856 +
            .chain(self.edits.iter().map(|e| e.node()))
2857 +
            .collect();
2858 +
        let mut retired: HashSet<(ChangeHash, u32)> = HashSet::new();
2859 +
        for op in &self.tree_ops {
2860 +
            match op {
2861 +
                TreeOp::Move { old, .. } => {
2862 +
                    retired.insert((old.change, old.index));
2863 +
                }
2864 +
                TreeOp::RemoveNode { placement, .. } => {
2865 +
                    retired.insert((placement.change, placement.index));
2866 +
                }
2867 +
                TreeOp::AddNode { .. } | TreeOp::SetMode { .. } => {}
2868 +
            }
2869 +
        }
2870 +
        let mut extra = Vec::new();
2871 +
        for (dup, shown) in unified {
2872 +
            if !touched.contains(shown) {
2873 +
                continue;
2874 +
            }
2875 +
            for (k, _) in self.base.tree().live_placements(dup) {
2876 +
                if retired.insert(k) {
2877 +
                    extra.push(TreeOp::RemoveNode {
2878 +
                        node: *dup,
2879 +
                        placement: op_ref(k),
2880 +
                    });
2881 +
                }
2882 +
            }
2883 +
        }
2884 +
        self.tree_ops.extend(extra);
2885 +
    }
2886 +
2887 +
    /// The node's other live placements at the same place as `k`, the one
2888 +
    /// the tree shows: two lines of history that put a node back where it
2889 +
    /// was each leave a placement, and the tree shows them as one file. A
2890 +
    /// change that removes or moves it retires them together, or the copy
2891 +
    /// left behind outlives its directory (oxygen#88).
2892 +
    fn same_place(&self, node: &NodeId, k: (ChangeHash, u32)) -> Vec<(ChangeHash, u32)> {
2893 +
        let tree = self.base.tree();
2894 +
        let Some(shown) = tree.placement(&k) else {
2895 +
            return Vec::new();
2896 +
        };
2897 +
        tree.live_placements(node)
2898 +
            .into_iter()
2899 +
            .filter(|(o, p)| *o != k && p.parent == shown.parent && p.name == shown.name)
2900 +
            .map(|(o, _)| o)
2901 +
            .collect()
2902 +
    }
2903 +
2328 2904
    fn remove_path(&mut self, path: &str, expected: &crate::state::FileState) {
2329 2905
        let entry = &self.base_snapshot.tree.entries[path];
2906 +
        let placement = entry.placement;
2330 2907
        self.tree_ops.push(TreeOp::RemoveNode {
2331 2908
            node: expected.node,
2332 -
            placement: op_ref(entry.placement),
2909 +
            placement: op_ref(placement),
2333 2910
        });
2911 +
        for k in self.same_place(&expected.node, placement) {
2912 +
            self.tree_ops.push(TreeOp::RemoveNode {
2913 +
                node: expected.node,
2914 +
                placement: op_ref(k),
2915 +
            });
2916 +
        }
2334 2917
        if path.contains(".oxy-conflict-") || expected.is_conflicted() {
2335 2918
            self.resolves = true;
2336 2919
        }
…
59 unchanged lines
2396 2979
                    if bytes == &*disk_text {
2397 2980
                        return Ok(());
2398 2981
                    }
2399 -
                    if has_conflict_markers(&e.content) {
2982 +
                    let regions = self.base.conflict_regions(&node);
2983 +
                    let plain: Vec<&[u8]> =
2984 +
                        regions.plain.iter().map(|(_, l)| l.as_slice()).collect();
2985 +
                    if has_own_conflict_markers(&e.content, bytes, &plain) {
2400 2986
                        self.statuses.push(PathStatus {
2401 2987
                            path: path.to_string(),
2402 2988
                            status: ChangeStatus::ConflictEdited,
…
4 unchanged lines
2407 2993
                    // with each region as a gap, then retire every region atom.
2408 2994
                    // Whatever the author kept from a region is re-inserted as
2409 2995
                    // new atoms, which leaves the graph totally ordered.
2410 -
                    let regions = self.base.conflict_regions(&node);
2411 2996
                    let mut edits = text_edits(node, &regions.plain, &disk_text);
2412 2997
                    if !regions.region_atoms.is_empty() {
2413 2998
                        edits.insert(
…
158 unchanged lines
2572 3157
                            old: current,
2573 3158
                            new: Some(blob),
2574 3159
                        });
3160 +
                        // New contents and a new executable bit at once: the
3161 +
                        // bit was dropped (VS Code's inno_updater.exe,
3162 +
                        // 2018-02-28, lost +x for 49 commits; oxygen#87).
3163 +
                        if e.kind == NodeKind::File && e.exec != expected.mode.exec {
3164 +
                            if let Some(old) = mode_atom {
3165 +
                                self.tree_ops.push(TreeOp::SetMode {
3166 +
                                    node,
3167 +
                                    old,
3168 +
                                    mode: Mode {
3169 +
                                        exec: e.exec,
3170 +
                                        opaque: true,
3171 +
                                    },
3172 +
                                });
3173 +
                            }
3174 +
                        }
2575 3175
                    }
2576 3176
                    status = Some(ChangeStatus::Modified);
2577 3177
                } else if e.kind == NodeKind::File && e.exec != expected.mode.exec {
…
141 unchanged lines
2719 3319
    out
2720 3320
}
2721 3321
3322 +
/// Whether `op` retires a node `base_snapshot` shows as one with another
3323 +
/// (spec §9.2): part of touching that node, never a removal the user made.
3324 +
fn is_unified_retirement(op: &TreeOp, base_snapshot: &Snapshot) -> bool {
3325 +
    matches!(op, TreeOp::RemoveNode { node, .. } if base_snapshot.tree.unified.contains_key(node))
3326 +
}
3327 +
2722 3328
fn path_of_node(
2723 3329
    n: &NodeId,
2724 3330
    by_node: &HashMap<NodeId, &crate::tree::TreeEntry>,
…
43 unchanged lines
2768 3374
    };
2769 3375
2770 3376
    let mut out = String::new();
2771 -
    for op in &change.tree_ops {
3377 +
    for op in change
3378 +
        .tree_ops
3379 +
        .iter()
3380 +
        .filter(|o| !is_unified_retirement(o, base_snapshot))
3381 +
    {
2772 3382
        match op {
2773 3383
            TreeOp::AddNode {
2774 3384
                node, kind, mode, ..
…
247 unchanged lines
modified crates/oxy-core/src/repo/strata.rs +14 −2
…
232 unchanged lines
233 233
            None => (RefsResponse::default(), None),
234 234
        };
235 235
        self.with_mirror(|mirror| {
236 +
            // Another process may have brought the mirror level since this
237 +
            // handle read its head.
238 +
            mirror.refresh()?;
236 239
            // The policy first: bundles are then imported under it.
237 240
            if let Some(sp) = &resp.policy {
238 241
                let have = mirror.current_policy()?.map(|(_, p)| p.policy.serial);
…
105 unchanged lines
344 347
        for t in &resp.tags {
345 348
            mirror.store.put_state(&StateObject::new(t.heads.clone()))?;
346 349
            mirror
347 -
                .adopt_tag(&mut view, t, policy.as_ref())
348 -
                .map_err(|e| RepoError::Msg(format!("{}: {e}", self.what())))?;
350 +
                .adopt_tag(
351 +
                    &mut view,
352 +
                    t,
353 +
                    policy.as_ref(),
354 +
                    &mut super::tags::TagCheck::Mirrored,
355 +
                )
356 +
                .map_err(|e| RepoError::Msg(format!("{}: {}", self.what(), e.reason)))?;
349 357
        }
350 358
        if view != op.view {
351 359
            mirror.record_op(&[op_ref], "strata sync", view)?;
…
60 unchanged lines
412 420
        self.with_mirror(|m| m.checkpoint_bundle_for(heads))
413 421
    }
414 422
423 +
    fn checkpoint_index(&self, heads: &[ChangeHash]) -> Result<Bundle> {
424 +
        self.with_mirror(|m| m.checkpoint_index_for(heads))
425 +
    }
426 +
415 427
    fn shallow(&self, heads: &[ChangeHash], depth: u32) -> Result<Vec<ChangeHash>> {
416 428
        self.with_mirror(|m| m.shallow_changes(heads, depth))
417 429
    }
…
941 unchanged lines
modified crates/oxy-core/src/repo/subrepos.rs +24 −17
…
181 unchanged lines
182 182
            }
183 183
        }
184 184
        let actual = read_pin(&dir, module.as_ref().and_then(|m| m.bookmark.as_deref()))?;
185 -
        let same = match (pin, &actual) {
186 -
            (Pin::Oxy { heads: a, .. }, Pin::Oxy { heads: b, .. }) => {
187 -
                let (mut a, mut b) = (a.clone(), b.clone());
188 -
                a.sort();
189 -
                b.sort();
190 -
                a == b
191 -
            }
192 -
            (Pin::Git { sha: a, .. }, Pin::Git { sha: b, .. }) => a == b,
193 -
            _ => false,
194 -
        };
185 +
        let same = pin.same_state(&actual);
195 186
        if same {
196 187
            return Ok(if cloned {
197 188
                SubrepoSync::Updated
…
49 unchanged lines
247 238
    /// Every subrepo of the working copy's base state.
248 239
    pub fn subrepos(&self) -> Result<Vec<SubrepoInfo>> {
249 240
        let (_, wc) = self.working_copy()?;
250 -
        let snap = self.load_state(&wc.deps)?.materialize();
241 +
        // The tree and the subrepos' pins only: a materialized state
242 +
        // rendered every file of the tree to find them, after each clone,
243 +
        // pull, switch and `workspace add` (in the VS Code history, ~70 s
244 +
        // and 11 GB; oxygen#137).
245 +
        let mut state = self.load_state_indexed(&wc.deps)?;
246 +
        let tree = self.tree_view(&mut state)?;
251 247
        let modules = self.modules_on_disk();
252 248
        let mut out = Vec::new();
253 -
        for (path, f) in &snap.files {
254 -
            if f.kind != NodeKind::Subrepo {
249 +
        for (path, e) in tree.entries.iter() {
250 +
            if e.kind != NodeKind::Subrepo {
255 251
                continue;
256 252
            }
257 -
            let FileContent::Opaque { blobs } = &f.content else {
253 +
            let Some(b) = state.live_blobs(&e.node).first().copied() else {
258 254
                continue;
259 255
            };
260 -
            let Some(b) = blobs.first() else { continue };
261 -
            let pin = Pin::decode(&self.read_blob(b)?)?;
256 +
            let pin = Pin::decode(&self.read_blob(&b)?)?;
262 257
            let dir = self.root.join(path);
263 258
            let present = dir.join(OXY_DIR).exists() || dir.join(".git").exists();
264 259
            let actual = if present {
…
20 unchanged lines
285 280
        }
286 281
        // Registered but not committed yet: pinned at what they are now.
287 282
        for (path, m) in &modules.entries {
288 -
            if snap.files.contains_key(path) {
283 +
            if tree.entries.contains_key(path) {
289 284
                continue;
290 285
            }
291 286
            let dir = self.root.join(path);
…
13 unchanged lines
305 300
        Ok(out)
306 301
    }
307 302
303 +
    /// `.oxymodules` entries with no subrepo behind them: never added, so
304 +
    /// there is no pin to clone or check out (oxygen#72).
305 +
    pub fn unpinned_modules(&self) -> Result<Vec<(String, Module)>> {
306 +
        let pinned: Vec<String> = self.subrepos()?.into_iter().map(|s| s.path).collect();
307 +
        Ok(self
308 +
            .modules_on_disk()
309 +
            .entries
310 +
            .into_iter()
311 +
            .filter(|(path, _)| !pinned.contains(path))
312 +
            .collect())
313 +
    }
314 +
308 315
    /// Clone `url` at `path`, register it in `.oxymodules`, and return the
309 316
    /// pin the next snapshot will record.
310 317
    pub fn submodule_add(&self, url: &str, path: &str, bookmark: Option<&str>) -> Result<Pin> {
…
113 unchanged lines
modified crates/oxy-core/src/repo/sync.rs +1376 −128
1 1
//! Serving and talking to remotes (spec §10.2, §16).
2 2
3 +
use super::tags::TagCheck;
3 4
use super::*;
4 5
5 6
/// What a push did, for hosts (hub spec §6.12).
…
21 unchanged lines
27 28
    pub refs: Vec<(String, Vec<ChangeHash>, usize)>,
28 29
    /// Tags adopted from the remote.
29 30
    pub new_tags: usize,
31 +
    /// Tags the remote withdrew, and so withdrawn here (oxygenhub#301).
32 +
    pub withdrawn_tags: Vec<String>,
30 33
    /// The remote has never presented a policy, so nothing was verified
31 34
    /// against one (spec §14.5: warn loudly).
32 35
    pub unverified: bool,
36 +
    /// What checking the host's transparency log found short of a fork
37 +
    /// (spec §14.8): attestations not logged yet, a log that could not be
38 +
    /// read. A fork fails the fetch instead.
39 +
    pub log_notes: Vec<String>,
40 +
    /// Bookmarks left as they were because their attestation could not be
41 +
    /// verified (its signer unknown here, or its signature bad), each with
42 +
    /// why (oxygenhub#266). The current bookmark is never skipped: its
43 +
    /// fetch fails instead.
44 +
    pub skipped: Vec<(String, String)>,
33 45
}
34 46
35 47
#[derive(Debug, Clone, PartialEq, Eq)]
…
6 unchanged lines
42 54
    /// Other bookmarks the pull moved: each local bookmark tracks the
43 55
    /// remote's of the same name.
44 56
    pub others: Vec<String>,
57 +
    /// What the fetch left as it was ([`FetchReport::skipped`]).
58 +
    pub skipped: Vec<(String, String)>,
45 59
}
46 60
61 +
/// The result a push of objects alone reports (a batch of a long history):
62 +
/// not a reference.
63 +
pub const OBJECTS_RESULT: &str = "objects";
64 +
65 +
/// Graph blobs asked for in one `objects` request when a shallow clone
66 +
/// fetches its checkpoint in parts (#137): their chunk lists and chunks,
67 +
/// a few MiB a request rather than one answer of gigabytes.
68 +
const CHECKPOINT_PART: usize = 2048;
69 +
47 70
impl Repo {
48 71
    // ------------------------------------------------------------ serving (spec §16)
49 72
…
35 unchanged lines
85 108
                _ => None,
86 109
            });
87 110
            let attestation = latest.filter(|a| matches!(&a.attest.body, AttestBody::Ref { new_state, .. } if new_state == state));
111 +
            let accepted = match &attestation {
112 +
                Some(a) => self.acceptance_of(&a.attest.hash())?,
113 +
                None => None,
114 +
            };
88 115
            out.push(RemoteRef {
89 116
                name: name.clone(),
90 117
                state: *state,
91 118
                heads: self.store.get_state(state)?.heads,
92 119
                attestation,
93 120
                last_serial,
121 +
                accepted,
122 +
                logged: None,
123 +
                integration: vec![],
94 124
            });
95 125
        }
96 126
        let policy = self.current_policy()?.map(|(_, sp)| sp);
…
12 unchanged lines
109 139
            tags: self.tag_refs()?,
110 140
            history,
111 141
            default_bookmark: Some(self.config.bookmark.clone()),
142 +
            log: None,
143 +
            witnesses: vec![],
144 +
            withdrawn: self.withdrawn_refs()?,
112 145
        })
113 146
    }
114 147
…
105 unchanged lines
220 253
        let mut done: BTreeSet<ChangeHash> = have.clone();
221 254
        done.extend(boundary.iter().copied());
222 255
        // Iterative post-order DFS.
256 +
        let shallow = self.is_shallow();
223 257
        let mut stack: Vec<(ChangeHash, bool)> = want.iter().rev().map(|h| (*h, false)).collect();
224 258
        while let Some((h, expanded)) = stack.pop() {
225 259
            if done.contains(&h) {
226 260
                continue;
227 261
            }
228 262
            if !self.store.has(&h) {
263 +
                // A shallow repository holds only part of its history; a
264 +
                // change it lacks is below what it cloned (in a depth-0
265 +
                // clone, any change of the checkpoint), which the remote
266 +
                // is assumed to hold, as it is for the boundary (#137).
267 +
                if shallow {
268 +
                    done.insert(h);
269 +
                    continue;
270 +
                }
229 271
                return fail(format!("change {} is not in this repository", h.short(12)));
230 272
            }
231 273
            if expanded {
…
14 unchanged lines
246 288
    /// Bundle changes (in the given dependency order) with their signatures,
247 289
    /// seals, identities, blobs, chunks.
248 290
    pub fn bundle_for(&self, changes: &[ChangeHash]) -> Result<Bundle> {
249 -
        self.bundle_for_filtered(changes, None)
291 +
        self.bundle_of(changes, None, false, false)
250 292
    }
251 293
252 294
    /// Objects by reference for the `objects` endpoint; a blob brings its chunks.
253 295
    pub fn objects_for(&self, refs: &[ObjectRef]) -> Result<Vec<(ObjectType, Vec<u8>)>> {
254 -
        let mut out = Vec::new();
255 -
        for r in refs {
256 -
            let (ty, body) = match self.store.get(r) {
257 -
                Ok(x) => x,
258 -
                Err(StoreError::NotFound(_)) => continue,
296 +
        // In two reads in pack order (#137): a checkpoint's graph blobs come
297 +
        // this way in parts, tens of thousands for a big state.
298 +
        let mut found = Vec::with_capacity(refs.len());
299 +
        for got in self.store.get_many(refs) {
300 +
            match got {
301 +
                Ok(x) => found.push(x),
302 +
                Err(StoreError::NotFound(_)) => {}
259 303
                Err(e) => return Err(e.into()),
260 -
            };
261 -
            if ty == ObjectType::Blob {
262 -
                for c in Blob::decode(&body)?.chunks {
263 -
                    if let Ok((cty, cb)) = self.store.get(&c) {
264 -
                        out.push((cty, cb));
265 -
                    }
266 -
                }
304 +
        let mut chunks = Vec::new();
305 +
        for (ty, body) in &found {
306 +
            if *ty == ObjectType::Blob {
307 +
                chunks.extend(Blob::decode(body)?.chunks);
308 +
            }
309 +
        }
310 +
        let mut out = Vec::with_capacity(found.len() + chunks.len());
311 +
        for got in self.store.get_many(&chunks) {
312 +
            match got {
313 +
                Ok(x) => out.push(x),
314 +
                Err(StoreError::NotFound(_)) => {}
315 +
                Err(e) => return Err(e.into()),
316 +
            }
317 +
        }
318 +
        out.extend(found);
267 319
            }
268 -
            out.push((ty, body));
269 320
        }
270 321
        Ok(out)
271 322
    }
272 323
273 324
    /// Like `bundle_for`, leaving out blobs larger than `max_blob` bytes
274 325
    /// (and their chunks) so the receiver fetches them lazily (spec §13.5).
326 +
    ///
327 +
    /// This is what a server answers a fetch with, so it also carries what
328 +
    /// verifies the attestations it serves beside it (see
329 +
    /// [`Repo::served_attest_signers`]), also when `changes` is empty.
275 330
    pub fn bundle_for_filtered(
276 331
        &self,
277 332
        changes: &[ChangeHash],
278 333
        max_blob: Option<u64>,
279 334
    ) -> Result<Bundle> {
280 -
        self.bundle_of(changes, max_blob, false)
335 +
        self.bundle_of(changes, max_blob, false, true)
281 336
    }
282 337
283 338
    /// Like [`Repo::bundle_for_filtered`] for a fetch between repositories
…
1 unchanged lines
285 340
    /// with their text blobs and chunks among the objects, so neither side
286 341
    /// decodes or encodes their text.
287 342
    pub fn bundle_raw_for(&self, changes: &[ChangeHash], max_blob: Option<u64>) -> Result<Bundle> {
288 -
        self.bundle_of(changes, max_blob, true)
343 +
        self.bundle_of(changes, max_blob, true, true)
344 +
    }
345 +
346 +
    /// The keys that signed the attestations this repository serves with
347 +
    /// its references: each bookmark's latest and each tag's. A fetch
348 +
    /// bundle carries their identities and certificates (spec §16.2), so a
349 +
    /// client can check an attestation whose signer signed none of the
350 +
    /// changes it receives, a delegate or a short-lived key that only
351 +
    /// moved a bookmark (oxygenhub#266). One pointer and one signature
352 +
    /// list per reference, computed once per operation head and kept
353 +
    /// (every bookmark move and tag is an operation): a client sends one
354 +
    /// fetch per bookmark, and on a bucket store each signature list is a
355 +
    /// LIST request, so recomputing per fetch cost metalgear2 (145
356 +
    /// bookmarks) 14 s a fetch.
357 +
    pub fn served_attest_signers(&self) -> Result<BTreeSet<KeyId>> {
358 +
        type Signers = std::collections::HashMap<(PathBuf, ObjectRef), BTreeSet<KeyId>>;
359 +
        static SERVED: std::sync::Mutex<Option<Signers>> = std::sync::Mutex::new(None);
360 +
        let key = (self.oxy.clone(), self.head_op()?.0);
361 +
        if let Some(keys) = SERVED
362 +
            .lock()
363 +
            .unwrap_or_else(|p| p.into_inner())
364 +
            .as_ref()
365 +
            .and_then(|m| m.get(&key))
366 +
        {
367 +
            return Ok(keys.clone());
368 +
        }
369 +
        let keys = self.attest_signers_now()?;
370 +
        let mut served = SERVED.lock().unwrap_or_else(|p| p.into_inner());
371 +
        let map = served.get_or_insert_with(Default::default);
372 +
        // One entry per repository: an older operation head is never asked
373 +
        // for again once a newer one is current.
374 +
        map.retain(|(oxy, _), _| *oxy != key.0);
375 +
        if map.len() >= 1024 {
376 +
            map.clear();
377 +
        }
378 +
        map.insert(key, keys.clone());
379 +
        Ok(keys)
380 +
    }
381 +
382 +
    fn attest_signers_now(&self) -> Result<BTreeSet<KeyId>> {
383 +
        let view = self.view()?;
384 +
        let pointers = view
385 +
            .bookmarks
386 +
            .keys()
387 +
            .map(|b| self.ref_attest_path(b))
388 +
            .chain(view.tags.keys().map(|t| self.tag_attest_path(t)));
389 +
        let mut keys = BTreeSet::new();
390 +
        for p in pointers {
391 +
            let Ok(text) = fs::read_to_string(&p) else {
392 +
                continue;
393 +
            };
394 +
            let Some(r) = ObjectRef::from_hex(text.trim()) else {
395 +
                continue;
396 +
            };
397 +
            keys.extend(self.store.signatures(&r)?.into_iter().map(|s| s.key));
398 +
        }
399 +
        Ok(keys)
289 400
    }
290 401
402 +
    /// `serving`: a fetch's answer, which carries the identities and
403 +
    /// certificates of [`Repo::served_attest_signers`] too.
291 404
    fn bundle_of(
292 405
        &self,
293 406
        changes: &[ChangeHash],
294 407
        max_blob: Option<u64>,
295 408
        raw: bool,
409 +
        serving: bool,
296 410
    ) -> Result<Bundle> {
297 411
        let mut bundle = Bundle::default();
298 412
        let mut objects: BTreeSet<ObjectRef> = BTreeSet::new();
…
24 unchanged lines
323 437
            for s in &sigs {
324 438
                add_object(&s.key, &mut objects);
325 439
            }
440 +
            // A replacement's new content only: its old content is the new
441 +
            // content of a change it depends on, which the receiver holds or
442 +
            // which this bundle carries (oxygen#136: removing ~1,100 binary
443 +
            // files sent them all again).
326 444
            for e in &c.edits {
327 -
                if let Edit::Replace { old, new, .. } = e {
328 -
                    for b in [old, new].into_iter().flatten() {
329 -
                        if self.store.has(b) {
330 -
                            if let Ok(blob) =
331 -
                                Blob::decode(&self.store.get_typed(b, ObjectType::Blob)?)
332 -
                            {
333 -
                                if max_blob.is_some_and(|m| blob.length > m) {
334 -
                                    continue;
335 -
                                }
336 -
                                objects.insert(*b);
337 -
                                objects.extend(blob.chunks.iter().copied());
445 +
                if let Edit::Replace { new: Some(b), .. } = e {
446 +
                    if self.store.has(b) {
447 +
                        if let Ok(blob) = Blob::decode(&self.store.get_typed(b, ObjectType::Blob)?)
448 +
                        {
449 +
                            if max_blob.is_some_and(|m| blob.length > m) {
450 +
                                continue;
338 451
                            }
452 +
                            objects.insert(*b);
453 +
                            objects.extend(blob.chunks.iter().copied());
339 454
                        }
340 455
                    }
341 456
                }
…
17 unchanged lines
359 474
            }
360 475
            bundle.attests.push(SignedAttest { attest, sigs });
361 476
        }
477 +
        let signers: Vec<KeyId> = if serving {
478 +
            let signers = self.served_attest_signers()?;
479 +
            for k in &signers {
480 +
                add_object(k, &mut objects);
481 +
            }
482 +
            signers.into_iter().collect()
483 +
        } else {
484 +
            Vec::new()
485 +
        };
362 486
        for r in objects {
363 487
            let (ty, body) = self.store.get(&r)?;
364 488
            bundle.objects.push((ty, body));
365 489
        }
490 +
        self.attach_certificates_for(&mut bundle, &signers)?;
366 491
        Ok(bundle)
367 492
    }
368 493
494 +
    /// Add the certificates of short-lived keys that signed what `bundle`
495 +
    /// carries, and their issuers' identities, so a receiver can tell who
496 +
    /// they stand for (spec §14.7). `also` names more signers (a reference
497 +
    /// attestation sent beside the bundle).
498 +
    pub(crate) fn attach_certificates_for(
499 +
        &self,
500 +
        bundle: &mut Bundle,
501 +
        also: &[KeyId],
502 +
    ) -> Result<()> {
503 +
        let mut keys: BTreeSet<KeyId> = also.iter().copied().collect();
504 +
        keys.extend(
505 +
            bundle
506 +
                .changes
507 +
                .iter()
508 +
                .flat_map(|(_, s)| s.iter().map(|s| s.key)),
509 +
        );
510 +
        keys.extend(
511 +
            bundle
512 +
                .raw_changes
513 +
                .iter()
514 +
                .flat_map(|(_, s)| s.iter().map(|s| s.key)),
515 +
        );
516 +
        keys.extend(
517 +
            bundle
518 +
                .attests
519 +
                .iter()
520 +
                .flat_map(|a| a.sigs.iter().map(|s| s.key)),
521 +
        );
522 +
        for k in keys {
523 +
            let Some(r) = self.store.certificate_of(&k)? else {
524 +
                continue;
525 +
            };
526 +
            let cert = self.store.get_certificate(&r)?;
527 +
            if bundle
528 +
                .certificates
529 +
                .iter()
530 +
                .any(|c| c.cert.hash() == cert.cert.hash())
531 +
            {
532 +
                continue;
533 +
            }
534 +
            // An agent's certificate brings its grant's signer too, so the
535 +
            // receiver can check the grant (oxygen#134).
536 +
            let grantor = cert.cert.grant.as_ref().map(|g| g.grant.grantor);
537 +
            for id in [Some(k), Some(cert.cert.issuer), grantor].iter().flatten() {
538 +
                if let Ok(body) = self.store.get_typed(id, ObjectType::Identity) {
539 +
                    if !bundle
540 +
                        .objects
541 +
                        .iter()
542 +
                        .any(|(t, b)| *t == ObjectType::Identity && *b == body)
543 +
                    {
544 +
                        bundle.objects.push((ObjectType::Identity, body));
545 +
                    }
546 +
                }
547 +
            }
548 +
            bundle.certificates.push(cert);
549 +
        }
550 +
        Ok(())
551 +
    }
552 +
553 +
    /// Keep the certificates a bundle brings once each verifies with the
554 +
    /// identity of the issuer it names; one that does not refuses the bundle.
555 +
    fn adopt_certificates(&self, certs: &[crate::cert::SignedCertificate]) -> Result<()> {
556 +
        for c in certs {
557 +
            let issuer = self.store.get_identity(&c.cert.issuer).map_err(|_| {
558 +
                RepoError::Msg(format!(
559 +
                    "a certificate names issuer {}, whose identity did not come with it",
560 +
                    c.cert.issuer.short(12)
561 +
                ))
562 +
            })?;
563 +
            c.verify(&issuer)
564 +
                .map_err(|e| RepoError::Msg(format!("a certificate does not verify: {e}")))?;
565 +
            self.store.put_identity(&c.cert.key)?;
566 +
            self.store.put_certificate(c)?;
567 +
        }
568 +
        Ok(())
569 +
    }
570 +
571 +
    /// A signer authorized for `reference` by what an issuer certified it
572 +
    /// as (spec §14.7): a short-lived key with a kept certificate that
573 +
    /// verifies with its issuer, gives `scope`, covers `at_ms` (a time the
574 +
    /// signer did not choose) and names a subject the policy grants.
575 +
    pub(crate) fn certified_authorized(
576 +
        &self,
577 +
        p: &crate::policy::Policy,
578 +
        signer: &KeyId,
579 +
        reference: &str,
580 +
        scope: Scope,
581 +
        at_ms: u64,
582 +
    ) -> bool {
583 +
        let Ok(Some(r)) = self.store.certificate_of(signer) else {
584 +
            return false;
585 +
        };
586 +
        let Ok(c) = self.store.get_certificate(&r) else {
587 +
            return false;
588 +
        };
589 +
        let Some(issuer) = p
590 +
            .identity(&c.cert.issuer)
591 +
            .cloned()
592 +
            .or_else(|| self.store.get_identity(&c.cert.issuer).ok())
593 +
        else {
594 +
            return false;
595 +
        };
596 +
        c.verify(&issuer).is_ok()
597 +
            && c.cert.key_id() == *signer
598 +
            && c.cert.covers(scope, at_ms)
599 +
            && p.subject_grant(reference, &c.cert.issuer, &c.cert.subject, scope)
600 +
    }
601 +
369 602
    /// Store a bundle's contents. With `strict`, every change must
370 603
    /// authenticate (signature or seal by its author or a delegate).
371 604
    pub fn import_bundle(&self, bundle: &Bundle, strict: bool) -> Result<Vec<ChangeHash>> {
…
43 unchanged lines
415 648
            .map(|(ty, body)| (*ty, body.as_slice()))
416 649
            .collect();
417 650
        self.store.put_many(&objects)?;
651 +
        self.adopt_certificates(&bundle.certificates)?;
418 652
        for sa in &bundle.attests {
419 653
            let r = self.store.put_attest(&sa.attest)?;
420 654
            for sig in &sa.sigs {
…
118 unchanged lines
539 773
        })
540 774
    }
541 775
776 +
    /// A lasting attestation (a tag) signed by a key the policy has since
777 +
    /// rotated away from (spec §14.2) is authorized as the key that
778 +
    /// succeeded it: a key the rule lists reaches the signer through
779 +
    /// `previous` links, and the attestation is dated (`at_ms`) before the
780 +
    /// signer's successor was made, so a superseded key makes nothing new.
781 +
    /// Revoked keys authorize nothing.
782 +
    pub(crate) fn predecessor_authorized(
783 +
        &self,
784 +
        p: &crate::policy::Policy,
785 +
        signer: &KeyId,
786 +
        reference: &str,
787 +
        at_ms: u64,
788 +
    ) -> bool {
789 +
        if p.is_revoked(signer) {
790 +
            return false;
791 +
        }
792 +
        let Some(rule) = p.rule_for(reference) else {
793 +
            return false;
794 +
        };
795 +
        let identity = |k: &KeyId| {
796 +
            p.identity(k)
797 +
                .cloned()
798 +
                .or_else(|| self.store.get_identity(k).ok())
799 +
        };
800 +
        rule.keys.iter().any(|k| {
801 +
            if p.is_revoked(k) {
802 +
                return false;
803 +
            }
804 +
            let mut cur = identity(k);
805 +
            for _ in 0..32 {
806 +
                let Some(id) = cur else {
807 +
                    return false;
808 +
                };
809 +
                match id.previous {
810 +
                    Some(prev) if prev == *signer => return at_ms < id.created,
811 +
                    Some(prev) if !p.is_revoked(&prev) => cur = identity(&prev),
812 +
                    _ => return false,
813 +
                }
814 +
            }
815 +
            false
816 +
        })
817 +
    }
818 +
542 819
    fn check_ref_attest(
543 820
        &self,
544 821
        sa: &SignedAttest,
…
3 unchanged lines
548 825
        last_serial: Option<u64>,
549 826
        allow_equal: bool,
550 827
    ) -> std::result::Result<u64, String> {
828 +
        let serial =
829 +
            self.check_ref_attest_body(sa, name, old_state, new_state, last_serial, allow_equal)?;
830 +
        self.verify_attest_signature(sa)?;
831 +
        Ok(serial)
832 +
    }
833 +
834 +
    /// What [`Repo::check_ref_attest`] checks short of the signature: the
835 +
    /// attestation describes this update of this repository with a serial
836 +
    /// past the last seen.
837 +
    fn check_ref_attest_body(
838 +
        &self,
839 +
        sa: &SignedAttest,
840 +
        name: &str,
841 +
        old_state: &ObjectRef,
842 +
        new_state: &ObjectRef,
843 +
        last_serial: Option<u64>,
844 +
        allow_equal: bool,
845 +
    ) -> std::result::Result<u64, String> {
551 846
        let AttestBody::Ref {
552 847
            name: n,
553 848
            old_state: o,
…
16 unchanged lines
570 865
                ));
571 866
            }
572 867
        }
868 +
        Ok(*serial)
869 +
    }
870 +
871 +
    /// An attestation's first signature verifies with its signer's
872 +
    /// identity, which must be stored here.
873 +
    fn verify_attest_signature(&self, sa: &SignedAttest) -> std::result::Result<(), String> {
573 874
        let r = sa.attest.hash();
574 875
        let sig = sa.sigs.first().ok_or("attestation carries no signature")?;
575 876
        let id = self
576 877
            .store
577 878
            .get_identity(&sig.key)
578 879
            .map_err(|_| format!("unknown key {}", sig.key.short(12)))?;
579 -
        verify_signature(&id, sig, Domain::Attest, &r).map_err(|e| e.to_string())?;
580 -
        Ok(*serial)
880 +
        verify_signature(&id, sig, Domain::Attest, &r).map_err(|e| e.to_string())
581 881
    }
582 882
583 883
    /// Accept a push (spec §16.4). Without a trust policy every repository
…
23 unchanged lines
607 907
        policy: Option<SignedPolicy>,
608 908
        tags: Vec<crate::protocol::TagRef>,
609 909
    ) -> Result<PushReport> {
910 +
        // Updates are checked against the head as it is now, not as this
911 +
        // handle first read it (oxygenhub#249): another push may have
912 +
        // landed since.
913 +
        self.refresh()?;
610 914
        let mut report = PushReport::default();
611 915
        let results = &mut report.results;
612 916
        if let Some(sp) = policy {
…
23 unchanged lines
636 940
            }
637 941
        }
638 942
        let policy = self.current_policy()?.map(|(_, sp)| sp.policy);
943 +
        // A push of objects alone (one batch of a long history, oxygen#95)
944 +
        // names no reference: its outcome is reported as `objects`.
945 +
        let objects_only = updates.is_empty() && tags.is_empty();
639 946
        match self.import_bundle(&bundle, policy.is_some()) {
640 -
            Ok(imported) => report.imported = imported,
947 +
            Ok(imported) => {
948 +
                if objects_only {
949 +
                    results.push(RefResult {
950 +
                        name: OBJECTS_RESULT.into(),
951 +
                        ok: true,
952 +
                        reason: format!("{} changes stored", imported.len()),
953 +
                    });
954 +
                }
955 +
                report.imported = imported;
956 +
            }
641 957
            Err(e) => {
958 +
                if objects_only {
959 +
                    results.push(RefResult {
960 +
                        name: OBJECTS_RESULT.into(),
961 +
                        ok: false,
962 +
                        reason: e.to_string(),
963 +
                    });
964 +
                }
642 965
                for u in updates {
643 966
                    results.push(RefResult {
644 967
                        name: u.name,
…
35 unchanged lines
680 1003
        }
681 1004
        for t in &tags {
682 1005
            let name = format!("tags/{}", t.name);
1006 +
            if t.is_withdrawal() {
1007 +
                match self.adopt_withdrawal(&mut view, t, policy.as_ref(), &mut TagCheck::Received)
1008 +
                {
1009 +
                    Ok(removed) => {
1010 +
                        // A withdrawal is an update to nothing, as a
1011 +
                        // bookmark's deletion is.
1012 +
                        if removed {
1013 +
                            report.updated.push(RefUpdated {
1014 +
                                name: name.clone(),
1015 +
                                old_state: t.state,
1016 +
                                new_state: ObjectRef::ZERO,
1017 +
                                heads: vec![],
1018 +
                            });
1019 +
                        }
1020 +
                        names.push(name.clone());
1021 +
                        results.push(RefResult {
1022 +
                            name,
1023 +
                            ok: true,
1024 +
                            reason: if removed {
1025 +
                                "withdrawn"
1026 +
                            } else {
1027 +
                                "already withdrawn"
1028 +
                            }
1029 +
                            .into(),
1030 +
                        });
1031 +
                    }
1032 +
                    Err(e) => results.push(RefResult {
1033 +
                        name,
1034 +
                        ok: false,
1035 +
                        reason: e.reason,
1036 +
                    }),
1037 +
                }
1038 +
                continue;
1039 +
            }
683 1040
            let new = !view.tags.contains_key(&t.name);
684 -
            match self.adopt_tag(&mut view, t, policy.as_ref()) {
1041 +
            match self.adopt_tag(&mut view, t, policy.as_ref(), &mut TagCheck::Received) {
685 1042
                Ok(()) => {
686 1043
                    // A tag new here is an update too, so a hub can start
687 1044
                    // its pipeline's tag jobs (oxygenflow#24); one it
…
16 unchanged lines
704 1061
                Err(e) => results.push(RefResult {
705 1062
                    name,
706 1063
                    ok: false,
707 -
                    reason: e,
1064 +
                    reason: e.reason,
708 1065
                }),
709 1066
            }
710 1067
        }
…
30 unchanged lines
741 1098
        for sig in &sa.sigs {
742 1099
            self.store.put_signature(&r, sig).map_err(|e| store(&e))?;
743 1100
        }
744 -
        self.set_ref_attest(name, &r).map_err(|e| store(&e))
1101 +
        self.set_ref_attest(name, &r).map_err(|e| store(&e))?;
1102 +
        // Signed by a short-lived key: a host records when it accepted it.
1103 +
        let certified = sa
1104 +
            .sigs
1105 +
            .first()
1106 +
            .is_some_and(|s| self.store.certificate_of(&s.key).ok().flatten().is_some());
1107 +
        if certified {
1108 +
            self.record_acceptance(&r).map_err(|e| store(&e))?;
1109 +
        }
1110 +
        Ok(())
1111 +
    }
1112 +
1113 +
    /// Record that this host accepted attestation `r` now, signed with its
1114 +
    /// acceptor key (spec §14.7); nothing without one.
1115 +
    pub(crate) fn record_acceptance(&self, r: &ObjectRef) -> Result<()> {
1116 +
        let Some(acceptor) = &self.acceptor else {
1117 +
            return Ok(());
1118 +
        };
1119 +
        let (id, sk) = (&acceptor.0, &acceptor.1);
1120 +
        let attest = Attest {
1121 +
            repo: self.id()?,
1122 +
            hlc: Hlc::now(),
1123 +
            body: AttestBody::Accepted { attest: *r },
1124 +
        };
1125 +
        let sig = sk.sign(Domain::Attest, &id.hash(), &attest.hash());
1126 +
        self.store.put_identity(id)?;
1127 +
        let a = self.store.put_attest(&attest)?;
1128 +
        self.store.put_signature(&a, &sig)?;
1129 +
        self.store.index_add_pub("accepted", r, &a)?;
1130 +
        Ok(())
1131 +
    }
1132 +
1133 +
    /// The host's acceptance of attestation `r`, if one was recorded.
1134 +
    pub fn acceptance_of(&self, r: &ObjectRef) -> Result<Option<SignedAttest>> {
1135 +
        let Some(a) = self.store.index_get_pub("accepted", r)?.into_iter().next() else {
1136 +
            return Ok(None);
1137 +
        };
1138 +
        Ok(Some(SignedAttest {
1139 +
            attest: self.store.get_attest(&a)?,
1140 +
            sigs: self.store.signatures(&a)?,
1141 +
        }))
1142 +
    }
1143 +
1144 +
    /// When a certified signer's attestation `sa` is checked on a fetch: the
1145 +
    /// host's acceptance time, if `accepted` is signed by the issuer of the
1146 +
    /// signer's certificate and accepts `sa`; else the attestation's own
1147 +
    /// time, which the signer chose.
1148 +
    pub(crate) fn certified_time(
1149 +
        &self,
1150 +
        signer: &KeyId,
1151 +
        sa: &SignedAttest,
1152 +
        accepted: Option<&SignedAttest>,
1153 +
    ) -> u64 {
1154 +
        self.issuer_acceptance(signer, sa, accepted)
1155 +
            .unwrap_or(sa.attest.hlc.wall_ms)
1156 +
    }
1157 +
1158 +
    /// The time of `accepted` when it accepts exactly `sa` and is signed by
1159 +
    /// the issuer of `signer`'s kept certificate (spec §14.7); else none.
1160 +
    pub(crate) fn issuer_acceptance(
1161 +
        &self,
1162 +
        signer: &KeyId,
1163 +
        sa: &SignedAttest,
1164 +
        accepted: Option<&SignedAttest>,
1165 +
    ) -> Option<u64> {
1166 +
        let acc = accepted?;
1167 +
        let c = self.store.certificate_of(signer).ok().flatten()?;
1168 +
        let cert = self.store.get_certificate(&c).ok()?;
1169 +
        let AttestBody::Accepted { attest } = &acc.attest.body else {
1170 +
            return None;
1171 +
        };
1172 +
        let sig = acc.sigs.iter().find(|s| s.key == cert.cert.issuer)?;
1173 +
        let issuer = self.store.get_identity(&cert.cert.issuer).ok()?;
1174 +
        (*attest == sa.attest.hash()
1175 +
            && verify_signature(&issuer, sig, Domain::Attest, &acc.attest.hash()).is_ok())
1176 +
        .then_some(acc.attest.hlc.wall_ms)
1177 +
    }
1178 +
1179 +
    /// Keep an acceptance this repository was given for attestation `r`
1180 +
    /// (one [`Repo::issuer_acceptance`] took), so it is served again with
1181 +
    /// what it accepts.
1182 +
    pub(crate) fn keep_acceptance(&self, r: &ObjectRef, acc: &SignedAttest) -> Result<()> {
1183 +
        let a = self.store.put_attest(&acc.attest)?;
1184 +
        for sig in &acc.sigs {
1185 +
            self.store.put_signature(&a, sig)?;
1186 +
        }
1187 +
        self.store.index_add_pub("accepted", r, &a)?;
1188 +
        Ok(())
1189 +
    }
1190 +
1191 +
    /// A fetched bookmark's attestation by a signer the policy does not
1192 +
    /// list itself: a delegate or a short-lived key, whose authority
1193 +
    /// depends on when it signed (spec §14.5, §14.8). Checked at the
1194 +
    /// attestation's integrated time when the host's log gives one; else
1195 +
    /// as before, at the host's acceptance for a certified signer, when it
1196 +
    /// countersigned one, or at the attestation's own time, and said so
1197 +
    /// when the host keeps a log. Failing at the integrated time but not
1198 +
    /// as before is a note while logs are new; a large gap between the
1199 +
    /// claimed and the integrated time is a note.
1200 +
    #[allow(clippy::too_many_arguments)]
1201 +
    pub(crate) fn authorize_in_time(
1202 +
        &self,
1203 +
        p: &crate::policy::Policy,
1204 +
        remote_name: &str,
1205 +
        r: &crate::protocol::RemoteRef,
1206 +
        sa: &SignedAttest,
1207 +
        signer: &KeyId,
1208 +
        log: &super::transparency::LogCheck,
1209 +
        notes: &mut Vec<String>,
1210 +
    ) -> Result<()> {
1211 +
        use super::transparency::{gap_note, judge, Timed};
1212 +
        let own = sa.attest.hlc.wall_ms;
1213 +
        let at = self.certified_time(signer, sa, r.accepted.as_ref());
1214 +
        let time = log.time_of(&r.name);
1215 +
        let check = |t_delegate: u64, t_cert: u64| {
1216 +
            self.delegate_authorized(p, signer, &r.name, t_delegate)
1217 +
                || self.certified_authorized(p, signer, &r.name, Scope::REFS, t_cert)
1218 +
        };
1219 +
        let before = if at != own {
1220 +
            "the host's acceptance"
1221 +
        } else {
1222 +
            "the time it claims"
1223 +
        };
1224 +
        let what = format!("{remote_name}/{}", r.name);
1225 +
        match judge(time.as_ref().ok().map(|i| i.ms()), |t| check(t, t), || check(own, at)) {
1226 +
            Timed::Refused => {
1227 +
                return fail(format!(
1228 +
                    "{what}: attested by {}, which the policy does not authorize for this bookmark; a policy owner can grant it with `oxy policy grant`",
1229 +
                    signer.short(12)
1230 +
                ))
1231 +
            }
1232 +
            Timed::Integrated => {}
1233 +
            Timed::Disagrees => notes.push(format!(
1234 +
                "{what}: {}'s authority had lapsed when witness {} first saw its attestation, though not at {before}; accepted while transparency logs are new",
1235 +
                signer.short(12),
1236 +
                time.as_ref().map(|i| i.witness.as_str()).unwrap_or("?")
1237 +
            )),
1238 +
            Timed::Fallback if log.kept => notes.push(format!(
1239 +
                "{what}: {} checked at {before}, not at a witnessed time: {}",
1240 +
                signer.short(12),
1241 +
                time.as_ref().err().map(String::as_str).unwrap_or("none")
1242 +
            )),
1243 +
            Timed::Fallback => {}
1244 +
        }
1245 +
        if let Some(n) = time.ok().and_then(|i| gap_note(own, i)) {
1246 +
            notes.push(format!("{what}'s attestation {n}"));
1247 +
        }
1248 +
        Ok(())
745 1249
    }
746 1250
747 1251
    /// A deletion (spec §16.4): `new_state` is zero. The bookmark must exist
…
33 unchanged lines
781 1285
                .first()
782 1286
                .map(|s| s.key)
783 1287
                .ok_or("attestation carries no signature")?;
1288 +
            // Received now: the receiver's own clock is the trusted time.
1289 +
            let now = Hlc::now().wall_ms;
784 1290
            if !p.authorizes(&signer, &u.name)
785 1291
                && !self.delegate_authorized(p, &signer, &u.name, sa.attest.hlc.wall_ms)
1292 +
                && !self.certified_authorized(p, &signer, &u.name, Scope::REFS, now)
786 1293
            {
787 1294
                return Err(format!(
788 1295
                    "key {} may not delete {:?} under this repository's policy",
…
68 unchanged lines
857 1364
                .first()
858 1365
                .map(|s| s.key)
859 1366
                .ok_or("attestation carries no signature")?;
1367 +
            // Received now: the receiver's own clock is the trusted time.
1368 +
            let now = Hlc::now().wall_ms;
860 1369
            if !p.authorizes(&signer, &u.name)
861 1370
                && !self.delegate_authorized(p, &signer, &u.name, sa.attest.hlc.wall_ms)
1371 +
                && !self.certified_authorized(p, &signer, &u.name, Scope::REFS, now)
862 1372
            {
863 1373
                return Err(format!(
864 1374
                    "key {} may not update {:?} under this repository's policy",
…
19 unchanged lines
884 1394
885 1395
    fn remote_url(&self, name: &str) -> Result<String> {
886 1396
        self.config.remotes.get(name).cloned().ok_or_else(|| {
887 -
            RepoError::Msg(format!(
888 -
                "no remote named {name:?}; add one with `oxy remote add`"
889 -
            ))
1397 +
            let names: Vec<&str> = self.config.remotes.keys().map(String::as_str).collect();
1398 +
            RepoError::Msg(match names.as_slice() {
1399 +
                [] => format!("no remote named {name:?}; add one with `oxy remote add`"),
1400 +
                _ if name == "origin" && names.len() > 1 && self.config.default_remote.is_none() => format!(
1401 +
                    "this repository has no remote named origin, and several others ({}): name one (`oxy pull {}`), or make one the default with `oxy config core.remote {}`",
1402 +
                    names.join(", "),
1403 +
                    names[0],
1404 +
                    names[0]
1405 +
                ),
1406 +
                _ => format!(
1407 +
                    "no remote named {name:?}; this repository has {}",
1408 +
                    names.join(", ")
1409 +
                ),
1410 +
            })
890 1411
        })
891 1412
    }
892 1413
…
33 unchanged lines
926 1447
            token: self.config.remote_tokens.get(name).cloned(),
927 1448
            ca: self.config.remote_cas.get(name).map(PathBuf::from),
928 1449
            cache_dir: Some(self.oxy.join("remotes").join(name).join("strata")),
1450 +
            ..RemoteOptions::default()
929 1451
        };
930 1452
        self.fill_credentials(&url, &mut options);
931 1453
        options
…
13 unchanged lines
945 1467
    /// Fill in what `options` lacks for `url`: `$OXY_TOKEN` and `$OXY_CA`,
946 1468
    /// then the user's credential store for its hub, then a remote on the
947 1469
    /// same hub in an enclosing checkout (a subrepo borrows its parent's).
1470 +
    /// When the token is not this remote's own or the environment's, the
1471 +
    /// other tokens enclosing checkouts hold for the hub are kept to try
1472 +
    /// if it is refused: a stale stored login must not hide a parent's
1473 +
    /// good token (oxygen#112).
948 1474
    fn fill_credentials(&self, url: &str, options: &mut RemoteOptions) {
949 1475
        if options.token.is_some() && options.ca.is_some() {
950 1476
            return;
951 1477
        }
1478 +
        let own = options.token.is_some();
1479 +
        let env = std::env::var("OXY_TOKEN").is_ok_and(|t| !t.is_empty());
1480 +
        let explicit = own || env;
952 1481
        let stored = self.credentials().options_for(url);
1482 +
        // A checkout's own token refused as expired or unknown (401) falls
1483 +
        // back to this computer's sign-in for the hub, never to another
1484 +
        // checkout's, and never when it is refused for permission (#155).
1485 +
        if own && !env {
1486 +
            if let Some(t) = stored
1487 +
                .token
1488 +
                .clone()
1489 +
                .filter(|t| options.token.as_ref() != Some(t))
1490 +
            {
1491 +
                options.fallback_tokens.push(t);
1492 +
                options.fallback_on_401_only = true;
1493 +
            }
1494 +
        }
953 1495
        options.token = options.token.take().or(stored.token);
954 1496
        options.ca = options.ca.take().or(stored.ca);
955 -
        if options.token.is_some() && options.ca.is_some() {
956 -
            return;
1497 +
        let enclosing = self.enclosing_credentials(url);
1498 +
        if options.token.is_none() {
1499 +
            options.token = enclosing.iter().find_map(|(t, _)| t.clone());
1500 +
        }
1501 +
        if options.ca.is_none() {
1502 +
            options.ca = enclosing.iter().find_map(|(_, c)| c.clone());
1503 +
        }
1504 +
        if !explicit {
1505 +
            for t in enclosing.into_iter().filter_map(|(t, _)| t) {
1506 +
                if options.token.as_ref() != Some(&t) && !options.fallback_tokens.contains(&t) {
1507 +
                    options.fallback_tokens.push(t);
1508 +
                }
1509 +
            }
957 1510
        }
1511 +
    }
1512 +
1513 +
    /// What to do when `remote`'s own token was refused (`message` says
1514 +
    /// HTTP 401) and so was this computer's sign-in for its hub, which it
1515 +
    /// falls back to (oxygen#155).
1516 +
    pub fn stale_token_note(&self, remote: &str, message: &str) -> Option<String> {
1517 +
        if !message.contains("HTTP 401") {
1518 +
            return None;
1519 +
        }
1520 +
        let own = self.config.remote_tokens.get(remote)?;
1521 +
        let url = self.config.remotes.get(remote)?;
1522 +
        let stored = self.credentials().options_for(url).token?;
1523 +
        (stored != *own).then(|| {
1524 +
            format!(
1525 +
                "the hub refused both this checkout's own token for {remote} (remote.{remote}.token) and your sign-in: \
1526 +
                 `oxy auth login HUB`, then `oxy config --unset remote.{remote}.token` so the sign-in is used"
1527 +
            )
1528 +
        })
1529 +
    }
1530 +
1531 +
    /// The tokens and CAs of remotes on `url`'s hub in enclosing
1532 +
    /// checkouts, nearest first.
1533 +
    fn enclosing_credentials(&self, url: &str) -> Vec<(Option<String>, Option<PathBuf>)> {
1534 +
        let mut out = Vec::new();
958 1535
        let Some(hub) = crate::credentials::hub_of(url) else {
959 -
            return;
1536 +
            return out;
960 1537
        };
961 1538
        let mut at = self.root.parent();
962 1539
        while let Some(dir) = at {
…
10 unchanged lines
973 1550
                if crate::credentials::hub_of(remote_url).as_deref() != Some(hub.as_str()) {
974 1551
                    continue;
975 1552
                }
976 -
                if options.token.is_none() {
977 -
                    options.token = cfg.get(&format!("remote.{name}.token")).cloned();
978 -
                }
979 -
                if options.ca.is_none() {
980 -
                    options.ca = cfg.get(&format!("remote.{name}.ca")).map(PathBuf::from);
981 -
                }
982 -
            }
983 -
            if options.token.is_some() && options.ca.is_some() {
984 -
                return;
1553 +
                out.push((
1554 +
                    cfg.get(&format!("remote.{name}.token")).cloned(),
1555 +
                    cfg.get(&format!("remote.{name}.ca")).map(PathBuf::from),
1556 +
                ));
985 1557
            }
986 1558
            at = dir.parent();
987 1559
        }
1560 +
        out
988 1561
    }
989 1562
990 1563
    /// Push seal attestations on their own (hub spec §7.4): a bundle with
…
51 unchanged lines
1042 1615
        }
1043 1616
    }
1044 1617
1045 -
    fn open_named_remote(&self, name: &str) -> Result<Box<dyn crate::protocol::Remote>> {
1618 +
    pub(crate) fn open_named_remote(&self, name: &str) -> Result<Box<dyn crate::protocol::Remote>> {
1046 1619
        open_remote(&self.remote_url(name)?, &self.remote_options(name))
1047 1620
    }
1048 1621
…
112 unchanged lines
1161 1734
    pub fn fetch(&self, remote_name: &str) -> Result<FetchReport> {
1162 1735
        let remote = self.open_named_remote(remote_name)?;
1163 1736
        self.check_remote_id(remote.as_ref(), remote_name, false)?;
1164 -
        let resp = remote.refs()?;
1737 +
        let resp = {
1738 +
            let _t = crate::timing::phase("fetch: refs");
1739 +
            remote.refs()?
1740 +
        };
1741 +
        // The host's transparency log (spec §14.8): a fork fails here.
1742 +
        let log_phase = crate::timing::phase("fetch: transparency log");
1743 +
        let log = self.check_log(
1744 +
            &self.remote_url(remote_name)?,
1745 +
            self.config.remote_cas.get(remote_name).map(Path::new),
1746 +
            &resp,
1747 +
        )?;
1748 +
        drop(log_phase);
1749 +
        let mut log_notes = log.notes.clone();
1165 1750
        // Trust policy (spec §14.5). Adopt the remote's policy when it is newer
1166 1751
        // and signed by roots we trust, or on first contact. A remote that
1167 1752
        // once presented a policy may never present none or an older one:
…
44 unchanged lines
1212 1797
        let mut view = op.view.clone();
1213 1798
        let have = self.known_heads(&view)?;
1214 1799
        let mut report = Vec::new();
1800 +
        let mut skipped = Vec::new();
1215 1801
        let max_blob = self.config.max_blob;
1216 -
        for r in resp.refs {
1802 +
        // One negotiation and one bundle for every bookmark's heads at
1803 +
        // once (oxygen#158): a bookmark at a time cost a round trip each and
1804 +
        // sent again the changes bookmarks share, as `have` did not grow
1805 +
        // between them (160 bookmarks: 160 bundles, 49k changes encoded for
1806 +
        // a few hundred). Heads already here, with every attestation's
1807 +
        // signer known, need no round trip at all.
1808 +
        let mut wanted: Vec<ChangeHash> = resp
1809 +
            .refs
1810 +
            .iter()
1811 +
            .flat_map(|r| r.heads.iter().copied())
1812 +
            .filter(|h| !self.store.has(h))
1813 +
            .collect();
1814 +
        wanted.sort();
1815 +
        wanted.dedup();
1816 +
        // A bundle also carries the identities and certificates of the keys
1817 +
        // that signed the remote's attestations (spec §16.2): with no change
1818 +
        // wanted, ask anyway when one of those signers is unknown here.
1819 +
        let unknown_signer = resp
1820 +
            .refs
1821 +
            .iter()
1822 +
            .filter_map(|r| r.attestation.as_ref())
1823 +
            .chain(resp.tags.iter().filter_map(|t| t.attestation.as_ref()))
1824 +
            .flat_map(|a| a.sigs.iter().map(|s| s.key))
1825 +
            .any(|k| {
1826 +
                self.store.get_identity(&k).is_err()
1827 +
                    && !matches!(self.store.certificate_of(&k), Ok(Some(_)))
1828 +
            });
1829 +
        let shallow = self.is_shallow();
1830 +
        let imported: Vec<ChangeHash> = if wanted.is_empty() && !unknown_signer {
1831 +
            Vec::new()
1832 +
        } else {
1217 1833
            let needed = {
1218 1834
                let _t = crate::timing::phase("fetch: negotiate");
1219 -
                remote.negotiate(&have, &r.heads)?
1835 +
                remote.negotiate(&have, &wanted)?
1220 1836
            };
1221 1837
            let bundle = {
1222 1838
                let _t = crate::timing::phase("fetch: bundle");
1223 1839
                remote.fetch(&needed, max_blob)?
1224 1840
            };
1225 -
            // Changes are not individually re-verified here: the bookmark's
1841 +
            // Changes are not individually re-verified here: each bookmark's
1226 1842
            // attestation (checked below) binds its whole state, and the
1227 1843
            // receiver of the push verified the added changes at that time.
1228 -
            let imported = {
1229 -
                let _t = crate::timing::phase("fetch: import");
1844 +
            // A shallow repository takes changes whose dependencies it never
1845 +
            // held (below what it cloned), as its clone did (#137).
1846 +
            let _t = crate::timing::phase("fetch: import");
1847 +
            if shallow {
1848 +
                self.import_bundle_shallow(&bundle, false)?
1849 +
            } else {
1230 1850
                self.import_bundle(&bundle, false)?
1231 -
            };
1232 -
            if max_blob.is_some() {
1233 -
                self.register_missing_blobs(&imported, remote_name)?;
1851 +
        };
1852 +
        if shallow {
1853 +
            self.refresh_boundary(&imported)?;
1854 +
        }
1855 +
        if max_blob.is_some() {
1856 +
            self.register_missing_blobs(&imported, remote_name)?;
1857 +
        }
1858 +
        let boundary = if shallow {
1859 +
            self.boundary()
1860 +
        } else {
1861 +
            BTreeSet::new()
1862 +
        };
1863 +
        // Each bookmark's new changes: those just imported that its heads
1864 +
        // reach, for the report.
1865 +
        let mut new_deps: HashMap<ChangeHash, Vec<ChangeHash>> = HashMap::new();
1866 +
        for h in &imported {
1867 +
            if let Ok(m) = self.store.get_change_meta(h) {
1868 +
                new_deps.insert(*h, m.deps);
1869 +
            }
1870 +
        }
1871 +
        let new_in = |heads: &[ChangeHash]| -> usize {
1872 +
            let mut seen: HashSet<ChangeHash> = HashSet::new();
1873 +
            let mut stack: Vec<ChangeHash> = heads
1874 +
                .iter()
1875 +
                .filter(|h| new_deps.contains_key(*h))
1876 +
                .copied()
1877 +
                .collect();
1878 +
            while let Some(h) = stack.pop() {
1879 +
                if seen.insert(h) {
1880 +
                    stack.extend(
1881 +
                        new_deps[&h]
1882 +
                            .iter()
1883 +
                            .filter(|d| new_deps.contains_key(*d) && !seen.contains(*d)),
1884 +
                    );
1885 +
                }
1886 +
            }
1887 +
            seen.len()
1888 +
        };
1889 +
        let checks = crate::timing::phase("fetch: check each bookmark");
1890 +
        for r in resp.refs {
1234 1891
            }
1235 1892
            for h in &r.heads {
1236 -
                if !self.store.has(h) {
1893 +
                if !self.store.has(h) && !boundary.contains(h) {
1237 1894
                    return fail(format!("remote did not send head {}", h.short(12)));
1238 1895
                }
1239 1896
            }
…
16 unchanged lines
1256 1913
                    .first()
1257 1914
                    .map(|s| s.key)
1258 1915
                    .ok_or_else(|| RepoError::Msg("attestation carries no signature".into()))?;
1259 -
                if !p.authorizes(&signer, &r.name)
1260 -
                    && !self.delegate_authorized(p, &signer, &r.name, sa.attest.hlc.wall_ms)
1261 -
                {
1262 -
                    return fail(format!(
1263 -
                        "{remote_name}/{}: attested by {}, which the policy does not authorize for this bookmark; a policy owner can grant it with `oxy policy grant`",
1264 -
                        r.name,
1265 -
                        signer.short(12)
1266 -
                    ));
1916 +
                if !p.authorizes(&signer, &r.name) {
1917 +
                    self.authorize_in_time(p, remote_name, &r, sa, &signer, &log, &mut log_notes)?;
1267 1918
                }
1268 1919
            }
1269 1920
            if let Some(sa) = &r.attestation {
…
7 unchanged lines
1277 1928
                };
1278 1929
                let (old_state, new_state) = (*old_state, *new_state);
1279 1930
                let serial = self
1280 -
                    .check_ref_attest(
1931 +
                    .check_ref_attest_body(
1281 1932
                        sa,
1282 1933
                        &r.name,
1283 1934
                        &old_state,
…
10 unchanged lines
1294 1945
                        remote_name, r.name
1295 1946
                    ));
1296 1947
                }
1948 +
                // A signature this client cannot verify (its signer unknown
1949 +
                // here, as from a host that did not send the identity, or
1950 +
                // bad) fails that bookmark alone: it stays as it was and
1951 +
                // the rest are fetched (oxygenhub#266). The current
1952 +
                // bookmark, which a pull would take, still fails the fetch.
1953 +
                // A policy's requirements were checked above and still fail.
1954 +
                if let Err(e) = self.verify_attest_signature(sa) {
1955 +
                    let why = format!("attestation for {remote_name}/{}: {e}", r.name);
1956 +
                    if r.name == self.config.bookmark {
1957 +
                        return fail(why);
1958 +
                    }
1959 +
                    skipped.push((r.name, why));
1960 +
                    continue;
1961 +
                }
1297 1962
                let ar = self.store.put_attest(&sa.attest)?;
1298 1963
                for sig in &sa.sigs {
1299 1964
                    self.store.put_signature(&ar, sig)?;
…
4 unchanged lines
1304 1969
                .entry(remote_name.to_string())
1305 1970
                .or_default()
1306 1971
                .insert(r.name.clone(), state);
1307 -
            report.push((r.name, r.heads, imported.len()));
1972 +
            let n = new_in(&r.heads);
1973 +
            report.push((r.name, r.heads, n));
1308 1974
        }
1975 +
        drop(checks);
1976 +
        let tags_phase = crate::timing::phase("fetch: tags");
1309 1977
        let mut new_tags = 0;
1310 1978
        for t in &resp.tags {
1311 1979
            if view.tags.contains_key(&t.name) {
1312 1980
                continue;
1313 1981
            }
1982 +
            // Withdrawn here: the name is not used again.
1983 +
            if self.tag_withdrawal(&t.name)?.is_some() {
1984 +
                continue;
1985 +
            }
1314 1986
            // The tag's changes must be here: fetch them if the bookmarks did not bring them.
1315 1987
            let missing: Vec<ChangeHash> = t
1316 1988
                .heads
…
7 unchanged lines
1324 1996
                self.import_bundle(&bundle, false)?;
1325 1997
            }
1326 1998
            self.store.put_state(&StateObject::new(t.heads.clone()))?;
1327 -
            self.adopt_tag(&mut view, t, policy.as_ref())
1328 -
                .map_err(|e| RepoError::Msg(format!("{remote_name} {}: {e}", t.name)))?;
1329 -
            new_tags += 1;
1999 +
            let mut check = TagCheck::Fetched {
2000 +
                remote: remote_name,
2001 +
                log: &log,
2002 +
                notes: &mut log_notes,
2003 +
            };
2004 +
            match self.adopt_tag(&mut view, t, policy.as_ref(), &mut check) {
2005 +
                Ok(()) => new_tags += 1,
2006 +
                // Nothing says when a short-lived key signed it: the tag is
2007 +
                // left out, said so, and the rest is fetched (oxygen#129).
2008 +
                Err(e) if e.untimed => {
2009 +
                    skipped.push((format!("tags/{}", t.name), e.reason));
2010 +
                }
2011 +
                Err(e) => return fail(format!("{remote_name} {}: {}", t.name, e.reason)),
2012 +
            }
2013 +
        }
2014 +
        // The remote withdrew a tag this repository has: so does this one,
2015 +
        // once the withdrawal is checked as a push's would be; one that is
2016 +
        // not is left as it is, said so (oxygenhub#301).
2017 +
        drop(tags_phase);
2018 +
        let mut withdrawn_tags = Vec::new();
2019 +
        for t in &resp.withdrawn {
2020 +
            if !view.tags.contains_key(&t.name) {
2021 +
                continue;
2022 +
            }
2023 +
            let mut check = TagCheck::Fetched {
2024 +
                remote: remote_name,
2025 +
                log: &log,
2026 +
                notes: &mut log_notes,
2027 +
            };
2028 +
            match self.adopt_withdrawal(&mut view, t, policy.as_ref(), &mut check) {
2029 +
                Ok(true) => withdrawn_tags.push(t.name.clone()),
2030 +
                Ok(false) => {}
2031 +
                Err(e) => skipped.push((
2032 +
                    format!("tags/{}", t.name),
2033 +
                    format!("withdrawn on {remote_name}, kept here: {}", e.reason),
2034 +
                )),
2035 +
            }
1330 2036
        }
1331 2037
        if view != op.view {
1332 2038
            self.record_op(&[op_ref], &format!("oxy fetch {remote_name}"), view)?;
…
14 unchanged lines
1347 2053
            refs: report,
1348 2054
            unverified,
1349 2055
            new_tags,
2056 +
            withdrawn_tags,
2057 +
            log_notes,
2058 +
            skipped,
1350 2059
        })
1351 2060
    }
1352 2061
…
15 unchanged lines
1368 2077
        if r.heads.is_empty() {
1369 2078
            return Ok(0);
1370 2079
        }
1371 -
        // Base: the remote's checkpoint of exactly these heads.
1372 -
        let cp_bundle = remote.checkpoint(&r.heads)?;
1373 -
        let mut cp_ref = None;
1374 -
        for (ty, body) in &cp_bundle.objects {
1375 -
            let put = self.store.put(*ty, body)?;
1376 -
            if *ty == ObjectType::Checkpoint {
1377 -
                cp_ref = Some(put);
1378 -
            }
1379 -
        }
2080 +
        // Base: the remote's checkpoint of exactly these heads, without its
2081 +
        // graph blobs, which come in parts below (#137).
2082 +
        let cp_bundle = {
2083 +
            let _t = crate::timing::phase("shallow: checkpoint request");
2084 +
            remote.checkpoint_index(&r.heads)?
2085 +
        };
2086 +
        // In one batch: a big state's checkpoint is tens of thousands of
2087 +
        // graph blobs and chunks, which stored one by one were each written
2088 +
        // loose and flushed (a VS Code clone ran at 27 objects a second).
2089 +
        let objects: Vec<(ObjectType, &[u8])> = cp_bundle
2090 +
            .objects
2091 +
            .iter()
2092 +
            .map(|(ty, body)| (*ty, body.as_slice()))
2093 +
            .collect();
2094 +
        let t_store = crate::timing::phase("shallow: store the checkpoint");
2095 +
        let batch = self.store.begin_batch()?;
2096 +
        let stored = self.store.put_many(&objects);
2097 +
        batch.finish()?;
2098 +
        drop(t_store);
2099 +
        let cp_ref = objects
2100 +
            .iter()
2101 +
            .zip(stored?)
2102 +
            .find(|((ty, _), _)| *ty == ObjectType::Checkpoint)
2103 +
            .map(|(_, r)| r);
1380 2104
        let Some(cp_ref) = cp_ref else {
1381 2105
            return fail(format!("{remote_name} sent no checkpoint"));
1382 2106
        };
2107 +
        // The graph blobs the payload names, with their chunks, a part at
2108 +
        // a time (#137): one answer for a big state was gigabytes, held
2109 +
        // whole on both sides. A remote that sent the whole checkpoint has
2110 +
        // left none to ask for.
2111 +
        let cp = crate::objects::Checkpoint::decode(
2112 +
            &self.store.get_typed(&cp_ref, ObjectType::Checkpoint)?,
2113 +
        )?;
2114 +
        let graphs: Vec<ObjectRef> = self
2115 +
            .checkpoint_graph_blobs(&cp)?
2116 +
            .into_iter()
2117 +
            .filter(|b| !self.store.has(b))
2118 +
            .collect();
2119 +
        {
2120 +
            let _t = crate::timing::phase("shallow: the checkpoint's graphs, in parts");
2121 +
            for part in graphs.chunks(CHECKPOINT_PART) {
2122 +
                let objects = remote.objects(part)?;
2123 +
                let objects: Vec<(ObjectType, &[u8])> = objects
2124 +
                    .iter()
2125 +
                    .filter(|(ty, _)| matches!(ty, ObjectType::Blob | ObjectType::Chunk))
2126 +
                    .map(|(ty, body)| (*ty, body.as_slice()))
2127 +
                    .collect();
2128 +
                let batch = self.store.begin_batch()?;
2129 +
                let stored = self.store.put_many(&objects);
2130 +
                batch.finish()?;
2131 +
                stored?;
2132 +
            }
2133 +
            if let Some(b) = graphs.iter().find(|b| !self.store.has(b)) {
2134 +
                return fail(format!(
2135 +
                    "{remote_name} did not send graph blob {}",
2136 +
                    b.short(12)
2137 +
                ));
2138 +
            }
2139 +
        }
1383 2140
        self.install_checkpoint(&bookmark, &cp_ref)?;
1384 -
        // History window.
1385 -
        let wanted = remote.shallow(&r.heads, depth)?;
1386 -
        let bundle = remote.fetch(&wanted, self.config.max_blob)?;
1387 -
        let imported = self.import_bundle_shallow(&bundle, false)?;
2141 +
        let _t_blobs = crate::timing::phase("shallow: the binary files' blobs");
2142 +
        // The blobs of the state's opaque files come with no change when the
2143 +
        // window holds none of the changes that set them (#137): fetch them
2144 +
        // now, or under `filter.max-blob` leave them to be fetched on read.
2145 +
        let base = State::from_checkpoint_history(&self.read_blob(&cp.payload)?)?;
2146 +
        let mut missing: Vec<ObjectRef> = base
2147 +
            .opaque_nodes()
2148 +
            .iter()
2149 +
            .flat_map(|n| base.live_blobs(n))
2150 +
            .filter(|b| !self.store.has(b))
2151 +
            .collect();
2152 +
        missing.sort();
2153 +
        missing.dedup();
2154 +
        if self.config.max_blob.is_some() {
2155 +
            for b in &missing {
2156 +
                self.register_lazy(b, remote_name)?;
2157 +
            }
2158 +
        } else {
2159 +
            for part in missing.chunks(512) {
2160 +
                let objects = remote.objects(part)?;
2161 +
                let objects: Vec<(ObjectType, &[u8])> = objects
2162 +
                    .iter()
2163 +
                    .filter(|(ty, _)| matches!(ty, ObjectType::Blob | ObjectType::Chunk))
2164 +
                    .map(|(ty, body)| (*ty, body.as_slice()))
2165 +
                    .collect();
2166 +
                let batch = self.store.begin_batch()?;
2167 +
                let stored = self.store.put_many(&objects);
2168 +
                batch.finish()?;
2169 +
                stored?;
2170 +
            }
2171 +
            if let Some(b) = missing.iter().find(|b| !self.store.has(b)) {
2172 +
                return fail(format!("{remote_name} did not send blob {}", b.short(12)));
2173 +
            }
2174 +
        }
2175 +
        // History window. Depth 0 takes none (#137): the checkpoint holds
2176 +
        // every change's metadata, and its heads become the boundary, so a
2177 +
        // state of many heads (33,835 in the VS Code import) costs nothing
2178 +
        // to fetch.
2179 +
        let imported = if depth == 0 {
2180 +
            let mut boundary = self.boundary();
2181 +
            boundary.extend(r.heads.iter().filter(|h| !self.store.has(h)).copied());
2182 +
            self.write_boundary(&boundary)?;
2183 +
            Vec::new()
2184 +
        } else {
2185 +
            let wanted = remote.shallow(&r.heads, depth)?;
2186 +
            let bundle = remote.fetch(&wanted, self.config.max_blob)?;
2187 +
            self.import_bundle_shallow(&bundle, false)?
2188 +
        };
1388 2189
        if self.config.max_blob.is_some() {
1389 2190
            self.register_missing_blobs(&imported, remote_name)?;
1390 2191
        }
…
26 unchanged lines
1417 2218
    /// remote's: no merge change, conflicts become reified (spec §9.4).
1418 2219
    pub fn pull(&self, remote_name: &str) -> Result<PullReport> {
1419 2220
        let _wc = self.lock_working_copy()?;
1420 -
        self.fetch(remote_name)?;
2221 +
        let skipped = self.fetch(remote_name)?.skipped;
1421 2222
        let (op_ref, op) = self.head_op()?;
1422 2223
        let current = self.config.bookmark.clone();
1423 2224
        let mut local_current = self.heads_of(&op.view, &current)?;
…
44 unchanged lines
1468 2269
                others.push(name);
1469 2270
            }
1470 2271
        }
1471 -
        if !changed && others.is_empty() {
1472 -
            return Ok(PullReport {
1473 -
                heads: local_current,
1474 -
                changed: false,
1475 -
                checked_out: false,
1476 -
                others,
1477 -
            });
2272 +
        if changed || !others.is_empty() {
2273 +
            self.record_op(&[op_ref], &format!("oxy pull {remote_name}"), view)?;
1478 2274
        }
1479 -
        self.record_op(&[op_ref], &format!("oxy pull {remote_name}"), view)?;
1480 -
        let mut checked_out = false;
1481 -
        if changed {
1482 -
            // Follow along if the working copy was clean and on the old heads.
1483 -
            let snap = self.snapshot("oxy pull")?;
1484 -
            let clean = snap.wc.tree_ops.is_empty() && snap.wc.edits.is_empty();
1485 -
            let mut base = snap.base.maximal(&snap.wc.deps);
1486 -
            base.sort();
1487 -
            if clean && base == local_current {
1488 -
                self.checkout(&current_heads, false, "oxy pull (checkout)")?;
1489 -
                checked_out = true;
1490 -
            }
1491 -
        }
2275 +
        // The working copy follows its bookmark whenever nothing would be
2276 +
        // lost, also when this pull brought nothing new but it had been
2277 +
        // left behind (oxygen#89).
2278 +
        let checked_out = self.follow_bookmark(&current_heads, &current, "oxy pull")?;
1492 2279
        Ok(PullReport {
1493 -
            heads: current_heads,
2280 +
            heads: if changed {
2281 +
                current_heads
2282 +
            } else {
2283 +
                local_current
2284 +
            },
1494 2285
            changed,
1495 2286
            checked_out,
1496 2287
            others,
2288 +
            skipped,
1497 2289
        })
1498 2290
    }
1499 2291
2292 +
    /// How many changes the working copy's bookmark has that its base
2293 +
    /// lacks, when the base is part of the bookmark's state (it fell behind,
2294 +
    /// and `oxy checkout` or `oxy pull` brings it up); `None` when it is on
2295 +
    /// the bookmark or holds changes the bookmark lacks. History is read
2296 +
    /// only when the base and the bookmark's heads differ (oxygen#89).
2297 +
    pub fn working_copy_behind(&self) -> Result<Option<usize>> {
2298 +
        let (_, wc) = self.working_copy()?;
2299 +
        let mut base = wc.deps.clone();
2300 +
        base.sort();
2301 +
        let mut heads = self.bookmark_heads()?;
2302 +
        heads.sort();
2303 +
        if base == heads || heads.is_empty() {
2304 +
            return Ok(None);
2305 +
        }
2306 +
        let state = self.load_state_history(&heads)?;
2307 +
        if !base.iter().all(|h| state.contains(h)) {
2308 +
            return Ok(None);
2309 +
        }
2310 +
        let had = if base.is_empty() {
2311 +
            0
2312 +
        } else {
2313 +
            self.load_state_history(&base)?.len()
2314 +
        };
2315 +
        Ok(Some(state.len().saturating_sub(had)).filter(|n| *n > 0))
2316 +
    }
2317 +
2318 +
    /// Bring the working copy to `heads`, the state of its bookmark
2319 +
    /// `bookmark`, when nothing would be lost: its base is part of that
2320 +
    /// state, so the bookmark only adds to it. Clean, it is checked out;
2321 +
    /// with uncommitted work, the work comes along when the new changes
2322 +
    /// leave its paths alone (as `git pull` does), else it stays. Before,
2323 +
    /// it followed only from exactly the bookmark's old heads, so a working
2324 +
    /// copy already behind (the bookmark moved by `bookmark set`, a landing
2325 +
    /// or a pull from another workspace) never caught up, and nothing said
2326 +
    /// so (oxygen#74, #89). Returns whether it moved.
2327 +
    pub fn follow_bookmark(
2328 +
        &self,
2329 +
        heads: &[ChangeHash],
2330 +
        bookmark: &str,
2331 +
        command: &str,
2332 +
    ) -> Result<bool> {
2333 +
        let snap = self.snapshot(command)?;
2334 +
        let clean = snap.wc.tree_ops.is_empty() && snap.wc.edits.is_empty();
2335 +
        let mut base = snap.base.maximal(&snap.wc.deps);
2336 +
        base.sort();
2337 +
        let mut target = heads.to_vec();
2338 +
        target.sort();
2339 +
        if base == target {
2340 +
            return Ok(false);
2341 +
        }
2342 +
        let state = self.load_state_history(heads)?;
2343 +
        if !base.iter().all(|h| state.contains(h)) {
2344 +
            // The working copy holds changes the bookmark lacks: not ours to move.
2345 +
            return Ok(false);
2346 +
        }
2347 +
        if clean {
2348 +
            self.checkout(heads, false, &format!("{command} (checkout)"))?;
2349 +
            Ok(true)
2350 +
        } else {
2351 +
            Ok(self
2352 +
                .checkout_carry(heads, bookmark, &format!("{command} (checkout --carry)"))
2353 +
                .is_ok())
2354 +
        }
2355 +
    }
2356 +
2357 +
    /// Withdraw tag `tag` on `remote_name` (`oxy push --delete tags/NAME`,
2358 +
    /// oxygenhub#301): withdrawn here first when it still is a tag here
2359 +
    /// (`oxy tag delete`), then the withdrawal alone is sent, with its
2360 +
    /// signer's identity and certificate.
2361 +
    pub fn push_tag_withdrawal(&self, remote_name: &str, tag: &str) -> Result<Vec<RefResult>> {
2362 +
        let remote = self.open_named_remote(remote_name)?;
2363 +
        self.check_remote_id(remote.as_ref(), remote_name, true)?;
2364 +
        let resp = remote.refs()?;
2365 +
        let name = format!("tags/{tag}");
2366 +
        if resp.withdrawn.iter().any(|t| t.name == tag) {
2367 +
            return Ok(vec![RefResult {
2368 +
                name,
2369 +
                ok: true,
2370 +
                reason: "already withdrawn".into(),
2371 +
            }]);
2372 +
        }
2373 +
        let Some(there) = resp.tags.iter().find(|t| t.name == tag) else {
2374 +
            return fail(format!("{remote_name} has no tag {tag}"));
2375 +
        };
2376 +
        if resp.policy.is_some() && self.secret_key()?.is_none() {
2377 +
            return fail(format!("{remote_name} enforces a trust policy; withdrawing a tag needs an identity (`oxy identity create`)"));
2378 +
        }
2379 +
        if self.tag_withdrawal(tag)?.is_none() {
2380 +
            match self.view()?.tags.get(tag) {
2381 +
                Some(st) if *st == there.state => {
2382 +
                    self.tag_delete(tag, "")?;
2383 +
                }
2384 +
                Some(_) => {
2385 +
                    return fail(format!(
2386 +
                        "tag {tag} names another state here than on {remote_name}"
2387 +
                    ))
2388 +
                }
2389 +
                None => {
2390 +
                    return fail(format!(
2391 +
                        "there is no tag {tag} here to withdraw: `oxy fetch {remote_name}` first"
2392 +
                    ))
2393 +
                }
2394 +
            }
2395 +
        }
2396 +
        let Some(w) = self
2397 +
            .withdrawn_refs()?
2398 +
            .into_iter()
2399 +
            .find(|w| w.name == tag && w.state == there.state)
2400 +
        else {
2401 +
            return fail(format!(
2402 +
                "tag {tag} was withdrawn here at another state than {remote_name}'s"
2403 +
            ));
2404 +
        };
2405 +
        let mut bundle = Bundle::default();
2406 +
        let signers: Vec<KeyId> = w
2407 +
            .attestation
2408 +
            .iter()
2409 +
            .flat_map(|a| a.sigs.iter().map(|s| s.key))
2410 +
            .collect();
2411 +
        for k in &signers {
2412 +
            if let Ok(id) = self.identity(k) {
2413 +
                bundle.objects.push((ObjectType::Identity, id.encode()));
2414 +
            }
2415 +
        }
2416 +
        self.attach_certificates_for(&mut bundle, &signers)?;
2417 +
        remote.push(bundle, vec![], None, vec![w])
2418 +
    }
2419 +
1500 2420
    /// Push the current bookmark (spec §16.3), sealing this author's unsigned
1501 2421
    /// changes and attesting the reference when a key is available.
1502 2422
    /// Delete `bookmark` on a remote (spec §16.4): an update to the zero
1503 2423
    /// state, attested when we have an identity. The local bookmark, if
1504 2424
    /// any, is kept; only the remote-tracking entry goes.
1505 2425
    pub fn push_delete(&self, remote_name: &str, bookmark: &str) -> Result<Vec<RefResult>> {
2426 +
        if let Some(tag) = bookmark.strip_prefix("tags/") {
2427 +
            return self.push_tag_withdrawal(remote_name, tag);
2428 +
        }
1506 2429
        let remote = self.open_named_remote(remote_name)?;
1507 2430
        self.check_remote_id(remote.as_ref(), remote_name, true)?;
1508 2431
        let resp = remote.refs()?;
…
43 unchanged lines
1552 2475
        Ok(results)
1553 2476
    }
1554 2477
2478 +
    /// Remove remote `name` (oxygen#86): every `remote.NAME.*` key of this
2479 +
    /// repository's config (its URL, token, CA and store settings), its
2480 +
    /// tracking bookmarks in the view, and a store remote's mirror under
2481 +
    /// `.oxy/remotes/NAME`. Nothing stays to be read as if it were current.
2482 +
    pub fn remove_remote(&mut self, name: &str) -> Result<()> {
2483 +
        if !self.config.remotes.contains_key(name) {
2484 +
            return fail(format!("no remote {name}: `oxy remote list` shows them"));
2485 +
        }
2486 +
        let prefix = format!("remote.{name}.");
2487 +
        let keys: Vec<String> = self
2488 +
            .config
2489 +
            .repo_layer()
2490 +
            .keys()
2491 +
            .filter(|k| k.starts_with(&prefix))
2492 +
            .cloned()
2493 +
            .collect();
2494 +
        for k in keys {
2495 +
            self.config.set(&k, "")?;
2496 +
        }
2497 +
        self.config.remotes.remove(name);
2498 +
        self.save_config()?;
2499 +
        let (op_ref, op) = self.head_op()?;
2500 +
        if op.view.remotes.contains_key(name) {
2501 +
            let mut view = op.view.clone();
2502 +
            view.remotes.remove(name);
2503 +
            self.record_op(&[op_ref], &format!("oxy remote remove {name}"), view)?;
2504 +
        }
2505 +
        let mirror = self.oxy.join("remotes").join(name);
2506 +
        if mirror.is_dir() {
2507 +
            fs::remove_dir_all(&mirror)?;
2508 +
        }
2509 +
        Ok(())
2510 +
    }
2511 +
2512 +
    /// Send `bundle`'s blobs and chunks ahead of it, in parts of about
2513 +
    /// `push.batch-bytes` of objects alone, when the whole would be larger
2514 +
    /// (oxygen#133): one change can bring more blobs than one request may
2515 +
    /// carry, and a proxy in front of a hub refuses a request past its limit
2516 +
    /// (Cloudflare's is 100 MB). What is left, the changes with their
2517 +
    /// identities and states, is returned to send as before. A receiver
2518 +
    /// stores objects as they come, and checks a change's blobs are stored.
2519 +
    fn send_objects_ahead(
2520 +
        &self,
2521 +
        remote: &dyn crate::protocol::Remote,
2522 +
        remote_name: &str,
2523 +
        mut bundle: Bundle,
2524 +
        progress: crate::store::Progress,
2525 +
    ) -> Result<Bundle> {
2526 +
        let limit = self.config.push_batch_bytes;
2527 +
        let changes: u64 = bundle
2528 +
            .changes
2529 +
            .iter()
2530 +
            .map(|(c, _)| c.encode().len() as u64)
2531 +
            .sum();
2532 +
        let objects: u64 = bundle.objects.iter().map(|(_, b)| b.len() as u64).sum();
2533 +
        if changes + objects <= limit {
2534 +
            return Ok(bundle);
2535 +
        }
2536 +
        let (mut kept, mut parts, mut part, mut bytes) = (Vec::new(), Vec::new(), Vec::new(), 0u64);
2537 +
        for (ty, body) in std::mem::take(&mut bundle.objects) {
2538 +
            if !matches!(ty, ObjectType::Blob | ObjectType::Chunk) {
2539 +
                kept.push((ty, body));
2540 +
                continue;
2541 +
            }
2542 +
            if !part.is_empty() && bytes + body.len() as u64 > limit {
2543 +
                parts.push(std::mem::take(&mut part));
2544 +
                bytes = 0;
2545 +
            }
2546 +
            bytes += body.len() as u64;
2547 +
            part.push((ty, body));
2548 +
        }
2549 +
        if !part.is_empty() {
2550 +
            parts.push(part);
2551 +
        }
2552 +
        let n = parts.len();
2553 +
        for (i, objects) in parts.into_iter().enumerate() {
2554 +
            progress(&format!(
2555 +
                "objects {} of {n}: {} blobs and chunks",
2556 +
                i + 1,
2557 +
                objects.len()
2558 +
            ));
2559 +
            let results = remote.push(
2560 +
                Bundle {
2561 +
                    objects,
2562 +
                    ..Default::default()
2563 +
                },
2564 +
                vec![],
2565 +
                None,
2566 +
                vec![],
2567 +
            )?;
2568 +
            match results.iter().find(|r| r.name == OBJECTS_RESULT) {
2569 +
                Some(r) if r.ok => {}
2570 +
                Some(r) => {
2571 +
                    return fail(format!(
2572 +
                        "{remote_name} refused objects part {} of {n}: {}",
2573 +
                        i + 1,
2574 +
                        r.reason
2575 +
                    ))
2576 +
                }
2577 +
                None => {
2578 +
                    return fail(format!(
2579 +
                        "{remote_name} does not take a push in parts, and this one is too large for one: update the server's oxy"
2580 +
                    ))
2581 +
                }
2582 +
            }
2583 +
        }
2584 +
        bundle.objects = kept;
2585 +
        Ok(bundle)
2586 +
    }
2587 +
2588 +
    /// `changes`, in dependency order, cut into batches of about
2589 +
    /// `push.batch-bytes` by what each change and its texts and blobs weigh.
2590 +
    /// There is always one, perhaps empty.
2591 +
    fn push_batches(&self, changes: &[ChangeHash]) -> Result<Vec<Vec<ChangeHash>>> {
2592 +
        let blob_len = |r: &ObjectRef| -> u64 {
2593 +
            self.store
2594 +
                .get_typed(r, ObjectType::Blob)
2595 +
                .ok()
2596 +
                .and_then(|b| Blob::decode(&b).ok())
2597 +
                .map_or(0, |b| b.length)
2598 +
        };
2599 +
        let (mut out, mut batch, mut bytes) = (Vec::new(), Vec::new(), 0u64);
2600 +
        for h in changes {
2601 +
            let raw = self.store.get_typed(h, ObjectType::Change)?;
2602 +
            let (c, texts) = Change::decode_skeleton(&raw, *h)?;
2603 +
            let mut weight = raw.len() as u64;
2604 +
            for t in &texts {
2605 +
                if let crate::objects::TextRef::Blob(r, _, _) = t {
2606 +
                    weight += blob_len(r);
2607 +
                }
2608 +
            }
2609 +
            for e in &c.edits {
2610 +
                if let Edit::Replace { new: Some(b), .. } = e {
2611 +
                    weight += blob_len(b);
2612 +
                }
2613 +
            }
2614 +
            if !batch.is_empty() && bytes + weight > self.config.push_batch_bytes {
2615 +
                out.push(std::mem::take(&mut batch));
2616 +
                bytes = 0;
2617 +
            }
2618 +
            batch.push(*h);
2619 +
            bytes += weight;
2620 +
        }
2621 +
        out.push(batch);
2622 +
        Ok(out)
2623 +
    }
2624 +
1555 2625
    pub fn push(&self, remote_name: &str, force: bool) -> Result<Vec<RefResult>> {
1556 2626
        let bookmark = self.config.bookmark.clone();
1557 2627
        self.push_bookmark(remote_name, &bookmark, force)
…
6 unchanged lines
1564 2634
        bookmark: &str,
1565 2635
        force: bool,
1566 2636
    ) -> Result<Vec<RefResult>> {
2637 +
        self.push_bookmark_with(remote_name, bookmark, force, &mut |_| {})
2638 +
    }
2639 +
2640 +
    /// `push_bookmark`, saying how a long push is going: one line per
2641 +
    /// batch sent (oxygen#95).
2642 +
    /// What a push may take the remote to hold already: the heads of the
2643 +
    /// bookmark pushed, and of every other bookmark it lists that are held
2644 +
    /// here, so a new bookmark made from main sends only what main lacks
2645 +
    /// (oxygen#142), not its whole history again.
2646 +
    pub(crate) fn remote_holds(
2647 +
        &self,
2648 +
        refs: &[crate::protocol::RemoteRef],
2649 +
        bookmark_heads: &[ChangeHash],
2650 +
    ) -> Vec<ChangeHash> {
2651 +
        let mut have: Vec<ChangeHash> = bookmark_heads.to_vec();
2652 +
        for h in refs.iter().flat_map(|r| &r.heads) {
2653 +
            if !have.contains(h) && self.store.has(h) {
2654 +
                have.push(*h);
2655 +
            }
2656 +
        }
2657 +
        have
2658 +
    }
2659 +
2660 +
    pub fn push_bookmark_with(
2661 +
        &self,
2662 +
        remote_name: &str,
2663 +
        bookmark: &str,
2664 +
        force: bool,
2665 +
        progress: crate::store::Progress,
2666 +
    ) -> Result<Vec<RefResult>> {
1567 2667
        let (op_ref, op) = self.head_op()?;
1568 2668
        if !op.view.bookmarks.contains_key(bookmark) {
1569 2669
            return fail(format!("no bookmark {bookmark}"));
…
30 unchanged lines
1600 2700
        if remote_has_policy && self.secret_key()?.is_none() {
1601 2701
            return fail(format!("{remote_name} enforces a trust policy; pushing needs an identity (`oxy identity create`)"));
1602 2702
        }
1603 -
        let remote_tags: BTreeSet<String> = resp.tags.iter().map(|t| t.name.clone()).collect();
2703 +
        let remote_tags: BTreeSet<String> = resp
2704 +
            .tags
2705 +
            .iter()
2706 +
            .chain(&resp.withdrawn)
2707 +
            .map(|t| t.name.clone())
2708 +
            .collect();
1604 2709
        let tags_to_send: Vec<crate::protocol::TagRef> = self
1605 2710
            .tag_refs()?
1606 2711
            .into_iter()
1607 2712
            .filter(|t| !remote_tags.contains(&t.name))
1608 2713
            .collect();
2714 +
        // Tags withdrawn here that the remote still has, at the same state
2715 +
        // (oxygenhub#301).
2716 +
        let withdrawals_to_send: Vec<crate::protocol::TagRef> = self
2717 +
            .withdrawn_refs()?
2718 +
            .into_iter()
2719 +
            .filter(|w| {
2720 +
                resp.tags
2721 +
                    .iter()
2722 +
                    .any(|t| t.name == w.name && t.state == w.state)
2723 +
            })
2724 +
            .collect();
1609 2725
        let refs = resp.refs;
1610 2726
        let empty = self.store.put_state(&StateObject::new(vec![]))?;
1611 2727
        let (remote_state, remote_heads, last_serial) =
…
8 unchanged lines
1620 2736
        // state is unchanged, so the current attestation is always by a key
1621 2737
        // the current policy authorizes (a revoked key's attestation would
1622 2738
        // otherwise be refused by every later fetch).
1623 -
        if new_state == remote_state && policy_to_send.is_none() && tags_to_send.is_empty() {
2739 +
        if new_state == remote_state
2740 +
            && policy_to_send.is_none()
2741 +
            && tags_to_send.is_empty()
2742 +
            && withdrawals_to_send.is_empty()
2743 +
        {
1624 2744
            return Ok(vec![RefResult {
1625 2745
                name: bookmark,
1626 2746
                ok: true,
1627 2747
                reason: "already up to date".into(),
1628 2748
            }]);
1629 2749
        }
2750 +
        // A shallow repository knows the changes on its boundary without
2751 +
        // holding them: a depth-0 clone's are the heads it cloned (#137).
2752 +
        let boundary = self.boundary();
1630 2753
        for h in &remote_heads {
1631 -
            if !self.store.has(h) {
2754 +
            if !self.store.has(h) && !boundary.contains(h) {
1632 2755
                return fail(format!(
1633 2756
                    "the remote has change {} which you do not; pull first (`oxy pull {remote_name}`), then push again",
1634 2757
                    h.short(12)
…
2 unchanged lines
1637 2760
        }
1638 2761
        if !force {
1639 2762
            let closure = self.closure_in_store(&local_heads)?;
1640 -
            if let Some(h) = remote_heads.iter().find(|h| !closure.contains(h)) {
2763 +
            if let Some(h) = remote_heads
2764 +
                .iter()
2765 +
                .find(|h| !closure.contains(h) && !boundary.contains(h))
2766 +
            {
1641 2767
                return fail(format!("not a fast-forward: remote head {} is not in your state; pull first or use --force", h.short(12)));
1642 2768
            }
1643 2769
        }
1644 -
        let to_send = self.negotiate(&remote_heads, &local_heads)?;
1645 -
        let mut bundle = self.bundle_for(&to_send)?;
2770 +
        let to_send = self.negotiate(&self.remote_holds(&refs, &remote_heads), &local_heads)?;
1646 2771
        let key = self.secret_key()?.map(|(k, _)| k);
1647 2772
        if let Some(key) = key {
1648 2773
            // Seal this author's changes that carry no valid authenticator,
1649 -
            // and adopt anonymous ones (zero author, spec §14.4).
2774 +
            // and adopt anonymous ones (zero author, spec §14.4). The seals
2775 +
            // are stored, so each batch's bundle carries those it needs.
1650 2776
            let mut unsigned = Vec::new();
1651 2777
            for h in &to_send {
1652 -
                let c = self.store.get_change(h)?;
1653 -
                if (c.author == key || c.author.is_zero())
2778 +
                let c = self.store.get_change_meta(h)?;
2779 +
                if self.seals_for_author(&c.author, &key)
1654 2780
                    && !matches!(self.authenticate_change(h)?, SignatureStatus::Valid { .. })
1655 2781
                {
1656 2782
                    unsigned.push(*h);
1657 2783
                }
1658 2784
            }
1659 2785
            for chunk in unsigned.chunks(crate::attest::MAX_SEAL_HASHES) {
1660 -
                let sa = self.sign_attest(&Attest::seal(self.id()?, chunk.to_vec()))?;
1661 -
                bundle.attests.push(sa);
2786 +
                self.sign_attest(&Attest::seal(self.id()?, chunk.to_vec()))?;
2787 +
            }
2788 +
        }
2789 +
        // A long history goes in batches, in dependency order, so neither
2790 +
        // side holds it whole and no request passes the remote's limit:
2791 +
        // all but the last carry objects alone; the last moves the bookmark
2792 +
        // (oxygen#95).
2793 +
        let batches = self.push_batches(&to_send)?;
2794 +
        let (last, earlier) = batches.split_last().expect("at least one batch");
2795 +
        for (i, batch) in earlier.iter().enumerate() {
2796 +
            progress(&format!(
2797 +
                "part {} of {}: {} changes",
2798 +
                i + 1,
2799 +
                batches.len(),
2800 +
                batch.len()
2801 +
            ));
2802 +
            let bundle = self.send_objects_ahead(
2803 +
                remote.as_ref(),
2804 +
                remote_name,
2805 +
                self.bundle_for(batch)?,
2806 +
                &mut *progress,
2807 +
            )?;
2808 +
            let results = remote.push(bundle, vec![], None, vec![])?;
2809 +
            match results.iter().find(|r| r.name == OBJECTS_RESULT) {
2810 +
                Some(r) if r.ok => {}
2811 +
                Some(r) => {
2812 +
                    return fail(format!(
2813 +
                        "{remote_name} refused part {} of the push: {}",
2814 +
                        i + 1,
2815 +
                        r.reason
2816 +
                    ))
2817 +
                }
2818 +
                None => {
2819 +
                    return fail(format!(
2820 +
                        "{remote_name} does not take a push in parts, and this one is too long for one: update the server's oxy"
2821 +
                    ))
2822 +
                }
1662 2823
            }
1663 -
            bundle = self.bundle_for(&to_send)?;
1664 2824
        }
2825 +
        if batches.len() > 1 {
2826 +
            progress(&format!(
2827 +
                "part {} of {}: {} changes",
2828 +
                batches.len(),
2829 +
                batches.len(),
2830 +
                last.len()
2831 +
            ));
2832 +
        }
2833 +
        let mut bundle = self.send_objects_ahead(
2834 +
            remote.as_ref(),
2835 +
            remote_name,
2836 +
            self.bundle_for(last)?,
2837 +
            &mut *progress,
2838 +
        )?;
1665 2839
        bundle.objects.push((
1666 2840
            ObjectType::State,
1667 2841
            StateObject::new(local_heads.clone()).encode(),
…
11 unchanged lines
1679 2853
            })?),
1680 2854
            None => None,
1681 2855
        };
2856 +
        // Whoever signed the move, and whom a delegate acts for, travel with
2857 +
        // it: a receiver that has never seen a change by this delegate could
2858 +
        // not otherwise check its signature (oxygenhub#129).
2859 +
        if attestation.is_some() {
2860 +
            for k in [self.signing_key(), self.config.author_key]
2861 +
                .into_iter()
2862 +
                .flatten()
2863 +
            {
2864 +
                if let Ok(id) = self.identity(&k) {
2865 +
                    let body = id.encode();
2866 +
                    if !bundle
2867 +
                        .objects
2868 +
                        .iter()
2869 +
                        .any(|(t, b)| *t == ObjectType::Identity && *b == body)
2870 +
                    {
2871 +
                        bundle.objects.push((ObjectType::Identity, body));
2872 +
                    }
2873 +
                }
2874 +
            }
2875 +
            // A short-lived signing key's certificate goes with it (§14.7).
2876 +
            let signers: Vec<KeyId> = self.signing_key().into_iter().collect();
2877 +
            self.attach_certificates_for(&mut bundle, &signers)?;
2878 +
        }
1682 2879
        let serial = attestation.as_ref().and_then(|a| match a.attest.body {
1683 2880
            AttestBody::Ref { serial, .. } => Some(serial),
1684 2881
            _ => None,
…
16 unchanged lines
1701 2898
                StateObject::new(t.heads.clone()).encode(),
1702 2899
            ));
1703 2900
        }
2901 +
        // A tag's signer travels with it, as the bookmark's does: a tag made
2902 +
        // by `oxy tag create` under a short-lived key was signed by a key the
2903 +
        // receiver has never seen, so its identity and certificate go too.
2904 +
        let tag_signers: Vec<KeyId> = tags_to_send
2905 +
            .iter()
2906 +
            .chain(&withdrawals_to_send)
2907 +
            .filter_map(|t| t.attestation.as_ref())
2908 +
            .flat_map(|a| a.sigs.iter().map(|s| s.key))
2909 +
            .collect();
2910 +
        for k in &tag_signers {
2911 +
            if let Ok(id) = self.identity(k) {
2912 +
                let body = id.encode();
2913 +
                if !bundle
2914 +
                    .objects
2915 +
                    .iter()
2916 +
                    .any(|(t, b)| *t == ObjectType::Identity && *b == body)
2917 +
                {
2918 +
                    bundle.objects.push((ObjectType::Identity, body));
2919 +
                }
2920 +
            }
2921 +
        }
2922 +
        if !tag_signers.is_empty() {
2923 +
            self.attach_certificates_for(&mut bundle, &tag_signers)?;
2924 +
        }
1704 2925
        if let Some(l) = &policy_to_send {
1705 2926
            let remote_serial = resp_policy_serial;
1706 2927
            bundle.policies = self
…
15 unchanged lines
1722 2943
                attestation,
1723 2944
            }],
1724 2945
            policy_to_send,
1725 -
            tags_to_send,
2946 +
            tags_to_send
2947 +
                .into_iter()
2948 +
                .chain(withdrawals_to_send)
2949 +
                .collect(),
1726 2950
        )?;
1727 2951
        if let (Some(serial), true) = (
1728 2952
            sent_serial,
…
52 unchanged lines
1781 3005
        sparse: &[String],
1782 3006
        depth: Option<u32>,
1783 3007
    ) -> Result<Repo> {
3008 +
        Ok(Self::clone_reporting(url, dest, max_blob, sparse, depth)?.0)
3009 +
    }
3010 +
3011 +
    /// Clone as [`Repo::clone_with`] (or [`Repo::clone_shallow`] with a
3012 +
    /// `depth`), also returning the bookmarks the fetch left out because
3013 +
    /// their attestation could not be verified ([`FetchReport::skipped`]),
3014 +
    /// for the caller to warn about. The bookmark checked out is never one
3015 +
    /// of them: that fails the clone.
3016 +
    pub fn clone_reporting(
3017 +
        url: &str,
3018 +
        dest: &Path,
3019 +
        max_blob: Option<u64>,
3020 +
        sparse: &[String],
3021 +
        depth: Option<u32>,
3022 +
    ) -> Result<(Repo, Vec<(String, String)>)> {
1784 3023
        let options = crate::credentials::Credentials::user().options_for(url);
1785 3024
        Self::clone_using(url, dest, max_blob, sparse, depth, &options)
1786 3025
    }
…
3 unchanged lines
1790 3029
    /// subrepo: it finds the same ones again through its parent.
1791 3030
    pub(crate) fn clone_as_subrepo(&self, url: &str, dest: &Path) -> Result<Repo> {
1792 3031
        let options = self.credentials_for(url);
1793 -
        Self::clone_using(url, dest, None, &[], None, &options)
3032 +
        Ok(Self::clone_using(url, dest, None, &[], None, &options)?.0)
1794 3033
    }
1795 3034
1796 3035
    /// The name of a remote of this repository on the same hub (scheme,
…
45 unchanged lines
1842 3081
        sparse: &[String],
1843 3082
        depth: Option<u32>,
1844 3083
        options: &RemoteOptions,
1845 -
    ) -> Result<Repo> {
3084 +
    ) -> Result<(Repo, Vec<(String, String)>)> {
1846 3085
        let remote = open_remote(url, options)?;
1847 3086
        let id = remote.repo_id()?;
1848 3087
        if id == crate::protocol::UNBOUND_ID {
…
13 unchanged lines
1862 3101
            repo.config.set("filter.paths", &sparse.join(","))?;
1863 3102
        }
1864 3103
        repo.save_config()?;
1865 -
        if let Some(depth) = depth {
3104 +
        let skipped = if let Some(depth) = depth {
1866 3105
            repo.fetch_shallow("origin", depth)?;
3106 +
            Vec::new()
1867 3107
        } else {
1868 -
            repo.fetch("origin")?;
1869 -
        }
3108 +
            repo.fetch("origin")?.skipped
3109 +
        };
1870 3110
        let (op_ref, op) = repo.head_op()?;
1871 -
        // The remote's default bookmark, else ours (main), else its only one.
1872 -
        let theirs = op.view.remotes.get("origin").cloned().unwrap_or_default();
3111 +
        // The remote's default bookmark, else ours (main), else its only
3112 +
        // one; a bookmark the fetch skipped counts, so the clone fails
3113 +
        // rather than check out another (oxygenhub#266).
3114 +
        let mut theirs = op.view.remotes.get("origin").cloned().unwrap_or_default();
3115 +
        for (name, _) in &skipped {
3116 +
            theirs.insert(name.clone(), ObjectRef::ZERO);
3117 +
        }
1873 3118
        // Asked of the remote as now configured, so a store-backed one
1874 3119
        // answers from the mirror the fetch just filled.
1875 3120
        let advertised = repo
…
14 unchanged lines
1890 3135
                    .flatten()
1891 3136
            })
1892 3137
            .unwrap_or_else(|| repo.config.bookmark.clone());
3138 +
        if let Some((_, why)) = skipped.iter().find(|(name, _)| *name == bookmark) {
3139 +
            return fail(format!("cannot check out {bookmark}: {why}"));
3140 +
        }
1893 3141
        let initial = repo.config.bookmark.clone();
1894 3142
        if bookmark != repo.config.bookmark {
1895 3143
            fs::create_dir_all(repo.workspace_bookmark_path().parent().unwrap())?;
…
11 unchanged lines
1907 3155
            repo.record_op(&[op_ref], &format!("oxy clone {url}"), view)?;
1908 3156
            repo.checkout(&heads, false, "oxy clone (checkout)")?;
1909 3157
        }
1910 -
        Ok(repo)
3158 +
        Ok((repo, skipped))
1911 3159
    }
1912 3160
}
1913 3161
…
10 unchanged lines
modified crates/oxy-core/src/repo/tags.rs +465 −16
1 1
//! Tags (spec §10.1, §14.4): immutable named states. A tag is a `tags/<name>`
2 2
//! entry in the view pointing at a state object, plus a signed `tag`
3 3
//! attestation when the author has a key. A name is never reassigned; a
4 -
//! server refuses a second attestation for an existing tag name.
4 +
//! server refuses a second attestation for an existing tag name. A tag is
5 +
//! withdrawn by a signed `untag` attestation (oxygenhub#301), kept beside
6 +
//! its creation; the name is not used again.
5 7
6 8
use super::*;
7 9
10 +
/// How a tag's signer is judged when it is adopted (spec §10.1, §14.7).
11 +
pub(crate) enum TagCheck<'a> {
12 +
    /// Received in a push: a short-lived key's certificate is checked at
13 +
    /// the receiver's own clock, or at its issuer's acceptance when the
14 +
    /// push carries one (a tag passed on from another host).
15 +
    Received,
16 +
    /// Fetched from `remote`: at the tag attestation's integrated time when
17 +
    /// the host's log gives one, else at its certificate issuer's
18 +
    /// acceptance; never at the time the tag claims, which its signer chose.
19 +
    Fetched {
20 +
        remote: &'a str,
21 +
        log: &'a super::transparency::LogCheck,
22 +
        notes: &'a mut Vec<String>,
23 +
    },
24 +
    /// Replayed into a Strata mirror from its store's record: at the
25 +
    /// acceptance when there is one, else at the tag's own time. The fetch
26 +
    /// from the mirror then judges it as every fetch does.
27 +
    Mirrored,
28 +
}
29 +
30 +
/// Why a tag was not adopted.
31 +
#[derive(Debug)]
32 +
pub(crate) struct TagRefusal {
33 +
    pub reason: String,
34 +
    /// Signed by a short-lived key, with nothing that says when to check
35 +
    /// its certificate: a fetch leaves the tag out and goes on.
36 +
    pub untimed: bool,
37 +
}
38 +
39 +
impl From<String> for TagRefusal {
40 +
    fn from(reason: String) -> TagRefusal {
41 +
        TagRefusal {
42 +
            reason,
43 +
            untimed: false,
44 +
        }
45 +
    }
46 +
}
47 +
48 +
impl From<&str> for TagRefusal {
49 +
    fn from(reason: &str) -> TagRefusal {
50 +
        reason.to_string().into()
51 +
    }
52 +
}
53 +
8 54
#[derive(Debug, Clone, PartialEq, Eq)]
9 55
pub struct TagInfo {
10 56
    pub name: String,
…
5 unchanged lines
16 62
}
17 63
18 64
impl Repo {
19 -
    fn tag_attest_path(&self, name: &str) -> PathBuf {
65 +
    pub(crate) fn tag_attest_path(&self, name: &str) -> PathBuf {
20 66
        self.oxy.join("attest").join("tags").join(name_file(name))
21 67
    }
22 68
…
21 unchanged lines
44 90
        write_atomic(&self.tag_attest_path(name), r.to_hex().as_bytes())
45 91
    }
46 92
93 +
    fn tag_withdrawal_path(&self, name: &str) -> PathBuf {
94 +
        self.oxy.join("attest").join("untags").join(name_file(name))
95 +
    }
96 +
97 +
    /// The withdrawal of tag `name`, when it was withdrawn here.
98 +
    pub fn tag_withdrawal(&self, name: &str) -> Result<Option<(ObjectRef, SignedAttest)>> {
99 +
        match fs::read_to_string(self.tag_withdrawal_path(name)) {
100 +
            Ok(h) => {
101 +
                let r = ObjectRef::from_hex(h.trim())
102 +
                    .ok_or_else(|| RepoError::Msg("corrupt tag withdrawal pointer".into()))?;
103 +
                Ok(Some((
104 +
                    r,
105 +
                    SignedAttest {
106 +
                        attest: self.store.get_attest(&r)?,
107 +
                        sigs: self.store.signatures(&r)?,
108 +
                    },
109 +
                )))
110 +
            }
111 +
            Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(None),
112 +
            Err(e) => Err(e.into()),
113 +
        }
114 +
    }
115 +
116 +
    fn set_tag_withdrawal(&self, name: &str, r: &ObjectRef) -> Result<()> {
117 +
        fs::create_dir_all(self.oxy.join("attest").join("untags"))?;
118 +
        write_atomic(&self.tag_withdrawal_path(name), r.to_hex().as_bytes())
119 +
    }
120 +
121 +
    /// Withdraw tag `name` (spec §10.1): an `untag` attestation naming its
122 +
    /// state, signed when this repository has a key, and the tag leaves the
123 +
    /// view. The next push sends the withdrawal to a remote that still has
124 +
    /// the tag, under a policy only when the key may create the tag there.
125 +
    pub fn tag_delete(&self, name: &str, reason: &str) -> Result<TagInfo> {
126 +
        let (op_ref, op) = self.head_op()?;
127 +
        let Some(state) = op.view.tags.get(name).copied() else {
128 +
            return fail(format!("no tag {name:?}"));
129 +
        };
130 +
        let heads = self.store.get_state(&state)?.heads;
131 +
        let attest = Attest {
132 +
            repo: self.id()?,
133 +
            hlc: Hlc::now(),
134 +
            body: AttestBody::Untag {
135 +
                name: name.into(),
136 +
                state,
137 +
                reason: reason.into(),
138 +
            },
139 +
        };
140 +
        let mut signer = None;
141 +
        let r = if self.secret_key()?.is_some() {
142 +
            let sa = self.sign_attest(&attest)?;
143 +
            signer = sa
144 +
                .sigs
145 +
                .first()
146 +
                .and_then(|s| self.store.get_identity(&s.key).ok())
147 +
                .map(|i| i.name);
148 +
            sa.attest.hash()
149 +
        } else {
150 +
            self.store.put_attest(&attest)?
151 +
        };
152 +
        self.set_tag_withdrawal(name, &r)?;
153 +
        let mut view = op.view.clone();
154 +
        view.tags.remove(name);
155 +
        self.record_op(&[op_ref], &format!("oxy tag delete {name}"), view)?;
156 +
        Ok(TagInfo {
157 +
            name: name.into(),
158 +
            state,
159 +
            heads,
160 +
            message: reason.into(),
161 +
            signer,
162 +
        })
163 +
    }
164 +
165 +
    /// Every tag withdrawn here, as `refs` reports them: the `untag`
166 +
    /// attestation as the attestation, with its acceptance when a
167 +
    /// short-lived key signed it.
168 +
    pub(crate) fn withdrawn_refs(&self) -> Result<Vec<crate::protocol::TagRef>> {
169 +
        let dir = self.oxy.join("attest").join("untags");
170 +
        let entries = match fs::read_dir(&dir) {
171 +
            Ok(e) => e,
172 +
            Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(vec![]),
173 +
            Err(e) => return Err(e.into()),
174 +
        };
175 +
        let mut out = Vec::new();
176 +
        for e in entries {
177 +
            let e = e?;
178 +
            let Ok(h) = fs::read_to_string(e.path()) else {
179 +
                continue;
180 +
            };
181 +
            let Some(r) = ObjectRef::from_hex(h.trim()) else {
182 +
                continue;
183 +
            };
184 +
            let sa = SignedAttest {
185 +
                attest: self.store.get_attest(&r)?,
186 +
                sigs: self.store.signatures(&r)?,
187 +
            };
188 +
            let AttestBody::Untag { name, state, .. } = &sa.attest.body else {
189 +
                continue;
190 +
            };
191 +
            let heads = self
192 +
                .store
193 +
                .get_state(state)
194 +
                .map(|s| s.heads)
195 +
                .unwrap_or_default();
196 +
            let accepted = self.tag_acceptance(&sa)?;
197 +
            let mut t = crate::protocol::TagRef::new(name.clone(), *state, heads, Some(sa));
198 +
            t.accepted = accepted;
199 +
            out.push(t);
200 +
        }
201 +
        out.sort_by(|a, b| a.name.cmp(&b.name));
202 +
        Ok(out)
203 +
    }
204 +
47 205
    /// Create a tag named `name` for the state of `heads` (spec §10.1).
48 206
    pub fn tag_create(&self, name: &str, heads: &[ChangeHash], message: &str) -> Result<TagInfo> {
49 207
        crate::objects::validate_name(name)
…
2 unchanged lines
52 210
        if op.view.tags.contains_key(name) {
53 211
            return fail(format!("tag {name:?} already exists; tags are immutable"));
54 212
        }
213 +
        if self.tag_withdrawal(name)?.is_some() {
214 +
            return fail(format!(
215 +
                "tag {name:?} was withdrawn; a withdrawn tag's name is not used again"
216 +
            ));
217 +
        }
55 218
        let st = self.load_state_history(heads)?;
56 219
        let heads = st.maximal(heads);
57 220
        if heads.is_empty() {
…
86 unchanged lines
144 307
                .tag_attest(&t.name)?
145 308
                .map(|(_, a)| a)
146 309
                .filter(|a| !a.sigs.is_empty());
147 -
            out.push(crate::protocol::TagRef {
148 -
                name: t.name.clone(),
149 -
                state: t.state,
150 -
                heads: t.heads,
151 -
                attestation,
152 -
            });
310 +
            // A short-lived key's tag goes with its issuer's acceptance
311 +
            // (spec §14.7), for whoever fetches it later.
312 +
            let accepted = match &attestation {
313 +
                Some(a) => self.tag_acceptance(a)?,
314 +
                None => None,
315 +
            };
316 +
            let mut tag = crate::protocol::TagRef::new(t.name, t.state, t.heads, attestation);
317 +
            tag.accepted = accepted;
318 +
            out.push(tag);
153 319
        }
154 320
        Ok(out)
155 321
    }
156 322
323 +
    /// The acceptance kept for tag attestation `sa`: its signer's
324 +
    /// certificate issuer's when one of those kept is, else the first.
325 +
    fn tag_acceptance(&self, sa: &SignedAttest) -> Result<Option<SignedAttest>> {
326 +
        let r = sa.attest.hash();
327 +
        let mut first = None;
328 +
        for a in self.store.index_get_pub("accepted", &r)? {
329 +
            let acc = SignedAttest {
330 +
                attest: self.store.get_attest(&a)?,
331 +
                sigs: self.store.signatures(&a)?,
332 +
            };
333 +
            let signer = sa.sigs.first().map(|s| s.key);
334 +
            if signer.is_some_and(|k| self.issuer_acceptance(&k, sa, Some(&acc)).is_some()) {
335 +
                return Ok(Some(acc));
336 +
            }
337 +
            first.get_or_insert(acc);
338 +
        }
339 +
        Ok(first)
340 +
    }
341 +
342 +
    /// Whether policy `p` authorizes the signer of tag `name`'s attestation
343 +
    /// `sa` without a certificate: a key its rule lists, a delegate of one,
344 +
    /// or a key one of them was rotated from (spec §14.5, §14.2). These are
345 +
    /// the only ways a client before oxygen#129 authorizes a tag.
346 +
    fn tag_signer_listed(&self, p: &crate::policy::Policy, name: &str, sa: &SignedAttest) -> bool {
347 +
        let Some(sig) = sa.sigs.first() else {
348 +
            return false;
349 +
        };
350 +
        let reference = format!("tags/{name}");
351 +
        let at = sa.attest.hlc.wall_ms;
352 +
        p.authorizes(&sig.key, &reference)
353 +
            || self.delegate_authorized(p, &sig.key, &reference, at)
354 +
            || self.predecessor_authorized(p, &sig.key, &reference, at)
355 +
    }
356 +
357 +
    /// Tailor a `refs` answer of this repository to a client that declared
358 +
    /// `features` (the [`FEATURES_HEADER`] value; `None` when it sent none).
359 +
    /// A client without [`FEATURE_CERTIFIED_TAGS`] (oxy 0.5.0 and older)
360 +
    /// refuses a tag signed by a short-lived key that only its certificate
361 +
    /// authorizes, and with it the whole clone or fetch (oxygen#131), so
362 +
    /// under a policy each tag whose signer the policy does not list, as a
363 +
    /// key, a delegate or a rotation predecessor, is left out: that client
364 +
    /// sees the repository as if the tag did not exist. A tag with no
365 +
    /// signed attestation is left as it is (every client refuses it alike),
366 +
    /// as is everything without a policy.
367 +
    ///
368 +
    /// [`FEATURES_HEADER`]: crate::protocol::FEATURES_HEADER
369 +
    /// [`FEATURE_CERTIFIED_TAGS`]: crate::protocol::FEATURE_CERTIFIED_TAGS
370 +
    pub fn refs_for_client(
371 +
        &self,
372 +
        refs: &mut crate::protocol::RefsResponse,
373 +
        features: Option<&str>,
374 +
    ) {
375 +
        if crate::protocol::declares_feature(features, crate::protocol::FEATURE_CERTIFIED_TAGS) {
376 +
            return;
377 +
        }
378 +
        let Some(sp) = refs.policy.as_ref() else {
379 +
            return;
380 +
        };
381 +
        refs.tags.retain(|t| match &t.attestation {
382 +
            Some(sa) if !sa.sigs.is_empty() => self.tag_signer_listed(&sp.policy, &t.name, sa),
383 +
            _ => true,
384 +
        });
385 +
    }
386 +
157 387
    /// Adopt a tag received from a remote or a push. Verifies the
158 388
    /// attestation when present (and requires one, by an authorized key,
159 389
    /// under a policy). An existing tag with a different state is refused.
390 +
    /// A short-lived key's tag is authorized by a subject grant, its
391 +
    /// certificate checked at the time `check` says (spec §14.7), and its
392 +
    /// issuer's acceptance is kept with it, or recorded when this host
393 +
    /// receives it.
160 394
    pub(crate) fn adopt_tag(
161 395
        &self,
162 396
        view: &mut View,
163 397
        t: &crate::protocol::TagRef,
164 398
        policy: Option<&crate::policy::Policy>,
165 -
    ) -> std::result::Result<(), String> {
399 +
        check: &mut TagCheck,
400 +
    ) -> std::result::Result<(), TagRefusal> {
401 +
        if self
402 +
            .tag_withdrawal(&t.name)
403 +
            .map_err(|e| e.to_string())?
404 +
            .is_some()
405 +
        {
406 +
            return Err(format!(
407 +
                "tag {:?} was withdrawn here; a withdrawn tag's name is not used again",
408 +
                t.name
409 +
            )
410 +
            .into());
411 +
        }
166 412
        if let Some(existing) = view.tags.get(&t.name) {
167 413
            return if *existing == t.state {
168 414
                Ok(())
…
1 unchanged lines
170 416
                Err(format!(
171 417
                    "tag {:?} already exists with a different state; tags are immutable",
172 418
                    t.name
173 -
                ))
419 +
                )
420 +
                .into())
174 421
            };
175 422
        }
176 423
        let state = self
…
6 unchanged lines
183 430
                    "tag {:?} names change {}, which is missing",
184 431
                    t.name,
185 432
                    h.short(12)
186 -
                ));
433 +
                )
434 +
                .into());
187 435
            }
188 436
        }
189 437
        if let Some(sa) = &t.attestation {
…
19 unchanged lines
209 457
                .get_identity(&sig.key)
210 458
                .map_err(|_| format!("unknown key {}", sig.key.short(12)))?;
211 459
            verify_signature(&id, sig, Domain::Attest, &r).map_err(|e| e.to_string())?;
460 +
            let certified = self.store.certificate_of(&sig.key).ok().flatten().is_some();
461 +
            // The issuer's acceptance this tag came with, when it is one.
462 +
            let accepted = self.issuer_acceptance(&sig.key, sa, t.accepted.as_ref());
212 463
            if let Some(p) = policy {
213 -
                let reference = format!("tags/{}", t.name);
214 -
                if !p.authorizes(&sig.key, &reference)
215 -
                    && !self.delegate_authorized(p, &sig.key, &reference, sa.attest.hlc.wall_ms)
464 +
                let listed = self.tag_signer_listed(p, &t.name, sa);
465 +
                if !listed
466 +
                    && !(certified
467 +
                        && self.tag_certified_in_time(p, t, sa, &sig.key, accepted, check)?)
216 468
                {
217 469
                    return Err(format!(
218 470
                        "key {} may not create tags/{} under this repository's policy",
219 471
                        sig.key.short(12),
220 472
                        t.name
221 -
                    ));
473 +
                    )
474 +
                    .into());
222 475
                }
223 476
            }
224 477
            let stored = self
…
5 unchanged lines
230 483
                    .put_signature(&stored, s)
231 484
                    .map_err(|e| e.to_string())?;
232 485
            }
486 +
            if certified {
487 +
                match (&t.accepted, accepted) {
488 +
                    (Some(acc), Some(_)) => self.keep_acceptance(&stored, acc),
489 +
                    // Received here: this host says when, as it does for a
490 +
                    // bookmark's attestation.
491 +
                    _ if matches!(check, TagCheck::Received) => self.record_acceptance(&stored),
492 +
                    _ => Ok(()),
493 +
                }
494 +
                .map_err(|e| e.to_string())?;
495 +
            }
233 496
            self.set_tag_attest(&t.name, &stored)
234 497
                .map_err(|e| e.to_string())?;
235 498
        } else if policy.is_some() {
236 499
            return Err(format!(
237 500
                "tag {:?} needs a signed attestation under this repository's policy",
238 501
                t.name
239 -
            ));
502 +
            )
503 +
            .into());
240 504
        }
241 505
        view.tags.insert(t.name.clone(), t.state);
242 506
        Ok(())
243 507
    }
508 +
509 +
    /// Adopt the withdrawal of tag `t` (its `attestation` the `untag`
510 +
    /// attestation), received in a push or fetched: the tag must be here
511 +
    /// with the state the withdrawal names, and under a policy its signer
512 +
    /// must be one that may create the tag, a short-lived key at the time
513 +
    /// `check` says (as for creating it, except that a fetch checks at the
514 +
    /// issuer's acceptance only). The tag leaves `view` and the withdrawal
515 +
    /// is kept. Whether the tag was here and is now withdrawn: a withdrawal
516 +
    /// already kept is `false`.
517 +
    pub(crate) fn adopt_withdrawal(
518 +
        &self,
519 +
        view: &mut View,
520 +
        t: &crate::protocol::TagRef,
521 +
        policy: Option<&crate::policy::Policy>,
522 +
        check: &mut TagCheck,
523 +
    ) -> std::result::Result<bool, TagRefusal> {
524 +
        let sa = t
525 +
            .attestation
526 +
            .as_ref()
527 +
            .ok_or("a tag withdrawal carries no attestation")?;
528 +
        let AttestBody::Untag { name, state, .. } = &sa.attest.body else {
529 +
            return Err("tag withdrawal has the wrong kind".into());
530 +
        };
531 +
        if *name != t.name || *state != t.state {
532 +
            return Err("tag withdrawal does not describe this tag".into());
533 +
        }
534 +
        if sa.attest.repo != self.id().map_err(|e| e.to_string())? {
535 +
            return Err("tag withdrawal is for a different repository".into());
536 +
        }
537 +
        if self
538 +
            .tag_withdrawal(&t.name)
539 +
            .map_err(|e| e.to_string())?
540 +
            .is_some()
541 +
        {
542 +
            return Ok(false);
543 +
        }
544 +
        match view.tags.get(&t.name) {
545 +
            None => return Err(format!("there is no tag {:?} here", t.name).into()),
546 +
            Some(st) if *st != t.state => {
547 +
                return Err(format!(
548 +
                    "the withdrawal names another state of tag {:?} than this repository's",
549 +
                    t.name
550 +
                )
551 +
                .into())
552 +
            }
553 +
            Some(_) => {}
554 +
        }
555 +
        let r = sa.attest.hash();
556 +
        let mut certified = false;
557 +
        if let Some(sig) = sa.sigs.first() {
558 +
            let id = self
559 +
                .store
560 +
                .get_identity(&sig.key)
561 +
                .map_err(|_| format!("unknown key {}", sig.key.short(12)))?;
562 +
            verify_signature(&id, sig, Domain::Attest, &r).map_err(|e| e.to_string())?;
563 +
            certified = self.store.certificate_of(&sig.key).ok().flatten().is_some();
564 +
            if let Some(p) = policy {
565 +
                let accepted = self.issuer_acceptance(&sig.key, sa, t.accepted.as_ref());
566 +
                let reference = format!("tags/{}", t.name);
567 +
                let in_time =
568 +
                    |ms: u64| self.certified_authorized(p, &sig.key, &reference, Scope::REFS, ms);
569 +
                let allowed = self.tag_signer_listed(p, &t.name, sa)
570 +
                    || (certified
571 +
                        && match check {
572 +
                            TagCheck::Received => {
573 +
                                in_time(Hlc::now().wall_ms) || accepted.is_some_and(in_time)
574 +
                            }
575 +
                            TagCheck::Mirrored => {
576 +
                                in_time(accepted.unwrap_or(sa.attest.hlc.wall_ms))
577 +
                            }
578 +
                            TagCheck::Fetched { remote, .. } => match accepted {
579 +
                                Some(ms) => in_time(ms),
580 +
                                None => {
581 +
                                    return Err(TagRefusal {
582 +
                                        reason: format!(
583 +
                                            "withdrawn by short-lived key {}, and {remote} gave no acceptance to check its certificate at",
584 +
                                            sig.key.short(12)
585 +
                                        ),
586 +
                                        untimed: true,
587 +
                                    })
588 +
                                }
589 +
                            },
590 +
                        });
591 +
                if !allowed {
592 +
                    return Err(format!(
593 +
                        "key {} may not withdraw tags/{} under this repository's policy",
594 +
                        sig.key.short(12),
595 +
                        t.name
596 +
                    )
597 +
                    .into());
598 +
                }
599 +
            }
600 +
        } else if policy.is_some() {
601 +
            return Err(format!(
602 +
                "withdrawing tag {:?} needs a signed attestation under this repository's policy",
603 +
                t.name
604 +
            )
605 +
            .into());
606 +
        }
607 +
        let stored = self
608 +
            .store
609 +
            .put_attest(&sa.attest)
610 +
            .map_err(|e| e.to_string())?;
611 +
        for s in &sa.sigs {
612 +
            self.store
613 +
                .put_signature(&stored, s)
614 +
                .map_err(|e| e.to_string())?;
615 +
        }
616 +
        if certified {
617 +
            match &t.accepted {
618 +
                Some(acc)
619 +
                    if sa.sigs.first().is_some_and(|s| {
620 +
                        self.issuer_acceptance(&s.key, sa, Some(acc)).is_some()
621 +
                    }) =>
622 +
                {
623 +
                    self.keep_acceptance(&stored, acc)
624 +
                }
625 +
                _ if matches!(check, TagCheck::Received) => self.record_acceptance(&stored),
626 +
                _ => Ok(()),
627 +
            }
628 +
            .map_err(|e| e.to_string())?;
629 +
        }
630 +
        self.set_tag_withdrawal(&t.name, &stored)
631 +
            .map_err(|e| e.to_string())?;
632 +
        view.tags.remove(&t.name);
633 +
        Ok(true)
634 +
    }
635 +
636 +
    /// Whether short-lived key `signer`'s certificate gives it tag `t`
637 +
    /// under policy `p` at the time `check` says (spec §14.7, §14.8):
638 +
    /// `accepted` is the issuer's acceptance time the tag came with.
639 +
    fn tag_certified_in_time(
640 +
        &self,
641 +
        p: &crate::policy::Policy,
642 +
        t: &crate::protocol::TagRef,
643 +
        sa: &SignedAttest,
644 +
        signer: &KeyId,
645 +
        accepted: Option<u64>,
646 +
        check: &mut TagCheck,
647 +
    ) -> std::result::Result<bool, TagRefusal> {
648 +
        use super::transparency::{judge, Timed};
649 +
        let reference = format!("tags/{}", t.name);
650 +
        let certified = |ms: u64| self.certified_authorized(p, signer, &reference, Scope::REFS, ms);
651 +
        Ok(match check {
652 +
            TagCheck::Received => certified(Hlc::now().wall_ms) || accepted.is_some_and(certified),
653 +
            TagCheck::Mirrored => certified(accepted.unwrap_or(sa.attest.hlc.wall_ms)),
654 +
            TagCheck::Fetched { remote, log, notes } => {
655 +
                let time = log.tag_time_of(&t.name);
656 +
                let integrated = time.as_ref().ok().map(|i| i.ms());
657 +
                if integrated.is_none() && accepted.is_none() {
658 +
                    return Err(TagRefusal {
659 +
                        reason: format!(
660 +
                            "signed by short-lived key {}, and {remote} gave no time to check its certificate at (neither its issuer's acceptance nor a witnessed log time: {}); a tag is never checked at the time it claims",
661 +
                            signer.short(12),
662 +
                            time.err().unwrap_or_default()
663 +
                        ),
664 +
                        untimed: true,
665 +
                    });
666 +
                }
667 +
                let what = format!("{remote} tag {}", t.name);
668 +
                match judge(integrated, certified, || accepted.is_some_and(certified)) {
669 +
                    Timed::Refused => false,
670 +
                    Timed::Integrated => true,
671 +
                    Timed::Disagrees => {
672 +
                        notes.push(format!(
673 +
                            "{what}: {}'s certificate had lapsed when witness {} first saw its attestation, though not at the issuer's acceptance; accepted while transparency logs are new",
674 +
                            signer.short(12),
675 +
                            time.as_ref().map(|i| i.witness.as_str()).unwrap_or("?")
676 +
                        ));
677 +
                        true
678 +
                    }
679 +
                    Timed::Fallback => {
680 +
                        if log.kept {
681 +
                            notes.push(format!(
682 +
                                "{what}: {} checked at the issuer's acceptance, not at a witnessed time: {}",
683 +
                                signer.short(12),
684 +
                                time.err().unwrap_or_default()
685 +
                            ));
686 +
                        }
687 +
                        true
688 +
                    }
689 +
                }
690 +
            }
691 +
        })
692 +
    }
244 693
}
modified crates/oxy-core/src/repo/tests.rs +4216 −20
…
175 unchanged lines
176 176
    (dir, repo, h1, h2, h3)
177 177
}
178 178
179 +
/// oxygen#126: a file whose text includes an example of oxy's markers (as
180 +
/// the spec's §9.5 does) can still have its conflict resolved; a real
181 +
/// marker left behind is still caught.
182 +
#[test]
183 +
fn marker_like_text_does_not_block_a_resolution() {
184 +
    hermetic();
185 +
    let dir = tempfile::tempdir().unwrap();
186 +
    let root = dir.path();
187 +
    let repo = Repo::init(root).unwrap();
188 +
    let example = "```\n<<<<<<< oxy conflict 1 of 2, side 1: change k7m2p9 \"Add retry loop\"\n... lines ...\n======= oxy conflict 1 of 2, side 2: change q3x8w1 \"Refactor client\"\n>>>>>>> oxy conflict 1 of 2 end\n```\n";
189 +
    write(
190 +
        root,
191 +
        "doc.md",
192 +
        format!("intro\n{example}b\nend\n").as_bytes(),
193 +
    );
194 +
    let h1 = repo.commit("base", &[]).unwrap();
195 +
    write(
196 +
        root,
197 +
        "doc.md",
198 +
        format!("intro\n{example}B\nend\n").as_bytes(),
199 +
    );
200 +
    repo.commit("upper", &[]).unwrap();
201 +
    repo.checkout(&[h1], false, "checkout").unwrap();
202 +
    write(
203 +
        root,
204 +
        "doc.md",
205 +
        format!("intro\n{example}X\nend\n").as_bytes(),
206 +
    );
207 +
    repo.commit("ex", &[]).unwrap();
208 +
    repo.checkout(&repo.bookmark_heads().unwrap(), false, "checkout main")
209 +
        .unwrap();
210 +
    let marked = fs::read_to_string(root.join("doc.md")).unwrap();
211 +
    assert!(marked.contains("oxy conflict 1 of 1"), "{marked}");
212 +
    // A real marker left: still caught.
213 +
    write(root, "doc.md", marked.replace("B\n", "BB\n").as_bytes());
214 +
    let s = repo.snapshot("status").unwrap();
215 +
    assert_eq!(
216 +
        statuses(&s),
217 +
        vec![("doc.md".into(), ChangeStatus::ConflictEdited)]
218 +
    );
219 +
    // Resolved, the example kept: a resolution.
220 +
    write(
221 +
        root,
222 +
        "doc.md",
223 +
        format!("intro\n{example}X\nend\n").as_bytes(),
224 +
    );
225 +
    let s = repo.snapshot("status").unwrap();
226 +
    assert_eq!(
227 +
        statuses(&s),
228 +
        vec![("doc.md".into(), ChangeStatus::Resolved)]
229 +
    );
230 +
    let h = repo.commit("resolve", &[]).unwrap();
231 +
    assert_eq!(
232 +
        repo.store().get_change(&h).unwrap().kind,
233 +
        ChangeKind::Resolution
234 +
    );
235 +
    assert!(!repo.load_state(&[h]).unwrap().materialize().is_conflicted());
236 +
}
237 +
238 +
/// oxygen#121: two changes each adding a line after the same line conflict;
239 +
/// the markers list the earlier change first, and `oxy resolve --both`
240 +
/// keeps both lines in that order (or the other with `--reverse`).
241 +
#[test]
242 +
fn adjacent_insertions_are_settled_by_keeping_both() {
243 +
    hermetic();
244 +
    for reverse in [false, true] {
245 +
        let dir = tempfile::tempdir().unwrap();
246 +
        let root = dir.path();
247 +
        let repo = Repo::init(root).unwrap();
248 +
        write(root, "f.txt", b"a\nb\n");
249 +
        let h1 = repo.commit("base", &[]).unwrap();
250 +
        write(root, "f.txt", b"a\nfirst\nb\n");
251 +
        repo.commit("first", &[]).unwrap();
252 +
        repo.checkout(&[h1], false, "checkout").unwrap();
253 +
        write(root, "f.txt", b"a\nsecond\nb\n");
254 +
        repo.commit("second", &[]).unwrap();
255 +
        repo.checkout(&repo.bookmark_heads().unwrap(), false, "checkout main")
256 +
            .unwrap();
257 +
        let marked = fs::read_to_string(root.join("f.txt")).unwrap();
258 +
        let (f, s) = (
259 +
            marked.find("first\n").unwrap(),
260 +
            marked.find("second\n").unwrap(),
261 +
        );
262 +
        assert!(f < s, "the earlier change's side comes first: {marked}");
263 +
264 +
        let snap = repo.snapshot("status").unwrap();
265 +
        let node = snap.base_snapshot.files["f.txt"].node;
266 +
        assert_eq!(snap.base.both_sides(&node).iter().flatten().count(), 1);
267 +
        let kept = repo.keep_both(&snap, &[], reverse).unwrap();
268 +
        assert_eq!(
269 +
            kept,
270 +
            vec![KeptBoth {
271 +
                path: "f.txt".into(),
272 +
                settled: vec![1],
273 +
                identical: vec![],
274 +
                left: vec![]
275 +
            }]
276 +
        );
277 +
        let want: &[u8] = if reverse {
278 +
            b"a\nsecond\nfirst\nb\n"
279 +
        } else {
280 +
            b"a\nfirst\nsecond\nb\n"
281 +
        };
282 +
        assert_eq!(fs::read(root.join("f.txt")).unwrap(), want);
283 +
        let s = repo.snapshot("status").unwrap();
284 +
        assert_eq!(statuses(&s), vec![("f.txt".into(), ChangeStatus::Resolved)]);
285 +
        let h = repo.commit("both", &[]).unwrap();
286 +
        assert_eq!(
287 +
            repo.store().get_change(&h).unwrap().kind,
288 +
            ChangeKind::Resolution
289 +
        );
290 +
        assert!(!repo.load_state(&[h]).unwrap().materialize().is_conflicted());
291 +
    }
292 +
}
293 +
294 +
/// oxygen#145: a file marked `merge=union` has its adjacent insertions
295 +
/// settled by the checkout itself, both kept, earliest first; the next
296 +
/// commit records the resolution. Other files keep their markers.
297 +
#[test]
298 +
fn union_files_keep_both_sides_on_checkout() {
299 +
    hermetic();
300 +
    let dir = tempfile::tempdir().unwrap();
301 +
    let root = dir.path();
302 +
    let repo = Repo::init(root).unwrap();
303 +
    write(root, ".oxyattributes", b"CHANGELOG.md merge=union\n");
304 +
    write(
305 +
        root,
306 +
        "CHANGELOG.md",
307 +
        b"# Changelog\n## Unreleased\n## 1.0\n",
308 +
    );
309 +
    write(root, "f.txt", b"a\nb\n");
310 +
    let h1 = repo.commit("base", &[]).unwrap();
311 +
    write(
312 +
        root,
313 +
        "CHANGELOG.md",
314 +
        b"# Changelog\n## Unreleased\n- one\n## 1.0\n",
315 +
    );
316 +
    write(root, "f.txt", b"a\none\nb\n");
317 +
    repo.commit("one", &[]).unwrap();
318 +
    repo.checkout(&[h1], false, "checkout").unwrap();
319 +
    write(
320 +
        root,
321 +
        "CHANGELOG.md",
322 +
        b"# Changelog\n## Unreleased\n- two\n## 1.0\n",
323 +
    );
324 +
    write(root, "f.txt", b"a\ntwo\nb\n");
325 +
    repo.commit("two", &[]).unwrap();
326 +
    let r = repo
327 +
        .checkout(&repo.bookmark_heads().unwrap(), false, "checkout main")
328 +
        .unwrap();
329 +
    assert!(r.conflicted);
330 +
    assert_eq!(r.kept_both.len(), 1, "{:?}", r.kept_both);
331 +
    assert_eq!(r.kept_both[0].path, "CHANGELOG.md");
332 +
    assert_eq!(
333 +
        fs::read_to_string(root.join("CHANGELOG.md")).unwrap(),
334 +
        "# Changelog\n## Unreleased\n- one\n- two\n## 1.0\n"
335 +
    );
336 +
    assert!(has_conflict_markers(&fs::read(root.join("f.txt")).unwrap()));
337 +
    let s = repo.snapshot("status").unwrap();
338 +
    assert_eq!(
339 +
        statuses(&s),
340 +
        vec![("CHANGELOG.md".into(), ChangeStatus::Resolved)]
341 +
    );
342 +
}
343 +
344 +
/// oxygen#146: two bookmarks that resolved the same conflict to the same
345 +
/// text agree. Merged, their resolutions are a conflict whose sides are
346 +
/// identical: a checkout keeps one, in any file, and a host settles it so
347 +
/// too; the next commit records it.
348 +
#[test]
349 +
fn identical_resolutions_on_two_bookmarks_settle_by_themselves() {
350 +
    hermetic();
351 +
    let dir = tempfile::tempdir().unwrap();
352 +
    let root = dir.path();
353 +
    let repo = Repo::init(root).unwrap();
354 +
    write(root, "f.txt", b"a\nx\nb\n");
355 +
    let base = repo.commit("base", &[]).unwrap();
356 +
    write(root, "f.txt", b"a\nX1\nb\n");
357 +
    let c1 = repo.commit("X1", &[]).unwrap();
358 +
    repo.checkout(&[base], false, "checkout").unwrap();
359 +
    write(root, "f.txt", b"a\nX2\nb\n");
360 +
    let c2 = repo.commit("X2", &[]).unwrap();
361 +
    // Each bookmark merges both and resolves the conflict the same way.
362 +
    repo.checkout(&[c1, c2], false, "checkout").unwrap();
363 +
    assert!(has_conflict_markers(&fs::read(root.join("f.txt")).unwrap()));
364 +
    write(root, "f.txt", b"a\nX\nb\n");
365 +
    let r1 = repo.commit("resolve on one", &[]).unwrap();
366 +
    repo.checkout(&[c1, c2], false, "checkout").unwrap();
367 +
    write(root, "f.txt", b"a\nX\nb\n");
368 +
    let r2 = repo.commit("resolve on two", &[]).unwrap();
369 +
    assert_ne!(r1, r2);
370 +
371 +
    let st = repo.load_state(&[r1, r2]).unwrap();
372 +
    assert!(
373 +
        st.materialize().is_conflicted(),
374 +
        "two resolutions, one place"
375 +
    );
376 +
    assert_eq!(
377 +
        Repo::union_resolution(&st, &st.materialize()),
378 +
        Some(vec![("f.txt".into(), b"a\nX\nb\n".to_vec())]),
379 +
        "a host settles it"
380 +
    );
381 +
    let r = repo.checkout(&[r1, r2], false, "checkout").unwrap();
382 +
    assert!(r.conflicted);
383 +
    assert_eq!(r.kept_both.len(), 1, "{:?}", r.kept_both);
384 +
    assert_eq!(
385 +
        (r.kept_both[0].path.as_str(), &r.kept_both[0].identical),
386 +
        ("f.txt", &vec![1])
387 +
    );
388 +
    assert_eq!(fs::read_to_string(root.join("f.txt")).unwrap(), "a\nX\nb\n");
389 +
    let s = repo.snapshot("status").unwrap();
390 +
    assert_eq!(statuses(&s), vec![("f.txt".into(), ChangeStatus::Resolved)]);
391 +
    let merged = repo.commit("merge", &[]).unwrap();
392 +
    let st = repo.load_state(&[merged]).unwrap();
393 +
    assert!(!st.materialize().is_conflicted());
394 +
}
395 +
396 +
/// oxygen#145: a host (OxygenHub landing a proposal) settles a state whose
397 +
/// only conflicts are adjacent insertions in union files; any other
398 +
/// conflict needs a person.
399 +
#[test]
400 +
fn a_host_settles_a_state_whose_conflicts_are_all_union() {
401 +
    hermetic();
402 +
    let dir = tempfile::tempdir().unwrap();
403 +
    let root = dir.path();
404 +
    let repo = Repo::init(root).unwrap();
405 +
    write(root, ".oxyattributes", b"*.md merge=union\n");
406 +
    write(root, "A.md", b"a\nb\n");
407 +
    write(root, "B.md", b"x\ny\n");
408 +
    let h1 = repo.commit("base", &[]).unwrap();
409 +
    write(root, "A.md", b"a\none\nb\n");
410 +
    write(root, "B.md", b"x\nY1\n");
411 +
    let h2 = repo.commit("one", &[]).unwrap();
412 +
    repo.checkout(&[h1], false, "checkout").unwrap();
413 +
    write(root, "A.md", b"a\ntwo\nb\n");
414 +
    let h3 = repo.commit("two", &[]).unwrap();
415 +
    let st = repo.load_state(&[h2, h3]).unwrap();
416 +
    assert_eq!(
417 +
        Repo::union_resolution(&st, &st.materialize()),
418 +
        Some(vec![("A.md".into(), b"a\none\ntwo\nb\n".to_vec())])
419 +
    );
420 +
    // B.md's line replaced two ways: a person decides.
421 +
    repo.checkout(&[h3], false, "checkout").unwrap();
422 +
    write(root, "B.md", b"x\nY2\n");
423 +
    let h4 = repo.commit("Y2", &[]).unwrap();
424 +
    let st = repo.load_state(&[h2, h4]).unwrap();
425 +
    assert!(st.materialize().is_conflicted());
426 +
    assert_eq!(Repo::union_resolution(&st, &st.materialize()), None);
427 +
}
428 +
429 +
/// oxygen#121: two changes that each replaced the same line are not an
430 +
/// adjacent insertion: keeping both would keep two versions of the line,
431 +
/// so `--both` leaves the markers, as it does for markers already edited.
432 +
#[test]
433 +
fn keeping_both_refuses_replaced_or_edited_lines() {
434 +
    hermetic();
435 +
    let (dir, repo, _, _, _) = conflicted_repo();
436 +
    let root = dir.path();
437 +
    repo.checkout(&repo.bookmark_heads().unwrap(), false, "checkout main")
438 +
        .unwrap();
439 +
    let marked = fs::read(root.join("f.txt")).unwrap();
440 +
    let snap = repo.snapshot("status").unwrap();
441 +
    let node = snap.base_snapshot.files["f.txt"].node;
442 +
    assert_eq!(snap.base.both_sides(&node), vec![None]);
443 +
    let kept = repo.keep_both(&snap, &["f.txt".into()], false).unwrap();
444 +
    assert!(kept[0].settled.is_empty());
445 +
    assert_eq!(kept[0].left[0].0, 1);
446 +
    assert!(kept[0].left[0].1.contains("both sides changed"), "{kept:?}");
447 +
    assert_eq!(fs::read(root.join("f.txt")).unwrap(), marked);
448 +
449 +
    // An adjacent insertion whose lines were edited between the markers.
450 +
    let sides = vec![Some(vec![b"x\n".to_vec(), b"y\n".to_vec()])];
451 +
    let text = b"a\n<<<<<<< oxy conflict 1 of 1, side 1 of 2: change k \"x\"\nx!\n======= oxy conflict 1 of 1, side 2 of 2: change q \"y\"\ny\n>>>>>>> oxy conflict 1 of 1 end\nb\n";
452 +
    let (out, settled, left) = super::snapshot::keep_both_text(text, &sides, false);
453 +
    assert_eq!(out, text.to_vec());
454 +
    assert!(settled.is_empty());
455 +
    assert!(left[0].1.contains("edited"), "{left:?}");
456 +
    // Unedited, in either order on disk: kept in the graph's order.
457 +
    let text = b"a\n<<<<<<< oxy conflict 1 of 1, side 1 of 2: change q \"y\"\ny\n======= oxy conflict 1 of 1, side 2 of 2: change k \"x\"\nx\n>>>>>>> oxy conflict 1 of 1 end\nb\n";
458 +
    let (out, settled, _) = super::snapshot::keep_both_text(text, &sides, false);
459 +
    assert_eq!(out, b"a\nx\ny\nb\n".to_vec());
460 +
    assert_eq!(settled, vec![1]);
461 +
}
462 +
179 463
#[test]
180 464
fn concurrent_edits_conflict_and_can_be_resolved() {
181 465
    hermetic();
…
66 unchanged lines
248 532
    );
249 533
}
250 534
535 +
/// oxygen#118 (Metal Gear): a resolution commits by paths, leaving other
536 +
/// work uncommitted, so long as it takes every file it resolves; and
537 +
/// `amend PATHS` folds only those paths into the head.
251 538
#[test]
539 +
fn resolutions_and_amends_take_paths() {
540 +
    hermetic();
541 +
    let (dir, repo, _h1, _h2, _h3) = conflicted_repo();
542 +
    let root = dir.path();
543 +
    repo.checkout(&repo.bookmark_heads().unwrap(), false, "checkout main")
544 +
        .unwrap();
545 +
    // The conflict resolved, and other work beside it.
546 +
    write(root, "f.txt", b"a\nX\nc\n");
547 +
    write(root, "notes.txt", b"pending\n");
548 +
    let s = repo.snapshot("status").unwrap();
549 +
    assert!(s.resolves);
550 +
    let err = repo
551 +
        .commit_selection_partial("resolve", &[], &Selection::Paths(vec!["notes.txt".into()]))
552 +
        .unwrap_err()
553 +
        .to_string();
554 +
    assert!(
555 +
        err.contains("takes every file it resolves") && err.contains("f.txt"),
556 +
        "{err}"
557 +
    );
558 +
    let (h4, partial) = repo
559 +
        .commit_selection_partial("resolve", &[], &Selection::Paths(vec!["f.txt".into()]))
560 +
        .unwrap();
561 +
    assert!(partial);
562 +
    assert_eq!(
563 +
        repo.store().get_change(&h4).unwrap().kind,
564 +
        ChangeKind::Resolution
565 +
    );
566 +
    assert!(!repo
567 +
        .load_state(&[h4])
568 +
        .unwrap()
569 +
        .materialize()
570 +
        .is_conflicted());
571 +
    let s = repo.snapshot("status").unwrap();
572 +
    assert_eq!(
573 +
        statuses(&s),
574 +
        vec![("notes.txt".into(), ChangeStatus::Added)],
575 +
        "the other work waits"
576 +
    );
577 +
578 +
    // amend PATHS: only those paths join the head; it keeps its ChangeId.
579 +
    // On the resolution, so the bookmark has one head to amend.
580 +
    let h5 = repo.commit("notes", &[h4]).unwrap();
581 +
    let id5 = repo.store().get_change(&h5).unwrap().change_id;
582 +
    write(root, "notes.txt", b"pending\nmore\n");
583 +
    write(root, "other.txt", b"later\n");
584 +
    let h6 = repo
585 +
        .amend_paths(Some("notes, amended"), &["notes.txt".into()])
586 +
        .unwrap();
587 +
    let c6 = repo.store().get_change(&h6).unwrap();
588 +
    assert_eq!((c6.change_id, c6.message.as_str()), (id5, "notes, amended"));
589 +
    assert_eq!(repo.bookmark_heads().unwrap(), vec![h6]);
590 +
    let snap6 = repo.load_state(&[h6]).unwrap().materialize();
591 +
    assert!(
592 +
        matches!(&snap6.files["notes.txt"].content, FileContent::Text { bytes, .. } if bytes == b"pending\nmore\n")
593 +
    );
594 +
    assert!(
595 +
        !snap6.files.contains_key("other.txt"),
596 +
        "other.txt stays out"
597 +
    );
598 +
    let s = repo.snapshot("status").unwrap();
599 +
    assert_eq!(
600 +
        statuses(&s),
601 +
        vec![("other.txt".into(), ChangeStatus::Added)]
602 +
    );
603 +
}
604 +
605 +
#[test]
252 606
fn resolution_may_keep_both_sides_or_neither() {
253 607
    hermetic();
254 608
    let (dir, repo, _h1, _h2, _h3) = conflicted_repo();
…
77 unchanged lines
332 686
    );
333 687
}
334 688
689 +
/// Each of two bookmarks adds `path` (a file written by `side` on each)
690 +
/// from the same base, and the second is merged into the first. Returns
691 +
/// the repository on `main`, the merge's heads, and the three changes.
692 +
fn merged_additions(
693 +
    side: impl Fn(&Path, bool),
694 +
) -> (tempfile::TempDir, Repo, Vec<ChangeHash>, [ChangeHash; 3]) {
695 +
    let dir = tempfile::tempdir().unwrap();
696 +
    let root = dir.path().to_path_buf();
697 +
    let mut repo = Repo::init(&root).unwrap();
698 +
    write(&root, "keep.txt", b"k\n");
699 +
    let h1 = repo.commit("base", &[]).unwrap();
700 +
    repo.bookmark_create("side", None).unwrap();
701 +
    side(&root, false);
702 +
    let ha = repo.commit("add on main", &[]).unwrap();
703 +
    repo.bookmark_switch("side", false).unwrap();
704 +
    side(&root, true);
705 +
    let hb = repo.commit("add on side", &[]).unwrap();
706 +
    repo.bookmark_switch("main", false).unwrap();
707 +
    let m = repo.merge("side").unwrap();
708 +
    assert!(m.changed && m.checked_out);
709 +
    (dir, repo, m.heads, [h1, ha, hb])
710 +
}
711 +
712 +
fn conflict_copies(root: &Path) -> Vec<String> {
713 +
    let mut out: Vec<String> = fs::read_dir(root)
714 +
        .unwrap()
715 +
        .map(|e| e.unwrap().file_name().to_string_lossy().into_owned())
716 +
        .filter(|n| n.contains(".oxy-conflict-"))
717 +
        .collect();
718 +
    out.sort();
719 +
    out
720 +
}
721 +
722 +
/// The same path added on two bookmarks with identical content is one
723 +
/// file after the merge (oxygen#101): no conflict, nothing to resolve,
724 +
/// nothing recorded; and editing or removing it later keeps it one.
335 725
#[test]
726 +
fn identical_additions_unify_on_merge() {
727 +
    hermetic();
728 +
    let (dir, repo, heads, [h1, ha, hb]) =
729 +
        merged_additions(|root, _| write(root, "x.txt", b"same\nlines\n"));
730 +
    let root = dir.path();
731 +
    // The tree graph has the name conflict; every reader sees one file.
732 +
    let st = repo.load_state(&heads).unwrap();
733 +
    assert!(st
734 +
        .tree()
735 +
        .view()
736 +
        .conflicts
737 +
        .iter()
738 +
        .any(|c| matches!(c, crate::tree::TreeConflict::Name { .. })));
739 +
    let snap = st.materialize();
740 +
    assert!(!snap.is_conflicted(), "{:?}", snap.tree.conflicts);
741 +
    assert_eq!(
742 +
        snap.files.keys().cloned().collect::<Vec<_>>(),
743 +
        ["keep.txt", "x.txt"]
744 +
    );
745 +
    assert!(
746 +
        matches!(&snap.files["x.txt"].content, FileContent::Text { bytes, .. } if bytes == b"same\nlines\n")
747 +
    );
748 +
    assert_eq!(snap.tree.unified.len(), 1);
749 +
    // On disk once, the working copy clean, nothing to commit.
750 +
    assert_eq!(fs::read(root.join("x.txt")).unwrap(), b"same\nlines\n");
751 +
    assert!(
752 +
        conflict_copies(root).is_empty(),
753 +
        "{:?}",
754 +
        conflict_copies(root)
755 +
    );
756 +
    let s = repo.snapshot("status").unwrap();
757 +
    assert!(s.statuses.is_empty(), "{:?}", statuses(&s));
758 +
    assert!(!s.changed && !s.resolves);
759 +
    assert!(!s.base_snapshot.is_conflicted());
760 +
    assert!(s.wc.tree_ops.is_empty() && s.wc.edits.is_empty());
761 +
    let json = repo.status_json(&s).unwrap();
762 +
    assert!(json.contains("\"conflicted\":false"), "{json}");
763 +
    assert!(repo.commit("nothing", &[]).is_err());
764 +
    assert_eq!(repo.file_at("x.txt", None).unwrap(), b"same\nlines\n");
765 +
    // A peer holding the same changes, added in another order, agrees.
766 +
    let mut peer = State::new();
767 +
    for h in [hb, ha, h1] {
768 +
        peer.add_unchecked(repo.store().get_change(&h).unwrap());
769 +
    }
770 +
    peer.validate().unwrap();
771 +
    let ps = peer.materialize();
772 +
    assert!(!ps.is_conflicted());
773 +
    assert_eq!(ps.tree.entries["x.txt"], snap.tree.entries["x.txt"]);
774 +
    assert_eq!(ps.tree.unified, snap.tree.unified);
775 +
    // The file's history is both additions.
776 +
    let log: Vec<ChangeHash> = repo
777 +
        .log_for_path("x.txt")
778 +
        .unwrap()
779 +
        .iter()
780 +
        .map(|e| e.hash)
781 +
        .collect();
782 +
    assert!(log.contains(&ha) && log.contains(&hb), "{log:?}");
783 +
784 +
    // Edited and committed: one file still, the duplicate retired with it.
785 +
    write(root, "x.txt", b"same\nedited\n");
786 +
    let s = repo.snapshot("status").unwrap();
787 +
    assert_eq!(statuses(&s), vec![("x.txt".into(), ChangeStatus::Modified)]);
788 +
    let he = repo.commit("edit x", &[]).unwrap();
789 +
    let c = repo.store().get_change(&he).unwrap();
790 +
    assert_eq!(c.kind, ChangeKind::Normal);
791 +
    let (dup, _) = snap.tree.unified.iter().next().unwrap();
792 +
    assert!(
793 +
        c.tree_ops
794 +
            .iter()
795 +
            .any(|o| matches!(o, TreeOp::RemoveNode { node, .. } if node == dup)),
796 +
        "{:?}",
797 +
        c.tree_ops
798 +
    );
799 +
    let d = repo.describe_change(&he).unwrap();
800 +
    assert!(!d.contains("D x.txt"), "{d}");
801 +
    let st = repo.load_state(&repo.bookmark_heads().unwrap()).unwrap();
802 +
    assert!(
803 +
        st.tree().view().conflicts.is_empty(),
804 +
        "nothing left to unify"
805 +
    );
806 +
    let snap = st.materialize();
807 +
    assert!(!snap.is_conflicted());
808 +
    assert!(
809 +
        matches!(&snap.files["x.txt"].content, FileContent::Text { bytes, .. } if bytes == b"same\nedited\n")
810 +
    );
811 +
    assert!(!repo.snapshot("status").unwrap().changed);
812 +
813 +
    // Removed and committed: gone.
814 +
    fs::remove_file(root.join("x.txt")).unwrap();
815 +
    repo.commit("drop x", &[]).unwrap();
816 +
    let snap = repo
817 +
        .load_state(&repo.bookmark_heads().unwrap())
818 +
        .unwrap()
819 +
        .materialize();
820 +
    assert!(!snap.is_conflicted(), "{:?}", snap.tree.conflicts);
821 +
    assert_eq!(snap.files.keys().cloned().collect::<Vec<_>>(), ["keep.txt"]);
822 +
    assert!(!root.join("x.txt").exists());
823 +
}
824 +
825 +
/// Removing, renaming, or committing just the path of a unified file
826 +
/// retires its duplicate with it, so the conflict never comes back.
827 +
#[test]
828 +
fn unified_additions_stay_one_when_removed_renamed_or_committed_alone() {
829 +
    hermetic();
830 +
    let add = |root: &Path, _: bool| write(root, "x.txt", b"same\n");
831 +
    for how in ["remove", "rename", "path", "hunk"] {
832 +
        let (dir, repo, _, _) = merged_additions(add);
833 +
        let root = dir.path();
834 +
        match how {
835 +
            "remove" => fs::remove_file(root.join("x.txt")).unwrap(),
836 +
            "rename" => fs::rename(root.join("x.txt"), root.join("y.txt")).unwrap(),
837 +
            _ => {
838 +
                write(root, "x.txt", b"same\nmore\n");
839 +
                write(root, "other.txt", b"o\n");
840 +
            }
841 +
        }
842 +
        let s = repo.snapshot("status").unwrap();
843 +
        let h = match how {
844 +
            "path" => repo
845 +
                .commit_selection("x alone", &[], &Selection::Paths(vec!["x.txt".into()]))
846 +
                .unwrap(),
847 +
            "hunk" => {
848 +
                let (_, hunks) = repo.hunks().unwrap();
849 +
                // The duplicate's retirement is no hunk of its own.
850 +
                assert!(hunks.iter().all(|h| !h.text.starts_with("D ")), "{hunks:?}");
851 +
                let x: Vec<usize> = hunks
852 +
                    .iter()
853 +
                    .filter(|h| h.path == "x.txt")
854 +
                    .map(|h| h.index)
855 +
                    .collect();
856 +
                assert!(!x.is_empty());
857 +
                repo.commit_selection("x hunks", &[], &Selection::Hunks(x))
858 +
                    .unwrap()
859 +
            }
860 +
            _ => repo.commit(how, &[]).unwrap(),
861 +
        };
862 +
        assert!(!s.base_snapshot.is_conflicted());
863 +
        let heads = repo.bookmark_heads().unwrap();
864 +
        assert!(heads.contains(&h));
865 +
        let snap = repo.load_state(&heads).unwrap().materialize();
866 +
        assert!(!snap.is_conflicted(), "{how}: {:?}", snap.tree.conflicts);
867 +
        let want: &[&str] = match how {
868 +
            "remove" => &["keep.txt"],
869 +
            "rename" => &["keep.txt", "y.txt"],
870 +
            _ => &["keep.txt", "x.txt"],
871 +
        };
872 +
        assert_eq!(
873 +
            snap.files.keys().cloned().collect::<Vec<_>>(),
874 +
            want,
875 +
            "{how}"
876 +
        );
877 +
        assert!(conflict_copies(root).is_empty(), "{how}");
878 +
    }
879 +
}
880 +
881 +
/// The same path added with different content, or a different mode, is a
882 +
/// name conflict as before.
883 +
#[test]
884 +
fn additions_that_differ_still_conflict() {
885 +
    hermetic();
886 +
    #[cfg(unix)]
887 +
    use std::os::unix::fs::PermissionsExt;
888 +
    let content =
889 +
        |root: &Path, side: bool| write(root, "x.txt", if side { b"two\n" } else { b"one\n" });
890 +
    let mode = |root: &Path, side: bool| {
891 +
        write(root, "x.txt", b"same\n");
892 +
        #[cfg(unix)]
893 +
        if side {
894 +
            fs::set_permissions(root.join("x.txt"), fs::Permissions::from_mode(0o755)).unwrap();
895 +
        }
896 +
    };
897 +
    let check = |what: &str, side: &dyn Fn(&Path, bool)| {
898 +
        let (dir, repo, heads, _) = merged_additions(side);
899 +
        let snap = repo.load_state(&heads).unwrap().materialize();
900 +
        assert!(snap.tree.unified.is_empty(), "{what}");
901 +
        assert!(
902 +
            snap.tree.conflicts.iter().any(
903 +
                |c| matches!(c, crate::tree::TreeConflict::Name { name, .. } if name == "x.txt")
904 +
            ),
905 +
            "{what}: {:?}",
906 +
            snap.tree.conflicts
907 +
        );
908 +
        assert_eq!(
909 +
            conflict_copies(dir.path()),
910 +
            ["x.txt.oxy-conflict-1", "x.txt.oxy-conflict-2"],
911 +
            "{what}"
912 +
        );
913 +
        assert!(!dir.path().join("x.txt").exists(), "{what}");
914 +
    };
915 +
    check("content", &content);
916 +
    #[cfg(unix)]
917 +
    check("mode", &mode);
918 +
    #[cfg(not(unix))]
919 +
    let _ = mode;
920 +
}
921 +
922 +
/// The same subrepo at the same pin, added on two bookmarks, is one
923 +
/// subrepo after the merge (oxygen#101), as is the identical
924 +
/// `.oxymodules` beside it.
925 +
#[test]
926 +
fn the_same_subrepo_added_on_two_bookmarks_unifies() {
927 +
    hermetic();
928 +
    let dir = tempfile::tempdir().unwrap();
929 +
    let root = dir.path();
930 +
    let lib_origin = Repo::init(&root.join("lib-origin")).unwrap();
931 +
    let lib_url = root.join("lib-origin").to_string_lossy().to_string();
932 +
    let lib = Repo::clone(&lib_url, &root.join("lib"))
933 +
        .unwrap()
934 +
        .with_home(root.join("home"));
935 +
    write(&root.join("lib"), "lib.txt", b"v1\n");
936 +
    lib.commit("lib v1", &[]).unwrap();
937 +
    assert!(lib.push("origin", false).unwrap()[0].ok);
938 +
    let _ = lib_origin;
939 +
940 +
    let app_dir = root.join("app");
941 +
    let mut app = Repo::init(&app_dir).unwrap().with_home(root.join("home"));
942 +
    write(&app_dir, "main.txt", b"app\n");
943 +
    app.commit("base", &[]).unwrap();
944 +
    app.bookmark_create("side", None).unwrap();
945 +
    app.submodule_add(&lib_url, "rstar-api", None).unwrap();
946 +
    let modules = fs::read(app_dir.join(".oxymodules")).unwrap();
947 +
    app.commit("rstar-api on main", &[]).unwrap();
948 +
    // On the other bookmark the nested repository is still there (it is
949 +
    // never deleted); registering it again adds it at the same pin.
950 +
    app.bookmark_switch("side", false).unwrap();
951 +
    assert!(app_dir.join("rstar-api/.oxy").is_dir());
952 +
    write(&app_dir, ".oxymodules", &modules);
953 +
    let s = app.snapshot("status").unwrap();
954 +
    assert_eq!(
955 +
        statuses(&s),
956 +
        vec![
957 +
            (".oxymodules".into(), ChangeStatus::Added),
958 +
            ("rstar-api".into(), ChangeStatus::Added)
959 +
        ]
960 +
    );
961 +
    app.commit("rstar-api on side", &[]).unwrap();
962 +
    app.bookmark_switch("main", false).unwrap();
963 +
964 +
    let m = app.merge("side").unwrap();
965 +
    assert!(m.changed && m.checked_out);
966 +
    let snap = app.load_state(&m.heads).unwrap().materialize();
967 +
    assert!(!snap.is_conflicted(), "{:?}", snap.tree.conflicts);
968 +
    assert_eq!(snap.tree.unified.len(), 2);
969 +
    assert_eq!(snap.files["rstar-api"].kind, NodeKind::Subrepo);
970 +
    assert!(conflict_copies(&app_dir).is_empty());
971 +
    let s = app.snapshot("status").unwrap();
972 +
    assert!(s.statuses.is_empty(), "{:?}", statuses(&s));
973 +
    assert!(!s.base_snapshot.is_conflicted());
974 +
    assert_eq!(
975 +
        fs::read_to_string(app_dir.join("rstar-api/lib.txt")).unwrap(),
976 +
        "v1\n"
977 +
    );
978 +
}
979 +
980 +
#[test]
336 981
fn identities_and_signed_commits() {
337 982
    hermetic();
338 983
    let dir = tempfile::tempdir().unwrap();
…
98 unchanged lines
437 1082
        CheckoutReport {
438 1083
            written: 2,
439 1084
            removed: 1,
1085 +
            removed_paths: vec!["bin.dat".into()],
440 1086
            conflicted: false,
441 1087
            notes: vec![],
442 1088
            lacks: 1,
1089 +
            kept_both: vec![],
443 1090
        }
444 1091
    );
445 1092
    assert_eq!(fs::read(root.join("a.txt")).unwrap(), b"one\n");
…
65 unchanged lines
511 1158
    write(&root.join("a"), "f.txt", b"a\n");
512 1159
    let h1 = a.commit("first", &[]).unwrap();
513 1160
    let results = a.push("origin", false).unwrap();
1161 +
    origin.refresh().unwrap();
514 1162
    assert!(results[0].ok, "{}", results[0].reason);
515 1163
    assert_eq!(origin.bookmark_heads().unwrap(), vec![h1]);
516 1164
    // The change was sealed at push and the reference attested.
…
14 unchanged lines
531 1179
    let first = serial_of(&att);
532 1180
    assert!(first > 1_000_000_000_000, "{first}");
533 1181
    let again = a.push("origin", false).unwrap();
1182 +
    origin.refresh().unwrap();
534 1183
    assert!(again[0].ok && again[0].reason.contains("up to date"));
535 1184
    let (_, att) = origin.latest_ref_attest("main").unwrap().unwrap();
536 1185
    assert_eq!(serial_of(&att), first, "a no-op push does not attest again");
…
11 unchanged lines
548 1197
    write(&root.join("b"), "f.txt", b"a\nb\n");
549 1198
    let h2 = b.commit("second", &[]).unwrap();
550 1199
    assert!(b.push("origin", false).unwrap()[0].ok);
1200 +
    origin.refresh().unwrap();
551 1201
    assert_eq!(origin.bookmark_heads().unwrap(), vec![h2]);
552 1202
553 1203
    // A commits an independent change; push is refused until A pulls.
…
8 unchanged lines
562 1212
    assert_eq!(fs::read(root.join("a/f.txt")).unwrap(), b"a\nb\n");
563 1213
    assert_eq!(fs::read(root.join("a/g.txt")).unwrap(), b"g\n");
564 1214
    let results = a.push("origin", false).unwrap();
1215 +
    origin.refresh().unwrap();
565 1216
    assert!(results[0].ok, "{}", results[0].reason);
566 1217
    let mut origin_heads = origin.bookmark_heads().unwrap();
567 1218
    origin_heads.sort();
…
11 unchanged lines
579 1230
    write(&root.join("a"), "f.txt", b"a\nA\n");
580 1231
    a.commit("a-edit", &[]).unwrap();
581 1232
    a.push("origin", false).unwrap();
1233 +
    origin.refresh().unwrap();
582 1234
    write(&root.join("b"), "f.txt", b"a\nB\n");
583 1235
    b.commit("b-edit", &[]).unwrap();
584 1236
    let pr = b.pull("origin").unwrap();
…
7 unchanged lines
592 1244
        ChangeKind::Resolution
593 1245
    );
594 1246
    assert!(b.push("origin", false).unwrap()[0].ok);
1247 +
    origin.refresh().unwrap();
595 1248
    a.pull("origin").unwrap();
596 1249
    assert_eq!(fs::read(root.join("a/f.txt")).unwrap(), b"a\nAB\n");
597 1250
    // The resolution depends on the conflicting edits, but nothing depends
…
369 unchanged lines
967 1620
    assert!(!snap.is_conflicted());
968 1621
}
969 1622
1623 +
/// oxygen#87: a binary file given new contents and the executable bit in
1624 +
/// one commit kept the contents and lost the bit.
1625 +
#[cfg(unix)]
1626 +
#[test]
1627 +
fn a_binary_file_changed_and_made_executable_at_once_keeps_both() {
1628 +
    use std::os::unix::fs::PermissionsExt;
1629 +
    hermetic();
1630 +
    let dir = tempfile::tempdir().unwrap();
1631 +
    let root = dir.path();
1632 +
    let repo = Repo::init(root).unwrap();
1633 +
    write(root, "tool.exe", b"MZ\0one");
1634 +
    repo.commit("add", &[]).unwrap();
1635 +
    write(root, "tool.exe", b"MZ\0two");
1636 +
    fs::set_permissions(root.join("tool.exe"), fs::Permissions::from_mode(0o755)).unwrap();
1637 +
    let h = repo.commit("update and +x", &[]).unwrap();
1638 +
    let snap = repo.load_state(&[h]).unwrap().materialize();
1639 +
    let f = &snap.files["tool.exe"];
1640 +
    assert!(f.mode.opaque && f.mode.exec, "{:?}", f.mode);
1641 +
    assert!(
1642 +
        repo.snapshot("status").unwrap().statuses.is_empty(),
1643 +
        "nothing left to commit"
1644 +
    );
1645 +
}
1646 +
1647 +
/// Keyless step 4 (oxygenhub#241): an agent signing with a short-lived
1648 +
/// key whose certificate an issuer gave pushes when the policy grants its
1649 +
/// subject, and a fetch accepts what it attested; an expired certificate,
1650 +
/// or a subject the policy does not grant, may not move the bookmark.
1651 +
#[test]
1652 +
fn a_certified_key_pushes_under_a_subject_grant() {
1653 +
    use crate::cert::{Certificate, SignedCertificate};
1654 +
    use crate::identity::Scope;
1655 +
    hermetic();
1656 +
    let dir = tempfile::tempdir().unwrap();
1657 +
    let root = dir.path();
1658 +
    let _origin = Repo::init(&root.join("origin")).unwrap();
1659 +
    let url = root.join("origin").to_string_lossy().to_string();
1660 +
    let mut a = Repo::clone(&url, &root.join("a"))
1661 +
        .unwrap()
1662 +
        .with_home(root.join("home-a"));
1663 +
    a.create_identity("Ada", "[email protected]", IdentityScope::User)
1664 +
        .unwrap();
1665 +
    write(&root.join("a"), "a.txt", b"a\n");
1666 +
    a.commit("a", &[]).unwrap();
1667 +
    a.policy_init().unwrap();
1668 +
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
1669 +
    let hub_sk = SecretKey::generate();
1670 +
    let hub = hub_sk.identity("hub", "[email protected]", 1);
1671 +
    a.policy_grant_subject(
1672 +
        "agent:claude@ada",
1673 +
        hub.clone(),
1674 +
        Scope::CHANGES | Scope::REFS,
1675 +
        None,
1676 +
    )
1677 +
    .unwrap();
1678 +
    assert!(a
1679 +
        .push("origin", false)
1680 +
        .unwrap()
1681 +
        .iter()
1682 +
        .any(|r| r.name == "policy" && r.ok));
1683 +
1684 +
    let now = Hlc::now().wall_ms;
1685 +
    // An agent repository whose key the hub certified for `subject`, from
1686 +
    // `from` to `to`.
1687 +
    let agent = |name: &str, subject: &str, from: u64, to: u64| {
1688 +
        let mut r = Repo::clone(&url, &root.join(name))
1689 +
            .unwrap()
1690 +
            .with_home(root.join(format!("home-{name}")));
1691 +
        let (_, id) = r
1692 +
            .create_identity(name, "[email protected]", IdentityScope::User)
1693 +
            .unwrap();
1694 +
        let cert = SignedCertificate::sign(
1695 +
            Certificate {
1696 +
                key: id,
1697 +
                issuer: hub.hash(),
1698 +
                subject: subject.into(),
1699 +
                claims: vec![],
1700 +
                scope: Scope::CHANGES | Scope::REFS,
1701 +
                not_before: from,
1702 +
                expires: to,
1703 +
                grant: None,
1704 +
            },
1705 +
            &hub,
1706 +
            &hub_sk,
1707 +
        );
1708 +
        r.store().put_identity(&hub).unwrap();
1709 +
        r.store().put_certificate(&cert).unwrap();
1710 +
        write(&root.join(name), &format!("{name}.txt"), b"agent work\n");
1711 +
        r.commit("work", &[]).unwrap();
1712 +
        r
1713 +
    };
1714 +
1715 +
    let claude = agent("claude", "agent:claude@ada", now - 60_000, now + 600_000);
1716 +
    let r = claude.push("origin", false).unwrap();
1717 +
    assert!(r.iter().all(|x| x.ok), "{r:?}");
1718 +
    // Ada's fetch accepts the bookmark the certified key attested.
1719 +
    a.pull("origin").unwrap();
1720 +
    assert!(a.bookmark_heads().unwrap().iter().any(|h| a
1721 +
        .store()
1722 +
        .get_change(h)
1723 +
        .map(|c| c.message == "work")
1724 +
        .unwrap_or(false)));
1725 +
1726 +
    // Step 4c: a host with an acceptor key records when it accepted the
1727 +
    // certified update and serves it; a fetch checks the certificate at
1728 +
    // that time only when the certificate's issuer signed it for exactly
1729 +
    // this attestation, else at the attestation's own time.
1730 +
    let host = Repo::open(&root.join("origin"))
1731 +
        .unwrap()
1732 +
        .with_acceptor(hub.clone(), hub_sk.clone());
1733 +
    let (attest_ref, sa) = host.latest_ref_attest("main").unwrap().unwrap();
1734 +
    host.record_acceptance(&attest_ref).unwrap();
1735 +
    let served = host.remote_refs().unwrap();
1736 +
    let acc = served
1737 +
        .refs
1738 +
        .iter()
1739 +
        .find(|r| r.name == "main")
1740 +
        .unwrap()
1741 +
        .accepted
1742 +
        .clone()
1743 +
        .expect("served with the refs");
1744 +
    let signer = sa.sigs[0].key;
1745 +
    assert_eq!(
1746 +
        a.certified_time(&signer, &sa, Some(&acc)),
1747 +
        acc.attest.hlc.wall_ms
1748 +
    );
1749 +
    assert_eq!(a.certified_time(&signer, &sa, None), sa.attest.hlc.wall_ms);
1750 +
    let stranger_sk = SecretKey::generate();
1751 +
    let stranger = stranger_sk.identity("stranger", "s@x", 1);
1752 +
    let mut forged = acc.clone();
1753 +
    forged.sigs = vec![stranger_sk.sign(Domain::Attest, &stranger.hash(), &acc.attest.hash())];
1754 +
    assert_eq!(
1755 +
        a.certified_time(&signer, &sa, Some(&forged)),
1756 +
        sa.attest.hlc.wall_ms,
1757 +
        "not the issuer's"
1758 +
    );
1759 +
    let mut elsewhere = acc.clone();
1760 +
    elsewhere.attest.body = AttestBody::Accepted {
1761 +
        attest: ObjectRef([7; 32]),
1762 +
    };
1763 +
    elsewhere.sigs = vec![hub_sk.sign(Domain::Attest, &hub.hash(), &elsewhere.attest.hash())];
1764 +
    assert_eq!(
1765 +
        a.certified_time(&signer, &sa, Some(&elsewhere)),
1766 +
        sa.attest.hlc.wall_ms,
1767 +
        "another attestation's"
1768 +
    );
1769 +
    a.pull("origin").unwrap();
1770 +
    // gc keeps the acceptance with the attestation.
1771 +
    host.gc(true).unwrap();
1772 +
    assert!(host.acceptance_of(&attest_ref).unwrap().is_some());
1773 +
1774 +
    let late = agent("late", "agent:claude@ada", now - 1_200_000, now - 600_000);
1775 +
    let r = late.push("origin", false).unwrap();
1776 +
    assert!(
1777 +
        !r[0].ok && r[0].reason.contains("may not update"),
1778 +
        "an expired certificate: {r:?}"
1779 +
    );
1780 +
    let other = agent("other", "agent:mallory@ada", now - 60_000, now + 600_000);
1781 +
    let r = other.push("origin", false).unwrap();
1782 +
    assert!(
1783 +
        !r[0].ok && r[0].reason.contains("may not update"),
1784 +
        "a subject not granted: {r:?}"
1785 +
    );
1786 +
}
1787 +
1788 +
/// A tag made under one short-lived key and pushed under another: its
1789 +
/// signer travels with the push (identity and certificate), so a receiver
1790 +
/// that has never seen that key accepts it. A tag-only push by a keyless
1791 +
/// agent was refused "unknown key" (found releasing OxygenFlow 0.2.0).
1792 +
#[test]
1793 +
fn a_tag_signed_by_an_earlier_short_lived_key_is_pushed() {
1794 +
    use crate::cert::{Certificate, SignedCertificate};
1795 +
    use crate::identity::Scope;
1796 +
    hermetic();
1797 +
    let dir = tempfile::tempdir().unwrap();
1798 +
    let root = dir.path();
1799 +
    let _origin = Repo::init(&root.join("origin")).unwrap();
1800 +
    let url = root.join("origin").to_string_lossy().to_string();
1801 +
    let mut a = Repo::clone(&url, &root.join("a"))
1802 +
        .unwrap()
1803 +
        .with_home(root.join("home-a"));
1804 +
    a.create_identity("Ada", "[email protected]", IdentityScope::User)
1805 +
        .unwrap();
1806 +
    write(&root.join("a"), "a.txt", b"a\n");
1807 +
    a.commit("a", &[]).unwrap();
1808 +
    a.policy_init().unwrap();
1809 +
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
1810 +
    let hub_sk = SecretKey::generate();
1811 +
    let hub = hub_sk.identity("hub", "[email protected]", 1);
1812 +
    a.policy_grant_subject(
1813 +
        "agent:claude@ada",
1814 +
        hub.clone(),
1815 +
        Scope::CHANGES | Scope::REFS,
1816 +
        Some("*"),
1817 +
    )
1818 +
    .unwrap();
1819 +
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
1820 +
1821 +
    let now = Hlc::now().wall_ms;
1822 +
    let mut agent = Repo::clone(&url, &root.join("agent"))
1823 +
        .unwrap()
1824 +
        .with_home(root.join("home-agent"));
1825 +
    agent.store().put_identity(&hub).unwrap();
1826 +
    let certify = |agent: &Repo, id| {
1827 +
        let cert = SignedCertificate::sign(
1828 +
            Certificate {
1829 +
                key: id,
1830 +
                issuer: hub.hash(),
1831 +
                subject: "agent:claude@ada".into(),
1832 +
                claims: vec![],
1833 +
                scope: Scope::CHANGES | Scope::REFS,
1834 +
                not_before: now - 60_000,
1835 +
                expires: now + 600_000,
1836 +
                grant: None,
1837 +
            },
1838 +
            &hub,
1839 +
            &hub_sk,
1840 +
        );
1841 +
        agent.store().put_certificate(&cert).unwrap();
1842 +
    };
1843 +
    // The tag, under the first short-lived key.
1844 +
    let (first, first_id) = agent
1845 +
        .create_identity("claude", "[email protected]", IdentityScope::User)
1846 +
        .unwrap();
1847 +
    certify(&agent, first_id);
1848 +
    let heads = agent.bookmark_heads().unwrap();
1849 +
    agent.tag_create("v1", &heads, "first").unwrap();
1850 +
    // The push, under a second one (a later process's ten-minute key).
1851 +
    let (second, second_id) = agent
1852 +
        .create_identity("claude", "[email protected]", IdentityScope::User)
1853 +
        .unwrap();
1854 +
    assert_ne!(first, second);
1855 +
    certify(&agent, second_id);
1856 +
    agent
1857 +
        .set_user_config("author.signer", &second.to_hex())
1858 +
        .unwrap();
1859 +
    let host = Repo::open(&root.join("origin")).unwrap();
1860 +
    assert!(
1861 +
        host.store().get_identity(&first).is_err(),
1862 +
        "the host has never seen the tag's key"
1863 +
    );
1864 +
    let r = agent.push("origin", false).unwrap();
1865 +
    assert!(r.iter().all(|x| x.ok), "{r:?}");
1866 +
    let host = Repo::open(&root.join("origin")).unwrap();
1867 +
    assert!(
1868 +
        host.tag_refs().unwrap().iter().any(|t| t.name == "v1"),
1869 +
        "the tag arrived"
1870 +
    );
1871 +
}
1872 +
1873 +
/// oxygen#129: a short-lived key's tag is checked on a fetch at a time its
1874 +
/// signer did not choose, its issuer's acceptance, never at the time it
1875 +
/// claims nor at the fetch's (when the ten-minute certificate has long
1876 +
/// expired). Without an acceptance it is left out and said so; with one
1877 +
/// inside the certificate's window it is fetched however much later, and
1878 +
/// a host it is pushed on to keeps and serves the acceptance.
1879 +
#[test]
1880 +
fn a_short_lived_key_s_tag_is_fetched_at_its_issuer_s_acceptance() {
1881 +
    use crate::cert::{Certificate, SignedCertificate};
1882 +
    use crate::identity::Scope;
1883 +
    hermetic();
1884 +
    let dir = tempfile::tempdir().unwrap();
1885 +
    let root = dir.path();
1886 +
    let _origin = Repo::init(&root.join("origin")).unwrap();
1887 +
    let url = root.join("origin").to_string_lossy().to_string();
1888 +
    let mut a = Repo::clone(&url, &root.join("a"))
1889 +
        .unwrap()
1890 +
        .with_home(root.join("home-a"));
1891 +
    a.create_identity("Ada", "[email protected]", IdentityScope::User)
1892 +
        .unwrap();
1893 +
    write(&root.join("a"), "a.txt", b"a\n");
1894 +
    a.commit("a", &[]).unwrap();
1895 +
    a.policy_init().unwrap();
1896 +
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
1897 +
    let hub_sk = SecretKey::generate();
1898 +
    let hub = hub_sk.identity("hub", "[email protected]", 1);
1899 +
    a.policy_grant_subject(
1900 +
        "agent:claude@ada",
1901 +
        hub.clone(),
1902 +
        Scope::CHANGES | Scope::REFS,
1903 +
        Some("*"),
1904 +
    )
1905 +
    .unwrap();
1906 +
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
1907 +
1908 +
    let now = Hlc::now().wall_ms;
1909 +
    // An agent whose key the hub certified from `from` to `to`.
1910 +
    let agent = |name: &str, from: u64, to: u64| {
1911 +
        let mut r = Repo::clone(&url, &root.join(name))
1912 +
            .unwrap()
1913 +
            .with_home(root.join(format!("home-{name}")));
1914 +
        r.store().put_identity(&hub).unwrap();
1915 +
        let (_, id) = r
1916 +
            .create_identity(name, "[email protected]", IdentityScope::User)
1917 +
            .unwrap();
1918 +
        let cert = SignedCertificate::sign(
1919 +
            Certificate {
1920 +
                key: id,
1921 +
                issuer: hub.hash(),
1922 +
                subject: "agent:claude@ada".into(),
1923 +
                claims: vec![],
1924 +
                scope: Scope::CHANGES | Scope::REFS,
1925 +
                not_before: from,
1926 +
                expires: to,
1927 +
                grant: None,
1928 +
            },
1929 +
            &hub,
1930 +
            &hub_sk,
1931 +
        );
1932 +
        r.store().put_certificate(&cert).unwrap();
1933 +
        r
1934 +
    };
1935 +
1936 +
    // A tag by a key certified now, pushed to a host that records no
1937 +
    // acceptance (a path on disk): it lands, but a fetch has no time to
1938 +
    // check its certificate at, so it is left out and the fetch goes on.
1939 +
    let claude = agent("claude", now - 60_000, now + 600_000);
1940 +
    let heads = claude.bookmark_heads().unwrap();
1941 +
    claude.tag_create("v1", &heads, "one").unwrap();
1942 +
    let r = claude.push("origin", false).unwrap();
1943 +
    assert!(r.iter().all(|x| x.ok), "{r:?}");
1944 +
    let b = Repo::clone(&url, &root.join("b"))
1945 +
        .unwrap()
1946 +
        .with_home(root.join("home-b"));
1947 +
    let rep = b.fetch("origin").unwrap();
1948 +
    assert!(
1949 +
        rep.skipped
1950 +
            .iter()
1951 +
            .any(|(n, why)| n == "tags/v1" && why.contains("no time to check its certificate at")),
1952 +
        "{rep:?}"
1953 +
    );
1954 +
    assert_eq!(b.tag_heads("v1").unwrap(), None);
1955 +
1956 +
    // The host's acceptance, signed by the certificate's issuer, served
1957 +
    // with the tag: fetched.
1958 +
    let host = Repo::open(&root.join("origin"))
1959 +
        .unwrap()
1960 +
        .with_acceptor(hub.clone(), hub_sk.clone());
1961 +
    let (v1, _) = host.tag_attest("v1").unwrap().unwrap();
1962 +
    host.record_acceptance(&v1).unwrap();
1963 +
    let served = host.tag_refs().unwrap();
1964 +
    assert!(served
1965 +
        .iter()
1966 +
        .any(|t| t.name == "v1" && t.accepted.is_some()));
1967 +
    let rep = b.fetch("origin").unwrap();
1968 +
    assert!(rep.skipped.is_empty(), "{rep:?}");
1969 +
    assert_eq!(b.tag_heads("v1").unwrap(), Some(heads.clone()));
1970 +
    // gc keeps the acceptance with the tag.
1971 +
    host.gc(true).unwrap();
1972 +
    assert!(host.acceptance_of(&v1).unwrap().is_some());
1973 +
1974 +
    // Long after: a key whose certificate expired an hour ago may not
1975 +
    // create a tag now.
1976 +
    let late = agent("late", now - 7_200_000, now - 3_600_000);
1977 +
    late.tag_create("v2", &heads, "two").unwrap();
1978 +
    let r = late.push("origin", false).unwrap();
1979 +
    assert!(
1980 +
        r.iter()
1981 +
            .any(|x| x.name == "tags/v2" && !x.ok && x.reason.contains("may not create")),
1982 +
        "{r:?}"
1983 +
    );
1984 +
    // With its issuer's acceptance from inside the window (as a hub gave
1985 +
    // it then, passed on to another host), it is taken at that time: by
1986 +
    // the host it is pushed to, which keeps it, and by a fresh clone.
1987 +
    let (v2, _) = late.tag_attest("v2").unwrap().unwrap();
1988 +
    let acc = Attest {
1989 +
        repo: late.id().unwrap(),
1990 +
        hlc: Hlc {
1991 +
            wall_ms: now - 5_400_000,
1992 +
            logical: 0,
1993 +
        },
1994 +
        body: AttestBody::Accepted { attest: v2 },
1995 +
    };
1996 +
    let sig = hub_sk.sign(Domain::Attest, &hub.hash(), &acc.hash());
1997 +
    // One by any other key gives no time.
1998 +
    let stranger_sk = SecretKey::generate();
1999 +
    let stranger = stranger_sk.identity("stranger", "[email protected]", 1);
2000 +
    let forged = stranger_sk.sign(Domain::Attest, &stranger.hash(), &acc.hash());
2001 +
    late.keep_acceptance(
2002 +
        &v2,
2003 +
        &SignedAttest {
2004 +
            attest: acc.clone(),
2005 +
            sigs: vec![forged],
2006 +
        },
2007 +
    )
2008 +
    .unwrap();
2009 +
    let r = late.push("origin", false).unwrap();
2010 +
    assert!(
2011 +
        r.iter().any(|x| x.name == "tags/v2" && !x.ok),
2012 +
        "not the issuer's: {r:?}"
2013 +
    );
2014 +
    let theirs = SignedAttest {
2015 +
        attest: acc,
2016 +
        sigs: vec![sig],
2017 +
    };
2018 +
    // Kept beside the stranger's: the issuer's is the one sent.
2019 +
    late.keep_acceptance(&v2, &theirs).unwrap();
2020 +
    let r = late.push("origin", false).unwrap();
2021 +
    // (Its re-attestation of main is refused: an expired key moves no
2022 +
    // bookmark.)
2023 +
    assert!(r.iter().any(|x| x.name == "tags/v2" && x.ok), "{r:?}");
2024 +
    let host = Repo::open(&root.join("origin")).unwrap();
2025 +
    let (hv2, _) = host.tag_attest("v2").unwrap().unwrap();
2026 +
    assert_eq!(hv2, v2);
2027 +
    assert!(host
2028 +
        .store()
2029 +
        .index_get_pub("accepted", &v2)
2030 +
        .unwrap()
2031 +
        .contains(&theirs.attest.hash()));
2032 +
    let c = Repo::clone(&url, &root.join("c"))
2033 +
        .unwrap()
2034 +
        .with_home(root.join("home-c"));
2035 +
    assert_eq!(c.tag_heads("v2").unwrap(), Some(heads.clone()));
2036 +
    assert_eq!(c.tag_heads("v1").unwrap(), Some(heads));
2037 +
}
2038 +
2039 +
/// oxygenhub#266: a short-lived key its issuer certified moves a bookmark
2040 +
/// and signs no change (a keyless agent naming a bookmark). A third
2041 +
/// party's clone gets the key's identity, its certificate and the
2042 +
/// issuer's identity with the fetch, so the policy's subject grant
2043 +
/// authorizes the move and the signature verifies.
2044 +
#[test]
2045 +
fn a_bookmark_moved_by_a_certified_key_alone_is_fetched() {
2046 +
    use crate::cert::{Certificate, SignedCertificate};
2047 +
    use crate::identity::Scope;
2048 +
    hermetic();
2049 +
    let dir = tempfile::tempdir().unwrap();
2050 +
    let root = dir.path();
2051 +
    let _origin = Repo::init(&root.join("origin")).unwrap();
2052 +
    let url = root.join("origin").to_string_lossy().to_string();
2053 +
    let mut a = Repo::clone(&url, &root.join("a"))
2054 +
        .unwrap()
2055 +
        .with_home(root.join("home-a"));
2056 +
    a.create_identity("Ada", "[email protected]", IdentityScope::User)
2057 +
        .unwrap();
2058 +
    write(&root.join("a"), "a.txt", b"a\n");
2059 +
    a.commit("a", &[]).unwrap();
2060 +
    a.policy_init().unwrap();
2061 +
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
2062 +
    let hub_sk = SecretKey::generate();
2063 +
    let hub = hub_sk.identity("hub", "[email protected]", 1);
2064 +
    a.policy_grant_subject(
2065 +
        "agent:claude@ada",
2066 +
        hub.clone(),
2067 +
        Scope::CHANGES | Scope::REFS,
2068 +
        None,
2069 +
    )
2070 +
    .unwrap();
2071 +
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
2072 +
2073 +
    let now = Hlc::now().wall_ms;
2074 +
    let mut agent = Repo::clone(&url, &root.join("agent"))
2075 +
        .unwrap()
2076 +
        .with_home(root.join("home-agent"));
2077 +
    let (key, id) = agent
2078 +
        .create_identity("claude", "[email protected]", IdentityScope::User)
2079 +
        .unwrap();
2080 +
    let cert = SignedCertificate::sign(
2081 +
        Certificate {
2082 +
            key: id,
2083 +
            issuer: hub.hash(),
2084 +
            subject: "agent:claude@ada".into(),
2085 +
            claims: vec![],
2086 +
            scope: Scope::CHANGES | Scope::REFS,
2087 +
            not_before: now - 60_000,
2088 +
            expires: now + 600_000,
2089 +
            grant: None,
2090 +
        },
2091 +
        &hub,
2092 +
        &hub_sk,
2093 +
    );
2094 +
    agent.store().put_identity(&hub).unwrap();
2095 +
    agent.store().put_certificate(&cert).unwrap();
2096 +
    let heads = agent.bookmark_heads().unwrap();
2097 +
    agent.bookmark_create("agent-work", Some(&heads)).unwrap();
2098 +
    let r = agent.push_bookmark("origin", "agent-work", false).unwrap();
2099 +
    assert!(r.iter().all(|x| x.ok), "{r:?}");
2100 +
    let host = Repo::open(&root.join("origin")).unwrap();
2101 +
    let (_, sa) = host.latest_ref_attest("agent-work").unwrap().unwrap();
2102 +
    assert_eq!(sa.sigs[0].key, key, "the certified key alone moved it");
2103 +
2104 +
    // A fetch naming no change brings the certificate and both identities.
2105 +
    let bundle = host.bundle_raw_for(&[], None).unwrap();
2106 +
    assert!(bundle.changes.is_empty() && bundle.raw_changes.is_empty());
2107 +
    assert_eq!(bundle.certificates.len(), 1);
2108 +
    for k in [key, hub.hash()] {
2109 +
        assert!(bundle
2110 +
            .objects
2111 +
            .iter()
2112 +
            .any(|(t, b)| *t == ObjectType::Identity && hash_object(*t, b) == k));
2113 +
    }
2114 +
2115 +
    let c = Repo::clone(&url, &root.join("c")).unwrap();
2116 +
    let tracked = c.view().unwrap().remotes["origin"].clone();
2117 +
    assert_eq!(
2118 +
        tracked.get("agent-work"),
2119 +
        host.view().unwrap().bookmarks.get("agent-work")
2120 +
    );
2121 +
    assert!(c.store().certificate_of(&key).unwrap().is_some());
2122 +
}
2123 +
2124 +
/// Trusted time (oxygenhub#240, spec §14.8): a fetched attestation by a
2125 +
/// short-lived key is judged at its integrated time, the earliest time a
2126 +
/// pinned witness cosigned a checkpoint holding it. A certificate valid
2127 +
/// then but expired now passes; a witness this user does not trust gives
2128 +
/// no time (the earlier rule, said so); a backdated attestation is noted;
2129 +
/// an answer without the new fields is judged as before.
2130 +
#[test]
2131 +
fn a_fetched_attestation_is_judged_at_its_integrated_time() {
2132 +
    use super::transparency::LogCheck;
2133 +
    use crate::cert::{Certificate, SignedCertificate};
2134 +
    use crate::identity::Scope;
2135 +
    use crate::protocol::{Integration, Logged, RefsResponse, RemoteRef};
2136 +
    use crate::tlog::{self, Checkpoint, Entry, Hash, Integrated, Note};
2137 +
    hermetic();
2138 +
    let dir = tempfile::tempdir().unwrap();
2139 +
    let root = dir.path();
2140 +
    let mut a = Repo::init(&root.join("a"))
2141 +
        .unwrap()
2142 +
        .with_home(root.join("home-a"));
2143 +
    a.create_identity("Ada", "[email protected]", IdentityScope::User)
2144 +
        .unwrap();
2145 +
    write(&root.join("a"), "a.txt", b"a\n");
2146 +
    a.commit("a", &[]).unwrap();
2147 +
    a.policy_init().unwrap();
2148 +
    let issuer_sk = SecretKey::generate();
2149 +
    let issuer = issuer_sk.identity("hub", "[email protected]", 1);
2150 +
    a.policy_grant_subject(
2151 +
        "agent:claude@ada",
2152 +
        issuer.clone(),
2153 +
        Scope::CHANGES | Scope::REFS,
2154 +
        None,
2155 +
    )
2156 +
    .unwrap();
2157 +
    let p = a.current_policy().unwrap().unwrap().1.policy;
2158 +
2159 +
    // An agent key the issuer certified for ten minutes two hours ago.
2160 +
    let now = Hlc::now().wall_ms;
2161 +
    let t0 = now - 2 * 3_600_000;
2162 +
    let agent_sk = SecretKey::generate();
2163 +
    let agent = agent_sk.identity("claude", "[email protected]", t0);
2164 +
    let cert = SignedCertificate::sign(
2165 +
        Certificate {
2166 +
            key: agent.clone(),
2167 +
            issuer: issuer.hash(),
2168 +
            subject: "agent:claude@ada".into(),
2169 +
            claims: vec![],
2170 +
            scope: Scope::CHANGES | Scope::REFS,
2171 +
            not_before: t0,
2172 +
            expires: t0 + 600_000,
2173 +
            grant: None,
2174 +
        },
2175 +
        &issuer,
2176 +
        &issuer_sk,
2177 +
    );
2178 +
    a.store().put_identity(&agent).unwrap();
2179 +
    a.store().put_certificate(&cert).unwrap();
2180 +
    let signer = agent.hash();
2181 +
    let attested = |claimed: u64| {
2182 +
        let attest = Attest {
2183 +
            repo: a.id().unwrap(),
2184 +
            hlc: Hlc {
2185 +
                wall_ms: claimed,
2186 +
                logical: 0,
2187 +
            },
2188 +
            body: AttestBody::Ref {
2189 +
                name: "main".into(),
2190 +
                old_state: ObjectRef([1; 32]),
2191 +
                new_state: ObjectRef([2; 32]),
2192 +
                serial: 1,
2193 +
            },
2194 +
        };
2195 +
        let sig = agent_sk.sign(Domain::Attest, &signer, &attest.hash());
2196 +
        SignedAttest {
2197 +
            attest,
2198 +
            sigs: vec![sig],
2199 +
        }
2200 +
    };
2201 +
2202 +
    // The hub's log: eight entries, the attestation's at 3, the latest
2203 +
    // checkpoint of 8, and one of 4 that witnesses cosigned.
2204 +
    let log_sk = SecretKey::generate();
2205 +
    let log_key = log_sk.identity("hub.example", "[email protected]", 1);
2206 +
    let origin = "hub.example/log";
2207 +
    let hub = "https://hub.example";
2208 +
    let url = "https://hub.example/ada/a";
2209 +
    let w_sk = SecretKey::generate();
2210 +
    let w = w_sk.identity("w", "[email protected]", 1);
2211 +
    let stranger_sk = SecretKey::generate();
2212 +
    let stranger = stranger_sk.identity("s", "[email protected]", 1);
2213 +
    let sa = attested(t0 + 300_000);
2214 +
    let entry_of = |i: u64| {
2215 +
        let (kind, object) = if i == 3 {
2216 +
            ("ref", sa.attest.hash())
2217 +
        } else {
2218 +
            ("seal", ObjectRef([i as u8; 32]))
2219 +
        };
2220 +
        Entry {
2221 +
            kind: kind.into(),
2222 +
            repo: sa.attest.repo,
2223 +
            object,
2224 +
            signers: vec![signer],
2225 +
            received: t0 + 300_000,
2226 +
        }
2227 +
        .encode()
2228 +
    };
2229 +
    let mut stored: Vec<Hash> = Vec::new();
2230 +
    for i in 0..8 {
2231 +
        let add = tlog::stored_hashes_for_leaf(i, tlog::leaf_hash(&entry_of(i)), &stored).unwrap();
2232 +
        stored.extend(add);
2233 +
    }
2234 +
    let note_of = |n: u64, cosigners: &[(&str, &Identity, &SecretKey, u64)]| {
2235 +
        let mut note = Checkpoint {
2236 +
            origin: origin.into(),
2237 +
            size: n,
2238 +
            root: tlog::tree_hash(n, &stored).unwrap(),
2239 +
            extensions: vec![],
2240 +
        }
2241 +
        .sign(&log_key, &log_sk)
2242 +
        .unwrap();
2243 +
        for (name, id, sk, t) in cosigners {
2244 +
            note.sigs
2245 +
                .extend(tlog::cosign(&note.text, name, id, sk, *t).unwrap());
2246 +
        }
2247 +
        note.encode()
2248 +
    };
2249 +
    let latest = note_of(8, &[]);
2250 +
    let witnessed_at = (t0 + 360_000) / 1000;
2251 +
    let cosigned = note_of(
2252 +
        4,
2253 +
        &[
2254 +
            ("w.example", &w, &w_sk, witnessed_at),
2255 +
            ("s.example", &stranger, &stranger_sk, 1),
2256 +
        ],
2257 +
    );
2258 +
    let answer = |integration: Vec<Integration>, witnesses: Vec<(String, Vec<u8>)>| RefsResponse {
2259 +
        refs: vec![RemoteRef {
2260 +
            name: "main".into(),
2261 +
            state: ObjectRef([2; 32]),
2262 +
            heads: vec![],
2263 +
            attestation: Some(sa.clone()),
2264 +
            last_serial: None,
2265 +
            accepted: None,
2266 +
            logged: Some(Logged {
2267 +
                index: 3,
2268 +
                entry: entry_of(3),
2269 +
                proof: tlog::prove_inclusion(8, 3, &stored).unwrap(),
2270 +
            }),
2271 +
            integration,
2272 +
        }],
2273 +
        log: Some(latest.clone()),
2274 +
        witnesses,
2275 +
        ..RefsResponse::default()
2276 +
    };
2277 +
    let proof4 = tlog::prove_inclusion(4, 3, &stored).unwrap();
2278 +
    let served = vec![Integration {
2279 +
        note: cosigned.clone(),
2280 +
        proof: proof4.clone(),
2281 +
    }];
2282 +
    let listed = vec![("w.example".to_string(), w.encode())];
2283 +
    // This user saw the hub's log before (pinned, so nothing is fetched).
2284 +
    a.remember_log(hub, &log_key, &Note::parse(&latest).unwrap())
2285 +
        .unwrap();
2286 +
2287 +
    // The witness is pinned on first sight, silently; the stranger's
2288 +
    // earlier cosignature counts for nothing.
2289 +
    let resp = answer(served.clone(), listed.clone());
2290 +
    let lc = a.check_log(url, None, &resp).unwrap();
2291 +
    assert!(lc.notes.is_empty(), "{:?}", lc.notes);
2292 +
    let t = lc.time_of("main").unwrap();
2293 +
    assert_eq!((t.time, t.witness.as_str()), (witnessed_at, "w.example"));
2294 +
    // The certificate expired an hour and fifty minutes ago, but covered
2295 +
    // the integrated time: accepted, and nothing to say.
2296 +
    assert!(cert.cert.expires < now);
2297 +
    let mut notes = Vec::new();
2298 +
    a.authorize_in_time(&p, "origin", &resp.refs[0], &sa, &signer, &lc, &mut notes)
2299 +
        .unwrap();
2300 +
    assert!(notes.is_empty(), "{notes:?}");
2301 +
2302 +
    // Only the stranger cosigned: no witnessed time, so the earlier rule
2303 +
    // (the attestation's own time, inside the window), said so.
2304 +
    let lonely = vec![Integration {
2305 +
        note: note_of(4, &[("s.example", &stranger, &stranger_sk, 1)]),
2306 +
        proof: proof4.clone(),
2307 +
    }];
2308 +
    let resp = answer(lonely, vec![]);
2309 +
    let lc = a.check_log(url, None, &resp).unwrap();
2310 +
    assert!(lc
2311 +
        .time_of("main")
2312 +
        .unwrap_err()
2313 +
        .contains("no witness this user trusts"));
2314 +
    let mut notes = Vec::new();
2315 +
    a.authorize_in_time(&p, "origin", &resp.refs[0], &sa, &signer, &lc, &mut notes)
2316 +
        .unwrap();
2317 +
    assert!(
2318 +
        notes.len() == 1 && notes[0].contains("not at a witnessed time"),
2319 +
        "{notes:?}"
2320 +
    );
2321 +
2322 +
    // A witness listed later under the pinned name with another key is
2323 +
    // pinned with a note, as a new log key is.
2324 +
    let resp = answer(
2325 +
        served.clone(),
2326 +
        vec![("w.example".into(), stranger.encode())],
2327 +
    );
2328 +
    let lc = a.check_log(url, None, &resp).unwrap();
2329 +
    assert!(
2330 +
        lc.notes.iter().any(|n| n.contains("another key")),
2331 +
        "{:?}",
2332 +
        lc.notes
2333 +
    );
2334 +
    assert!(
2335 +
        lc.time_of("main").is_err(),
2336 +
        "w's cosignature is by its old key"
2337 +
    );
2338 +
    a.check_log(url, None, &answer(served.clone(), listed.clone()))
2339 +
        .unwrap();
2340 +
2341 +
    // An old peer's answer: no log, no witnesses, no integration. Judged
2342 +
    // as before, silently.
2343 +
    let mut old = answer(vec![], vec![]);
2344 +
    old.log = None;
2345 +
    old.refs[0].logged = None;
2346 +
    let lc = a.check_log(url, None, &old).unwrap();
2347 +
    assert!(!lc.kept && lc.notes.is_empty());
2348 +
    let mut notes = Vec::new();
2349 +
    a.authorize_in_time(&p, "origin", &old.refs[0], &sa, &signer, &lc, &mut notes)
2350 +
        .unwrap();
2351 +
    assert!(notes.is_empty(), "{notes:?}");
2352 +
2353 +
    // Judged at given integrated times.
2354 +
    let at = |secs_after_t0: u64| {
2355 +
        let mut times = std::collections::HashMap::new();
2356 +
        times.insert(
2357 +
            "main".to_string(),
2358 +
            Ok(Integrated {
2359 +
                time: (t0 + secs_after_t0 * 1000) / 1000,
2360 +
                witness: "w.example".into(),
2361 +
                size: 4,
2362 +
            }),
2363 +
        );
2364 +
        LogCheck {
2365 +
            kept: true,
2366 +
            times,
2367 +
            ..LogCheck::default()
2368 +
        }
2369 +
    };
2370 +
    let judged = |sa: &SignedAttest, lc: &LogCheck| {
2371 +
        let mut notes = Vec::new();
2372 +
        a.authorize_in_time(&p, "origin", &old.refs[0], sa, &signer, lc, &mut notes)
2373 +
            .map(|_| notes)
2374 +
    };
2375 +
    // Claimed inside the window, first witnessed an hour later, after
2376 +
    // the certificate expired: backdated, or held back. Accepted by the
2377 +
    // earlier rule while logs are new, with both said.
2378 +
    let notes = judged(&sa, &at(3600)).unwrap();
2379 +
    assert!(
2380 +
        notes
2381 +
            .iter()
2382 +
            .any(|n| n.contains("had lapsed when witness w.example"))
2383 +
            && notes
2384 +
                .iter()
2385 +
                .any(|n| n.contains("55 minutes before witness")),
2386 +
        "{notes:?}"
2387 +
    );
2388 +
    // Claimed after the certificate expired, but witnessed inside its
2389 +
    // window: the clock the signer did not choose decides. The gap is
2390 +
    // still noted.
2391 +
    let fast = attested(now);
2392 +
    let notes = judged(&fast, &at(400)).unwrap();
2393 +
    assert!(
2394 +
        notes.len() == 1 && notes[0].contains("after witness w.example saw it"),
2395 +
        "{notes:?}"
2396 +
    );
2397 +
    // Claimed and witnessed after it expired: refused, as before.
2398 +
    assert!(judged(&fast, &at(3600)).is_err());
2399 +
    // A gap within ten minutes is nothing to note.
2400 +
    assert!(judged(&sa, &at(500)).unwrap().is_empty());
2401 +
}
2402 +
2403 +
/// Keyless step 5 (oxygenhub#241): an agent's fresh key, certified for a
2404 +
/// subject the author granted, seals changes in the author's name; with
2405 +
/// no grant, a stranger's, or one that had ended, it is neither the
2406 +
/// author nor a delegate.
2407 +
/// oxygen#135: an owner renews or rotates with every delegation, and a
2408 +
/// repository may name the newest key as the author; the agent's grant by
2409 +
/// the earlier key still counts, but not one that key made after.
970 2410
#[test]
2411 +
fn a_grant_outlives_its_owner_s_rotation() {
2412 +
    use crate::cert::{Certificate, Grant, SignedCertificate, SignedGrant};
2413 +
    use crate::identity::Scope;
2414 +
    hermetic();
2415 +
    let dir = tempfile::tempdir().unwrap();
2416 +
    let root = dir.path();
2417 +
    let mut a = Repo::init(&root.join("a"))
2418 +
        .unwrap()
2419 +
        .with_home(root.join("home-a"));
2420 +
    let (old, _) = a
2421 +
        .create_identity("Ada", "[email protected]", IdentityScope::User)
2422 +
        .unwrap();
2423 +
    let (_, old_sk) = a.secret_key().unwrap().unwrap();
2424 +
    let old_id = a.identity(&old).unwrap();
2425 +
    let hub_sk = SecretKey::generate();
2426 +
    let hub = hub_sk.identity("hub", "[email protected]", 1);
2427 +
    let now = Hlc::now().wall_ms;
2428 +
    let grant = |from: u64| {
2429 +
        SignedGrant::sign(
2430 +
            Grant {
2431 +
                grantor: old,
2432 +
                subject: "agent:claude@ada".into(),
2433 +
                issuer: hub.hash(),
2434 +
                scope: Scope::CHANGES | Scope::SEAL,
2435 +
                not_before: from,
2436 +
                expires: now + 86_400_000,
2437 +
            },
2438 +
            &old_id,
2439 +
            &old_sk,
2440 +
        )
2441 +
    };
2442 +
    let before = grant(now - 60_000);
2443 +
    let after = grant(now + 60_000);
2444 +
    let rot = a.renew_identity(None).unwrap();
2445 +
    assert_ne!(rot.new, old);
2446 +
    let agent = |name: &str, g: SignedGrant| {
2447 +
        let mut r = Repo::init(&root.join(name))
2448 +
            .unwrap()
2449 +
            .with_home(root.join(format!("home-{name}")));
2450 +
        let (key, id) = r
2451 +
            .create_identity(name, "[email protected]", IdentityScope::User)
2452 +
            .unwrap();
2453 +
        for i in [&old_id, &rot.identity, &hub] {
2454 +
            r.store().put_identity(i).unwrap();
2455 +
        }
2456 +
        r.set_user_config("author.key", &rot.new.to_hex()).unwrap();
2457 +
        r.set_user_config("author.signer", &key.to_hex()).unwrap();
2458 +
        let cert = SignedCertificate::sign(
2459 +
            Certificate {
2460 +
                key: id,
2461 +
                issuer: hub.hash(),
2462 +
                subject: "agent:claude@ada".into(),
2463 +
                claims: vec![],
2464 +
                scope: Scope::CHANGES | Scope::REFS | Scope::SEAL,
2465 +
                not_before: now - 60_000,
2466 +
                expires: now + 600_000,
2467 +
                grant: Some(g),
2468 +
            },
2469 +
            &hub,
2470 +
            &hub_sk,
2471 +
        );
2472 +
        r.store().put_certificate(&cert).unwrap();
2473 +
        write(&root.join(name), "work.txt", b"agent work\n");
2474 +
        let h = r.commit("work", &[]).unwrap();
2475 +
        assert_eq!(r.store().get_change_meta(&h).unwrap().author, rot.new);
2476 +
        r.seal(&[h]).unwrap();
2477 +
        (r, h)
2478 +
    };
2479 +
    let (r, h) = agent("kept", before);
2480 +
    assert!(
2481 +
        matches!(
2482 +
            r.authenticate_change(&h).unwrap(),
2483 +
            SignatureStatus::Valid { .. }
2484 +
        ),
2485 +
        "{:?}",
2486 +
        r.authenticate_change(&h).unwrap()
2487 +
    );
2488 +
    let (r, h) = agent("late", after);
2489 +
    assert!(
2490 +
        matches!(r.authenticate_change(&h).unwrap(), SignatureStatus::Invalid(e) if e.contains("neither")),
2491 +
        "a grant the old key made after the rotation: {:?}",
2492 +
        r.authenticate_change(&h).unwrap()
2493 +
    );
2494 +
}
2495 +
2496 +
#[test]
2497 +
fn a_certified_key_acts_for_its_author_under_a_grant() {
2498 +
    use crate::cert::{Certificate, Grant, SignedCertificate, SignedGrant};
2499 +
    use crate::identity::{Identity, Scope};
2500 +
    hermetic();
2501 +
    let dir = tempfile::tempdir().unwrap();
2502 +
    let root = dir.path();
2503 +
    let _origin = Repo::init(&root.join("origin")).unwrap();
2504 +
    let url = root.join("origin").to_string_lossy().to_string();
2505 +
    let mut a = Repo::clone(&url, &root.join("a"))
2506 +
        .unwrap()
2507 +
        .with_home(root.join("home-a"));
2508 +
    let (ada, _) = a
2509 +
        .create_identity("Ada", "[email protected]", IdentityScope::User)
2510 +
        .unwrap();
2511 +
    write(&root.join("a"), "a.txt", b"a\n");
2512 +
    a.commit("a", &[]).unwrap();
2513 +
    a.policy_init().unwrap();
2514 +
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
2515 +
    let hub_sk = SecretKey::generate();
2516 +
    let hub = hub_sk.identity("hub", "[email protected]", 1);
2517 +
    a.policy_grant_subject(
2518 +
        "agent:claude@ada",
2519 +
        hub.clone(),
2520 +
        Scope::CHANGES | Scope::REFS,
2521 +
        None,
2522 +
    )
2523 +
    .unwrap();
2524 +
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
2525 +
    let (_, ada_sk) = a.secret_key().unwrap().unwrap();
2526 +
    let ada_id = a.identity(&ada).unwrap();
2527 +
2528 +
    let now = Hlc::now().wall_ms;
2529 +
    let grant = |by: &Identity, sk: &SecretKey, from: u64, to: u64| {
2530 +
        SignedGrant::sign(
2531 +
            Grant {
2532 +
                grantor: by.hash(),
2533 +
                subject: "agent:claude@ada".into(),
2534 +
                issuer: hub.hash(),
2535 +
                scope: Scope::CHANGES | Scope::SEAL,
2536 +
                not_before: from,
2537 +
                expires: to,
2538 +
            },
2539 +
            by,
2540 +
            sk,
2541 +
        )
2542 +
    };
2543 +
    // Ada's work as an agent runs it: Ada the author, a fresh key the hub
2544 +
    // certified (with `g` inside) the signer.
2545 +
    let agent = |name: &str, g: Option<SignedGrant>| {
2546 +
        let mut r = Repo::clone(&url, &root.join(name))
2547 +
            .unwrap()
2548 +
            .with_home(root.join(format!("home-{name}")));
2549 +
        let (key, id) = r
2550 +
            .create_identity(name, "[email protected]", IdentityScope::User)
2551 +
            .unwrap();
2552 +
        r.set_user_config("author.key", &ada.to_hex()).unwrap();
2553 +
        r.set_user_config("author.signer", &key.to_hex()).unwrap();
2554 +
        let cert = SignedCertificate::sign(
2555 +
            Certificate {
2556 +
                key: id,
2557 +
                issuer: hub.hash(),
2558 +
                subject: "agent:claude@ada".into(),
2559 +
                claims: vec![],
2560 +
                scope: Scope::CHANGES | Scope::REFS | Scope::SEAL,
2561 +
                not_before: now - 60_000,
2562 +
                expires: now + 600_000,
2563 +
                grant: g,
2564 +
            },
2565 +
            &hub,
2566 +
            &hub_sk,
2567 +
        );
2568 +
        r.store().put_identity(&hub).unwrap();
2569 +
        r.store().put_certificate(&cert).unwrap();
2570 +
        write(&root.join(name), &format!("{name}.txt"), b"agent work\n");
2571 +
        let h = r.commit("work", &[]).unwrap();
2572 +
        assert_eq!(r.store().get_change_meta(&h).unwrap().author, ada);
2573 +
        (r, key, h)
2574 +
    };
2575 +
2576 +
    let (claude, key, h) = agent(
2577 +
        "claude",
2578 +
        Some(grant(&ada_id, &ada_sk, now - 60_000, now + 86_400_000)),
2579 +
    );
2580 +
    let r = claude.push("origin", false).unwrap();
2581 +
    assert!(r.iter().all(|x| x.ok), "{r:?}");
2582 +
    let host = Repo::open(&root.join("origin")).unwrap();
2583 +
    assert!(
2584 +
        matches!(
2585 +
            host.authenticate_change(&h).unwrap(),
2586 +
            SignatureStatus::Valid { key: k, sealed: true, name, .. } if k == key && name == "agent:claude@ada"
2587 +
        ),
2588 +
        "named by its certificate's subject"
2589 +
    );
2590 +
    a.pull("origin").unwrap();
2591 +
    assert!(matches!(
2592 +
        a.authenticate_change(&h).unwrap(),
2593 +
        SignatureStatus::Valid { key: k, .. } if k == key
2594 +
    ));
2595 +
2596 +
    let (none, _, _) = agent("none", None);
2597 +
    let r = none.push("origin", false).unwrap();
2598 +
    assert!(r.iter().all(|x| !x.ok), "no grant: {r:?}");
2599 +
    let mallory_sk = SecretKey::generate();
2600 +
    let mallory = mallory_sk.identity("Mallory", "[email protected]", 1);
2601 +
    let (stranger, _, _) = agent(
2602 +
        "stranger",
2603 +
        Some(grant(&mallory, &mallory_sk, now - 60_000, now + 600_000)),
2604 +
    );
2605 +
    let r = stranger.push("origin", false).unwrap();
2606 +
    assert!(r.iter().all(|x| !x.ok), "a stranger's grant: {r:?}");
2607 +
    let (ended, _, eh) = agent(
2608 +
        "ended",
2609 +
        Some(grant(&ada_id, &ada_sk, now - 7_200_000, now - 3_600_000)),
2610 +
    );
2611 +
    let r = ended.push("origin", false).unwrap();
2612 +
    assert!(r.iter().all(|x| !x.ok), "an ended grant: {r:?}");
2613 +
    assert!(matches!(
2614 +
        ended.authenticate_change(&eh).unwrap(),
2615 +
        SignatureStatus::Invalid(e) if e.contains("grant did not cover")
2616 +
    ));
2617 +
}
2618 +
2619 +
#[test]
971 2620
fn trust_policy_governs_pushes_and_fetches() {
972 2621
    hermetic();
973 2622
    let dir = tempfile::tempdir().unwrap();
…
14 unchanged lines
988 2637
    write(&root.join("b"), "b.txt", b"b\n");
989 2638
    b.commit("unsigned before policy", &[]).unwrap();
990 2639
    assert!(b.push("origin", false).unwrap()[0].ok);
2640 +
    origin.refresh().unwrap();
991 2641
992 2642
    // A creates the policy and pushes it; origin now enforces it.
993 2643
    write(&root.join("a"), "a.txt", b"a\n");
…
2 unchanged lines
996 2646
    assert_eq!(sp.policy.serial, 1);
997 2647
    a.pull("origin").unwrap();
998 2648
    let results = a.push("origin", false).unwrap();
2649 +
    origin.refresh().unwrap();
999 2650
    assert!(results.iter().all(|r| r.ok), "{results:?}");
1000 2651
    assert_eq!(origin.current_policy().unwrap().unwrap().1.policy.serial, 1);
1001 2652
…
12 unchanged lines
1014 2665
    let h_b2 = b.commit("b2 as Bob", &[]).unwrap();
1015 2666
    assert_eq!(b.store().get_change(&h_b2).unwrap().author, key_b);
1016 2667
    let r = b.push("origin", false).unwrap();
2668 +
    origin.refresh().unwrap();
1017 2669
    assert!(
1018 2670
        !r[0].ok && r[0].reason.contains("may not update"),
1019 2671
        "{:?}",
…
4 unchanged lines
1024 2676
    let sp = a.policy_grant(id_b.clone(), None).unwrap();
1025 2677
    assert_eq!(sp.policy.serial, 2);
1026 2678
    let r = a.push("origin", false).unwrap();
2679 +
    origin.refresh().unwrap();
1027 2680
    assert!(r.iter().any(|x| x.name == "policy" && x.ok), "{r:?}");
1028 2681
    // B must fetch the new policy before pushing.
1029 2682
    assert!(b
…
8 unchanged lines
1038 2691
    b.pull("origin").unwrap();
1039 2692
    assert_eq!(b.current_policy().unwrap().unwrap().1.policy.serial, 2);
1040 2693
    let r = b.push("origin", false).unwrap();
2694 +
    origin.refresh().unwrap();
1041 2695
    assert!(r[0].ok, "{:?}", r);
1042 2696
    // Origin verified B's change under the policy.
1043 2697
    let heads = origin.bookmark_heads().unwrap();
…
36 unchanged lines
1080 2734
    a.pull("origin").unwrap();
1081 2735
    a.policy_revoke(&key_b).unwrap();
1082 2736
    let r = a.push("origin", false).unwrap();
2737 +
    origin.refresh().unwrap();
1083 2738
    assert!(r.iter().any(|x| x.name == "policy" && x.ok), "{r:?}");
1084 2739
    b.pull("origin").unwrap();
1085 2740
    write(&root.join("b"), "b.txt", b"b3\n");
1086 2741
    b.commit("b3", &[]).unwrap();
1087 2742
    let r = b.push("origin", false).unwrap();
2743 +
    origin.refresh().unwrap();
1088 2744
    assert!(!r[0].ok, "{:?}", r);
1089 2745
1090 2746
    // A clone by a third party receives and verifies everything.
…
912 unchanged lines
2003 3659
        "the state still has everything"
2004 3660
    );
2005 3661
    assert!(a.push("origin", false).unwrap()[0].ok);
3662 +
    origin.refresh().unwrap();
2006 3663
    // Files created outside the scope are ignored by status.
2007 3664
    write(&root.join("a"), "docs/new.md", b"ignored\n");
2008 3665
    assert!(a.snapshot("status").unwrap().statuses.is_empty());
…
25 unchanged lines
2034 3691
    let _ = h1;
2035 3692
}
2036 3693
3694 +
/// #137: a depth-0 clone takes no change, so the blobs of the state's
3695 +
/// binary files came with nothing and the checkout failed ("object … not
3696 +
/// found"); they are fetched with the checkpoint, or left lazy under
3697 +
/// `--max-blob`.
2037 3698
#[test]
3699 +
fn a_depth_0_clone_brings_its_binary_files() {
3700 +
    hermetic();
3701 +
    let dir = tempfile::tempdir().unwrap();
3702 +
    let root = dir.path();
3703 +
    let origin_dir = root.join("origin");
3704 +
    let origin = Repo::init(&origin_dir)
3705 +
        .unwrap()
3706 +
        .with_home(root.join("home"));
3707 +
    let png: Vec<u8> = (0..4000u32)
3708 +
        .map(|i| (i * 7 % 251) as u8)
3709 +
        .chain([0, 0xff])
3710 +
        .collect();
3711 +
    // The logo comes with a.txt's first line, and the head edits that line,
3712 +
    // so the change that set the logo is below the head, not a head itself.
3713 +
    write(&origin_dir, "logo.png", &png);
3714 +
    write(&origin_dir, "a.txt", b"first\n");
3715 +
    origin.commit("logo", &[]).unwrap();
3716 +
    write(&origin_dir, "a.txt", b"later\n");
3717 +
    origin.commit("later", &[]).unwrap();
3718 +
    assert_eq!(origin.bookmark_heads().unwrap().len(), 1);
3719 +
    let url = origin_dir.to_string_lossy().to_string();
3720 +
    Repo::clone_shallow(&url, &root.join("a"), 0, None, &[]).unwrap();
3721 +
    assert_eq!(fs::read(root.join("a/logo.png")).unwrap(), png);
3722 +
    let lazy = Repo::clone_shallow(&url, &root.join("b"), 0, Some(0), &[]).unwrap();
3723 +
    assert_eq!(fs::read(root.join("b/a.txt")).unwrap(), b"later\n");
3724 +
    let blob = {
3725 +
        let heads = lazy.bookmark_heads().unwrap();
3726 +
        let st = lazy.load_state_history(&heads).unwrap();
3727 +
        st.opaque_nodes()
3728 +
            .iter()
3729 +
            .flat_map(|n| st.live_blobs(n))
3730 +
            .next()
3731 +
            .unwrap()
3732 +
    };
3733 +
    assert_eq!(lazy.read_blob(&blob).unwrap(), png, "fetched on read");
3734 +
    // Depth 0 takes no change: the head is the boundary; the log comes from
3735 +
    // the checkpoint; a change on top commits and pushes.
3736 +
    let mut a = Repo::open(&root.join("a"))
3737 +
        .unwrap()
3738 +
        .with_home(root.join("home"));
3739 +
    let head = origin.bookmark_heads().unwrap()[0];
3740 +
    assert!(!a.store().has(&head), "no change fetched");
3741 +
    assert!(a.boundary().contains(&head));
3742 +
    assert_eq!(
3743 +
        a.log()
3744 +
            .unwrap()
3745 +
            .iter()
3746 +
            .filter(|e| !e.is_working_copy)
3747 +
            .count(),
3748 +
        2,
3749 +
        "the log from the checkpoint"
3750 +
    );
3751 +
    write(&root.join("a"), "a.txt", b"later\nmine\n");
3752 +
    a.config.set("remote.origin.url", &url).unwrap();
3753 +
    a.save_config().unwrap();
3754 +
    let mine = a.commit("mine", &[]).unwrap();
3755 +
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
3756 +
    origin.refresh().unwrap();
3757 +
    assert!(origin.store().has(&mine));
3758 +
    assert_eq!(origin.bookmark_heads().unwrap(), vec![mine]);
3759 +
    // And pulls what comes after.
3760 +
    origin
3761 +
        .checkout(&origin.bookmark_heads().unwrap(), false, "co")
3762 +
        .unwrap();
3763 +
    write(&origin_dir, "a.txt", b"later\nmine\ntheirs\n");
3764 +
    origin.snapshot("t").unwrap();
3765 +
    origin.commit("theirs", &[]).unwrap();
3766 +
    a.pull("origin").unwrap();
3767 +
    assert_eq!(
3768 +
        fs::read(root.join("a/a.txt")).unwrap(),
3769 +
        b"later\nmine\ntheirs\n"
3770 +
    );
3771 +
}
3772 +
3773 +
#[test]
2038 3774
fn shallow_clone_and_deepen() {
2039 3775
    hermetic();
2040 3776
    let dir = tempfile::tempdir().unwrap();
…
45 unchanged lines
2086 3822
    write(&root.join("a"), "a.txt", b"0\n1\n2\n3\n4\n5\n6\n7\n8\n");
2087 3823
    let h_new = a.commit("c8", &[]).unwrap();
2088 3824
    assert!(a.push("origin", false).unwrap()[0].ok);
3825 +
    origin.refresh().unwrap();
2089 3826
    assert_eq!(origin.bookmark_heads().unwrap(), vec![h_new]);
2090 3827
    // Fetching new work from origin into the shallow clone works too.
2091 3828
    write(&origin_dir, "b.txt", b"b\n");
…
28 unchanged lines
2120 3857
}
2121 3858
2122 3859
#[test]
3860 +
fn a_tag_is_withdrawn_by_a_signed_withdrawal() {
3861 +
    hermetic();
3862 +
    let dir = tempfile::tempdir().unwrap();
3863 +
    let root = dir.path();
3864 +
    let origin = Repo::init(&root.join("origin")).unwrap();
3865 +
    let url = root.join("origin").to_string_lossy().to_string();
3866 +
    let mut a = Repo::clone(&url, &root.join("a"))
3867 +
        .unwrap()
3868 +
        .with_home(root.join("home-a"));
3869 +
    a.create_identity("Ada", "[email protected]", IdentityScope::User)
3870 +
        .unwrap();
3871 +
    write(&root.join("a"), "f.txt", b"1\n");
3872 +
    let h1 = a.commit("one", &[]).unwrap();
3873 +
    a.tag_create("v1", &[h1], "first release").unwrap();
3874 +
    a.tag_create("v2", &[h1], "").unwrap();
3875 +
    a.policy_init().unwrap();
3876 +
    a.push("origin", false).unwrap();
3877 +
    origin.refresh().unwrap();
3878 +
    let original = a
3879 +
        .tag_refs()
3880 +
        .unwrap()
3881 +
        .into_iter()
3882 +
        .find(|t| t.name == "v1")
3883 +
        .unwrap();
3884 +
    let b = Repo::clone(&url, &root.join("b"))
3885 +
        .unwrap()
3886 +
        .with_home(root.join("home-b"));
3887 +
    assert_eq!(b.tag_heads("v1").unwrap(), Some(vec![h1]));
3888 +
3889 +
    // A key the policy does not let create the tag may not withdraw it.
3890 +
    let mut c = Repo::clone(&url, &root.join("c"))
3891 +
        .unwrap()
3892 +
        .with_home(root.join("home-c"));
3893 +
    c.create_identity("Cy", "[email protected]", IdentityScope::User)
3894 +
        .unwrap();
3895 +
    let r = c.push_delete("origin", "tags/v1").unwrap();
3896 +
    assert!(
3897 +
        !r[0].ok && r[0].reason.contains("may not withdraw"),
3898 +
        "{r:?}"
3899 +
    );
3900 +
    assert_eq!(c.tag_heads("v1").unwrap(), None, "withdrawn there first");
3901 +
    origin.refresh().unwrap();
3902 +
    assert_eq!(origin.tag_heads("v1").unwrap(), Some(vec![h1]));
3903 +
3904 +
    // Its creator's key may: the tag goes, the withdrawal stays.
3905 +
    let r = a.push_delete("origin", "tags/v1").unwrap();
3906 +
    assert!(r[0].ok && r[0].reason == "withdrawn", "{r:?}");
3907 +
    origin.refresh().unwrap();
3908 +
    assert_eq!(origin.tag_heads("v1").unwrap(), None);
3909 +
    assert_eq!(a.tag_heads("v1").unwrap(), None);
3910 +
    assert!(origin.tag_withdrawal("v1").unwrap().is_some());
3911 +
    assert!(origin.tag_attest("v1").unwrap().is_some(), "kept beside it");
3912 +
    let refs = origin.remote_refs().unwrap();
3913 +
    assert!(refs.tags.iter().all(|t| t.name != "v1"));
3914 +
    assert_eq!(refs.withdrawn.len(), 1);
3915 +
    let r = a.push_delete("origin", "tags/v1").unwrap();
3916 +
    assert!(r[0].ok && r[0].reason == "already withdrawn", "{r:?}");
3917 +
3918 +
    // A fetch withdraws it too, and the name is never used again.
3919 +
    let rep = b.fetch("origin").unwrap();
3920 +
    assert_eq!(rep.withdrawn_tags, vec!["v1".to_string()]);
3921 +
    assert_eq!(b.tag_heads("v1").unwrap(), None);
3922 +
    assert!(b.tag_create("v1", &[h1], "").is_err());
3923 +
    let r = origin
3924 +
        .receive_push(Bundle::default(), vec![], None, vec![original])
3925 +
        .unwrap();
3926 +
    assert!(!r[0].ok && r[0].reason.contains("withdrawn"), "{r:?}");
3927 +
3928 +
    // `oxy tag delete`, then a plain push, sends it.
3929 +
    a.tag_delete("v2", "tagged too early").unwrap();
3930 +
    let r = a.push("origin", false).unwrap();
3931 +
    assert!(
3932 +
        r.iter()
3933 +
            .any(|x| x.name == "tags/v2" && x.ok && x.reason == "withdrawn"),
3934 +
        "{r:?}"
3935 +
    );
3936 +
    origin.refresh().unwrap();
3937 +
    assert_eq!(origin.tag_heads("v2").unwrap(), None);
3938 +
    let (_, w) = origin.tag_withdrawal("v2").unwrap().unwrap();
3939 +
    assert!(
3940 +
        matches!(&w.attest.body, AttestBody::Untag { reason, .. } if reason == "tagged too early")
3941 +
    );
3942 +
}
3943 +
3944 +
#[test]
2123 3945
fn tags_are_immutable_named_states_and_travel() {
2124 3946
    hermetic();
2125 3947
    let dir = tempfile::tempdir().unwrap();
…
23 unchanged lines
2149 3971
    assert_eq!(a.resolve_heads("main").unwrap(), vec![h2]);
2150 3972
    let (_, sa) = a.tag_attest("v1").unwrap().unwrap();
2151 3973
    assert_eq!(sa.sigs[0].key, key_a);
3974 +
    // Without a policy every client is sent every tag, whatever features
3975 +
    // it declares (oxygen#131).
3976 +
    let mut refs = a.remote_refs().unwrap();
3977 +
    a.refs_for_client(&mut refs, None);
3978 +
    assert_eq!(refs.tags.len(), 1);
2152 3979
    // Checkout by tag.
2153 3980
    a.checkout(&a.resolve_heads("v1").unwrap(), false, "co")
2154 3981
        .unwrap();
…
3 unchanged lines
2158 3985
2159 3986
    // Push carries the tag; a second attestation for the name is refused.
2160 3987
    let r = a.push("origin", false).unwrap();
3988 +
    origin.refresh().unwrap();
2161 3989
    assert!(r.iter().any(|x| x.name == "tags/v1" && x.ok), "{r:?}");
2162 3990
    assert_eq!(origin.tag_heads("v1").unwrap(), Some(vec![h1]));
2163 3991
    let mut forged = a
…
21 unchanged lines
2185 4013
    assert!(b.tags().unwrap()[0].signer.as_deref() == Some("Ada"));
2186 4014
    a.tag_create("v2", &[h2], "second").unwrap();
2187 4015
    a.push("origin", false).unwrap();
4016 +
    origin.refresh().unwrap();
2188 4017
    let rep = b.fetch("origin").unwrap();
2189 4018
    assert_eq!(rep.new_tags, 1);
2190 4019
    assert_eq!(b.tag_heads("v2").unwrap(), Some(vec![h2]));
…
1 unchanged lines
2192 4021
    // Under a policy, an unattested tag is refused and an attested one by the root passes.
2193 4022
    a.policy_init().unwrap();
2194 4023
    a.push("origin", false).unwrap();
2195 -
    let unattested = crate::protocol::TagRef {
2196 -
        name: "v3".into(),
2197 -
        state: origin
4024 +
    origin.refresh().unwrap();
4025 +
    let unattested = crate::protocol::TagRef::new(
4026 +
        "v3".into(),
4027 +
        origin
2198 4028
            .store()
2199 4029
            .put_state(&StateObject::new(vec![h2]))
2200 4030
            .unwrap(),
2201 -
        heads: vec![h2],
2202 -
        attestation: None,
2203 -
    };
4031 +
        vec![h2],
4032 +
        None,
4033 +
    );
2204 4034
    let r = origin
2205 4035
        .receive_push(Bundle::default(), vec![], None, vec![unattested])
2206 4036
        .unwrap();
2207 4037
    assert!(!r[0].ok && r[0].reason.contains("attestation"), "{r:?}");
2208 4038
    a.tag_create("v3", &[h2], "").unwrap();
2209 4039
    let r = a.push("origin", false).unwrap();
4040 +
    origin.refresh().unwrap();
2210 4041
    assert!(r.iter().any(|x| x.name == "tags/v3" && x.ok), "{r:?}");
2211 4042
    assert!(a.gc(true).is_ok());
2212 4043
    assert_eq!(a.tags().unwrap().len(), 3, "tags survive gc");
4044 +
4045 +
    // After a rotation the policy names only the new key, yet a fresh clone
4046 +
    // still takes the tags the old key made before it; the old key makes
4047 +
    // nothing new.
4048 +
    let rot = a.rotate_identity().unwrap();
4049 +
    let r = a.push("origin", false).unwrap();
4050 +
    origin.refresh().unwrap();
4051 +
    assert!(r.iter().all(|x| x.ok), "{r:?}");
4052 +
    let c = Repo::clone(&url, &root.join("c"))
4053 +
        .unwrap()
4054 +
        .with_home(root.join("home-c"));
4055 +
    assert_eq!(c.tag_heads("v3").unwrap(), Some(vec![h2]));
4056 +
    let (_, sp) = c.current_policy().unwrap().unwrap();
4057 +
    assert!(!sp.policy.authorizes(&key_a, "tags/v9"));
4058 +
    let made = c.identity(&rot.new).unwrap().created;
4059 +
    assert!(c.predecessor_authorized(&sp.policy, &key_a, "tags/v9", made - 1));
4060 +
    assert!(!c.predecessor_authorized(&sp.policy, &key_a, "tags/v9", made));
4061 +
    assert!(!c.predecessor_authorized(&sp.policy, &rot.new, "tags/v9", 0));
2213 4062
}
2214 4063
2215 4064
#[test]
…
9 unchanged lines
2225 4074
2226 4075
    // A second workspace elsewhere, checked out on the bookmark.
2227 4076
    let ws_dir = root.join("ws2");
4077 +
    let ops = repo.op_log().unwrap().len();
2228 4078
    let ws = repo.workspace_add(&ws_dir, None).unwrap();
4079 +
    assert_eq!(
4080 +
        ws.op_log().unwrap().len(),
4081 +
        ops + 2,
4082 +
        "the add and its checkout, and no snapshot finding the tree deleted (oxygen#137)"
4083 +
    );
2229 4084
    assert!(ws_dir.join(".oxy").is_file());
2230 4085
    assert_eq!(fs::read(ws_dir.join("a.txt")).unwrap(), b"a\n");
2231 4086
    assert_ne!(ws.workspace(), repo.workspace());
…
13 unchanged lines
2245 4100
        "the first workspace is untouched"
2246 4101
    );
2247 4102
    let h2 = ws.commit("b from ws2", &[]).unwrap();
4103 +
    repo.refresh().unwrap();
2248 4104
    let mut heads = repo.bookmark_heads().unwrap();
2249 4105
    heads.sort();
2250 4106
    let mut want = vec![h1, h2];
…
278 unchanged lines
2529 4385
    let h1 = a.commit("one", &[]).unwrap();
2530 4386
    assert!(a.policy_init().unwrap().applied);
2531 4387
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
4388 +
    origin.refresh().unwrap();
2532 4389
2533 4390
    // Expiry: an identity that had expired when the change was made does not authenticate it.
2534 4391
    let mut e = Repo::init(&root.join("e"))
…
49 unchanged lines
2584 4441
    );
2585 4442
    // Origin accepts the rotated policy (signed by the old root) and the new key's change.
2586 4443
    let res = a.push("origin", false).unwrap();
4444 +
    origin.refresh().unwrap();
2587 4445
    assert!(res.iter().all(|x| x.ok), "{res:?}");
2588 4446
    assert_eq!(origin.current_policy().unwrap().unwrap().1.policy.serial, 2);
2589 4447
    let key_a = r.new;
…
22 unchanged lines
2612 4470
        "already a delegate"
2613 4471
    );
2614 4472
    assert!(a.push("origin", false).unwrap().iter().all(|x| x.ok));
4473 +
    origin.refresh().unwrap();
2615 4474
    write(&root.join("a"), "a.txt", b"a3\n");
2616 4475
    let h3 = a.commit("three", &[]).unwrap();
2617 4476
    assert!(
…
19 unchanged lines
2637 4496
    assert!(a.delegate_remove(&key_b).is_err());
2638 4497
    let key_a3 = r.new;
2639 4498
    let res = a.push("origin", false).unwrap();
4499 +
    origin.refresh().unwrap();
2640 4500
    assert!(res.iter().all(|x| x.ok), "{res:?}");
2641 4501
2642 4502
    // Multi-signer policy: Bob becomes a root and the threshold rises to 2,
…
3 unchanged lines
2646 4506
    assert!(a.policy_set_threshold(3).is_err());
2647 4507
    assert!(a.policy_set_threshold(2).unwrap().applied);
2648 4508
    assert!(a.push("origin", false).unwrap().iter().all(|x| x.ok));
4509 +
    origin.refresh().unwrap();
2649 4510
    b.pull("origin").unwrap();
2650 4511
    assert_eq!(
2651 4512
        b.current_policy().unwrap().unwrap().1.policy.root_threshold,
…
37 unchanged lines
2689 4550
        .unwrap()
2690 4551
        .iter()
2691 4552
        .any(|x| x.name == "policy" && x.ok));
4553 +
    origin.refresh().unwrap();
2692 4554
    a.pull("origin").unwrap();
2693 4555
    assert_eq!(
2694 4556
        a.current_policy().unwrap().unwrap().1.policy.serial,
…
586 unchanged lines
3281 5143
    write(&r, "a.txt", b"a\n");
3282 5144
    let h_a = repo.commit("a", &[]).unwrap();
3283 5145
    assert!(repo.push("origin", false).unwrap()[0].ok);
5146 +
    origin.refresh().unwrap();
3284 5147
3285 5148
    // A bookmark at the working copy's base, then work on it.
3286 5149
    assert!(repo.bookmark_create("bad name..", None).is_err());
…
52 unchanged lines
3339 5202
    assert!(repo.merge("main").is_err());
3340 5203
    // The other bookmark can be pushed by name and appears at origin.
3341 5204
    assert!(repo.push_bookmark("origin", "feature", false).unwrap()[0].ok);
5205 +
    origin.refresh().unwrap();
3342 5206
    assert_eq!(origin.resolve_heads("feature").unwrap(), vec![h_b]);
3343 5207
    assert!(repo.push("origin", false).unwrap()[0].ok);
5208 +
    origin.refresh().unwrap();
3344 5209
    assert_eq!(
3345 5210
        repo.bookmarks()
3346 5211
            .unwrap()
…
160 unchanged lines
3507 5372
        !clone.snapshot("status").unwrap().changed,
3508 5373
        "an unregistered nested repository is ignored"
3509 5374
    );
5375 +
5376 +
    // Registered again, it is pinned again; removed and committed, then
5377 +
    // registered again with no snapshot between, the change that adds it
5378 +
    // back pins it too: the removal's marker is spent (oxygen#152).
5379 +
    let modules = format!("[libs/lib]\nurl = {lib_url}\n");
5380 +
    let pinned = |c: ChangeHash| {
5381 +
        clone
5382 +
            .load_state(&[c])
5383 +
            .unwrap()
5384 +
            .materialize()
5385 +
            .files
5386 +
            .get("libs/lib")
5387 +
            .map(|n| n.kind)
5388 +
    };
5389 +
    write(&clone_dir, ".oxymodules", modules.as_bytes());
5390 +
    let c = clone.commit("lib again", &[]).unwrap();
5391 +
    assert_eq!(pinned(c), Some(NodeKind::Subrepo));
5392 +
    clone.submodule_remove("libs/lib").unwrap();
5393 +
    let c = clone.commit("drop lib again", &[]).unwrap();
5394 +
    assert_eq!(pinned(c), None);
5395 +
    write(&clone_dir, ".oxymodules", modules.as_bytes());
5396 +
    let c = clone.commit("lib back", &[]).unwrap();
5397 +
    assert_eq!(
5398 +
        pinned(c),
5399 +
        Some(NodeKind::Subrepo),
5400 +
        "pinned in the change that registers it"
5401 +
    );
5402 +
    assert!(!clone.snapshot("status").unwrap().changed);
3510 5403
}
3511 5404
3512 5405
#[test]
…
514 unchanged lines
4027 5920
    a.tag_create("v1", &[h2], "first release").unwrap();
4028 5921
    a.checkpoint().unwrap();
4029 5922
    assert!(a.push("origin", false).unwrap()[0].ok);
5923 +
    origin.refresh().unwrap();
4030 5924
4031 5925
    // Sound: signed changes, a seal, a tag, a checkpoint, a ref attestation.
4032 5926
    for repo in [&a, &origin] {
…
350 unchanged lines
4383 6277
        .unwrap()
4384 6278
        .contains("oxy identity create"));
4385 6279
    assert!(find(&r, "repository", Health::Ok).is_some());
6280 +
    // oxygen#127: no policy is said, with who sets one.
6281 +
    assert!(find(&r, "trust", Health::Warn)
6282 +
        .unwrap()
6283 +
        .hint
6284 +
        .unwrap()
6285 +
        .contains("oxy policy init"));
4386 6286
4387 6287
    repo.create_identity("Ada", "[email protected]", IdentityScope::Repo)
4388 6288
        .unwrap();
6289 +
    repo.policy_init().unwrap();
6290 +
    let r = repo.doctor(false);
6291 +
    let trust = find(&r, "trust", Health::Ok).expect("a fresh policy is fine");
6292 +
    assert!(
6293 +
        trust.message.contains("serial 1") && trust.message.contains("1 root"),
6294 +
        "{trust:?}"
6295 +
    );
4389 6296
    repo.config.set("author.name", "Ada").unwrap();
4390 6297
    repo.config.set("author.email", "[email protected]").unwrap();
4391 6298
    repo.config
…
703 unchanged lines
5095 7002
    write(&r, "a.txt", b"a\n");
5096 7003
    let h_a = repo.commit("a", &[]).unwrap();
5097 7004
    assert!(repo.push("origin", false).unwrap()[0].ok);
7005 +
    origin.refresh().unwrap();
5098 7006
    repo.bookmark_create("feature", None).unwrap();
5099 7007
    repo.bookmark_create("feature/x", None).unwrap();
5100 7008
    repo.bookmark_switch("feature/x", false).unwrap();
…
1 unchanged lines
5102 7010
    let h_b = repo.commit("b", &[]).unwrap();
5103 7011
    for name in ["feature", "feature/x"] {
5104 7012
        let res = repo.push_bookmark("origin", name, false).unwrap();
7013 +
        origin.refresh().unwrap();
5105 7014
        assert!(res[0].ok, "{name}: {res:?}");
5106 7015
    }
5107 7016
    assert_eq!(origin.resolve_heads("feature").unwrap(), vec![h_a]);
…
32 unchanged lines
5140 7049
    write(&r, "a.txt", b"a\n");
5141 7050
    repo.commit("a", &[]).unwrap();
5142 7051
    assert!(repo.push("origin", false).unwrap()[0].ok);
7052 +
    origin.refresh().unwrap();
5143 7053
    repo.bookmark_create("feature/x", None).unwrap();
5144 7054
    assert!(repo.push_bookmark("origin", "feature/x", false).unwrap()[0].ok);
7055 +
    origin.refresh().unwrap();
5145 7056
    assert!(origin.resolve_heads("feature/x").is_ok());
5146 7057
5147 7058
    let res = repo.push_delete("origin", "feature/x").unwrap();
7059 +
    origin.refresh().unwrap();
5148 7060
    assert!(res[0].ok, "{res:?}");
5149 7061
    assert!(!origin.view().unwrap().bookmarks.contains_key("feature/x"));
5150 7062
    let (_, att) = origin.latest_ref_attest("feature/x").unwrap().unwrap();
…
14 unchanged lines
5165 7077
5166 7078
    // Pushing it again creates it again, past the deletion's serial.
5167 7079
    let res = repo.push_bookmark("origin", "feature/x", false).unwrap();
7080 +
    origin.refresh().unwrap();
5168 7081
    assert!(res[0].ok, "{res:?}");
5169 7082
    assert!(origin.view().unwrap().bookmarks.contains_key("feature/x"));
5170 7083
5171 7084
    // Nothing to delete, and never the remote's own bookmark.
5172 7085
    assert!(repo.push_delete("origin", "nope").is_err());
5173 7086
    let res = repo.push_delete("origin", "main").unwrap();
7087 +
    origin.refresh().unwrap();
5174 7088
    assert!(
5175 7089
        !res[0].ok && res[0].reason.contains("default bookmark"),
5176 7090
        "{res:?}"
…
19 unchanged lines
5196 7110
    write(root, "a.txt", b"a\nmore\n");
5197 7111
    write(root, "new.txt", b"new\n");
5198 7112
    fs::remove_file(root.join("gone.txt")).unwrap();
5199 -
    let (_, carried) = repo.bookmark_switch_carry("feature").unwrap();
7113 +
    let (report, carried) = repo.bookmark_switch_carry("feature").unwrap();
5200 7114
    assert_eq!(carried, vec!["a.txt", "gone.txt", "new.txt"]);
7115 +
    // oxygen#120: the new file was carried, not removed.
7116 +
    assert_eq!(
7117 +
        (report.removed, report.removed_paths.len()),
7118 +
        (0, 0),
7119 +
        "{report:?}"
7120 +
    );
5201 7121
    assert_eq!(repo.config.bookmark, "feature");
5202 7122
    assert_eq!(fs::read(root.join("a.txt")).unwrap(), b"a\nmore\n");
5203 7123
    assert_eq!(fs::read(root.join("new.txt")).unwrap(), b"new\n");
…
60 unchanged lines
5264 7184
        err.contains("modified a.txt") && err.contains("removed b.txt") && err.contains("--carry"),
5265 7185
        "{err}"
5266 7186
    );
5267 -
    // --carry takes them along: h1 has a.txt as the base has it, and no b.txt
5268 -
    // is carried as absent (h1 has none either, so that is refused).
5269 -
    let err = repo
5270 -
        .checkout_carry(&[h1], "the first change", "checkout")
5271 -
        .unwrap_err()
5272 -
        .to_string();
5273 -
    assert!(
5274 -
        err.contains("b.txt") && err.contains("the first change"),
5275 -
        "{err}"
5276 -
    );
5277 -
    write(root, "b.txt", b"b\n");
7187 +
    // --carry takes them along: h1 has a.txt as the base has it, and the
7188 +
    // removed b.txt is already as h1 has it (none), so that is no conflict
7189 +
    // either (oxygen#100).
5278 7190
    let (_, carried) = repo
5279 7191
        .checkout_carry(&[h1], "the first change", "checkout")
5280 7192
        .unwrap();
…
295 unchanged lines
5576 7488
        .to_string()
5577 7489
        .contains("does not verify"));
5578 7490
5579 -
    // gc keeps the receipt, its runner and its output while the state lives.
7491 +
    // A receipt signed by a short-lived key keeps its certificate and the
7492 +
    // issuer's identity, and reads back with them (oxygenhub#241).
7493 +
    let hub_sk = SecretKey::generate();
7494 +
    let hub = hub_sk.identity("oxygenhub.io", "[email protected]", 1);
7495 +
    let job_sk = SecretKey::generate();
7496 +
    let job = job_sk.identity("job 2 test", "[email protected]", 1);
7497 +
    let mut test_receipt = receipt.clone();
7498 +
    test_receipt.job_name = "test".into();
7499 +
    let plain = SignedReceipt::sign(
7500 +
        test_receipt,
7501 +
        repo.id().unwrap(),
7502 +
        state,
7503 +
        "demo#2",
7504 +
        &job,
7505 +
        &job_sk,
7506 +
    );
7507 +
    let now = plain.attest.hlc.wall_ms;
7508 +
    let cert = crate::cert::SignedCertificate::sign(
7509 +
        crate::cert::Certificate {
7510 +
            key: job.clone(),
7511 +
            issuer: hub.hash(),
7512 +
            subject: "repo:andy/demo:bookmark:main".into(),
7513 +
            claims: vec![],
7514 +
            scope: crate::identity::Scope::SEAL,
7515 +
            not_before: now - 1_000,
7516 +
            expires: now + 600_000,
7517 +
            grant: None,
7518 +
        },
7519 +
        &hub,
7520 +
        &hub_sk,
7521 +
    );
7522 +
    let certified = plain.with_certificate(cert, hub.clone());
7523 +
    repo.add_receipt(&certified).unwrap();
7524 +
    let found = repo.receipts(&heads).unwrap();
7525 +
    let got = found
7526 +
        .iter()
7527 +
        .find(|i| i.signed.receipt.job_name == "test")
7528 +
        .unwrap();
7529 +
    assert_eq!(got.signed, certified);
7530 +
    assert!(got.problem.is_none(), "{:?}", got.problem);
7531 +
7532 +
    // gc keeps the receipts, their runners, the certificate and its issuer,
7533 +
    // and the output while the state lives.
5580 7534
    repo.gc(true).unwrap();
5581 -
    assert_eq!(repo.receipts(&heads).unwrap().len(), 1);
7535 +
    let found = repo.receipts(&heads).unwrap();
7536 +
    assert_eq!(found.len(), 2);
7537 +
    let got = found
7538 +
        .iter()
7539 +
        .find(|i| i.signed.receipt.job_name == "test")
7540 +
        .unwrap();
7541 +
    assert_eq!(got.signed, certified, "the certificate survives gc");
7542 +
    assert!(got.problem.is_none());
5582 7543
    assert_eq!(repo.read_blob(&out).unwrap(), b"built\n");
5583 7544
    assert!(repo.fsck().unwrap().problems.is_empty());
5584 7545
}
5585 7546
5586 7547
#[test]
7548 +
fn approvals_are_kept_with_their_state_and_survive_gc() {
7549 +
    hermetic();
7550 +
    let dir = tempfile::tempdir().unwrap();
7551 +
    let root = dir.path();
7552 +
    let repo = Repo::init(root).unwrap().with_home(root.join("home"));
7553 +
    write(root, "a.txt", b"1\n");
7554 +
    repo.commit("A", &[]).unwrap();
7555 +
    let heads = repo.bookmark_heads().unwrap();
7556 +
    let state = repo.store().put_state(&StateObject::new(heads)).unwrap();
7557 +
    let plan = repo.store_blob(b"the reviewed plan\n").unwrap();
7558 +
    let sk = SecretKey::generate();
7559 +
    let andy = sk.identity("Andy", "[email protected]", 1);
7560 +
    let statement = crate::approval::Statement {
7561 +
        repo: repo.id().unwrap(),
7562 +
        run: "https://hub.example/andy/infra/flow/8".into(),
7563 +
        job: "apply".into(),
7564 +
        job_hash: ObjectRef([1; 32]),
7565 +
        state,
7566 +
        inputs: vec![plan],
7567 +
        environment: "prod".into(),
7568 +
        approve: true,
7569 +
    };
7570 +
    let sa = crate::approval::SignedApproval::sign(&statement, &andy, &sk);
7571 +
    let r = repo.add_approval(&sa).unwrap();
7572 +
    assert_eq!(repo.store().approvals_for(&state).unwrap(), vec![r]);
7573 +
    // Another repository's approval, or a changed one, is refused.
7574 +
    let elsewhere = crate::approval::SignedApproval::sign(
7575 +
        &crate::approval::Statement {
7576 +
            repo: [0; 16],
7577 +
            ..statement.clone()
7578 +
        },
7579 +
        &andy,
7580 +
        &sk,
7581 +
    );
7582 +
    assert!(repo
7583 +
        .add_approval(&elsewhere)
7584 +
        .unwrap_err()
7585 +
        .to_string()
7586 +
        .contains("another repository"));
7587 +
    let mut forged = sa.clone();
7588 +
    if let AttestBody::Approval { environment, .. } = &mut forged.attest.body {
7589 +
        *environment = "dev".into();
7590 +
    }
7591 +
    assert!(repo
7592 +
        .add_approval(&forged)
7593 +
        .unwrap_err()
7594 +
        .to_string()
7595 +
        .contains("does not verify"));
7596 +
    // gc keeps it, its signature and its signer while the state lives.
7597 +
    repo.gc(true).unwrap();
7598 +
    assert_eq!(repo.store().approvals_for(&state).unwrap(), vec![r]);
7599 +
    assert_eq!(repo.store().get_attest(&r).unwrap(), sa.attest);
7600 +
    assert_eq!(repo.store().signatures(&r).unwrap(), sa.sigs);
7601 +
    assert!(repo.fsck().unwrap().problems.is_empty());
7602 +
}
7603 +
7604 +
#[test]
5587 7605
fn status_leaves_an_unchanged_manifest_alone() {
5588 7606
    hermetic();
5589 7607
    let dir = tempfile::tempdir().unwrap();
…
220 unchanged lines
5810 7828
    write(&root.join("b"), "base.txt", b"base\n");
5811 7829
    b.commit("base", &[]).unwrap();
5812 7830
    assert!(b.push("origin", false).unwrap()[0].ok);
7831 +
    origin.refresh().unwrap();
5813 7832
5814 7833
    // A feature bookmark with one change, pushed.
5815 7834
    b.bookmark_create("feat", None).unwrap();
…
1 unchanged lines
5817 7836
    write(&root.join("b"), "x.txt", b"x\n");
5818 7837
    let x = b.commit("x", &[]).unwrap();
5819 7838
    assert!(b.push("origin", false).unwrap()[0].ok);
7839 +
    origin.refresh().unwrap();
5820 7840
    b.bookmark_switch("main", false).unwrap();
5821 7841
5822 7842
    // The remote lands it: its main moves to the feature's state.
…
63 unchanged lines
5886 7906
    write(&root.join("a"), "m.txt", b"m\n");
5887 7907
    let m = a.commit("m", &[]).unwrap();
5888 7908
    assert!(a.push("origin", false).unwrap()[0].ok);
7909 +
    // oxygen#117: a remote's bookmark merges without switching to it.
7910 +
    let f = Repo::clone(&url, &root.join("f")).unwrap();
7911 +
    assert!(!f.view().unwrap().bookmarks.contains_key("develop"));
7912 +
    let merged = f.merge("origin/develop").unwrap();
7913 +
    assert!(merged.changed, "{merged:?}");
7914 +
    assert!(
7915 +
        f.load_state_indexed(&f.bookmark_heads().unwrap())
7916 +
            .unwrap()
7917 +
            .contains(&d1),
7918 +
        "main holds develop's change"
7919 +
    );
7920 +
    assert!(f
7921 +
        .merge("origin/nowhere")
7922 +
        .unwrap_err()
7923 +
        .to_string()
7924 +
        .contains("no bookmark origin/nowhere"));
5889 7925
    let r = c.pull("origin").unwrap();
5890 7926
    assert!(r.changed && r.checked_out, "{r:?}");
5891 7927
    assert_eq!(r.heads, vec![d2]);
…
169 unchanged lines
6061 8097
    let (wc, base, snapshot) = repo.working_copy_with_base().unwrap();
6062 8098
    assert_eq!(render_change(&wc, &base, &snapshot), slow);
6063 8099
    assert_eq!(repo.diff_files(None).unwrap().len(), 3);
8100 +
8101 +
    // Committed, nothing changes: the diff is empty without loading the
8102 +
    // base (oxygen#94), and the same as rendering it.
8103 +
    repo.commit("all", &[]).unwrap();
8104 +
    repo.snapshot_status("oxy status").unwrap();
8105 +
    assert!(repo.status_by_manifest().unwrap().is_some());
8106 +
    assert!(repo
8107 +
        .working_copy_with_base_unless_unchanged()
8108 +
        .unwrap()
8109 +
        .is_none());
8110 +
    let (wc, base, snapshot) = repo.working_copy_with_base().unwrap();
8111 +
    assert_eq!(render_change(&wc, &base, &snapshot), "");
8112 +
    assert!(repo.diff_files(None).unwrap().is_empty());
6064 8113
}
6065 8114
6066 8115
/// oxygen#41: a manifest written from a walk keeps every directory's time;
…
1286 unchanged lines
7353 9402
        .delegate_add(d1, crate::identity::Scope::CHANGES, None)
7354 9403
        .is_err());
7355 9404
}
9405 +
9406 +
/// Two unrelated histories that both add the same file, joined by two
9407 +
/// independent merges on two branches, then those branches merged: Git
9408 +
/// merges identical additions cleanly, and the import must too (#71,
9409 +
/// found in VS Code's history).
9410 +
#[test]
9411 +
fn git_import_joins_unrelated_histories_that_add_the_same_file() {
9412 +
    // Batches of one: every merge renders apart from the state, where a
9413 +
    // file whose graph the state had built came out empty (#71).
9414 +
    crate::repo::git::set_merge_batch_for_tests(Some(1));
9415 +
    hermetic();
9416 +
    if std::process::Command::new("git")
9417 +
        .arg("--version")
9418 +
        .output()
9419 +
        .is_err()
9420 +
    {
9421 +
        eprintln!("git not installed; skipping");
9422 +
        return;
9423 +
    }
9424 +
    let dir = tempfile::tempdir().unwrap();
9425 +
    let root = dir.path();
9426 +
    let g = root.join("src.git");
9427 +
    fs::create_dir_all(&g).unwrap();
9428 +
    let git = |args: &[&str]| -> String {
9429 +
        let out = std::process::Command::new("git")
9430 +
            .arg("-C")
9431 +
            .arg(&g)
9432 +
            .args([
9433 +
                "-c",
9434 +
                "user.name=Grace",
9435 +
                "-c",
9436 +
                "[email protected]",
9437 +
                "-c",
9438 +
                "commit.gpgsign=false",
9439 +
            ])
9440 +
            .args(args)
9441 +
            .output()
9442 +
            .unwrap();
9443 +
        assert!(
9444 +
            out.status.success(),
9445 +
            "git {args:?}: {}",
9446 +
            String::from_utf8_lossy(&out.stderr)
9447 +
        );
9448 +
        String::from_utf8_lossy(&out.stdout).trim().to_string()
9449 +
    };
9450 +
    let text = b"/* header */\nconst a = 1;\nconst b = 2;\n";
9451 +
    // History A: main.
9452 +
    git(&["init", "-q", "-b", "main"]);
9453 +
    write(&g, "gulpfile.js", text);
9454 +
    write(&g, "a.txt", b"a\n");
9455 +
    git(&["add", "."]);
9456 +
    git(&["commit", "-q", "-m", "A root"]);
9457 +
    // History B: an unrelated root adding the same file, identical.
9458 +
    git(&["checkout", "-q", "--orphan", "fork"]);
9459 +
    git(&["rm", "-q", "-r", "--cached", "."]);
9460 +
    for f in ["gulpfile.js", "a.txt"] {
9461 +
        let _ = fs::remove_file(g.join(f));
9462 +
    }
9463 +
    write(&g, "gulpfile.js", text);
9464 +
    write(&g, "b.txt", b"b\n");
9465 +
    git(&["add", "."]);
9466 +
    git(&["commit", "-q", "-m", "B root"]);
9467 +
    // Two branches from main, each joining B on its own.
9468 +
    git(&["checkout", "-q", "-b", "one", "main"]);
9469 +
    git(&[
9470 +
        "merge",
9471 +
        "-q",
9472 +
        "--no-ff",
9473 +
        "--allow-unrelated-histories",
9474 +
        "-m",
9475 +
        "join on one",
9476 +
        "fork",
9477 +
    ]);
9478 +
    git(&["checkout", "-q", "-b", "two", "main"]);
9479 +
    write(&g, "a.txt", b"a\ntwo\n");
9480 +
    git(&["commit", "-q", "-am", "work on two"]);
9481 +
    git(&[
9482 +
        "merge",
9483 +
        "-q",
9484 +
        "--no-ff",
9485 +
        "--allow-unrelated-histories",
9486 +
        "-m",
9487 +
        "join on two",
9488 +
        "fork",
9489 +
    ]);
9490 +
    // Then the two joins meet, and the file is reformatted after.
9491 +
    git(&["checkout", "-q", "main"]);
9492 +
    git(&["merge", "-q", "--no-ff", "-m", "merge one", "one"]);
9493 +
    git(&["merge", "-q", "--no-ff", "-m", "merge two", "two"]);
9494 +
    write(
9495 +
        &g,
9496 +
        "gulpfile.js",
9497 +
        b"/* header */\nconst a = 1;\nconst b = 3;\n",
9498 +
    );
9499 +
    git(&["commit", "-q", "-am", "after"]);
9500 +
9501 +
    let dest = root.join("dest");
9502 +
    let repo = Repo::init(&dest).unwrap().with_home(root.join("home"));
9503 +
    repo.import_git(&g, "main").unwrap();
9504 +
    let v = repo.verify_git_import(&g, "main").unwrap();
9505 +
    assert!(v.mismatches.is_empty(), "{:?}", v.mismatches);
9506 +
    assert_eq!(
9507 +
        fs::read(dest.join("gulpfile.js")).unwrap(),
9508 +
        b"/* header */\nconst a = 1;\nconst b = 3;\n"
9509 +
    );
9510 +
}
9511 +
9512 +
/// As above, but the two joins keep different copies (each branch had a
9513 +
/// different version when it joined), and later work edits each copy:
9514 +
/// the shape of VS Code's c67ef57cda90 (#71).
9515 +
#[test]
9516 +
fn git_import_joins_where_each_join_kept_a_different_copy() {
9517 +
    // Batches of one: every merge renders apart from the state, where a
9518 +
    // file whose graph the state had built came out empty (#71).
9519 +
    crate::repo::git::set_merge_batch_for_tests(Some(1));
9520 +
    hermetic();
9521 +
    if std::process::Command::new("git")
9522 +
        .arg("--version")
9523 +
        .output()
9524 +
        .is_err()
9525 +
    {
9526 +
        eprintln!("git not installed; skipping");
9527 +
        return;
9528 +
    }
9529 +
    let dir = tempfile::tempdir().unwrap();
9530 +
    let root = dir.path();
9531 +
    let g = root.join("src.git");
9532 +
    fs::create_dir_all(&g).unwrap();
9533 +
    let run = |args: &[&str]| {
9534 +
        std::process::Command::new("git")
9535 +
            .arg("-C")
9536 +
            .arg(&g)
9537 +
            .args([
9538 +
                "-c",
9539 +
                "user.name=Grace",
9540 +
                "-c",
9541 +
                "[email protected]",
9542 +
                "-c",
9543 +
                "commit.gpgsign=false",
9544 +
            ])
9545 +
            .args(args)
9546 +
            .output()
9547 +
            .unwrap()
9548 +
    };
9549 +
    let git = |args: &[&str]| -> String {
9550 +
        let out = run(args);
9551 +
        assert!(
9552 +
            out.status.success(),
9553 +
            "git {args:?}: {}",
9554 +
            String::from_utf8_lossy(&out.stderr)
9555 +
        );
9556 +
        String::from_utf8_lossy(&out.stdout).trim().to_string()
9557 +
    };
9558 +
    git(&["init", "-q", "-b", "main"]);
9559 +
    write(
9560 +
        &g,
9561 +
        "gulpfile.js",
9562 +
        b"/* header */\nconst a = 1;\nconst b = 2;\n",
9563 +
    );
9564 +
    git(&["add", "."]);
9565 +
    git(&["commit", "-q", "-m", "A root"]);
9566 +
    write(
9567 +
        &g,
9568 +
        "gulpfile.js",
9569 +
        b"/* header */\nconst a = 1;\n\nconst b = 2;\n",
9570 +
    );
9571 +
    git(&["commit", "-q", "-am", "whitespace"]);
9572 +
    git(&["checkout", "-q", "--orphan", "fork"]);
9573 +
    git(&["rm", "-q", "-r", "--cached", "."]);
9574 +
    let _ = fs::remove_file(g.join("gulpfile.js"));
9575 +
    write(
9576 +
        &g,
9577 +
        "gulpfile.js",
9578 +
        b"/* header */\nconst a = 1;\nconst b = 2;\nconst c = 3;\n",
9579 +
    );
9580 +
    git(&["add", "."]);
9581 +
    git(&["commit", "-q", "-m", "B root"]);
9582 +
    // Branch one joins B keeping main's copy; branch two keeps B's.
9583 +
    for (branch, keep) in [("one", "ours"), ("two", "theirs")] {
9584 +
        git(&["checkout", "-q", "-b", branch, "main"]);
9585 +
        let _ = run(&[
9586 +
            "merge",
9587 +
            "-q",
9588 +
            "--no-ff",
9589 +
            "--allow-unrelated-histories",
9590 +
            "-m",
9591 +
            "join",
9592 +
            "fork",
9593 +
        ]);
9594 +
        git(&["checkout", &format!("--{keep}"), "gulpfile.js"]);
9595 +
        git(&["add", "gulpfile.js"]);
9596 +
        git(&["commit", "-q", "-m", &format!("join on {branch}")]);
9597 +
    }
9598 +
    // Each branch then edits its copy.
9599 +
    git(&["checkout", "-q", "one"]);
9600 +
    write(
9601 +
        &g,
9602 +
        "gulpfile.js",
9603 +
        b"/* header */\nconst a = 1;\n\nconst b = 2;\n// tslint\n",
9604 +
    );
9605 +
    git(&["commit", "-q", "-am", "tslint on one"]);
9606 +
    git(&["checkout", "-q", "two"]);
9607 +
    write(
9608 +
        &g,
9609 +
        "gulpfile.js",
9610 +
        b"/* header */\nconst a = 1;\nconst b = 2;\nconst c = 3;\n// fmt\n",
9611 +
    );
9612 +
    write(&g, "extra.txt", b"x\n");
9613 +
    write(&g, "more.txt", b"m\n");
9614 +
    git(&["add", "."]);
9615 +
    git(&["commit", "-q", "-am", "fmt on two"]);
9616 +
    // They meet: resolved by hand.
9617 +
    git(&["checkout", "-q", "one"]);
9618 +
    let _ = run(&["merge", "-q", "--no-ff", "-m", "merge two", "two"]);
9619 +
    write(
9620 +
        &g,
9621 +
        "gulpfile.js",
9622 +
        b"/* header */\nconst a = 1;\nconst b = 2;\nconst c = 3;\n// tslint\n// fmt\n",
9623 +
    );
9624 +
    git(&["add", "gulpfile.js"]);
9625 +
    git(&["commit", "-q", "-m", "merge two into one"]);
9626 +
9627 +
    let dest = root.join("dest");
9628 +
    let repo = Repo::init(&dest).unwrap().with_home(root.join("home"));
9629 +
    repo.import_git(&g, "one").unwrap();
9630 +
    let v = repo.verify_git_import(&g, "one").unwrap();
9631 +
    assert!(v.mismatches.is_empty(), "{:?}", v.mismatches);
9632 +
}
9633 +
9634 +
/// oxygen#68: on a store that is not this disk's directory, the operation
9635 +
/// head is the store's reference record, moved by compare-and-swap. Two
9636 +
/// handles recording from the same head both get through, merged; a swap
9637 +
/// from a stale version is refused; a second repository cannot start on a
9638 +
/// store that already holds one.
9639 +
#[test]
9640 +
fn over_a_store_the_operation_head_moves_by_compare_and_swap() {
9641 +
    use crate::strata::{MemoryStore, Store as _};
9642 +
    let backend = std::sync::Arc::new(MemoryStore::new());
9643 +
    crate::store::TEST_BACKEND.with(|t| *t.borrow_mut() = Some(backend.clone()));
9644 +
    let dir = tempfile::tempdir().unwrap();
9645 +
    let a = Repo::init(dir.path()).unwrap();
9646 +
    assert!(
9647 +
        !dir.path().join(".oxy/ops/head").exists(),
9648 +
        "the head is not a file"
9649 +
    );
9650 +
    let (h0, op0) = a.head_op().unwrap();
9651 +
    let record = backend
9652 +
        .read_ref(HEAD_RECORD)
9653 +
        .unwrap()
9654 +
        .expect("the head is the store's record");
9655 +
    assert_eq!(String::from_utf8(record.0).unwrap(), h0.to_hex());
9656 +
9657 +
    let b = Repo::open(dir.path()).unwrap();
9658 +
    let state = a.store.put_state(&StateObject::new(vec![])).unwrap();
9659 +
    let mut va = op0.view.clone();
9660 +
    va.bookmarks.insert("from-a".into(), state);
9661 +
    let mut vb = op0.view.clone();
9662 +
    vb.bookmarks.insert("from-b".into(), state);
9663 +
    let rb = b.record_op(&[h0], "bookmark from-b", vb).unwrap();
9664 +
    assert_eq!(b.head_ref().unwrap(), rb);
9665 +
    // a recorded from h0 too: the head moved, so the two are merged.
9666 +
    let ra = a.record_op(&[h0], "bookmark from-a", va).unwrap();
9667 +
    let (head, op) = Repo::open(dir.path()).unwrap().head_op().unwrap();
9668 +
    assert_eq!(head, ra);
9669 +
    assert!(
9670 +
        op.command.ends_with("(merged with a concurrent operation)"),
9671 +
        "{}",
9672 +
        op.command
9673 +
    );
9674 +
    assert_eq!(op.parents, vec![rb, op.parents[1]]);
9675 +
    assert!(op.view.bookmarks.contains_key("from-a") && op.view.bookmarks.contains_key("from-b"));
9676 +
9677 +
    // A swap from a version that is no longer current moves nothing.
9678 +
    let (h, v) = a.read_head().unwrap();
9679 +
    assert!(b.swap_head(&rb, v.as_ref()).unwrap());
9680 +
    assert!(!a.swap_head(&h, v.as_ref()).unwrap(), "stale");
9681 +
    a.refresh().unwrap();
9682 +
    assert_eq!(a.head_ref().unwrap(), rb);
9683 +
9684 +
    let other = tempfile::tempdir().unwrap();
9685 +
    let err = Repo::init(other.path())
9686 +
        .err()
9687 +
        .expect("the store already holds a log")
9688 +
        .to_string();
9689 +
    assert!(
9690 +
        err.contains("already holds a repository's operation log"),
9691 +
        "{err}"
9692 +
    );
9693 +
    crate::store::TEST_BACKEND.with(|t| *t.borrow_mut() = None);
9694 +
}
9695 +
9696 +
/// oxygen#68: a repository kept on a store (`storage.url`) holds its
9697 +
/// objects and operation head there, with `.oxy/cache` in front: it works
9698 +
/// as one on disk does, reopens, and survives losing its cache.
9699 +
#[test]
9700 +
fn a_repository_runs_on_a_store_with_a_cache_in_front() {
9701 +
    hermetic();
9702 +
    if std::env::var_os("OXY_STORE_BACKEND").is_some() {
9703 +
        return;
9704 +
    }
9705 +
    let dir = tempfile::tempdir().unwrap();
9706 +
    let (root, bucket) = (dir.path().join("work"), dir.path().join("bucket"));
9707 +
    let url = format!("dir://{}", bucket.display());
9708 +
    let repo = Repo::init_on(&root, [7; 16], Some(&url)).unwrap();
9709 +
    assert_eq!(repo.config.storage_url.as_deref(), Some(url.as_str()));
9710 +
    write(&root, "a.txt", b"one\n");
9711 +
    let h1 = repo.commit("First", &[]).unwrap();
9712 +
    write(&root, "a.txt", b"one\ntwo\n");
9713 +
    let h2 = repo.commit("Second", &[]).unwrap();
9714 +
    assert!(
9715 +
        !root.join(".oxy/ops/head").exists(),
9716 +
        "the head is in the store"
9717 +
    );
9718 +
    assert!(bucket.join("ops/head").is_file());
9719 +
    assert!(root.join(".oxy/cache").is_dir());
9720 +
    drop(repo);
9721 +
9722 +
    // Reopened, and again with no cache at all: everything comes from the store.
9723 +
    for drop_cache in [false, true] {
9724 +
        if drop_cache {
9725 +
            std::fs::remove_dir_all(root.join(".oxy/cache")).unwrap();
9726 +
        }
9727 +
        let repo = Repo::open(&root).unwrap();
9728 +
        let log: Vec<String> = repo
9729 +
            .op_log()
9730 +
            .unwrap()
9731 +
            .into_iter()
9732 +
            .map(|(_, op)| op.command)
9733 +
            .collect();
9734 +
        assert!(log.len() >= 3, "{log:?}");
9735 +
        assert_eq!(repo.store().get_change(&h1).unwrap().message, "First");
9736 +
        let heads = repo
9737 +
            .store()
9738 +
            .get_state(&repo.view().unwrap().bookmarks["main"])
9739 +
            .unwrap()
9740 +
            .heads;
9741 +
        assert_eq!(heads, vec![h2]);
9742 +
        assert_eq!(
9743 +
            repo.load_state(&heads).unwrap().materialize().files.len(),
9744 +
            1
9745 +
        );
9746 +
    }
9747 +
9748 +
    // A second repository cannot start on the same store.
9749 +
    let err = Repo::init_on(&dir.path().join("other"), [8; 16], Some(&url))
9750 +
        .err()
9751 +
        .unwrap()
9752 +
        .to_string();
9753 +
    assert!(
9754 +
        err.contains("already holds a repository's operation log"),
9755 +
        "{err}"
9756 +
    );
9757 +
    let err = Repo::init_on(&dir.path().join("third"), [9; 16], Some("nope://x"))
9758 +
        .err()
9759 +
        .unwrap()
9760 +
        .to_string();
9761 +
    assert!(err.contains("cannot open that kind of store"), "{err}");
9762 +
}
9763 +
9764 +
/// oxygen#75: the prompt's quick check agrees with status in a repository
9765 +
/// with a subrepo: clean after a commit, dirty when the subrepo moves to
9766 +
/// another state, clean again once that pin is committed. (It compared the
9767 +
/// subrepo's directory as if it were a file, so it was always dirty.)
9768 +
#[test]
9769 +
fn the_quick_dirty_check_compares_a_subrepo_by_its_pin() {
9770 +
    hermetic();
9771 +
    let dir = tempfile::tempdir().unwrap();
9772 +
    let root = dir.path();
9773 +
    let _lib_origin = Repo::init(&root.join("lib-origin")).unwrap();
9774 +
    let lib_url = root.join("lib-origin").to_string_lossy().to_string();
9775 +
    let lib = Repo::clone(&lib_url, &root.join("lib"))
9776 +
        .unwrap()
9777 +
        .with_home(root.join("home"));
9778 +
    write(&root.join("lib"), "lib.txt", b"v1\n");
9779 +
    lib.commit("lib v1", &[]).unwrap();
9780 +
    assert!(lib.push("origin", false).unwrap()[0].ok);
9781 +
    let app_dir = root.join("app");
9782 +
    let app = Repo::init(&app_dir).unwrap().with_home(root.join("home"));
9783 +
    write(&app_dir, "main.txt", b"app\n");
9784 +
    app.submodule_add(&lib_url, "libs/lib", None).unwrap();
9785 +
    app.commit("app with lib", &[]).unwrap();
9786 +
    // Out of the racy window, so the stat-only path decides.
9787 +
    std::thread::sleep(std::time::Duration::from_millis(2100));
9788 +
    let reopen = || Repo::open(&app_dir).unwrap();
9789 +
    assert_eq!(
9790 +
        reopen().quick_dirty().unwrap(),
9791 +
        Some(false),
9792 +
        "clean, as status says"
9793 +
    );
9794 +
    let nested = Repo::open(&app_dir.join("libs/lib")).unwrap();
9795 +
    write(&app_dir.join("libs/lib"), "lib.txt", b"v2\n");
9796 +
    nested.commit("lib v2", &[]).unwrap();
9797 +
    assert_eq!(
9798 +
        reopen().quick_dirty().unwrap(),
9799 +
        Some(true),
9800 +
        "the subrepo moved"
9801 +
    );
9802 +
    reopen().commit("pin lib v2", &[]).unwrap();
9803 +
    std::thread::sleep(std::time::Duration::from_millis(2100));
9804 +
    assert_eq!(
9805 +
        reopen().quick_dirty().unwrap(),
9806 +
        Some(false),
9807 +
        "the new pin is committed"
9808 +
    );
9809 +
}
9810 +
9811 +
#[test]
9812 +
fn a_module_entry_that_was_never_added_is_reported_as_unpinned() {
9813 +
    // oxygen#72: a .oxymodules section with no subrepo behind it.
9814 +
    hermetic();
9815 +
    let dir = tempfile::tempdir().unwrap();
9816 +
    let root = dir.path();
9817 +
    let _lib_origin = Repo::init(&root.join("lib-origin")).unwrap();
9818 +
    let lib_url = root.join("lib-origin").to_string_lossy().to_string();
9819 +
    let lib = Repo::clone(&lib_url, &root.join("lib"))
9820 +
        .unwrap()
9821 +
        .with_home(root.join("home"));
9822 +
    write(&root.join("lib"), "lib.txt", b"v1\n");
9823 +
    lib.commit("lib v1", &[]).unwrap();
9824 +
    assert!(lib.push("origin", false).unwrap()[0].ok);
9825 +
    let app_dir = root.join("app");
9826 +
    let app = Repo::init(&app_dir).unwrap().with_home(root.join("home"));
9827 +
    write(
9828 +
        &app_dir,
9829 +
        ".oxymodules",
9830 +
        format!("[libs/lib]\nurl = {lib_url}\n").as_bytes(),
9831 +
    );
9832 +
    let unpinned = app.unpinned_modules().unwrap();
9833 +
    assert_eq!(unpinned.len(), 1);
9834 +
    assert_eq!(unpinned[0].0, "libs/lib");
9835 +
    assert_eq!(unpinned[0].1.url, lib_url);
9836 +
    assert!(app.submodule_update(true, &[]).unwrap().is_empty());
9837 +
    app.submodule_add(&lib_url, "libs/lib", None).unwrap();
9838 +
    assert!(app.unpinned_modules().unwrap().is_empty(), "added now");
9839 +
}
9840 +
9841 +
#[test]
9842 +
fn the_bookmark_revset_function_names_only_bookmarks() {
9843 +
    // oxygen#73: bookmark(NAME) is a bookmark's heads, never a tag or prefix.
9844 +
    hermetic();
9845 +
    let dir = tempfile::tempdir().unwrap();
9846 +
    let repo = Repo::init(dir.path()).unwrap();
9847 +
    write(dir.path(), "a.txt", b"a\n");
9848 +
    let a = repo.commit("A", &[]).unwrap();
9849 +
    write(dir.path(), "b.txt", b"b\n");
9850 +
    let b = repo.commit("B", &[]).unwrap();
9851 +
    let mut heads = repo.revset("bookmark(main)").unwrap().hashes;
9852 +
    heads.sort();
9853 +
    let mut want = repo.bookmark_heads().unwrap();
9854 +
    want.sort();
9855 +
    assert_eq!(heads, want);
9856 +
    assert!(heads.contains(&b));
9857 +
    assert_eq!(repo.revset("main").unwrap().hashes.len(), heads.len());
9858 +
    // A change prefix is a symbol, not a bookmark.
9859 +
    let prefix = repo.store().get_change_meta(&a).unwrap().change_id.short(8);
9860 +
    assert!(repo.revset(&prefix).is_ok());
9861 +
    let e = repo
9862 +
        .revset(&format!("bookmark({prefix})"))
9863 +
        .unwrap_err()
9864 +
        .to_string();
9865 +
    assert!(e.contains("no such bookmark"), "{e}");
9866 +
}
9867 +
9868 +
/// oxygen#140: a replace conflict is settled from the working copy, as
9869 +
/// `oxy resolve PATH` does: the path's content on disk becomes the
9870 +
/// resolution, and the commit that records it is a Resolution.
9871 +
#[test]
9872 +
fn a_replace_conflict_is_settled_from_the_working_copy() {
9873 +
    hermetic();
9874 +
    let dir = tempfile::tempdir().unwrap();
9875 +
    let root = dir.path();
9876 +
    let mut repo = Repo::init(root).unwrap().with_home(root.join("home"));
9877 +
    let blob = |n: u8| {
9878 +
        let mut v = vec![0u8; 64];
9879 +
        v[1] = n;
9880 +
        v
9881 +
    };
9882 +
    write(root, "img.bin", &blob(1));
9883 +
    write(root, "a.txt", b"a\n");
9884 +
    repo.commit("add img", &[]).unwrap();
9885 +
    repo.bookmark_create("other", None).unwrap();
9886 +
    write(root, "img.bin", &blob(2));
9887 +
    repo.commit("img two", &[]).unwrap();
9888 +
    repo.bookmark_switch("other", false).unwrap();
9889 +
    write(root, "img.bin", &blob(3));
9890 +
    repo.commit("img three", &[]).unwrap();
9891 +
    repo.bookmark_switch("main", false).unwrap();
9892 +
    repo.merge("other").unwrap();
9893 +
    // Without a choice, nothing the working copy holds settles it, and
9894 +
    // other work cannot be committed past it.
9895 +
    write(root, "img.bin", &blob(9));
9896 +
    write(root, "a.txt", b"a\nmore\n");
9897 +
    assert!(
9898 +
        repo.commit("edit a", &[]).is_err(),
9899 +
        "refused while conflicted"
9900 +
    );
9901 +
    let snap = repo.snapshot("oxy resolve").unwrap();
9902 +
    assert!(
9903 +
        repo.choose_resolutions(&snap.base_snapshot, &["a.txt".to_string()])
9904 +
            .is_err(),
9905 +
        "a.txt has no replace conflict"
9906 +
    );
9907 +
    repo.choose_resolutions(&snap.base_snapshot, &["img.bin".to_string()])
9908 +
        .unwrap();
9909 +
    let snap = repo.snapshot("oxy resolve").unwrap();
9910 +
    assert!(snap
9911 +
        .statuses
9912 +
        .iter()
9913 +
        .any(|s| s.path == "img.bin" && s.status == ChangeStatus::Resolved));
9914 +
    let c = repo.commit("settle img", &[]).unwrap();
9915 +
    assert_eq!(
9916 +
        repo.store().get_change(&c).unwrap().kind,
9917 +
        ChangeKind::Resolution
9918 +
    );
9919 +
    let after = repo
9920 +
        .load_state(&repo.bookmark_heads().unwrap())
9921 +
        .unwrap()
9922 +
        .materialize();
9923 +
    assert!(!after.is_conflicted());
9924 +
    assert_eq!(
9925 +
        after.files["img.bin"].content,
9926 +
        FileContent::Opaque {
9927 +
            blobs: vec![Repo::blob_ref(&blob(9))]
9928 +
        }
9929 +
    );
9930 +
    assert!(
9931 +
        matches!(&after.files["a.txt"].content, FileContent::Text { bytes, .. } if bytes == b"a\nmore\n")
9932 +
    );
9933 +
    // Settled: the choice is forgotten, and the working copy is clean.
9934 +
    let snap = repo.snapshot("status").unwrap();
9935 +
    assert!(snap.statuses.is_empty(), "{:?}", snap.statuses);
9936 +
    assert!(repo.resolutions().unwrap().is_empty());
9937 +
}
9938 +
9939 +
/// oxygen#140: the same for a subrepo pin moved two ways at once.
9940 +
#[test]
9941 +
fn a_pin_conflict_is_settled_from_the_working_copy() {
9942 +
    hermetic();
9943 +
    let dir = tempfile::tempdir().unwrap();
9944 +
    let root = dir.path();
9945 +
    let _lib_origin = Repo::init(&root.join("lib-origin")).unwrap();
9946 +
    let lib_url = root.join("lib-origin").to_string_lossy().to_string();
9947 +
    let lib = Repo::clone(&lib_url, &root.join("lib"))
9948 +
        .unwrap()
9949 +
        .with_home(root.join("home"));
9950 +
    write(&root.join("lib"), "lib.txt", b"v1\n");
9951 +
    lib.commit("lib v1", &[]).unwrap();
9952 +
    assert!(lib.push("origin", false).unwrap()[0].ok);
9953 +
    let app_dir = root.join("app");
9954 +
    let mut app = Repo::init(&app_dir).unwrap().with_home(root.join("home"));
9955 +
    write(&app_dir, "main.txt", b"app\n");
9956 +
    app.submodule_add(&lib_url, "libs/lib", None).unwrap();
9957 +
    app.commit("pin v1", &[]).unwrap();
9958 +
    let nested = Repo::open(&app_dir.join("libs/lib"))
9959 +
        .unwrap()
9960 +
        .with_home(root.join("home"));
9961 +
    fn pin(app: &Repo, nested: &Repo, text: &[u8], message: &str) {
9962 +
        write(nested.root(), "lib.txt", text);
9963 +
        nested.commit(message, &[]).unwrap();
9964 +
        app.commit(&format!("pin {message}"), &[]).unwrap();
9965 +
    }
9966 +
    app.bookmark_create("other", None).unwrap();
9967 +
    pin(&app, &nested, b"v2\n", "v2");
9968 +
    app.bookmark_switch("other", false).unwrap();
9969 +
    pin(&app, &nested, b"v3\n", "v3");
9970 +
    app.bookmark_switch("main", false).unwrap();
9971 +
    app.merge("other").unwrap();
9972 +
    let merged = app
9973 +
        .load_state(&app.bookmark_heads().unwrap())
9974 +
        .unwrap()
9975 +
        .materialize();
9976 +
    assert!(
9977 +
        matches!(&merged.files["libs/lib"].content, FileContent::Opaque { blobs } if blobs.len() == 2),
9978 +
        "a pin conflict"
9979 +
    );
9980 +
    // The nested repository moves on once more; that pin is the choice.
9981 +
    write(&app_dir.join("libs/lib"), "lib.txt", b"v4\n");
9982 +
    nested.commit("v4", &[]).unwrap();
9983 +
    let snap = app.snapshot("oxy resolve").unwrap();
9984 +
    app.choose_resolutions(&snap.base_snapshot, &["libs/lib".to_string()])
9985 +
        .unwrap();
9986 +
    let c = app.commit("settle the pin", &[]).unwrap();
9987 +
    assert_eq!(
9988 +
        app.store().get_change(&c).unwrap().kind,
9989 +
        ChangeKind::Resolution
9990 +
    );
9991 +
    let after = app
9992 +
        .load_state(&app.bookmark_heads().unwrap())
9993 +
        .unwrap()
9994 +
        .materialize();
9995 +
    assert!(!after.is_conflicted());
9996 +
    assert!(app.snapshot("status").unwrap().statuses.is_empty());
9997 +
}
9998 +
9999 +
#[test]
10000 +
fn carrying_work_over_a_subrepo_already_at_the_target_pin_is_no_conflict() {
10001 +
    // oxygen#74: a stale state with .oxymodules but no pin, a newer one
10002 +
    // pinning the subrepo, and the nested repository on disk at that pin.
10003 +
    hermetic();
10004 +
    let dir = tempfile::tempdir().unwrap();
10005 +
    let root = dir.path();
10006 +
    let _lib_origin = Repo::init(&root.join("lib-origin")).unwrap();
10007 +
    let lib_url = root.join("lib-origin").to_string_lossy().to_string();
10008 +
    let lib = Repo::clone(&lib_url, &root.join("lib"))
10009 +
        .unwrap()
10010 +
        .with_home(root.join("home"));
10011 +
    write(&root.join("lib"), "lib.txt", b"v1\n");
10012 +
    lib.commit("lib v1", &[]).unwrap();
10013 +
    assert!(lib.push("origin", false).unwrap()[0].ok);
10014 +
    let app_dir = root.join("app");
10015 +
    let app = Repo::init(&app_dir).unwrap().with_home(root.join("home"));
10016 +
    write(&app_dir, "main.txt", b"app\n");
10017 +
    app.submodule_add(&lib_url, "libs/lib", None).unwrap();
10018 +
    // The stale state: registered in .oxymodules, never pinned.
10019 +
    fs::rename(app_dir.join("libs/lib"), root.join("aside")).unwrap();
10020 +
    app.commit("initial", &[]).unwrap();
10021 +
    let stale = app.bookmark_heads().unwrap();
10022 +
    fs::rename(root.join("aside"), app_dir.join("libs/lib")).unwrap();
10023 +
    app.commit("pin lib", &[]).unwrap();
10024 +
    let newer = app.bookmark_heads().unwrap();
10025 +
    app.checkout(&stale, false, "test").unwrap();
10026 +
    write(&app_dir, "notes.txt", b"mine\n");
10027 +
    let (_, carried) = app.checkout_carry(&newer, "main", "test").unwrap();
10028 +
    assert_eq!(carried, vec!["notes.txt".to_string()]);
10029 +
    assert!(app_dir.join("notes.txt").exists());
10030 +
}
10031 +
10032 +
/// oxygen#142: a new bookmark made from main sends what main lacks, not
10033 +
/// its whole history again.
10034 +
#[test]
10035 +
fn a_new_bookmark_sends_only_what_the_remote_lacks() {
10036 +
    hermetic();
10037 +
    let dir = tempfile::tempdir().unwrap();
10038 +
    let root = dir.path();
10039 +
    let _origin = Repo::init(&root.join("origin")).unwrap();
10040 +
    let url = root.join("origin").to_string_lossy().to_string();
10041 +
    let mut one = Repo::clone(&url, &root.join("one"))
10042 +
        .unwrap()
10043 +
        .with_home(root.join("home"));
10044 +
    for n in 0..5 {
10045 +
        write(&root.join("one"), "a.txt", format!("{n}\n").as_bytes());
10046 +
        one.commit(&format!("main {n}"), &[]).unwrap();
10047 +
    }
10048 +
    assert!(one.push("origin", false).unwrap()[0].ok);
10049 +
    one.bookmark_create("feature", None).unwrap();
10050 +
    one.bookmark_switch("feature", false).unwrap();
10051 +
    write(&root.join("one"), "b.txt", b"feature\n");
10052 +
    let added = one.commit("feature", &[]).unwrap();
10053 +
    let heads = one.bookmark_heads().unwrap();
10054 +
    let refs = one
10055 +
        .open_named_remote("origin")
10056 +
        .unwrap()
10057 +
        .refs()
10058 +
        .unwrap()
10059 +
        .refs;
10060 +
    assert!(
10061 +
        refs.iter().all(|r| r.name != "feature"),
10062 +
        "new to the remote"
10063 +
    );
10064 +
    let sent = one
10065 +
        .negotiate(&one.remote_holds(&refs, &[]), &heads)
10066 +
        .unwrap();
10067 +
    assert_eq!(sent, vec![added], "only the change main lacks");
10068 +
    assert!(one.push_bookmark("origin", "feature", false).unwrap()[0].ok);
10069 +
    let two = Repo::clone(&url, &root.join("two")).unwrap();
10070 +
    assert!(two.store().has(&added));
10071 +
}
10072 +
10073 +
#[test]
10074 +
fn a_bookmark_only_behind_its_remote_catches_up() {
10075 +
    // oxygen#74: switch takes the remote's state rather than a stale one.
10076 +
    hermetic();
10077 +
    let dir = tempfile::tempdir().unwrap();
10078 +
    let root = dir.path();
10079 +
    let _origin = Repo::init(&root.join("origin")).unwrap();
10080 +
    let url = root.join("origin").to_string_lossy().to_string();
10081 +
    let one = Repo::clone(&url, &root.join("one"))
10082 +
        .unwrap()
10083 +
        .with_home(root.join("home"));
10084 +
    write(&root.join("one"), "a.txt", b"a\n");
10085 +
    one.commit("A", &[]).unwrap();
10086 +
    assert!(one.push("origin", false).unwrap()[0].ok);
10087 +
    let two = Repo::clone(&url, &root.join("two"))
10088 +
        .unwrap()
10089 +
        .with_home(root.join("home"));
10090 +
    write(&root.join("two"), "b.txt", b"b\n");
10091 +
    two.commit("B", &[]).unwrap();
10092 +
    assert!(two.push("origin", false).unwrap()[0].ok);
10093 +
    assert_eq!(one.catch_up_bookmark("main").unwrap(), None, "not fetched");
10094 +
    one.fetch("origin").unwrap();
10095 +
    assert_eq!(one.catch_up_bookmark("main").unwrap(), Some(1));
10096 +
    assert_eq!(one.bookmarks().unwrap()[0].behind, Some(0));
10097 +
    // Local work the remote lacks: left alone.
10098 +
    write(&root.join("one"), "c.txt", b"c\n");
10099 +
    one.commit("C", &[]).unwrap();
10100 +
    write(&root.join("two"), "d.txt", b"d\n");
10101 +
    two.commit("D", &[]).unwrap();
10102 +
    assert!(two.push("origin", false).unwrap()[0].ok);
10103 +
    one.fetch("origin").unwrap();
10104 +
    assert_eq!(one.catch_up_bookmark("main").unwrap(), None);
10105 +
}
10106 +
10107 +
#[test]
10108 +
fn a_bookmark_is_compared_with_a_remote_that_has_it_when_there_is_no_origin() {
10109 +
    // oxygen#83: remotes earth and gcs, origin removed.
10110 +
    hermetic();
10111 +
    let dir = tempfile::tempdir().unwrap();
10112 +
    let root = dir.path();
10113 +
    let _up = Repo::init(&root.join("up")).unwrap();
10114 +
    let url = root.join("up").to_string_lossy().to_string();
10115 +
    let other = Repo::clone(&url, &root.join("other"))
10116 +
        .unwrap()
10117 +
        .with_home(root.join("home"));
10118 +
    write(&root.join("other"), "a.txt", b"a\n");
10119 +
    other.commit("a", &[]).unwrap();
10120 +
    assert!(other.push("origin", false).unwrap()[0].ok);
10121 +
    let mut one = Repo::clone(&url, &root.join("one"))
10122 +
        .unwrap()
10123 +
        .with_home(root.join("home"));
10124 +
    one.config.set("remote.earth.url", &url).unwrap();
10125 +
    one.config.set("remote.gcs.url", &url).unwrap();
10126 +
    one.config.set("remote.origin.url", "").unwrap();
10127 +
    one.save_config().unwrap();
10128 +
    write(&root.join("other"), "b.txt", b"b\n");
10129 +
    other.commit("b", &[]).unwrap();
10130 +
    assert!(other.push("origin", false).unwrap()[0].ok);
10131 +
    one.fetch("earth").unwrap();
10132 +
    let b = one
10133 +
        .bookmarks()
10134 +
        .unwrap()
10135 +
        .into_iter()
10136 +
        .find(|b| b.name == "main")
10137 +
        .unwrap();
10138 +
    assert_eq!(
10139 +
        (b.remote.as_deref(), b.behind),
10140 +
        (Some("earth"), Some(1)),
10141 +
        "a stale origin copy is not used"
10142 +
    );
10143 +
    assert_eq!(
10144 +
        one.default_remote(),
10145 +
        "origin",
10146 +
        "no default chosen among several"
10147 +
    );
10148 +
    let e = one.pull(&one.default_remote()).unwrap_err().to_string();
10149 +
    assert!(e.contains("earth, gcs") && e.contains("core.remote"), "{e}");
10150 +
    one.config.set("core.remote", "earth").unwrap();
10151 +
    assert_eq!(one.default_remote(), "earth");
10152 +
}
10153 +
10154 +
#[test]
10155 +
fn a_missing_author_identity_says_how_to_fix_it() {
10156 +
    // oxygen#82: an agent home's author.key naming an identity the
10157 +
    // repository does not hold.
10158 +
    hermetic();
10159 +
    let dir = tempfile::tempdir().unwrap();
10160 +
    let mut repo = Repo::init(dir.path()).unwrap();
10161 +
    let missing = KeyId::from_hex(&"ab".repeat(32)).unwrap();
10162 +
    repo.config.set("author.key", &missing.to_hex()).unwrap();
10163 +
    let e = repo.identity(&missing).unwrap_err().to_string();
10164 +
    assert!(
10165 +
        e.contains("this repository has no identity") && e.contains("oxy config author.key"),
10166 +
        "{e}"
10167 +
    );
10168 +
    assert!(e.contains("this repository's config"), "{e}");
10169 +
}
10170 +
10171 +
#[test]
10172 +
fn a_working_copy_behind_its_bookmark_is_flagged_and_pull_brings_it_up() {
10173 +
    // oxygen#89 (supermarioland, 2026-10-09): the bookmark moved on without
10174 +
    // the working copy, a pull with nothing new left it there, and status
10175 +
    // said "clean".
10176 +
    hermetic();
10177 +
    let dir = tempfile::tempdir().unwrap();
10178 +
    let root = dir.path();
10179 +
    let _up = Repo::init(&root.join("up")).unwrap();
10180 +
    let url = root.join("up").to_string_lossy().to_string();
10181 +
    let other = Repo::clone(&url, &root.join("other"))
10182 +
        .unwrap()
10183 +
        .with_home(root.join("home"));
10184 +
    write(&root.join("other"), "a.txt", b"a\n");
10185 +
    other.commit("a", &[]).unwrap();
10186 +
    assert!(other.push("origin", false).unwrap()[0].ok);
10187 +
    let one = Repo::clone(&url, &root.join("one"))
10188 +
        .unwrap()
10189 +
        .with_home(root.join("home"));
10190 +
    write(&root.join("other"), "b.txt", b"b\n");
10191 +
    other.commit("b", &[]).unwrap();
10192 +
    assert!(other.push("origin", false).unwrap()[0].ok);
10193 +
    one.fetch("origin").unwrap();
10194 +
    // The bookmark moves; the working copy does not.
10195 +
    let theirs = one.resolve_heads("origin/main").unwrap();
10196 +
    one.bookmark_set("main", &theirs).unwrap();
10197 +
    assert!(!root.join("one/b.txt").exists());
10198 +
    assert_eq!(one.working_copy_behind().unwrap(), Some(1));
10199 +
    let r = one.pull("origin").unwrap();
10200 +
    assert!(
10201 +
        !r.changed && r.checked_out,
10202 +
        "nothing new, but the working copy caught up"
10203 +
    );
10204 +
    assert!(root.join("one/b.txt").exists());
10205 +
    assert_eq!(one.working_copy_behind().unwrap(), None);
10206 +
}
10207 +
10208 +
#[test]
10209 +
fn touched_nodes_cache_answers_as_the_changes_do() {
10210 +
    hermetic();
10211 +
    let dir = tempfile::tempdir().unwrap();
10212 +
    let repo = Repo::init(dir.path()).unwrap();
10213 +
    write(dir.path(), "a.txt", b"one\n");
10214 +
    write(dir.path(), "b.txt", b"x\n");
10215 +
    repo.snapshot("t").unwrap();
10216 +
    let c1 = repo.commit("first", &[]).unwrap();
10217 +
    write(dir.path(), "b.txt", b"y\n");
10218 +
    repo.snapshot("t").unwrap();
10219 +
    repo.commit("b only", &[]).unwrap();
10220 +
    fs::rename(dir.path().join("a.txt"), dir.path().join("c.txt")).unwrap();
10221 +
    write(dir.path(), "c.txt", b"one\ntwo\n");
10222 +
    repo.snapshot("t").unwrap();
10223 +
    let c3 = repo.commit("rename and edit", &[]).unwrap();
10224 +
    // Uncommitted work, so the working copy touches a file too.
10225 +
    write(dir.path(), "c.txt", b"one\ntwo\nthree\n");
10226 +
    repo.snapshot("t").unwrap();
10227 +
10228 +
    let path = dir.path().join(".oxy/index/touched");
10229 +
    let hashes = |es: Vec<LogEntry>| es.iter().map(|e| e.hash).collect::<Vec<_>>();
10230 +
    let cold = hashes(repo.log_for_path("c.txt").unwrap());
10231 +
    assert!(path.exists(), "the first query writes the cache");
10232 +
    let warm = hashes(repo.log_for_path("c.txt").unwrap());
10233 +
    assert_eq!(cold, warm, "the cache answers as the changes do");
10234 +
    let (wc, _) = repo.working_copy().unwrap();
10235 +
    assert_eq!(
10236 +
        cold,
10237 +
        vec![wc, c3, c1],
10238 +
        "renames followed, the working copy included"
10239 +
    );
10240 +
10241 +
    // Working-copy changes are not kept: they are replaced by every snapshot.
10242 +
    let cached = super::touched::read_touched_for_test(&fs::read(&path).unwrap()).unwrap();
10243 +
    assert!(!cached.contains(&wc), "a working-copy change is not cached");
10244 +
    assert!(cached.contains(&c1) && cached.contains(&c3));
10245 +
10246 +
    // A damaged cache is ignored, not trusted.
10247 +
    let mut bytes = fs::read(&path).unwrap();
10248 +
    let last = bytes.len() - 40;
10249 +
    bytes[last] ^= 1;
10250 +
    fs::write(&path, &bytes).unwrap();
10251 +
    assert_eq!(hashes(repo.log_for_path("c.txt").unwrap()), cold);
10252 +
    let touches = repo.log_revset("touches(\"c.txt\") ~ @").unwrap();
10253 +
    assert_eq!(hashes(touches), vec![c3, c1]);
10254 +
}
10255 +
10256 +
/// oxygen#88 (vscode 9ab2056393c8): two lines of history that each move a
10257 +
/// file back where it was leave two placements at one place, which the
10258 +
/// tree shows as one file. Removing it, and then its directory, retires
10259 +
/// both, so no copy is left under a directory that is gone.
10260 +
#[test]
10261 +
fn git_import_removes_every_copy_of_a_file_at_one_place() {
10262 +
    hermetic();
10263 +
    if std::process::Command::new("git")
10264 +
        .arg("--version")
10265 +
        .output()
10266 +
        .is_err()
10267 +
    {
10268 +
        eprintln!("git not installed; skipping");
10269 +
        return;
10270 +
    }
10271 +
    let dir = tempfile::tempdir().unwrap();
10272 +
    let root = dir.path();
10273 +
    let g = root.join("src.git");
10274 +
    fs::create_dir_all(&g).unwrap();
10275 +
    let git = |args: &[&str]| {
10276 +
        let out = std::process::Command::new("git")
10277 +
            .arg("-C")
10278 +
            .arg(&g)
10279 +
            .args([
10280 +
                "-c",
10281 +
                "user.name=Grace",
10282 +
                "-c",
10283 +
                "[email protected]",
10284 +
                "-c",
10285 +
                "commit.gpgsign=false",
10286 +
            ])
10287 +
            .args(args)
10288 +
            .output()
10289 +
            .unwrap();
10290 +
        assert!(
10291 +
            out.status.success(),
10292 +
            "git {args:?}: {}",
10293 +
            String::from_utf8_lossy(&out.stderr)
10294 +
        );
10295 +
    };
10296 +
    git(&["init", "-q", "-b", "main"]);
10297 +
    write(&g, "d/list.txt", b"one\ntwo\nthree\nfour\n");
10298 +
    write(&g, "keep.txt", b"k\n");
10299 +
    git(&["add", "."]);
10300 +
    git(&["commit", "-qm", "base"]);
10301 +
    fs::create_dir_all(g.join("d/sub")).unwrap();
10302 +
    git(&["mv", "d/list.txt", "d/sub/list.txt"]);
10303 +
    git(&["commit", "-qm", "into sub"]);
10304 +
    git(&["checkout", "-qb", "side"]);
10305 +
    git(&["mv", "d/sub/list.txt", "d/list.txt"]);
10306 +
    git(&["commit", "-qm", "back, on side"]);
10307 +
    git(&["checkout", "-q", "main"]);
10308 +
    git(&["mv", "d/sub/list.txt", "d/list.txt"]);
10309 +
    git(&["commit", "-qm", "back, on main"]);
10310 +
    git(&["merge", "-q", "--no-edit", "side"]);
10311 +
    git(&["rm", "-rq", "d"]);
10312 +
    git(&["commit", "-qm", "d goes"]);
10313 +
10314 +
    let repo = Repo::init(&root.join("o")).unwrap();
10315 +
    repo.import_git(&g, "main").unwrap();
10316 +
    let v = repo.verify_git_import(&g, "main").unwrap();
10317 +
    assert!(v.mismatches.is_empty(), "{:?}", v.mismatches);
10318 +
    let st = repo
10319 +
        .load_state_history(&repo.bookmark_heads().unwrap())
10320 +
        .unwrap();
10321 +
    let view = st.tree().view();
10322 +
    assert!(view.conflicts.is_empty(), "{:?}", view.conflicts);
10323 +
    assert_eq!(view.entries.keys().collect::<Vec<_>>(), vec!["keep.txt"]);
10324 +
}
10325 +
10326 +
/// oxygen#94: a state index's file histories are read, and checked by
10327 +
/// their own hashes, only when a file is built. One found damaged is an
10328 +
/// error that says so, the index is removed, and the next read rebuilds it.
10329 +
#[test]
10330 +
fn a_damaged_file_history_in_the_state_index_is_rebuilt() {
10331 +
    hermetic();
10332 +
    let dir = tempfile::tempdir().unwrap();
10333 +
    let repo = Repo::init(dir.path()).unwrap();
10334 +
    for i in 0..6 {
10335 +
        write(
10336 +
            dir.path(),
10337 +
            &format!("f{i}.txt"),
10338 +
            format!("one {i}\n").as_bytes(),
10339 +
        );
10340 +
    }
10341 +
    repo.snapshot("t").unwrap();
10342 +
    repo.commit("first", &[]).unwrap();
10343 +
    for i in 0..6 {
10344 +
        write(
10345 +
            dir.path(),
10346 +
            &format!("f{i}.txt"),
10347 +
            format!("one {i}\ntwo\n").as_bytes(),
10348 +
        );
10349 +
    }
10350 +
    repo.snapshot("t").unwrap();
10351 +
    repo.commit("second", &[]).unwrap();
10352 +
    let want: Vec<Vec<u8>> = (0..6)
10353 +
        .map(|i| format!("one {i}\ntwo\n").into_bytes())
10354 +
        .collect();
10355 +
    // The index of the working copy's base, holding every file's history
10356 +
    // (a few changes behind, an index is replayed onto, not rewritten).
10357 +
    let (_, wc) = repo.working_copy().unwrap();
10358 +
    let base = repo.load_state_history(&wc.deps).unwrap();
10359 +
    repo.write_state_index(&wc.deps, &base);
10360 +
    for (i, w) in want.iter().enumerate() {
10361 +
        assert_eq!(&repo.file_at(&format!("f{i}.txt"), None).unwrap(), w);
10362 +
    }
10363 +
    let index = repo.state_indexes();
10364 +
    assert!(!index.is_empty(), "a state index was written");
10365 +
    // The histories are last in the file: damage the last of them.
10366 +
    let mut bytes = fs::read(&index[0]).unwrap();
10367 +
    let last = bytes.len() - 1;
10368 +
    bytes[last] ^= 0x40;
10369 +
    fs::write(&index[0], &bytes).unwrap();
10370 +
10371 +
    let reads: Vec<_> = (0..6)
10372 +
        .map(|i| repo.file_at(&format!("f{i}.txt"), None))
10373 +
        .collect();
10374 +
    let damaged: Vec<String> = reads
10375 +
        .iter()
10376 +
        .filter_map(|r| r.as_ref().err().map(|e| e.to_string()))
10377 +
        .collect();
10378 +
    assert_eq!(damaged.len(), 1, "{damaged:?}");
10379 +
    assert!(
10380 +
        damaged[0].contains("state index was damaged")
10381 +
            && damaged[0].contains("run the command again"),
10382 +
        "{}",
10383 +
        damaged[0]
10384 +
    );
10385 +
    // Every file reads right again, from an index rebuilt from history.
10386 +
    for (i, w) in want.iter().enumerate() {
10387 +
        assert_eq!(&repo.file_at(&format!("f{i}.txt"), None).unwrap(), w);
10388 +
    }
10389 +
}
10390 +
10391 +
/// oxygen#95: a push longer than `push.batch-bytes` goes in parts, each
10392 +
/// objects alone and in dependency order, and the last moves the bookmark.
10393 +
/// The remote ends with every change and the same state, and says how each
10394 +
/// part went.
10395 +
#[test]
10396 +
fn a_long_push_goes_in_parts() {
10397 +
    hermetic();
10398 +
    let dir = tempfile::tempdir().unwrap();
10399 +
    let root = dir.path();
10400 +
    let origin = root.join("origin");
10401 +
    Repo::init_bare_unbound(&origin).unwrap();
10402 +
    let main = root.join("main");
10403 +
    let mut repo = Repo::init(&main).unwrap();
10404 +
    repo.config
10405 +
        .set("remote.origin.url", &origin.to_string_lossy())
10406 +
        .unwrap();
10407 +
    // One byte: every change is a part of its own.
10408 +
    repo.config.set("push.batch-bytes", "1").unwrap();
10409 +
    repo.save_config().unwrap();
10410 +
    for i in 0..6 {
10411 +
        write(
10412 +
            &main,
10413 +
            &format!("f{i}.txt"),
10414 +
            format!("file {i}\n").as_bytes(),
10415 +
        );
10416 +
        repo.commit(&format!("c{i}"), &[]).unwrap();
10417 +
    }
10418 +
    let mut parts = Vec::new();
10419 +
    let results = repo
10420 +
        .push_bookmark_with("origin", "main", false, &mut |m| parts.push(m.to_string()))
10421 +
        .unwrap();
10422 +
    assert!(
10423 +
        results.iter().any(|r| r.ok && r.name == "main"),
10424 +
        "{results:?}"
10425 +
    );
10426 +
    assert_eq!(parts.len(), 6, "{parts:?}");
10427 +
    assert_eq!(parts[0], "part 1 of 6: 1 changes");
10428 +
10429 +
    let remote = Repo::open(&origin).unwrap();
10430 +
    let heads = repo.bookmark_heads().unwrap();
10431 +
    assert_eq!(remote.bookmark_heads().unwrap(), heads);
10432 +
    assert_eq!(
10433 +
        remote.log_state(&heads).unwrap().len(),
10434 +
        repo.log_state(&heads).unwrap().len()
10435 +
    );
10436 +
    // A push of objects alone reports how it went, as `objects`.
10437 +
    let got = remote
10438 +
        .receive_push(Bundle::default(), vec![], None, vec![])
10439 +
        .unwrap();
10440 +
    assert_eq!(got.len(), 1);
10441 +
    assert!(got[0].ok && got[0].name == OBJECTS_RESULT, "{got:?}");
10442 +
}
10443 +
10444 +
/// oxygen#133: one change whose blobs pass `push.batch-bytes` sends them
10445 +
/// first, in parts of objects alone, so no request is larger than about
10446 +
/// that (a proxy in front of a hub refuses one past 100 MB); the change
10447 +
/// and the bookmark follow, and a clone has every file.
10448 +
#[test]
10449 +
fn a_heavy_change_sends_its_blobs_in_parts() {
10450 +
    hermetic();
10451 +
    let dir = tempfile::tempdir().unwrap();
10452 +
    let root = dir.path();
10453 +
    let origin = root.join("origin");
10454 +
    Repo::init_bare_unbound(&origin).unwrap();
10455 +
    let main = root.join("main");
10456 +
    let mut repo = Repo::init(&main).unwrap();
10457 +
    repo.config
10458 +
        .set("remote.origin.url", &origin.to_string_lossy())
10459 +
        .unwrap();
10460 +
    repo.config.set("push.batch-bytes", "50000").unwrap();
10461 +
    repo.save_config().unwrap();
10462 +
    let mut x: u64 = 0x9e3779b97f4a7c15;
10463 +
    let mut noise = |n: usize| -> Vec<u8> {
10464 +
        (0..n)
10465 +
            .map(|_| {
10466 +
                x ^= x << 13;
10467 +
                x ^= x >> 7;
10468 +
                x ^= x << 17;
10469 +
                (x >> 24) as u8
10470 +
            })
10471 +
            .collect()
10472 +
    };
10473 +
    let files: Vec<Vec<u8>> = (0..4).map(|_| noise(40_000)).collect();
10474 +
    for (i, f) in files.iter().enumerate() {
10475 +
        write(&main, &format!("bin{i}.dat"), f);
10476 +
    }
10477 +
    repo.commit("four blobs", &[]).unwrap();
10478 +
    let mut parts = Vec::new();
10479 +
    let results = repo
10480 +
        .push_bookmark_with("origin", "main", false, &mut |m| parts.push(m.to_string()))
10481 +
        .unwrap();
10482 +
    assert!(
10483 +
        results.iter().any(|r| r.ok && r.name == "main"),
10484 +
        "{results:?}"
10485 +
    );
10486 +
    assert!(
10487 +
        parts.len() >= 3 && parts[0].starts_with("objects 1 of"),
10488 +
        "{parts:?}"
10489 +
    );
10490 +
    let back = Repo::clone(&origin.to_string_lossy(), &root.join("back")).unwrap();
10491 +
    for (i, f) in files.iter().enumerate() {
10492 +
        assert_eq!(&fs::read(root.join(format!("back/bin{i}.dat"))).unwrap(), f);
10493 +
    }
10494 +
    drop(back);
10495 +
}
10496 +
10497 +
/// oxygen#136: a bundle carries a replaced binary file's new content
10498 +
/// only. Its old content came with the change that added it, so removing
10499 +
/// or replacing a file sends no old blob again; the remote and a clone of
10500 +
/// it are whole.
10501 +
#[test]
10502 +
fn a_replacement_does_not_send_the_old_content_again() {
10503 +
    hermetic();
10504 +
    let dir = tempfile::tempdir().unwrap();
10505 +
    let root = dir.path();
10506 +
    let origin = root.join("origin");
10507 +
    Repo::init_bare_unbound(&origin).unwrap();
10508 +
    let main = root.join("main");
10509 +
    let mut repo = Repo::init(&main).unwrap();
10510 +
    repo.config
10511 +
        .set("remote.origin.url", &origin.to_string_lossy())
10512 +
        .unwrap();
10513 +
    repo.save_config().unwrap();
10514 +
    let blobs = |b: &Bundle| {
10515 +
        b.objects
10516 +
            .iter()
10517 +
            .filter(|(ty, _)| *ty == ObjectType::Blob)
10518 +
            .count()
10519 +
    };
10520 +
    write(&main, "bin.dat", b"\0\x01first binary");
10521 +
    write(&main, "gone.dat", b"\0\x02to be removed");
10522 +
    let added = repo.commit("add", &[]).unwrap();
10523 +
    assert!(repo.push("origin", false).unwrap()[0].ok);
10524 +
    write(&main, "bin.dat", b"\0\x01second binary");
10525 +
    fs::remove_file(main.join("gone.dat")).unwrap();
10526 +
    let changed = repo.commit("replace and remove", &[]).unwrap();
10527 +
    assert_eq!(blobs(&repo.bundle_for(&[changed]).unwrap()), 1);
10528 +
    assert_eq!(blobs(&repo.bundle_for(&[added, changed]).unwrap()), 3);
10529 +
    assert!(repo.push("origin", false).unwrap()[0].ok);
10530 +
    let origin = Repo::open(&origin).unwrap();
10531 +
    assert_eq!(origin.fsck().unwrap().problems, vec![]);
10532 +
    let back = Repo::clone(&origin.root().to_string_lossy(), &root.join("back")).unwrap();
10533 +
    assert_eq!(back.fsck().unwrap().problems, vec![]);
10534 +
    assert_eq!(
10535 +
        fs::read(root.join("back/bin.dat")).unwrap(),
10536 +
        b"\0\x01second binary"
10537 +
    );
10538 +
    assert!(!root.join("back/gone.dat").exists());
10539 +
}
10540 +
10541 +
/// oxygen#97: with `git.claim`, an import authors the importing identity's
10542 +
/// own commits (Git author email = its email, any case) with its key, and
10543 +
/// leaves anyone else's with the zero key; the history verifies as before.
10544 +
#[test]
10545 +
fn git_import_claims_your_own_commits() {
10546 +
    hermetic();
10547 +
    if std::process::Command::new("git")
10548 +
        .arg("--version")
10549 +
        .output()
10550 +
        .is_err()
10551 +
    {
10552 +
        eprintln!("git not installed; skipping");
10553 +
        return;
10554 +
    }
10555 +
    let dir = tempfile::tempdir().unwrap();
10556 +
    let root = dir.path();
10557 +
    let g = root.join("src.git");
10558 +
    fs::create_dir_all(&g).unwrap();
10559 +
    let git = |who: &str, args: &[&str]| {
10560 +
        let (name, email) = who.split_once(' ').unwrap();
10561 +
        let out = std::process::Command::new("git")
10562 +
            .arg("-C")
10563 +
            .arg(&g)
10564 +
            .args([
10565 +
                "-c",
10566 +
                &format!("user.name={name}"),
10567 +
                "-c",
10568 +
                &format!("user.email={email}"),
10569 +
                "-c",
10570 +
                "commit.gpgsign=false",
10571 +
            ])
10572 +
            .args(args)
10573 +
            .output()
10574 +
            .unwrap();
10575 +
        assert!(
10576 +
            out.status.success(),
10577 +
            "git {args:?}: {}",
10578 +
            String::from_utf8_lossy(&out.stderr)
10579 +
        );
10580 +
    };
10581 +
    git("Grace [email protected]", &["init", "-q", "-b", "main"]);
10582 +
    write(&g, "a.txt", b"a\n");
10583 +
    git("Grace [email protected]", &["add", "."]);
10584 +
    git("Grace [email protected]", &["commit", "-qm", "mine"]);
10585 +
    write(&g, "b.txt", b"b\n");
10586 +
    git("Ada [email protected]", &["add", "."]);
10587 +
    git("Ada [email protected]", &["commit", "-qm", "someone else's"]);
10588 +
10589 +
    let mut repo = Repo::init(&root.join("o")).unwrap();
10590 +
    let (key, _) = repo
10591 +
        .create_identity("Grace", "[email protected]", IdentityScope::Repo)
10592 +
        .unwrap();
10593 +
    repo.config.set("git.claim", "true").unwrap();
10594 +
    repo.save_config().unwrap();
10595 +
    let repo = Repo::open(&root.join("o")).unwrap();
10596 +
    repo.import_git(&g, "main").unwrap();
10597 +
    let v = repo.verify_git_import(&g, "main").unwrap();
10598 +
    assert!(v.mismatches.is_empty(), "{:?}", v.mismatches);
10599 +
    let heads = repo.bookmark_heads().unwrap();
10600 +
    let by_message: std::collections::HashMap<String, KeyId> = repo
10601 +
        .log_state(&heads)
10602 +
        .unwrap()
10603 +
        .into_iter()
10604 +
        .map(|e| {
10605 +
            (
10606 +
                e.change.message.lines().next().unwrap_or("").to_string(),
10607 +
                e.change.author,
10608 +
            )
10609 +
        })
10610 +
        .collect();
10611 +
    assert_eq!(by_message["mine"], key, "her own commit, any case of email");
10612 +
    assert_eq!(by_message["someone else's"], ObjectRef::ZERO, "not hers");
10613 +
}
10614 +
10615 +
/// oxygen#86: removing a remote removes all of it: every `remote.NAME.*`
10616 +
/// key (URL, token, CA), its tracking bookmarks, and nothing else.
10617 +
#[test]
10618 +
fn removing_a_remote_leaves_nothing_of_it() {
10619 +
    hermetic();
10620 +
    let dir = tempfile::tempdir().unwrap();
10621 +
    let origin = dir.path().join("origin");
10622 +
    Repo::init_bare_unbound(&origin).unwrap();
10623 +
    let main = dir.path().join("main");
10624 +
    let mut repo = Repo::init(&main).unwrap();
10625 +
    repo.config
10626 +
        .set("remote.origin.url", &origin.to_string_lossy())
10627 +
        .unwrap();
10628 +
    repo.config
10629 +
        .set("remote.origin.token", "secret-ish")
10630 +
        .unwrap();
10631 +
    repo.config
10632 +
        .set("remote.keep.url", &origin.to_string_lossy())
10633 +
        .unwrap();
10634 +
    repo.save_config().unwrap();
10635 +
    write(&main, "a.txt", b"a\n");
10636 +
    repo.commit("a", &[]).unwrap();
10637 +
    assert!(repo.push("origin", false).unwrap()[0].ok);
10638 +
    assert!(
10639 +
        repo.view().unwrap().remotes.contains_key("origin"),
10640 +
        "a tracking bookmark"
10641 +
    );
10642 +
10643 +
    repo.remove_remote("origin").unwrap();
10644 +
    let again = Repo::open(&main).unwrap();
10645 +
    assert!(
10646 +
        !again
10647 +
            .config
10648 +
            .repo_layer()
10649 +
            .keys()
10650 +
            .any(|k| k.starts_with("remote.origin.")),
10651 +
        "{:?}",
10652 +
        again.config.repo_layer()
10653 +
    );
10654 +
    assert!(!again.config.remotes.contains_key("origin"));
10655 +
    assert!(
10656 +
        !again.view().unwrap().remotes.contains_key("origin"),
10657 +
        "its tracking bookmarks go"
10658 +
    );
10659 +
    assert!(
10660 +
        again.config.remotes.contains_key("keep"),
10661 +
        "another remote stays"
10662 +
    );
10663 +
    let mut again = again;
10664 +
    assert!(
10665 +
        again.remove_remote("origin").is_err(),
10666 +
        "a second removal is refused"
10667 +
    );
10668 +
}
10669 +
10670 +
/// oxygenhub#126: a blob a host stored for itself (a job's artifact) is
10671 +
/// unreachable from history, so gc prunes it, unless the host names it:
10672 +
/// then it and its chunks stay, and it reads back whole.
10673 +
#[test]
10674 +
fn gc_keeps_the_blobs_a_host_names() {
10675 +
    hermetic();
10676 +
    let dir = tempfile::tempdir().unwrap();
10677 +
    let repo = Repo::init(dir.path()).unwrap();
10678 +
    write(dir.path(), "a.txt", b"a\n");
10679 +
    repo.commit("a", &[]).unwrap();
10680 +
    // Large enough to be several chunks.
10681 +
    let body: Vec<u8> = (0..400_000u32)
10682 +
        .flat_map(|i| (i.wrapping_mul(2654435761)).to_le_bytes())
10683 +
        .collect();
10684 +
    let kept = repo.store_blob(&body).unwrap();
10685 +
    let lost = repo.store_blob(b"an artifact nobody names").unwrap();
10686 +
    repo.gc_keeping(true, &[kept], &mut |_| {}).unwrap();
10687 +
    assert_eq!(repo.read_blob(&kept).unwrap(), body, "named: kept whole");
10688 +
    assert!(!repo.store().has(&lost), "not named: pruned");
10689 +
    repo.gc(true).unwrap();
10690 +
    assert!(!repo.store().has(&kept), "a plain gc prunes it");
10691 +
}
10692 +
10693 +
/// oxygen#99: undo in one workspace undoes that workspace's last command,
10694 +
/// never another workspace's, says what it undid, and goes further back
10695 +
/// when run again.
10696 +
#[test]
10697 +
fn undo_is_scoped_to_its_workspace() {
10698 +
    hermetic();
10699 +
    let dir = tempfile::tempdir().unwrap();
10700 +
    let root = dir.path().join("main");
10701 +
    let a = Repo::init(&root).unwrap();
10702 +
    write(&root, "base.txt", b"base\n");
10703 +
    let base = a.commit("base", &[]).unwrap();
10704 +
    a.bookmark_create("feature", None).unwrap();
10705 +
    let ws_dir = dir.path().join("ws");
10706 +
    let heads = a.resolve_heads("feature").unwrap();
10707 +
    let b = a
10708 +
        .workspace_add_on(&ws_dir, Some(&heads), Some("feature"))
10709 +
        .unwrap();
10710 +
10711 +
    // A commits on main, then B commits on feature: B's is the newest.
10712 +
    write(&root, "a.txt", b"a\n");
10713 +
    let ca = a.commit("from a", &[]).unwrap();
10714 +
    write(&ws_dir, "b.txt", b"b\n");
10715 +
    let cb = b.commit("from b", &[]).unwrap();
10716 +
10717 +
    // A's undo undoes A's commit and leaves B's.
10718 +
    let u = a.undo().unwrap();
10719 +
    assert_eq!(u.command, "oxy commit -m \"from a\"");
10720 +
    assert_eq!(u.others_kept, 1, "B's newer commit is left");
10721 +
    assert!(!u.changes.is_empty());
10722 +
    let view = a.view().unwrap();
10723 +
    // (Independent changes commute, so a bookmark's heads may hold base too.)
10724 +
    assert_eq!(a.heads_of(&view, "main").unwrap(), vec![base]);
10725 +
    assert!(
10726 +
        a.heads_of(&view, "feature").unwrap().contains(&cb),
10727 +
        "B's commit stands"
10728 +
    );
10729 +
    assert!(
10730 +
        a.store().get_change(&ca).is_ok(),
10731 +
        "the change itself is still in the store"
10732 +
    );
10733 +
    // A's working copy was clean, so its files followed; B's are untouched.
10734 +
    assert!(!root.join("a.txt").exists());
10735 +
    assert!(a.snapshot("oxy status").unwrap().statuses.is_empty());
10736 +
    assert!(ws_dir.join("b.txt").exists());
10737 +
10738 +
    // B's undo still finds B's own commit, under A's undo.
10739 +
    let u = b.undo().unwrap();
10740 +
    assert_eq!(u.command, "oxy commit -m \"from b\"");
10741 +
    assert!(!b
10742 +
        .heads_of(&b.view().unwrap(), "feature")
10743 +
        .unwrap()
10744 +
        .contains(&cb));
10745 +
    assert_eq!(
10746 +
        a.heads_of(&a.view().unwrap(), "main").unwrap(),
10747 +
        vec![base],
10748 +
        "A's undo stands"
10749 +
    );
10750 +
10751 +
    // A's next undo goes past the commit it already undid, to adding B's
10752 +
    // workspace: refused, since B has worked there since.
10753 +
    let e = a.undo().unwrap_err().to_string();
10754 +
    assert!(e.contains("the workspace at") && e.contains("ws"), "{e}");
10755 +
}
10756 +
10757 +
/// oxygen#99: two workspaces on one bookmark: an undo takes back only its
10758 +
/// own commit's head, and the other workspace's head stays.
10759 +
#[test]
10760 +
fn undo_on_a_shared_bookmark_keeps_the_other_workspaces_head() {
10761 +
    hermetic();
10762 +
    let dir = tempfile::tempdir().unwrap();
10763 +
    let root = dir.path().join("main");
10764 +
    let a = Repo::init(&root).unwrap();
10765 +
    write(&root, "base.txt", b"base\n");
10766 +
    a.commit("base", &[]).unwrap();
10767 +
    let ws_dir = dir.path().join("ws");
10768 +
    let b = a.workspace_add(&ws_dir, None).unwrap();
10769 +
    write(&root, "a.txt", b"a\n");
10770 +
    let ca = a.commit("from a", &[]).unwrap();
10771 +
    write(&ws_dir, "b.txt", b"b\n");
10772 +
    let cb = b.commit("from b", &[]).unwrap();
10773 +
    let u = a.undo().unwrap();
10774 +
    assert_eq!(u.command, "oxy commit -m \"from a\"");
10775 +
    let heads = a.heads_of(&a.view().unwrap(), "main").unwrap();
10776 +
    assert!(heads.contains(&cb) && !heads.contains(&ca), "{heads:?}");
10777 +
}
10778 +
10779 +
/// oxygen#99: an undo that would also take back what another workspace
10780 +
/// did since to the same bookmark is refused, naming it.
10781 +
#[test]
10782 +
fn undo_refuses_to_take_back_another_workspaces_work() {
10783 +
    hermetic();
10784 +
    let dir = tempfile::tempdir().unwrap();
10785 +
    let root = dir.path().join("main");
10786 +
    let a = Repo::init(&root).unwrap();
10787 +
    write(&root, "base.txt", b"base\n");
10788 +
    a.commit("base", &[]).unwrap();
10789 +
    let ws_dir = dir.path().join("ws");
10790 +
    let b = a.workspace_add(&ws_dir, None).unwrap();
10791 +
    // Both on main: A commits, then B builds on it, editing A's file.
10792 +
    write(&root, "a.txt", b"a\n");
10793 +
    a.commit("from a", &[]).unwrap();
10794 +
    b.refresh().unwrap();
10795 +
    let heads = b.resolve_heads("main").unwrap();
10796 +
    b.checkout(&heads, false, "oxy checkout").unwrap();
10797 +
    write(&ws_dir, "a.txt", b"a, then b\n");
10798 +
    b.commit("from b", &[]).unwrap();
10799 +
    let e = a.undo().unwrap_err().to_string();
10800 +
    assert!(
10801 +
        e.contains("bookmark main") && e.contains("builds on"),
10802 +
        "{e}"
10803 +
    );
10804 +
    assert!(e.contains("oxy op restore"), "{e}");
10805 +
}
10806 +
10807 +
/// oxygen#99: undoing a commit that was already pushed says so: the remote
10808 +
/// still has it.
10809 +
#[test]
10810 +
fn undo_says_when_the_commit_was_pushed() {
10811 +
    hermetic();
10812 +
    let dir = tempfile::tempdir().unwrap();
10813 +
    let root = dir.path();
10814 +
    Repo::init(&root.join("origin")).unwrap();
10815 +
    let url = root.join("origin").to_string_lossy().to_string();
10816 +
    let repo = Repo::clone(&url, &root.join("work"))
10817 +
        .unwrap()
10818 +
        .with_home(root.join("home"));
10819 +
    write(&root.join("work"), "a.txt", b"a\n");
10820 +
    repo.commit("a", &[]).unwrap();
10821 +
    assert!(repo.push("origin", false).unwrap()[0].ok);
10822 +
    // The push itself is passed over: the remote keeps what it was sent.
10823 +
    let u = repo.undo().unwrap();
10824 +
    assert_eq!(u.command, "oxy commit -m \"a\"");
10825 +
    assert_eq!(u.pushed, vec![("main".to_string(), "origin".to_string())]);
10826 +
}
10827 +
10828 +
/// oxygen#98: a new untracked file over snapshot.max-new-file-size is left
10829 +
/// out of the snapshot and named; tracked files and files already in the
10830 +
/// working copy are not.
10831 +
#[test]
10832 +
fn large_new_files_are_left_out_of_snapshots() {
10833 +
    hermetic();
10834 +
    let dir = tempfile::tempdir().unwrap();
10835 +
    let root = dir.path();
10836 +
    let mut repo = Repo::init(root).unwrap();
10837 +
    repo.config.max_new_file = 1024;
10838 +
    write(root, "small.txt", b"small\n");
10839 +
    write(root, "big.bin", &vec![0u8; 4096]);
10840 +
    let s = repo.snapshot("oxy status").unwrap();
10841 +
    let paths: Vec<_> = s.statuses.iter().map(|p| p.path.as_str()).collect();
10842 +
    assert_eq!(paths, ["small.txt"]);
10843 +
    assert_eq!(repo.skipped_large(), vec![("big.bin".to_string(), 4096)]);
10844 +
    repo.commit("small", &[]).unwrap();
10845 +
10846 +
    // A tracked file may grow past the limit.
10847 +
    write(root, "small.txt", &vec![b'x'; 4096]);
10848 +
    let s = repo.snapshot("oxy status").unwrap();
10849 +
    assert!(s.statuses.iter().any(|p| p.path == "small.txt"));
10850 +
    // With no limit the big file is taken in, and stays once it is.
10851 +
    repo.config.max_new_file = 0;
10852 +
    let s = repo.snapshot("oxy status").unwrap();
10853 +
    assert!(s.statuses.iter().any(|p| p.path == "big.bin"));
10854 +
    repo.config.max_new_file = 1024;
10855 +
    let s = repo.snapshot("oxy status").unwrap();
10856 +
    assert!(
10857 +
        s.statuses.iter().any(|p| p.path == "big.bin"),
10858 +
        "already in the working copy"
10859 +
    );
10860 +
    assert!(repo.skipped_large().is_empty());
10861 +
}
10862 +
10863 +
/// oxygen#98: a file only a past working-copy snapshot held is pruned once
10864 +
/// snapshots expire, and kept while they have not.
10865 +
#[test]
10866 +
fn gc_prunes_what_only_expired_snapshots_held() {
10867 +
    hermetic();
10868 +
    let dir = tempfile::tempdir().unwrap();
10869 +
    let root = dir.path();
10870 +
    let mut repo = Repo::init(root).unwrap();
10871 +
    repo.config.gc_grace_days = 0;
10872 +
    write(root, "a.txt", b"a\n");
10873 +
    repo.commit("a", &[]).unwrap();
10874 +
    let image: Vec<u8> = (0..200_000u32).map(|i| (i % 7) as u8).collect();
10875 +
    write(root, "image.tar", &image);
10876 +
    repo.snapshot("oxy status").unwrap();
10877 +
    std::fs::remove_file(root.join("image.tar")).unwrap();
10878 +
    repo.snapshot("oxy status").unwrap();
10879 +
    let blob = Repo::blob_ref(&image);
10880 +
    assert!(repo.store().has(&blob));
10881 +
    repo.gc(true).unwrap();
10882 +
    assert!(repo.store().has(&blob), "a recent snapshot keeps it");
10883 +
    repo.config.snapshot_retention_days = 0;
10884 +
    repo.gc(true).unwrap();
10885 +
    assert!(!repo.store().has(&blob), "an expired snapshot does not");
10886 +
    assert_eq!(
10887 +
        repo.file_at("a.txt", None).unwrap(),
10888 +
        b"a\n",
10889 +
        "committed files stay"
10890 +
    );
10891 +
}
10892 +
10893 +
/// oxygen#100: uncommitted work byte-for-byte what the bookmark switched to
10894 +
/// already has is no conflict: the switch goes through, nothing is lost,
10895 +
/// and other carried work still comes along.
10896 +
#[test]
10897 +
fn carrying_work_the_target_already_has_is_no_conflict() {
10898 +
    hermetic();
10899 +
    let dir = tempfile::tempdir().unwrap();
10900 +
    let root = dir.path();
10901 +
    let mut repo = Repo::init(root).unwrap();
10902 +
    write(root, "a.txt", b"one\n");
10903 +
    write(root, "b.txt", b"b\n");
10904 +
    repo.commit("one", &[]).unwrap();
10905 +
    repo.bookmark_create("landed", None).unwrap();
10906 +
    // On landed, a.txt becomes "two" (as a proposal landing would).
10907 +
    repo.bookmark_switch("landed", false).unwrap();
10908 +
    write(root, "a.txt", b"two\n");
10909 +
    repo.commit("two", &[]).unwrap();
10910 +
    repo.bookmark_switch("main", false).unwrap();
10911 +
    assert_eq!(std::fs::read(root.join("a.txt")).unwrap(), b"one\n");
10912 +
    // The same edit, uncommitted, on main; and another edit besides.
10913 +
    write(root, "a.txt", b"two\n");
10914 +
    write(root, "b.txt", b"b, edited\n");
10915 +
    let (_, carried) = repo.bookmark_switch_carry("landed").unwrap();
10916 +
    assert_eq!(
10917 +
        carried,
10918 +
        vec!["b.txt".to_string()],
10919 +
        "only the work landed lacks is carried"
10920 +
    );
10921 +
    assert_eq!(std::fs::read(root.join("a.txt")).unwrap(), b"two\n");
10922 +
    assert_eq!(std::fs::read(root.join("b.txt")).unwrap(), b"b, edited\n");
10923 +
    let s = repo.snapshot("oxy status").unwrap();
10924 +
    let paths: Vec<_> = s.statuses.iter().map(|p| p.path.as_str()).collect();
10925 +
    assert_eq!(paths, ["b.txt"]);
10926 +
10927 +
    // A real difference is still refused.
10928 +
    repo.bookmark_switch("main", true).unwrap();
10929 +
    write(root, "a.txt", b"three\n");
10930 +
    let e = repo
10931 +
        .bookmark_switch_carry("landed")
10932 +
        .unwrap_err()
10933 +
        .to_string();
10934 +
    assert!(e.contains("a.txt") && e.contains("--discard"), "{e}");
10935 +
}
10936 +
10937 +
/// oxygen#103: a full change hash as `--json` prints it, without `#`,
10938 +
/// resolves, so JSON output feeds back into commands.
10939 +
#[test]
10940 +
fn a_full_hash_resolves_without_its_hash_sign() {
10941 +
    hermetic();
10942 +
    let dir = tempfile::tempdir().unwrap();
10943 +
    let root = dir.path();
10944 +
    let repo = Repo::init(root).unwrap();
10945 +
    write(root, "a.txt", b"a\n");
10946 +
    let h = repo.commit("a", &[]).unwrap();
10947 +
    assert_eq!(repo.resolve_change(&h.to_hex()).unwrap(), h);
10948 +
    assert_eq!(repo.resolve_change(&format!("#{}", h.to_hex())).unwrap(), h);
10949 +
    assert!(repo
10950 +
        .resolve_change(&"0".repeat(64))
10951 +
        .unwrap_err()
10952 +
        .to_string()
10953 +
        .contains("no change matches"));
10954 +
}
10955 +
10956 +
#[test]
10957 +
fn a_pin_change_moves_only_the_pin() {
10958 +
    hermetic();
10959 +
    let dir = tempfile::tempdir().unwrap();
10960 +
    let root = dir.path();
10961 +
    let _lib_origin = Repo::init(&root.join("lib-origin")).unwrap();
10962 +
    let lib_url = root.join("lib-origin").to_string_lossy().to_string();
10963 +
    let lib = Repo::clone(&lib_url, &root.join("lib"))
10964 +
        .unwrap()
10965 +
        .with_home(root.join("home"));
10966 +
    write(&root.join("lib"), "lib.txt", b"v1\n");
10967 +
    lib.commit("lib v1", &[]).unwrap();
10968 +
    assert!(lib.push("origin", false).unwrap()[0].ok);
10969 +
    let _app_origin = Repo::init(&root.join("app-origin")).unwrap();
10970 +
    let app_dir = root.join("app");
10971 +
    let app = Repo::clone(&root.join("app-origin").to_string_lossy(), &app_dir)
10972 +
        .unwrap()
10973 +
        .with_home(root.join("home"));
10974 +
    write(&app_dir, "main.txt", b"app\n");
10975 +
    let pin1 = app.submodule_add(&lib_url, "libs/lib", None).unwrap();
10976 +
    let a1 = app.commit("app with lib", &[]).unwrap();
10977 +
    // The library moves on.
10978 +
    write(&root.join("lib"), "more.txt", b"v2\n");
10979 +
    let l2 = lib.commit("lib v2", &[]).unwrap();
10980 +
    let crate::subrepo::Pin::Oxy { repo, bookmark, .. } = pin1 else {
10981 +
        panic!("an Oxygen pin")
10982 +
    };
10983 +
    let pin2 = crate::subrepo::Pin::Oxy {
10984 +
        repo,
10985 +
        heads: vec![l2],
10986 +
        bookmark,
10987 +
    };
10988 +
    let author = app.store().get_change(&a1).unwrap().author;
10989 +
    let change = app
10990 +
        .build_pin_change(&[a1], "libs/lib", &pin2, "Move lib to v2", author)
10991 +
        .unwrap();
10992 +
    let mut state = app.load_state(&[a1]).unwrap();
10993 +
    state.add(change).unwrap();
10994 +
    let files = state.materialize().files;
10995 +
    assert_eq!(files["libs/lib"].kind, NodeKind::Subrepo);
10996 +
    let FileContent::Opaque { blobs } = &files["libs/lib"].content else {
10997 +
        panic!("a pin is opaque")
10998 +
    };
10999 +
    let read = crate::subrepo::Pin::decode(&app.read_blob(&blobs[0]).unwrap()).unwrap();
11000 +
    assert_eq!(read, pin2, "the new pin");
11001 +
    assert!(
11002 +
        matches!(&files["main.txt"].content, FileContent::Text { bytes, .. } if bytes.as_slice() == b"app\n"),
11003 +
        "nothing else changes"
11004 +
    );
11005 +
    // Only a subrepo's pin can be moved this way.
11006 +
    let e = app.build_pin_change(&[a1], "main.txt", &pin2, "x", author);
11007 +
    assert!(format!("{:?}", e.unwrap_err()).contains("not a subrepo"));
11008 +
    let e = app.build_pin_change(&[a1], "libs/none", &pin2, "x", author);
11009 +
    assert!(format!("{:?}", e.unwrap_err()).contains("does not exist"));
11010 +
}
11011 +
11012 +
/// oxygen#147: a subrepo at its pinned state is unmodified whatever
11013 +
/// bookmark it sits on, so the parent's status is clean and a bookmark
11014 +
/// switch goes ahead; a new state in it is still a change.
11015 +
#[test]
11016 +
fn a_subrepo_at_its_pinned_state_on_another_bookmark_is_unmodified() {
11017 +
    hermetic();
11018 +
    let dir = tempfile::tempdir().unwrap();
11019 +
    let root = dir.path();
11020 +
    let _lib_origin = Repo::init(&root.join("lib-origin")).unwrap();
11021 +
    let lib_url = root.join("lib-origin").to_string_lossy().to_string();
11022 +
    let lib = Repo::clone(&lib_url, &root.join("lib"))
11023 +
        .unwrap()
11024 +
        .with_home(root.join("home"));
11025 +
    write(&root.join("lib"), "lib.txt", b"v1\n");
11026 +
    lib.commit("lib v1", &[]).unwrap();
11027 +
    assert!(lib.push("origin", false).unwrap()[0].ok);
11028 +
    let _app_origin = Repo::init(&root.join("app-origin")).unwrap();
11029 +
    let app_dir = root.join("app");
11030 +
    let mut app = Repo::clone(&root.join("app-origin").to_string_lossy(), &app_dir)
11031 +
        .unwrap()
11032 +
        .with_home(root.join("home"));
11033 +
    write(&app_dir, "main.txt", b"app\n");
11034 +
    app.submodule_add(&lib_url, "libs/lib", None).unwrap();
11035 +
    let a1 = app.commit("app with lib", &[]).unwrap();
11036 +
11037 +
    // The nested repository moves to another bookmark at the same heads.
11038 +
    let mut nested = Repo::open(&app_dir.join("libs/lib"))
11039 +
        .unwrap()
11040 +
        .with_home(root.join("home"));
11041 +
    let heads = nested.bookmark_heads().unwrap();
11042 +
    nested.bookmark_create("feature", Some(&heads)).unwrap();
11043 +
    nested.bookmark_switch("feature", false).unwrap();
11044 +
    let snap = app.snapshot("status").unwrap();
11045 +
    assert!(
11046 +
        !snap.changed,
11047 +
        "only the bookmark differs: {:?}",
11048 +
        snap.statuses
11049 +
    );
11050 +
    app.bookmark_create("other", Some(&[a1])).unwrap();
11051 +
    app.bookmark_switch("other", false).unwrap();
11052 +
11053 +
    // A new state in it is a change, as before.
11054 +
    let nested = Repo::open(&app_dir.join("libs/lib"))
11055 +
        .unwrap()
11056 +
        .with_home(root.join("home"));
11057 +
    write(&app_dir.join("libs/lib"), "lib.txt", b"v2\n");
11058 +
    nested.commit("lib v2", &[]).unwrap();
11059 +
    let snap = app.snapshot("status").unwrap();
11060 +
    assert_eq!(
11061 +
        snap.statuses
11062 +
            .iter()
11063 +
            .map(|s| (s.path.as_str(), s.status))
11064 +
            .collect::<Vec<_>>(),
11065 +
        vec![("libs/lib", ChangeStatus::Modified)]
11066 +
    );
11067 +
}
11068 +
11069 +
/// oxygenhub#249: a handle reads the operation head once and keeps it,
11070 +
/// so one command or request never mixes two heads; another handle's
11071 +
/// operation shows only after `refresh`. Its own operations move it, and
11072 +
/// one recorded from the head it kept merges with what landed since, as
11073 +
/// any concurrent operation does.
11074 +
#[test]
11075 +
fn one_handle_keeps_one_head_until_it_refreshes() {
11076 +
    hermetic();
11077 +
    let dir = tempfile::tempdir().unwrap();
11078 +
    let root = dir.path();
11079 +
    let writer = Repo::init(root).unwrap();
11080 +
    write(root, "a.txt", b"one\n");
11081 +
    let h1 = writer.commit("one", &[]).unwrap();
11082 +
    let reader = Repo::open(root).unwrap();
11083 +
    let before = reader.view().unwrap();
11084 +
    assert_eq!(reader.bookmark_heads().unwrap(), vec![h1]);
11085 +
    write(root, "a.txt", b"two\n");
11086 +
    let h2 = writer.commit("two", &[]).unwrap();
11087 +
    assert_eq!(
11088 +
        writer.bookmark_heads().unwrap(),
11089 +
        vec![h2],
11090 +
        "its own operation"
11091 +
    );
11092 +
    assert_eq!(reader.view().unwrap(), before, "the head it read first");
11093 +
    assert_eq!(reader.bookmark_heads().unwrap(), vec![h1]);
11094 +
    reader.refresh().unwrap();
11095 +
    assert_eq!(reader.bookmark_heads().unwrap(), vec![h2]);
11096 +
11097 +
    // A handle recording from an older head merges with the newer one.
11098 +
    let stale = Repo::open(root).unwrap();
11099 +
    assert_eq!(stale.bookmark_heads().unwrap(), vec![h2]);
11100 +
    write(root, "a.txt", b"three\n");
11101 +
    let h3 = writer.commit("three", &[]).unwrap();
11102 +
    stale.bookmark_create("feature", Some(&[h1])).unwrap();
11103 +
    let view = stale.view().unwrap();
11104 +
    assert!(view.bookmarks.contains_key("feature"));
11105 +
    assert_eq!(
11106 +
        stale.bookmark_heads().unwrap(),
11107 +
        vec![h3],
11108 +
        "merged, not lost"
11109 +
    );
11110 +
    let fresh = Repo::open(root).unwrap();
11111 +
    assert_eq!(fresh.view().unwrap(), view);
11112 +
}
11113 +
11114 +
/// oxygenhub#249 over a store (a bucket's compare-and-swap): the same.
11115 +
#[test]
11116 +
fn over_a_store_one_handle_keeps_one_head() {
11117 +
    hermetic();
11118 +
    let backend = std::sync::Arc::new(crate::strata::MemoryStore::new());
11119 +
    crate::store::TEST_BACKEND.with(|t| *t.borrow_mut() = Some(backend.clone()));
11120 +
    let dir = tempfile::tempdir().unwrap();
11121 +
    let a = Repo::init(dir.path()).unwrap();
11122 +
    let (h0, _) = a.head_op().unwrap();
11123 +
    let b = Repo::open(dir.path()).unwrap();
11124 +
    assert_eq!(b.head_op().unwrap().0, h0);
11125 +
    let state = a.store.put_state(&StateObject::new(vec![])).unwrap();
11126 +
    let mut view = a.view().unwrap();
11127 +
    view.bookmarks.insert("x".into(), state);
11128 +
    let r = a.record_op(&[h0], "bookmark x", view).unwrap();
11129 +
    assert_eq!(a.head_op().unwrap().0, r);
11130 +
    assert_eq!(b.head_op().unwrap().0, h0, "b keeps its head");
11131 +
    b.refresh().unwrap();
11132 +
    assert_eq!(b.head_op().unwrap().0, r);
11133 +
    crate::store::TEST_BACKEND.with(|t| *t.borrow_mut() = None);
11134 +
}
11135 +
11136 +
/// oxygenhub#249: gc retires what it repacks rather than deleting it; a
11137 +
/// reader opened before reads on, then and after the next gc deletes the
11138 +
/// retired files once their hour is up. Younger ones are kept.
11139 +
#[test]
11140 +
fn gc_retires_and_a_later_gc_deletes_after_the_hour() {
11141 +
    hermetic();
11142 +
    let dir = tempfile::tempdir().unwrap();
11143 +
    let root = dir.path();
11144 +
    let repo = Repo::init(root).unwrap();
11145 +
    let mut heads = Vec::new();
11146 +
    for i in 0..4 {
11147 +
        write(root, &format!("f{i}.txt"), format!("{i}\n").as_bytes());
11148 +
        heads.push(repo.commit(&format!("c{i}"), &[]).unwrap());
11149 +
    }
11150 +
    let reader = Repo::open(root).unwrap();
11151 +
    assert_eq!(reader.log().unwrap().len(), 5);
11152 +
    let r = repo.gc(true).unwrap();
11153 +
    assert_eq!(r.retired, crate::store::RetiredReport::default());
11154 +
    assert_eq!(repo.store().loose_count().unwrap(), 0);
11155 +
    assert_eq!(reader.log().unwrap().len(), 5, "read on from what it had");
11156 +
    // Nothing is an hour old yet: a gc keeps it all.
11157 +
    let r = repo.gc(true).unwrap();
11158 +
    assert_eq!((r.retired.packs, r.retired.loose), (0, 0));
11159 +
    assert!(r.retired.kept > 0, "{r:?}");
11160 +
    write(root, "f9.txt", b"9\n");
11161 +
    heads.push(repo.commit("c9", &[]).unwrap());
11162 +
    crate::store::reader_tests::age_retired(
11163 +
        repo.store(),
11164 +
        crate::store::RETIRED_FOR + std::time::Duration::from_secs(60),
11165 +
    );
11166 +
    let r = repo.gc(true).unwrap();
11167 +
    assert!(r.retired.packs >= 1 && r.retired.loose > 0, "{r:?}");
11168 +
    assert_eq!(r.retired.kept, 0, "{r:?}");
11169 +
    let r = repo.gc(true).unwrap();
11170 +
    assert_eq!((r.retired.packs, r.retired.loose), (0, 0));
11171 +
    assert!(r.retired.kept > 0, "what the last gc retired is young");
11172 +
    // The reader opened before all of it still reads everything, from
11173 +
    // the packs it finds when what it loaded is gone.
11174 +
    assert_eq!(reader.log().unwrap().len(), 5);
11175 +
    for h in &heads[..4] {
11176 +
        reader.store().get_change(h).unwrap();
11177 +
    }
11178 +
    reader.refresh().unwrap();
11179 +
    assert_eq!(reader.log().unwrap().len(), 6);
11180 +
    let fresh = Repo::open(root).unwrap();
11181 +
    assert_eq!(
11182 +
        fresh.store().pack_count(),
11183 +
        1,
11184 +
        "retired packs are not loaded"
11185 +
    );
11186 +
}
11187 +
11188 +
/// oxygenhub#249: a repository opens when its store's pack listing names
11189 +
/// an index that is not there (deleted between the listing and the read).
11190 +
#[test]
11191 +
fn a_repository_opens_when_a_listed_index_is_gone() {
11192 +
    hermetic();
11193 +
    let backend = crate::store::reader_tests::racing(true);
11194 +
    let dyn_backend: std::sync::Arc<crate::store::Backend> = backend.clone();
11195 +
    crate::store::TEST_BACKEND.with(|t| *t.borrow_mut() = Some(dyn_backend));
11196 +
    let dir = tempfile::tempdir().unwrap();
11197 +
    let repo = Repo::init(dir.path()).unwrap();
11198 +
    write(dir.path(), "a.txt", b"a\n");
11199 +
    let h = repo.commit("a", &[]).unwrap();
11200 +
    repo.gc(true).unwrap();
11201 +
    let again = Repo::open(dir.path()).unwrap();
11202 +
    assert_eq!(again.store().pack_count(), 1);
11203 +
    assert_eq!(again.bookmark_heads().unwrap(), vec![h]);
11204 +
    // And when one is deleted between the listing and the read: a copy of
11205 +
    // the live pack, whose index goes as the store opens.
11206 +
    use crate::strata::Store as _;
11207 +
    let idx = backend
11208 +
        .list("pack-")
11209 +
        .unwrap()
11210 +
        .into_iter()
11211 +
        .map(|e| e.key)
11212 +
        .find(|k| k.ends_with(".idx") && k != "pack-0000.idx")
11213 +
        .unwrap();
11214 +
    let stem = idx.trim_end_matches(".idx");
11215 +
    for ext in ["pack", "idx"] {
11216 +
        let bytes = backend.get(&format!("{stem}.{ext}")).unwrap();
11217 +
        backend
11218 +
            .put_new(&format!("pack-copy.{ext}"), &bytes)
11219 +
            .unwrap();
11220 +
    }
11221 +
    *backend.doomed.lock().unwrap() = Some("pack-copy.idx".into());
11222 +
    let again = Repo::open(dir.path()).unwrap();
11223 +
    assert_eq!(again.store().pack_count(), 1);
11224 +
    assert_eq!(again.bookmark_heads().unwrap(), vec![h]);
11225 +
    assert!(!backend.has("pack-copy.idx").unwrap());
11226 +
    crate::store::TEST_BACKEND.with(|t| *t.borrow_mut() = None);
11227 +
}
11228 +
11229 +
/// oxygen#82: a home with `author.for` authors as the identity its hub
11230 +
/// last named (kept in `keys/author.for`), not its stale `author.key`; a
11231 +
/// repository's own `author.key` pin still wins, and without `author.for`
11232 +
/// the note is ignored.
11233 +
#[test]
11234 +
fn author_for_follows_the_key_the_hub_named() {
11235 +
    let dir = tempfile::tempdir().unwrap();
11236 +
    let home = dir.path().join("home");
11237 +
    fs::create_dir_all(home.join("keys")).unwrap();
11238 +
    let user = home.join("config.toml");
11239 +
    let repo_cfg = dir.path().join("config");
11240 +
    let (old, new, pin) = (
11241 +
        KeyId::from_hex(&"a".repeat(64)).unwrap(),
11242 +
        KeyId::from_hex(&"b".repeat(64)).unwrap(),
11243 +
        KeyId::from_hex(&"c".repeat(64)).unwrap(),
11244 +
    );
11245 +
    fs::write(&user, format!("author.key = \"{}\"\n", old.to_hex())).unwrap();
11246 +
    fs::write(
11247 +
        home.join("keys").join(super::config::AUTHOR_FOR_FILE),
11248 +
        format!("{}\n", new.to_hex()),
11249 +
    )
11250 +
    .unwrap();
11251 +
    fs::write(&repo_cfg, "").unwrap();
11252 +
    assert_eq!(
11253 +
        Config::load(Some(&user), &repo_cfg).unwrap().author_key,
11254 +
        Some(old),
11255 +
        "no author.for: the note is ignored"
11256 +
    );
11257 +
    fs::write(
11258 +
        &user,
11259 +
        format!(
11260 +
            "author.key = \"{}\"\nauthor.for = \"andymccall\"\n",
11261 +
            old.to_hex()
11262 +
        ),
11263 +
    )
11264 +
    .unwrap();
11265 +
    assert_eq!(
11266 +
        Config::load(Some(&user), &repo_cfg).unwrap().author_key,
11267 +
        Some(new)
11268 +
    );
11269 +
    fs::write(&repo_cfg, format!("author.key = \"{}\"\n", pin.to_hex())).unwrap();
11270 +
    assert_eq!(
11271 +
        Config::load(Some(&user), &repo_cfg).unwrap().author_key,
11272 +
        Some(pin),
11273 +
        "a repository's pin wins"
11274 +
    );
11275 +
}
11276 +
11277 +
#[test]
11278 +
fn git_export_graph_is_deterministic_and_merges_heads() {
11279 +
    hermetic();
11280 +
    if std::process::Command::new("git")
11281 +
        .arg("--version")
11282 +
        .output()
11283 +
        .is_err()
11284 +
    {
11285 +
        eprintln!("git not installed; skipping");
11286 +
        return;
11287 +
    }
11288 +
    let dir = tempfile::tempdir().unwrap();
11289 +
    let root = dir.path();
11290 +
    let mut repo = Repo::init(&root.join("r"))
11291 +
        .unwrap()
11292 +
        .with_home(root.join("home"));
11293 +
    repo.create_identity("Ada Lovelace", "[email protected]", IdentityScope::User)
11294 +
        .unwrap();
11295 +
    let r = root.join("r");
11296 +
    write(&r, "README.md", b"hello\n");
11297 +
    let h1 = repo.commit("first", &[]).unwrap();
11298 +
    write(&r, "README.md", b"hello\nworld\n");
11299 +
    write(&r, "src/main.rs", b"fn main() {}\n");
11300 +
    let h2 = repo.commit("second", &[]).unwrap();
11301 +
    repo.tag_create("v1", &[h2], "the first release").unwrap();
11302 +
    // An independent change: main gets two heads.
11303 +
    repo.checkout(&[h1], false, "checkout").unwrap();
11304 +
    write(&r, "other.txt", b"o\n");
11305 +
    let h3 = repo.commit("third", &[]).unwrap();
11306 +
    repo.bookmark_create("gone", Some(&[h1])).unwrap();
11307 +
11308 +
    let git = |git_dir: &Path, args: &[&str]| -> String {
11309 +
        let out = std::process::Command::new("git")
11310 +
            .arg("-C")
11311 +
            .arg(git_dir)
11312 +
            .args(args)
11313 +
            .output()
11314 +
            .unwrap();
11315 +
        assert!(
11316 +
            out.status.success(),
11317 +
            "git {args:?}: {}",
11318 +
            String::from_utf8_lossy(&out.stderr)
11319 +
        );
11320 +
        String::from_utf8_lossy(&out.stdout).trim().to_string()
11321 +
    };
11322 +
    let a = root.join("a.git");
11323 +
    let rep = repo.git_export_graph(&a).unwrap();
11324 +
    assert_eq!((rep.commits, rep.tags), (3, 1), "{rep:?}");
11325 +
    assert_eq!(rep.branches, 2, "main and gone: {rep:?}");
11326 +
11327 +
    // Each change is a commit whose parents are exactly its dependencies'
11328 +
    // commits (Oxygen's dependencies, not the order changes were made in).
11329 +
    let by_hash = |hash: ChangeHash| {
11330 +
        git(
11331 +
            &a,
11332 +
            &[
11333 +
                "log",
11334 +
                "--all",
11335 +
                "--format=%H",
11336 +
                &format!("--grep=Oxy-Hash: {}", hash.to_hex()),
11337 +
            ],
11338 +
        )
11339 +
    };
11340 +
    let (c1, c2, c3) = (by_hash(h1), by_hash(h2), by_hash(h3));
11341 +
    for (h, c) in [(h1, &c1), (h2, &c2), (h3, &c3)] {
11342 +
        let mut want: Vec<String> = repo
11343 +
            .store()
11344 +
            .get_change_meta(&h)
11345 +
            .unwrap()
11346 +
            .deps
11347 +
            .iter()
11348 +
            .map(|d| by_hash(*d))
11349 +
            .collect();
11350 +
        want.sort();
11351 +
        let mut have: Vec<String> = git(&a, &["rev-list", "--parents", "-1", c])
11352 +
            .split(' ')
11353 +
            .skip(1)
11354 +
            .map(String::from)
11355 +
            .collect();
11356 +
        have.sort();
11357 +
        assert_eq!(have, want, "{h:?}'s commit's parents");
11358 +
    }
11359 +
    assert!(
11360 +
        !repo.store().get_change_meta(&h2).unwrap().deps.is_empty(),
11361 +
        "second edits first's README"
11362 +
    );
11363 +
    assert_eq!(
11364 +
        git(&a, &["show", &format!("{c2}:src/main.rs")]),
11365 +
        "fn main() {}"
11366 +
    );
11367 +
    assert!(
11368 +
        git(&a, &["ls-tree", "--name-only", &c3])
11369 +
            .lines()
11370 +
            .all(|l| l != "src"),
11371 +
        "a change's tree is its own closure, not the bookmark's"
11372 +
    );
11373 +
    assert_eq!(
11374 +
        git(&a, &["log", "-1", "--format=%an <%ae>", &c1]),
11375 +
        "Ada Lovelace <[email protected]>"
11376 +
    );
11377 +
11378 +
    // main has two heads: a merge of both, whose tree is the union.
11379 +
    let main = git(&a, &["rev-parse", "main"]);
11380 +
    let mut parents: Vec<String> = git(&a, &["rev-list", "--parents", "-1", "main"])
11381 +
        .split(' ')
11382 +
        .skip(1)
11383 +
        .map(String::from)
11384 +
        .collect();
11385 +
    parents.sort();
11386 +
    let mut want = vec![c2.clone(), c3.clone()];
11387 +
    want.sort();
11388 +
    assert_eq!(parents, want, "the merge's parents are the heads");
11389 +
    assert_eq!(
11390 +
        git(&a, &["ls-tree", "--name-only", "main"]),
11391 +
        "README.md\nother.txt\nsrc"
11392 +
    );
11393 +
    assert!(git(&a, &["log", "-1", "--format=%B", "main"]).contains("Merge of 2 heads"));
11394 +
    assert_eq!(
11395 +
        git(&a, &["rev-parse", "v1^{commit}"]),
11396 +
        c2,
11397 +
        "the tag is its head's commit"
11398 +
    );
11399 +
    assert_eq!(git(&a, &["symbolic-ref", "HEAD"]), "refs/heads/main");
11400 +
11401 +
    // Again: nothing new. A bookmark deleted takes its branch with it.
11402 +
    repo.bookmark_delete("gone").unwrap();
11403 +
    let rep = repo.git_export_graph(&a).unwrap();
11404 +
    assert_eq!(rep.commits, 0, "{rep:?}");
11405 +
    assert!(
11406 +
        git(&a, &["for-each-ref", "--format=%(refname)", "refs/heads"])
11407 +
            .lines()
11408 +
            .all(|r| r != "refs/heads/gone")
11409 +
    );
11410 +
    assert_eq!(
11411 +
        git(&a, &["rev-parse", "main"]),
11412 +
        main,
11413 +
        "the merge is made once"
11414 +
    );
11415 +
11416 +
    // The same changes elsewhere give the same commits.
11417 +
    let other = Repo::clone(r.to_str().unwrap(), &root.join("clone"))
11418 +
        .unwrap()
11419 +
        .with_home(root.join("home"));
11420 +
    let b = root.join("b.git");
11421 +
    other.git_export_graph(&b).unwrap();
11422 +
    assert_eq!(
11423 +
        git(&b, &["rev-parse", "main"]),
11424 +
        main,
11425 +
        "deterministic across repositories"
11426 +
    );
11427 +
    assert_eq!(git(&b, &["rev-parse", "v1^{commit}"]), c2);
11428 +
}
11429 +
11430 +
#[test]
11431 +
fn a_fetch_of_many_bookmarks_takes_shared_changes_once() {
11432 +
    // oxygen#158: every bookmark's heads are fetched in one exchange, so
11433 +
    // changes bookmarks share are not sent again for each; each bookmark
11434 +
    // still says how many of its changes were new, and a fetch with
11435 +
    // nothing new imports nothing.
11436 +
    hermetic();
11437 +
    let dir = tempfile::tempdir().unwrap();
11438 +
    let root = dir.path();
11439 +
    drop(Repo::init(&root.join("origin")).unwrap());
11440 +
    let url = root.join("origin").to_string_lossy().to_string();
11441 +
    let mut a = Repo::clone(&url, &root.join("a"))
11442 +
        .unwrap()
11443 +
        .with_home(root.join("home-a"));
11444 +
    let w = root.join("a");
11445 +
    write(&w, "f.txt", b"one\n");
11446 +
    a.commit("one", &[]).unwrap();
11447 +
    write(&w, "f.txt", b"one\ntwo\n");
11448 +
    a.commit("two", &[]).unwrap();
11449 +
    a.push("origin", false).unwrap();
11450 +
    // Five bookmarks on main's history, each one change of its own.
11451 +
    for i in 0..5 {
11452 +
        a.bookmark_create(&format!("feat{i}"), None).unwrap();
11453 +
        a.bookmark_switch(&format!("feat{i}"), false).unwrap();
11454 +
        write(&w, &format!("feat{i}.txt"), format!("{i}\n").as_bytes());
11455 +
        a.commit(&format!("feat {i}"), &[]).unwrap();
11456 +
        a.push("origin", false).unwrap();
11457 +
        a.bookmark_switch("main", false).unwrap();
11458 +
    }
11459 +
11460 +
    let b = Repo::clone(&url, &root.join("b"))
11461 +
        .unwrap()
11462 +
        .with_home(root.join("home-b"));
11463 +
    let report = b.fetch("origin").unwrap();
11464 +
    assert_eq!(report.refs.len(), 6, "every bookmark: {:?}", report.refs);
11465 +
    assert!(
11466 +
        report.refs.iter().all(|(_, _, n)| *n == 0),
11467 +
        "the clone brought them all: {:?}",
11468 +
        report.refs
11469 +
    );
11470 +
11471 +
    // One new change on one bookmark: counted there, nowhere else.
11472 +
    a.bookmark_switch("feat3", false).unwrap();
11473 +
    write(&w, "feat3.txt", b"3\nmore\n");
11474 +
    a.commit("feat 3 again", &[]).unwrap();
11475 +
    a.push("origin", false).unwrap();
11476 +
    let fetches = || crate::protocol::LOCAL_FETCHES.with(|n| n.get());
11477 +
    let at = fetches();
11478 +
    let report = b.fetch("origin").unwrap();
11479 +
    assert_eq!(fetches() - at, 1, "one bundle for all six bookmarks");
11480 +
    let counts: Vec<(String, usize)> = report
11481 +
        .refs
11482 +
        .iter()
11483 +
        .map(|(n, _, c)| (n.clone(), *c))
11484 +
        .collect();
11485 +
    assert!(counts.contains(&("feat3".to_string(), 1)), "{counts:?}");
11486 +
    assert_eq!(
11487 +
        counts.iter().filter(|(_, c)| *c > 0).count(),
11488 +
        1,
11489 +
        "only feat3 had anything new: {counts:?}"
11490 +
    );
11491 +
11492 +
    // Nothing new: no bundle at all, every bookmark still listed.
11493 +
    let at = fetches();
11494 +
    let report = b.fetch("origin").unwrap();
11495 +
    assert_eq!(fetches() - at, 0, "nothing to fetch, no round trip");
11496 +
    assert_eq!(report.refs.len(), 6);
11497 +
    assert!(
11498 +
        report.refs.iter().all(|(_, _, n)| *n == 0),
11499 +
        "{:?}",
11500 +
        report.refs
11501 +
    );
11502 +
}
11503 +
11504 +
#[test]
11505 +
fn a_stale_pinned_key_cannot_fork_the_identity() {
11506 +
    // oxygen#163: a checkout pinned author.key to a key that a rotation made
11507 +
    // elsewhere had already succeeded; delegating there built a second
11508 +
    // successor of it, splitting the identity in two.
11509 +
    hermetic();
11510 +
    let dir = tempfile::tempdir().unwrap();
11511 +
    let root = dir.path();
11512 +
    let home = root.join("home");
11513 +
    let mut a = Repo::init(&root.join("a")).unwrap().with_home(&home);
11514 +
    let (first, _) = a
11515 +
        .create_identity("Ada", "[email protected]", IdentityScope::User)
11516 +
        .unwrap();
11517 +
    let r = a.rotate_identity().unwrap();
11518 +
    assert_eq!((r.old, a.config.author_key), (first, Some(r.new)));
11519 +
    assert_eq!(a.successors(&first).unwrap(), vec![r.new]);
11520 +
    assert_eq!(a.newest(&first).unwrap(), r.new);
11521 +
11522 +
    // Another checkout still pins the first key.
11523 +
    let mut b = Repo::init(&root.join("b")).unwrap().with_home(&home);
11524 +
    b.config.set("author.key", &first.to_hex()).unwrap();
11525 +
    b.save_config().unwrap();
11526 +
    let mut b = Repo::open(&root.join("b")).unwrap().with_home(&home);
11527 +
    assert_eq!(b.config.author_key, Some(first));
11528 +
    for e in [
11529 +
        b.rotate_identity().unwrap_err().to_string(),
11530 +
        b.renew_identity(None).unwrap_err().to_string(),
11531 +
    ] {
11532 +
        assert!(e.contains("fork") && e.contains(&r.new.short(12)), "{e}");
11533 +
        assert!(
11534 +
            e.contains(&format!("oxy config author.key {}", r.new.to_hex())),
11535 +
            "{e}"
11536 +
        );
11537 +
    }
11538 +
    assert_eq!(
11539 +
        b.successors(&first).unwrap(),
11540 +
        vec![r.new],
11541 +
        "nothing was made"
11542 +
    );
11543 +
11544 +
    // Pointed at the newest key, it rotates as usual.
11545 +
    b.config.set("author.key", &r.new.to_hex()).unwrap();
11546 +
    b.save_config().unwrap();
11547 +
    let mut b = Repo::open(&root.join("b")).unwrap().with_home(&home);
11548 +
    let r2 = b.rotate_identity().unwrap();
11549 +
    assert_eq!(r2.old, r.new);
11550 +
    assert_eq!(b.newest(&first).unwrap(), r2.new);
11551 +
}
added crates/oxy-core/src/repo/touched.rs +163 −0
1 +
//! The nodes each change touches, kept beside the store (oxygen#91).
2 +
//!
3 +
//! Finding the changes that touched a file (`oxy log --path`, the
4 +
//! `touches()` revset) otherwise reads and decodes every change of the
5 +
//! history, its lines included. A change is immutable, so its entry never
6 +
//! goes stale: one missing is read from the store once and added. Like
7 +
//! Git's changed-path filters it is derived data, and deleting
8 +
//! `.oxy/index/touched` loses nothing but time.
9 +
10 +
use super::*;
11 +
use crate::fasthash::FastMap;
12 +
use crate::ids::NodeId;
13 +
14 +
/// The cache, loaded; [`Touched::save`] writes it back when it grew.
15 +
pub(crate) struct Touched {
16 +
    nodes: FastMap<ChangeHash, Box<[NodeId]>>,
17 +
    /// Working-copy changes, which a snapshot replaces: kept for this
18 +
    /// command only, so the cache does not fill with dead entries.
19 +
    transient: FastMap<ChangeHash, Box<[NodeId]>>,
20 +
    added: usize,
21 +
    path: PathBuf,
22 +
}
23 +
24 +
impl Touched {
25 +
    /// The nodes `h` touches, from the cache or else from the store; `None`
26 +
    /// when the change is not stored (beyond a shallow boundary).
27 +
    pub(crate) fn of(&mut self, repo: &Repo, h: &ChangeHash) -> Option<&[NodeId]> {
28 +
        if !self.nodes.contains_key(h) && !self.transient.contains_key(h) {
29 +
            // The skeleton names every node; no text is read.
30 +
            let (c, _) = repo.store.get_change_lazy(h).ok()?;
31 +
            self.keep(*h, &c);
32 +
        }
33 +
        self.nodes
34 +
            .get(h)
35 +
            .or_else(|| self.transient.get(h))
36 +
            .map(|n| &n[..])
37 +
    }
38 +
39 +
    fn keep(&mut self, h: ChangeHash, c: &Change) {
40 +
        if c.kind == ChangeKind::WorkingCopy {
41 +
            self.transient.insert(h, nodes_of(c));
42 +
        } else {
43 +
            self.nodes.insert(h, nodes_of(c));
44 +
            self.added += 1;
45 +
        }
46 +
    }
47 +
48 +
    /// Read every one of `hashes` the cache lacks.
49 +
    pub(crate) fn fill(&mut self, repo: &Repo, hashes: &[ChangeHash]) {
50 +
        use rayon::prelude::*;
51 +
        let missing: Vec<ChangeHash> = hashes
52 +
            .iter()
53 +
            .filter(|h| !self.nodes.contains_key(*h) && !self.transient.contains_key(*h))
54 +
            .copied()
55 +
            .collect();
56 +
        // Read a batch at a time, then decode skeletons (no text) on every
57 +
        // core. A change the store lacks is left out, as `of` leaves it.
58 +
        for batch in missing.chunks(8192) {
59 +
            let read: Vec<(ChangeHash, Change)> = repo
60 +
                .store
61 +
                .get_many(batch)
62 +
                .into_par_iter()
63 +
                .zip(batch.par_iter())
64 +
                .filter_map(|(got, h)| match got {
65 +
                    Ok((ObjectType::Change, bytes)) => {
66 +
                        Some((*h, Change::decode_skeleton(&bytes, *h).ok()?.0))
67 +
                    }
68 +
                    _ => None,
69 +
                })
70 +
                .collect();
71 +
            for (h, c) in &read {
72 +
                self.keep(*h, c);
73 +
            }
74 +
        }
75 +
    }
76 +
77 +
    /// Whether `h` touches any of `nodes` (false when it is not stored).
78 +
    pub(crate) fn touches(&mut self, repo: &Repo, h: &ChangeHash, nodes: &HashSet<NodeId>) -> bool {
79 +
        self.of(repo, h)
80 +
            .is_some_and(|n| n.iter().any(|n| nodes.contains(n)))
81 +
    }
82 +
83 +
    /// Write the cache back if it grew; quietly nothing when it cannot be.
84 +
    pub(crate) fn save(self) {
85 +
        if self.added == 0 {
86 +
            return;
87 +
        }
88 +
        let _t = crate::timing::phase("write: touched nodes");
89 +
        let mut w = crate::flat::Writer::default();
90 +
        w.count(self.nodes.len());
91 +
        for (h, nodes) in &self.nodes {
92 +
            w.object(h);
93 +
            w.count(nodes.len());
94 +
            for n in nodes.iter() {
95 +
                w.fixed(&n.0);
96 +
            }
97 +
        }
98 +
        // Checked whole on reading: a damaged entry would hide a change.
99 +
        let sum = *blake3::hash(&w.0).as_bytes();
100 +
        w.fixed(&sum);
101 +
        let _ = fs::create_dir_all(self.path.parent().unwrap_or(&self.path));
102 +
        let _ = write_cache(&self.path, &w.0);
103 +
    }
104 +
}
105 +
106 +
/// The nodes a change's tree operations and edits name, each once.
107 +
fn nodes_of(c: &Change) -> Box<[NodeId]> {
108 +
    let mut nodes: Vec<NodeId> = c
109 +
        .tree_ops
110 +
        .iter()
111 +
        .map(|o| o.node())
112 +
        .chain(c.edits.iter().map(|e| e.node()))
113 +
        .collect();
114 +
    nodes.sort();
115 +
    nodes.dedup();
116 +
    nodes.into_boxed_slice()
117 +
}
118 +
119 +
impl Repo {
120 +
    /// The touched-nodes cache: empty when there is none yet or it does
121 +
    /// not read back whole.
122 +
    pub(crate) fn touched(&self) -> Touched {
123 +
        let _t = crate::timing::phase("load: touched nodes");
124 +
        let path = self.oxy.join("index").join("touched");
125 +
        let nodes = fs::read(&path)
126 +
            .ok()
127 +
            .and_then(|b| read_touched(&b))
128 +
            .unwrap_or_default();
129 +
        Touched {
130 +
            nodes,
131 +
            transient: FastMap::default(),
132 +
            added: 0,
133 +
            path,
134 +
        }
135 +
    }
136 +
}
137 +
138 +
fn read_touched(bytes: &[u8]) -> Option<FastMap<ChangeHash, Box<[NodeId]>>> {
139 +
    let (body, sum) = bytes.split_at_checked(bytes.len().checked_sub(32)?)?;
140 +
    if blake3::hash(body).as_bytes() != sum {
141 +
        return None;
142 +
    }
143 +
    let mut r = crate::flat::Reader::new(body);
144 +
    let n = r.count().ok()?;
145 +
    let mut out = FastMap::default();
146 +
    out.reserve(n);
147 +
    for _ in 0..n {
148 +
        let h = r.object().ok()?;
149 +
        let k = r.count().ok()?;
150 +
        let mut nodes = Vec::with_capacity(k);
151 +
        for _ in 0..k {
152 +
            nodes.push(NodeId(r.fixed::<16>().ok()?));
153 +
        }
154 +
        out.insert(h, nodes.into_boxed_slice());
155 +
    }
156 +
    r.at_end().then_some(out)
157 +
}
158 +
159 +
/// The changes a cache file holds, for tests.
160 +
#[cfg(test)]
161 +
pub(crate) fn read_touched_for_test(bytes: &[u8]) -> Option<HashSet<ChangeHash>> {
162 +
    read_touched(bytes).map(|m| m.into_keys().collect())
163 +
}
added crates/oxy-core/src/repo/transparency.rs +623 −0
1 +
//! Checking a host's transparency log on fetch (spec §14.8, oxygenhub#238).
2 +
//!
3 +
//! A host that keeps a log sends its latest checkpoint with the refs, and
4 +
//! for each bookmark's attestation its entry and an inclusion proof. This
5 +
//! user remembers, per hub, the log's key and the last checkpoint seen
6 +
//! (`<oxy home>/tlog/<hub>`): the key on first sight, then never another.
7 +
//! Each new checkpoint must extend the last one, which oxy proves from the
8 +
//! hub's public tiles (C2SP tlog-tiles); one that does not means the hub
9 +
//! showed two histories, and the fetch fails. An attestation that is not
10 +
//! in the log yet, or whose proof does not verify, is only reported while
11 +
//! the log is new (oxygenhub docs/transparency.md, decision 3).
12 +
//!
13 +
//! The witnesses that cosign a hub's checkpoints are pinned the same way
14 +
//! (`<oxy home>/tlog/<hub>.witnesses`), and give each bookmark's
15 +
//! attestation an *integrated time* (oxygenhub#240): the earliest time a
16 +
//! trusted witness cosigned a checkpoint holding it. A fetch checks
17 +
//! delegations and certificates at that time when it has one.
18 +
19 +
use super::*;
20 +
use crate::credentials::hub_of;
21 +
use crate::protocol::http_get;
22 +
use crate::tlog::{self, Checkpoint, Entry, Hash, HashReader, Integrated, Note, Tile, TlogError};
23 +
use std::cell::RefCell;
24 +
25 +
/// Tiles fetched, by level and number, with the width fetched.
26 +
type TileCache = RefCell<HashMap<(u32, u64), (Tile, Vec<u8>)>>;
27 +
28 +
/// A hub's log as its tiles serve it, for a tree of `size` entries.
29 +
struct Tiles<'a> {
30 +
    hub: &'a str,
31 +
    size: u64,
32 +
    options: &'a RemoteOptions,
33 +
    cache: TileCache,
34 +
}
35 +
36 +
impl HashReader for Tiles<'_> {
37 +
    fn read_hashes(&self, indexes: &[u64]) -> std::result::Result<Vec<Hash>, TlogError> {
38 +
        let mut out = Vec::with_capacity(indexes.len());
39 +
        for &i in indexes {
40 +
            let (t, ..) = Tile::for_index(i);
41 +
            let key = (t.level, t.n);
42 +
            if !self.cache.borrow().contains_key(&key) {
43 +
                let held =
44 +
                    (self.size >> (tlog::TILE_HEIGHT * t.level)) - (t.n << tlog::TILE_HEIGHT);
45 +
                let tile = Tile {
46 +
                    width: held.min(tlog::TILE_WIDTH) as u32,
47 +
                    ..t
48 +
                };
49 +
                let data = http_get(&format!("{}/log/{}", self.hub, tile.path()), self.options)
50 +
                    .map_err(|e| TlogError::Read(e.to_string()))?;
51 +
                self.cache.borrow_mut().insert(key, (tile, data));
52 +
            }
53 +
            let cache = self.cache.borrow();
54 +
            let (tile, data) = &cache[&key];
55 +
            out.push(tile.hash_at(data, i)?);
56 +
        }
57 +
        Ok(out)
58 +
    }
59 +
}
60 +
61 +
/// What this user last saw of a hub's log.
62 +
struct Seen {
63 +
    identity: Identity,
64 +
    checkpoint: Checkpoint,
65 +
}
66 +
67 +
fn file_name(hub: &str) -> String {
68 +
    hub.chars()
69 +
        .map(|c| {
70 +
            if c.is_ascii_alphanumeric() || c == '.' || c == '-' {
71 +
                c
72 +
            } else {
73 +
                '_'
74 +
            }
75 +
        })
76 +
        .collect()
77 +
}
78 +
79 +
impl Repo {
80 +
    fn tlog_file(&self, hub: &str) -> PathBuf {
81 +
        self.user_config
82 +
            .parent()
83 +
            .map(Path::to_path_buf)
84 +
            .unwrap_or_else(default_home)
85 +
            .join("tlog")
86 +
            .join(file_name(hub))
87 +
    }
88 +
89 +
    fn seen_log(&self, hub: &str) -> Option<Seen> {
90 +
        let text = fs::read_to_string(self.tlog_file(hub)).ok()?;
91 +
        let (key, note) = text.split_once('\n')?;
92 +
        let identity = Identity::decode(&crate::dsse::base64_decode(key)?).ok()?;
93 +
        let note = Note::parse(note).ok()?;
94 +
        Some(Seen {
95 +
            checkpoint: Checkpoint::parse(&note.text).ok()?,
96 +
            identity,
97 +
        })
98 +
    }
99 +
100 +
    pub(crate) fn remember_log(&self, hub: &str, identity: &Identity, note: &Note) -> Result<()> {
101 +
        let path = self.tlog_file(hub);
102 +
        if let Some(dir) = path.parent() {
103 +
            fs::create_dir_all(dir)?;
104 +
        }
105 +
        let text = format!(
106 +
            "{}\n{}",
107 +
            crate::dsse::base64(&identity.encode()),
108 +
            note.encode()
109 +
        );
110 +
        crate::durable::write_atomic(&path, text.as_bytes())?;
111 +
        Ok(())
112 +
    }
113 +
114 +
    /// Take a checkpoint note `text` from `hub` as the newest seen, once it
115 +
    /// verifies with the hub's pinned log key and extends the last one
116 +
    /// seen. `Err` when the hub showed two histories; `Ok(Err(why))` when
117 +
    /// it could not be checked (the last one seen is kept).
118 +
    pub(crate) fn accept_checkpoint(
119 +
        &self,
120 +
        hub: &str,
121 +
        text: &str,
122 +
        options: &RemoteOptions,
123 +
    ) -> Result<std::result::Result<Accepted, String>> {
124 +
        let unchecked = |why: String| Ok(Err(why));
125 +
        let note = match Note::parse(text) {
126 +
            Ok(n) => n,
127 +
            Err(e) => return unchecked(e.to_string()),
128 +
        };
129 +
        let cp = match Checkpoint::parse(&note.text) {
130 +
            Ok(c) => c,
131 +
            Err(e) => return unchecked(e.to_string()),
132 +
        };
133 +
        let fetch_key = || {
134 +
            http_get(&format!("{hub}/.well-known/oxyhub-key"), options)
135 +
                .and_then(|b| Ok(Identity::decode(&b)?))
136 +
        };
137 +
        let mut notes = Vec::new();
138 +
        let mut seen = self.seen_log(hub);
139 +
        let mut identity = match &seen {
140 +
            Some(s) => s.identity.clone(),
141 +
            None => match fetch_key() {
142 +
                Ok(i) => i,
143 +
                Err(e) => return unchecked(format!("its key could not be fetched ({e})")),
144 +
            },
145 +
        };
146 +
        if let Err(e) = note.verify(&cp.origin, &identity) {
147 +
            // Signed by another key than the one pinned: a hub given a new
148 +
            // key, or one starting another log. Not proof of a fork, so a
149 +
            // loud note while logs are new (decision 3), and the new key is
150 +
            // pinned; a fork under one key is what fails.
151 +
            let Some(old) = &seen else {
152 +
                return unchecked(e.to_string());
153 +
            };
154 +
            match fetch_key() {
155 +
                Ok(new) if note.verify(&cp.origin, &new).is_ok() => {
156 +
                    notes.push(format!(
157 +
                        "{hub}'s log is signed by another key than before ({} now, {} then): if its operator did not announce a new key, it may be showing others a different log",
158 +
                        new.hash().short(12),
159 +
                        old.identity.hash().short(12)
160 +
                    ));
161 +
                    identity = new;
162 +
                    seen = None;
163 +
                }
164 +
                _ => return unchecked(format!("its checkpoint does not verify ({e})")),
165 +
            }
166 +
        }
167 +
        let two = |why: String| {
168 +
            fail(format!(
169 +
                "{hub} showed two histories: {why}. Do not trust what it serves; tell its operator. \
170 +
                 (What this user saw of its log is in {}.)",
171 +
                self.tlog_file(hub).display()
172 +
            ))
173 +
        };
174 +
        if let Some(s) = &seen {
175 +
            let old = &s.checkpoint;
176 +
            if old.origin != cp.origin {
177 +
                // A hub moved to another name keeps its key: a new log as
178 +
                // far as this user can tell, said so.
179 +
                notes.push(format!(
180 +
                    "{hub}'s log is now {}, not {}: checking it afresh",
181 +
                    cp.origin, old.origin
182 +
                ));
183 +
            } else if cp.size < old.size {
184 +
                return two(format!(
185 +
                    "its log has {} entries, fewer than the {} it showed before",
186 +
                    cp.size, old.size
187 +
                ));
188 +
            } else if cp.size == old.size && cp.root != old.root {
189 +
                return two(format!("two different logs of {} entries", cp.size));
190 +
            } else if cp.size > old.size && old.size > 0 {
191 +
                let tiles = Tiles {
192 +
                    hub,
193 +
                    size: cp.size,
194 +
                    options,
195 +
                    cache: RefCell::new(HashMap::new()),
196 +
                };
197 +
                match tlog::prove_consistency(cp.size, old.size, &tiles).and_then(|p| {
198 +
                    tlog::check_consistency(&p, cp.size, &cp.root, old.size, &old.root)
199 +
                }) {
200 +
                    Ok(()) => {}
201 +
                    Err(TlogError::ProofFailed) | Err(TlogError::BadIndex(_)) => {
202 +
                        return two(format!(
203 +
                            "its log of {} entries does not extend the one of {} it showed before",
204 +
                            cp.size, old.size
205 +
                        ))
206 +
                    }
207 +
                    // Keep the older checkpoint: the next fetch tries again.
208 +
                    Err(e) => return unchecked(format!("its tiles could not be read ({e})")),
209 +
                }
210 +
            }
211 +
        }
212 +
        self.remember_log(hub, &identity, &note)?;
213 +
        Ok(Ok(Accepted {
214 +
            checkpoint: cp,
215 +
            identity,
216 +
            note,
217 +
            notes,
218 +
        }))
219 +
    }
220 +
221 +
    fn witnesses_file(&self, hub: &str) -> PathBuf {
222 +
        let mut p = self.tlog_file(hub);
223 +
        p.set_file_name(format!("{}.witnesses", file_name(hub)));
224 +
        p
225 +
    }
226 +
227 +
    /// The witnesses this user pinned for `hub`: each one's name in
228 +
    /// cosignatures and its identity.
229 +
    pub(crate) fn pinned_witnesses(&self, hub: &str) -> Vec<(String, Identity)> {
230 +
        let Ok(text) = fs::read_to_string(self.witnesses_file(hub)) else {
231 +
            return vec![];
232 +
        };
233 +
        text.lines()
234 +
            .filter_map(|l| {
235 +
                let (name, key) = l.split_once(' ')?;
236 +
                let id = Identity::decode(&crate::dsse::base64_decode(key)?).ok()?;
237 +
                Some((name.to_string(), id))
238 +
            })
239 +
            .collect()
240 +
    }
241 +
242 +
    /// Pin the witnesses `hub` lists (name and identity bytes), as its log
243 +
    /// key is pinned: on first sight silently, later with a note for a
244 +
    /// witness that is new or has another key than before. A witness the
245 +
    /// hub no longer lists stays pinned. Returns every pinned witness.
246 +
    pub(crate) fn pin_witnesses(
247 +
        &self,
248 +
        hub: &str,
249 +
        listed: &[(String, Vec<u8>)],
250 +
        notes: &mut Vec<String>,
251 +
    ) -> Result<Vec<(String, Identity)>> {
252 +
        let path = self.witnesses_file(hub);
253 +
        let first = !path.exists();
254 +
        let mut pinned = self.pinned_witnesses(hub);
255 +
        let mut changed = false;
256 +
        for (name, bytes) in listed {
257 +
            if name.is_empty() || name.contains(char::is_whitespace) {
258 +
                notes.push(format!(
259 +
                    "{hub} lists a witness named {name:?}, which cannot name one"
260 +
                ));
261 +
                continue;
262 +
            }
263 +
            let Ok(id) = Identity::decode(bytes) else {
264 +
                notes.push(format!(
265 +
                    "{hub} lists witness {name} with a key that does not decode"
266 +
                ));
267 +
                continue;
268 +
            };
269 +
            match pinned.iter_mut().find(|(n, _)| n == name) {
270 +
                Some((_, old)) if old.hash() == id.hash() => {}
271 +
                Some((_, old)) => {
272 +
                    notes.push(format!(
273 +
                        "{hub}'s witness {name} has another key than before ({} now, {} then): if its operator did not announce one, {hub} may be naming a witness of its own",
274 +
                        id.hash().short(12),
275 +
                        old.hash().short(12)
276 +
                    ));
277 +
                    *old = id;
278 +
                    changed = true;
279 +
                }
280 +
                None => {
281 +
                    if !first {
282 +
                        notes.push(format!(
283 +
                            "{hub} is now cosigned by witness {name} (key {}), trusted from now on",
284 +
                            id.hash().short(12)
285 +
                        ));
286 +
                    }
287 +
                    pinned.push((name.clone(), id));
288 +
                    changed = true;
289 +
                }
290 +
            }
291 +
        }
292 +
        if changed {
293 +
            if let Some(dir) = path.parent() {
294 +
                fs::create_dir_all(dir)?;
295 +
            }
296 +
            let text: String = pinned
297 +
                .iter()
298 +
                .map(|(n, id)| format!("{n} {}\n", crate::dsse::base64(&id.encode())))
299 +
                .collect();
300 +
            crate::durable::write_atomic(&path, text.as_bytes())?;
301 +
        }
302 +
        Ok(pinned)
303 +
    }
304 +
305 +
    /// Check what a refs answer from `url` says about its host's log, and
306 +
    /// read each bookmark attestation's integrated time. Fails only when
307 +
    /// the host showed two histories under one key; everything else is a
308 +
    /// note.
309 +
    pub(crate) fn check_log(
310 +
        &self,
311 +
        url: &str,
312 +
        ca: Option<&Path>,
313 +
        resp: &RefsResponse,
314 +
    ) -> Result<LogCheck> {
315 +
        let mut out = LogCheck::default();
316 +
        let (Some(text), Some(hub)) = (&resp.log, hub_of(url)) else {
317 +
            return Ok(out);
318 +
        };
319 +
        out.kept = true;
320 +
        let options = RemoteOptions {
321 +
            ca: ca.map(Path::to_path_buf),
322 +
            ..RemoteOptions::default()
323 +
        };
324 +
        let accepted = match self.accept_checkpoint(&hub, text, &options)? {
325 +
            Ok(c) => c,
326 +
            Err(why) => {
327 +
                out.notes
328 +
                    .push(format!("{hub}'s transparency log was not checked: {why}"));
329 +
                out.unchecked = Some(why);
330 +
                return Ok(out);
331 +
            }
332 +
        };
333 +
        let cp = &accepted.checkpoint;
334 +
        out.notes.extend(accepted.notes.iter().cloned());
335 +
        let witnesses = self.pin_witnesses(&hub, &resp.witnesses, &mut out.notes)?;
336 +
        let mut missing = 0;
337 +
        // Each attestation's place in the log, and its integrated time.
338 +
        let mut place = |name: &str,
339 +
                         kind: &str,
340 +
                         a: &SignedAttest,
341 +
                         logged: Option<&crate::protocol::Logged>,
342 +
                         integration: &[crate::protocol::Integration]|
343 +
         -> std::result::Result<Integrated, String> {
344 +
            let Some(l) = logged else {
345 +
                missing += 1;
346 +
                return Err("it is not in the log yet".into());
347 +
            };
348 +
            let fits = Entry::decode(&l.entry).is_ok_and(|e| {
349 +
                e.kind == kind && e.object == a.attest.hash() && e.repo == a.attest.repo
350 +
            });
351 +
            let included = fits
352 +
                && tlog::check_inclusion(
353 +
                    &l.proof,
354 +
                    cp.size,
355 +
                    &cp.root,
356 +
                    l.index,
357 +
                    &tlog::leaf_hash(&l.entry),
358 +
                )
359 +
                .is_ok();
360 +
            if !included {
361 +
                out.notes.push(format!(
362 +
                    "{hub}'s log entry {} for {name} does not prove its attestation is in the log",
363 +
                    l.index
364 +
                ));
365 +
            }
366 +
            if !fits {
367 +
                Err("its log entry is not its attestation's".to_string())
368 +
            } else if witnesses.is_empty() {
369 +
                Err(format!("{hub} names no witness"))
370 +
            } else {
371 +
                let cosigned: Vec<(&str, &[Hash])> = integration
372 +
                    .iter()
373 +
                    .map(|i| (i.note.as_str(), i.proof.as_slice()))
374 +
                    .collect();
375 +
                tlog::integrated_time(
376 +
                    &l.entry,
377 +
                    l.index,
378 +
                    &cosigned,
379 +
                    &cp.origin,
380 +
                    &accepted.identity,
381 +
                    &witnesses,
382 +
                )
383 +
            }
384 +
        };
385 +
        let mut times = HashMap::new();
386 +
        for r in &resp.refs {
387 +
            let Some(a) = &r.attestation else { continue };
388 +
            let time = place(&r.name, "ref", a, r.logged.as_ref(), &r.integration);
389 +
            times.insert(r.name.clone(), time);
390 +
        }
391 +
        // Tags too (oxygen#129): a short-lived key's tag is checked at its
392 +
        // integrated time on every later fetch.
393 +
        let mut tag_times = HashMap::new();
394 +
        for t in &resp.tags {
395 +
            let Some(a) = &t.attestation else { continue };
396 +
            let name = format!("tags/{}", t.name);
397 +
            let time = place(&name, "tag", a, t.logged.as_ref(), &t.integration);
398 +
            tag_times.insert(t.name.clone(), time);
399 +
        }
400 +
        out.times = times;
401 +
        out.tag_times = tag_times;
402 +
        if missing > 0 {
403 +
            out.notes.push(format!(
404 +
                "{} of {hub}'s attestations {} not in its transparency log yet (logged within seconds of a push; checked on a later fetch)",
405 +
                missing,
406 +
                if missing == 1 { "is" } else { "are" }
407 +
            ));
408 +
        }
409 +
        Ok(out)
410 +
    }
411 +
}
412 +
413 +
/// A checkpoint `accept_checkpoint` took: the checkpoint, the log key it
414 +
/// verified with, its note as served, and what accepting it noted.
415 +
pub(crate) struct Accepted {
416 +
    pub checkpoint: Checkpoint,
417 +
    pub identity: Identity,
418 +
    pub note: Note,
419 +
    pub notes: Vec<String>,
420 +
}
421 +
422 +
/// What a refs answer said about its host's log.
423 +
#[derive(Debug, Default)]
424 +
pub(crate) struct LogCheck {
425 +
    /// Whether the host keeps a log (it sent a checkpoint).
426 +
    pub kept: bool,
427 +
    /// Why the log could not be checked, when it could not.
428 +
    pub unchecked: Option<String>,
429 +
    pub notes: Vec<String>,
430 +
    /// Each attested bookmark's integrated time, or why it has none.
431 +
    pub times: HashMap<String, std::result::Result<Integrated, String>>,
432 +
    /// Each attested tag's, by tag name.
433 +
    pub tag_times: HashMap<String, std::result::Result<Integrated, String>>,
434 +
}
435 +
436 +
impl LogCheck {
437 +
    /// Bookmark `name`'s integrated time, or why there is none.
438 +
    pub fn time_of(&self, name: &str) -> std::result::Result<&Integrated, String> {
439 +
        self.lookup(&self.times, name)
440 +
    }
441 +
442 +
    /// Tag `name`'s integrated time, or why there is none.
443 +
    pub fn tag_time_of(&self, name: &str) -> std::result::Result<&Integrated, String> {
444 +
        self.lookup(&self.tag_times, name)
445 +
    }
446 +
447 +
    fn lookup<'a>(
448 +
        &self,
449 +
        times: &'a HashMap<String, std::result::Result<Integrated, String>>,
450 +
        name: &str,
451 +
    ) -> std::result::Result<&'a Integrated, String> {
452 +
        if let Some(why) = &self.unchecked {
453 +
            return Err(format!("its host's log was not checked ({why})"));
454 +
        }
455 +
        if !self.kept {
456 +
            return Err("its host keeps no transparency log".into());
457 +
        }
458 +
        match times.get(name) {
459 +
            Some(Ok(i)) => Ok(i),
460 +
            Some(Err(why)) => Err(why.clone()),
461 +
            None => Err("it has no log entry".into()),
462 +
        }
463 +
    }
464 +
}
465 +
466 +
/// How a check that depends on when an attestation was signed came out
467 +
/// (spec §14.8, *Integrated time*).
468 +
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
469 +
pub(crate) enum Timed {
470 +
    /// Passed at the integrated time.
471 +
    Integrated,
472 +
    /// No integrated time: passed by the earlier rule (the issuer's
473 +
    /// acceptance, else the attestation's own time).
474 +
    Fallback,
475 +
    /// Failed at the integrated time but passed by the earlier rule:
476 +
    /// accepted with a note while logs are new.
477 +
    Disagrees,
478 +
    /// Failed.
479 +
    Refused,
480 +
}
481 +
482 +
/// Judge a check (`at`, made at a time in milliseconds) at the integrated
483 +
/// time when there is one, else by the earlier rule (`before`).
484 +
pub(crate) fn judge(
485 +
    integrated: Option<u64>,
486 +
    at: impl Fn(u64) -> bool,
487 +
    before: impl FnOnce() -> bool,
488 +
) -> Timed {
489 +
    match integrated {
490 +
        Some(t) if at(t) => Timed::Integrated,
491 +
        Some(_) => {
492 +
            if before() {
493 +
                Timed::Disagrees
494 +
            } else {
495 +
                Timed::Refused
496 +
            }
497 +
        }
498 +
        None => {
499 +
            if before() {
500 +
                Timed::Fallback
501 +
            } else {
502 +
                Timed::Refused
503 +
            }
504 +
        }
505 +
    }
506 +
}
507 +
508 +
/// A note when an attestation's claimed time (`claimed`, its own clock)
509 +
/// is more than `tlog::TIME_TOLERANCE_MS` from its integrated time.
510 +
pub(crate) fn gap_note(claimed: u64, integrated: &Integrated) -> Option<String> {
511 +
    let t = integrated.ms();
512 +
    let minutes = |d: u64| d.div_ceil(60_000);
513 +
    if claimed.saturating_add(tlog::TIME_TOLERANCE_MS) < t {
514 +
        Some(format!(
515 +
            "claims to be from {} minutes before witness {} first saw it: backdated, held back before it was logged, or signed on a slow clock",
516 +
            minutes(t - claimed),
517 +
            integrated.witness
518 +
        ))
519 +
    } else if claimed > t.saturating_add(tlog::TIME_TOLERANCE_MS) {
520 +
        Some(format!(
521 +
            "claims to be from {} minutes after witness {} saw it: signed on a fast clock",
522 +
            minutes(claimed - t),
523 +
            integrated.witness
524 +
        ))
525 +
    } else {
526 +
        None
527 +
    }
528 +
}
529 +
530 +
/// A hub's whole log, checked (`oxy tlog verify`).
531 +
#[derive(Clone, Debug)]
532 +
pub struct LogAudit {
533 +
    pub hub: String,
534 +
    pub checkpoint: Checkpoint,
535 +
    /// Every entry, numbered from 0.
536 +
    pub entries: Vec<Entry>,
537 +
    /// What accepting the checkpoint noted (a new log key, a new name).
538 +
    pub notes: Vec<String>,
539 +
    /// The trusted witnesses that cosigned the checkpoint, with their
540 +
    /// times in seconds, earliest first.
541 +
    pub witnessed: Vec<(String, u64)>,
542 +
}
543 +
544 +
impl Repo {
545 +
    /// Fetch the log of `url`'s hub whole: its checkpoint, accepted as a
546 +
    /// fetch accepts one, and every entry bundle, whose entries must make
547 +
    /// the root the checkpoint signs.
548 +
    pub fn audit_log(&self, url: &str, ca: Option<&Path>) -> Result<LogAudit> {
549 +
        let hub =
550 +
            hub_of(url).ok_or_else(|| RepoError::Msg(format!("{url} is not a hub's address")))?;
551 +
        let options = RemoteOptions {
552 +
            ca: ca.map(Path::to_path_buf),
553 +
            ..RemoteOptions::default()
554 +
        };
555 +
        let tl = |e: TlogError| RepoError::Msg(e.to_string());
556 +
        let text = String::from_utf8(http_get(&format!("{hub}/log/checkpoint"), &options)?)
557 +
            .map_err(|_| RepoError::Msg(format!("{hub}'s checkpoint is not text")))?;
558 +
        let accepted = match self.accept_checkpoint(&hub, &text, &options)? {
559 +
            Ok(c) => c,
560 +
            Err(why) => return fail(format!("{hub}'s log could not be checked: {why}")),
561 +
        };
562 +
        let mut notes = accepted.notes;
563 +
        // The witnesses it names (one `<name> <base64 identity>` line
564 +
        // each), pinned as a fetch pins them; a hub that does not list
565 +
        // them leaves those pinned before.
566 +
        let listed: Vec<(String, Vec<u8>)> = http_get(&format!("{hub}/log/witnesses"), &options)
567 +
            .ok()
568 +
            .and_then(|b| String::from_utf8(b).ok())
569 +
            .map(|t| {
570 +
                t.lines()
571 +
                    .filter_map(|l| {
572 +
                        let (n, k) = l.split_once(' ')?;
573 +
                        Some((n.to_string(), crate::dsse::base64_decode(k.trim())?))
574 +
                    })
575 +
                    .collect()
576 +
            })
577 +
            .unwrap_or_default();
578 +
        let witnesses = self.pin_witnesses(&hub, &listed, &mut notes)?;
579 +
        let mut witnessed = accepted.note.cosignatures(&witnesses);
580 +
        witnessed.sort_by_key(|(_, t)| *t);
581 +
        let cp = accepted.checkpoint;
582 +
        let mut entries = Vec::with_capacity(cp.size as usize);
583 +
        let mut stored: Vec<Hash> = Vec::new();
584 +
        let mut n = 0u64;
585 +
        while n < cp.size {
586 +
            let t = Tile {
587 +
                level: 0,
588 +
                n: n >> tlog::TILE_HEIGHT,
589 +
                width: (cp.size - n).min(tlog::TILE_WIDTH) as u32,
590 +
            };
591 +
            let bytes = http_get(&format!("{hub}/log/{}", t.entries_path()), &options)?;
592 +
            let bundle = tlog::decode_bundle(&bytes).map_err(tl)?;
593 +
            if bundle.len() != t.width as usize {
594 +
                return fail(format!(
595 +
                    "{hub}'s entry bundle {} holds {} entries, not {}",
596 +
                    t.entries_path(),
597 +
                    bundle.len(),
598 +
                    t.width
599 +
                ));
600 +
            }
601 +
            for b in bundle {
602 +
                let add =
603 +
                    tlog::stored_hashes_for_leaf(n, tlog::leaf_hash(&b), &stored).map_err(tl)?;
604 +
                stored.extend(add);
605 +
                entries.push(Entry::decode(&b).map_err(tl)?);
606 +
                n += 1;
607 +
            }
608 +
        }
609 +
        if tlog::tree_hash(cp.size, &stored).map_err(tl)? != cp.root {
610 +
            return fail(format!(
611 +
                "{hub} serves entries that do not make the root it signed for {} entries: its log is not the one it attests",
612 +
                cp.size
613 +
            ));
614 +
        }
615 +
        Ok(LogAudit {
616 +
            hub,
617 +
            checkpoint: cp,
618 +
            entries,
619 +
            notes,
620 +
            witnessed,
621 +
        })
622 +
    }
623 +
}
modified crates/oxy-core/src/repo/walk.rs +3 −1
…
184 unchanged lines
185 185
    let mut mine = Listing::default();
186 186
    mine.dirs.push((rel.clone(), mtime));
187 187
    for (e, name) in entries.into_iter().zip(names) {
188 -
        if name == OXY_DIR {
188 +
        // `.git` is never content: a side-by-side Git repository
189 +
        // (`git.colocate`), or Git's own beside an import.
190 +
        if name == OXY_DIR || name.eq_ignore_ascii_case(".git") {
189 191
            continue;
190 192
        }
191 193
        let Ok(ft) = e.file_type() else { continue };
…
136 unchanged lines
modified crates/oxy-core/src/repo/workspace.rs +9 −2
…
200 unchanged lines
201 201
            return fail(format!("{} exists and is not empty", dir.display()));
202 202
        }
203 203
        fs::create_dir_all(dir)?;
204 -
        let dir = dir.canonicalize()?;
204 +
        let dir = crate::portable::canonical(dir)?;
205 205
        if dir == self.root || dir.starts_with(&self.root) || self.root.starts_with(&dir) {
206 206
            return fail("a workspace cannot be inside another workspace of the same repository");
207 207
        }
…
2 unchanged lines
210 210
            None => self.bookmark_heads()?,
211 211
        };
212 212
        let id = WorkspaceId::random();
213 -
        let wc = self.new_working_copy(heads.clone(), "");
213 +
        // On nothing, as the directory is, until the checkout: on `heads`,
214 +
        // the checkout's first snapshot found every file of the tree
215 +
        // deleted, rendering the whole tree to say so (195 s of a 467 s
216 +
        // `oxy workspace add` in the VS Code history, oxygen#137).
217 +
        let wc = self.new_working_copy(Vec::new(), "");
214 218
        let wc_hash = self.store.put_change(&wc)?;
215 219
        let (op_ref, op) = self.head_op()?;
216 220
        let mut view = op.view.clone();
…
129 unchanged lines
346 350
            std::time::Duration::from_secs(60),
347 351
        )?;
348 352
        HELD.with(|h| h.borrow_mut().insert(path.clone(), 1));
353 +
        // Whoever held the lock may have moved the operation head: what
354 +
        // is done under it starts from the head as it is now (oxygenhub#249).
355 +
        self.forget_head();
349 356
        Ok(WorkingCopyLock {
350 357
            path,
351 358
            _file: Some(file),
…
3 unchanged lines
modified crates/oxy-core/src/state.rs +801 −182
…
10 unchanged lines
11 11
};
12 12
use crate::tree::{TreeGraph, TreeView};
13 13
use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet};
14 -
use std::sync::{Arc, Mutex};
14 +
use std::sync::Arc;
15 15
16 16
/// Checkpoint payload version 2 (spec §11): file graphs by reference.
17 17
pub const CHECKPOINT_INDEX_VERSION: u64 = 2;
…
22 unchanged lines
40 40
    pub kind: ChangeKind,
41 41
}
42 42
43 +
/// The log's order of changes: newest first by clock, then by hash, so
44 +
/// it is total.
45 +
pub fn log_order(a: (&ChangeHash, &Hlc), b: (&ChangeHash, &Hlc)) -> std::cmp::Ordering {
46 +
    b.1.cmp(a.1).then(b.0.cmp(a.0))
47 +
}
48 +
43 49
impl ChangeMeta {
44 50
    /// In the flat layout of local indexes and caches ([`crate::flat`]).
45 51
    pub(crate) fn write_flat(&self, w: &mut crate::flat::Writer) {
…
44 unchanged lines
90 96
pub struct State {
91 97
    /// Shared between clones, like `files` and `deferred`: a state kept
92 98
    /// beside another that differs by a few changes shares the rest.
93 -
    changes: HashMap<ChangeHash, Arc<ChangeMeta>>,
99 +
    changes: crate::fasthash::FastMap<ChangeHash, Arc<ChangeMeta>>,
94 100
    tree: TreeGraph,
95 101
    /// Shared between clones: a clone copies a graph only when an edit
96 102
    /// changes it (`Arc::make_mut`), so keeping several states that differ
97 103
    /// by a few changes costs little more than one.
98 104
    files: crate::fasthash::FastMap<NodeId, Arc<FileGraph>>,
99 105
    replaces: HashMap<NodeId, Vec<ReplaceRecord>>,
100 -
    /// Memoised transitive dependencies. A mutex, not a `RefCell`, so a
101 -
    /// loaded state can be shared across threads by a host that caches it.
102 -
    closure: Mutex<HashMap<ChangeHash, Arc<HashSet<ChangeHash>>>>,
106 +
    /// Memoised transitive dependencies, bounded by the hashes they hold
107 +
    /// (each set is the whole history below a change, so keeping one for
108 +
    /// every change asked about grew with the square of history: #25). A
109 +
    /// mutex, not a `RefCell`, so a loaded state can be shared across
110 +
    /// threads by a host that caches it.
111 +
    closure: std::sync::RwLock<ClosureMemo>,
112 +
    /// Each change's level: 0 with no dependencies, else one more than its
113 +
    /// highest dependency. A change can only depend on changes of lower
114 +
    /// level, which bounds the walk [`DepOracle::depends_on`] does.
115 +
    levels: std::sync::RwLock<crate::fasthash::FastMap<ChangeHash, u32>>,
103 116
    /// `None` for a full state. For a history-only state (loaded for its
104 117
    /// changes and tree), the file graphs not built yet, by node: `files`
105 118
    /// holds only those loaded with [`State::load_graphs`].
106 119
    deferred: Option<crate::fasthash::FastMap<NodeId, Arc<Deferred>>>,
120 +
    /// The tree as readers see it ([`State::view`]), once worked out for
121 +
    /// a tree with name conflicts; cleared by every change added.
122 +
    unified_view: std::sync::OnceLock<TreeView>,
107 123
}
108 124
109 125
use crate::objects::TextRef;
…
3 unchanged lines
113 129
/// order they were applied (spec §11). Building it later gives the same
114 130
/// graph, since graph construction does not depend on order (§8).
115 131
///
116 -
/// Read from a state index, it stays encoded until first used: a command
117 -
/// that builds a few files decodes only their histories.
132 +
/// Read from a state index, it stays in the index, unread, until first
133 +
/// used: a command that builds a few files reads and checks only their
134 +
/// histories (oxygen#94).
118 135
#[derive(Clone, Default)]
119 136
struct Deferred {
120 -
    body: std::sync::OnceLock<DeferredBody>,
121 -
    /// The body as the index encodes it, decoded into `body` on first use.
122 -
    raw: Option<(Arc<[u8]>, std::ops::Range<usize>)>,
137 +
    /// The history: `raw` read and decoded, with `more` after it. An
138 +
    /// error when `raw` does not match its hash.
139 +
    body: std::sync::OnceLock<Result<DeferredBody, String>>,
140 +
    /// The history as an index holds it, not read yet.
141 +
    raw: Option<RawHistory>,
142 +
    /// Changes added since the index was read, kept beside `raw`, so that
143 +
    /// adding one neither reads nor decodes the history.
144 +
    more: Vec<(ChangeHash, Vec<Edit>)>,
145 +
    more_texts: Vec<TextRef>,
123 146
    /// Its graph key, once worked out (or read from an index); reset by
124 147
    /// any edit added.
125 148
    key: std::sync::OnceLock<[u8; 32]>,
126 149
}
127 150
151 +
/// Where a state index's file histories are: the index file itself, read
152 +
/// on first use, or bytes in memory.
153 +
#[derive(Clone)]
154 +
pub enum Histories {
155 +
    Bytes(Arc<Vec<u8>>),
156 +
    /// The file, where its histories start, and how many bytes they take.
157 +
    File(Arc<std::fs::File>, u64, u64),
158 +
}
159 +
160 +
impl Histories {
161 +
    fn len(&self) -> u64 {
162 +
        match self {
163 +
            Histories::Bytes(b) => b.len() as u64,
164 +
            Histories::File(_, _, len) => *len,
165 +
        }
166 +
    }
167 +
168 +
    fn read(&self, at: u64, len: usize) -> std::io::Result<Vec<u8>> {
169 +
        match self {
170 +
            Histories::Bytes(b) => Ok(b[at as usize..at as usize + len].to_vec()),
171 +
            Histories::File(f, start, _) => {
172 +
                let mut buf = vec![0u8; len];
173 +
                read_exact_at(f, &mut buf, start + at)?;
174 +
                Ok(buf)
175 +
            }
176 +
        }
177 +
    }
178 +
}
179 +
180 +
#[cfg(unix)]
181 +
fn read_exact_at(f: &std::fs::File, buf: &mut [u8], at: u64) -> std::io::Result<()> {
182 +
    std::os::unix::fs::FileExt::read_exact_at(f, buf, at)
183 +
}
184 +
185 +
#[cfg(windows)]
186 +
fn read_exact_at(f: &std::fs::File, mut buf: &mut [u8], mut at: u64) -> std::io::Result<()> {
187 +
    while !buf.is_empty() {
188 +
        match std::os::windows::fs::FileExt::seek_read(f, buf, at)? {
189 +
            0 => return Err(std::io::ErrorKind::UnexpectedEof.into()),
190 +
            n => {
191 +
                buf = &mut buf[n..];
192 +
                at += n as u64;
193 +
            }
194 +
        }
195 +
    }
196 +
    Ok(())
197 +
}
198 +
199 +
/// One file's history in a state index: where it is and its hash.
200 +
#[derive(Clone)]
201 +
struct RawHistory {
202 +
    from: Histories,
203 +
    at: u64,
204 +
    len: usize,
205 +
    sum: [u8; 32],
206 +
}
207 +
208 +
impl RawHistory {
209 +
    /// The history's bytes, checked against the hash the index holds.
210 +
    fn bytes(&self) -> Result<Vec<u8>, ObjectError> {
211 +
        let damaged = |why: String| ObjectError::DamagedIndex(why);
212 +
        let b = self
213 +
            .from
214 +
            .read(self.at, self.len)
215 +
            .map_err(|e| damaged(format!("a file's history cannot be read: {e}")))?;
216 +
        if *blake3::hash(&b).as_bytes() != self.sum {
217 +
            return Err(damaged("a file's history does not match its hash".into()));
218 +
        }
219 +
        Ok(b)
220 +
    }
221 +
}
222 +
223 +
/// The closures kept, and how many hashes they hold in all.
224 +
#[derive(Default)]
225 +
struct ClosureMemo {
226 +
    sets: HashMap<ChangeHash, Arc<HashSet<ChangeHash>>>,
227 +
    held: usize,
228 +
}
229 +
230 +
/// The most hashes the closure memo holds before it starts again: a few
231 +
/// hundred megabytes at most.
232 +
const CLOSURE_MEMO_HASHES: usize = 4_000_000;
233 +
128 234
#[derive(Clone, Default)]
129 235
struct DeferredBody {
130 236
    base: Option<DeferredBase>,
…
6 unchanged lines
137 243
impl Deferred {
138 244
    fn from_body(body: DeferredBody) -> Deferred {
139 245
        Deferred {
140 -
            body: std::sync::OnceLock::from(body),
246 +
            body: std::sync::OnceLock::from(Ok(body)),
141 247
            ..Deferred::default()
142 248
        }
143 249
    }
144 250
145 -
    fn body(&self) -> &DeferredBody {
146 -
        self.body.get_or_init(|| match &self.raw {
147 -
            // The index was checked against its hash when read, and was
148 -
            // written by `write_index`: it decodes.
149 -
            Some((bytes, range)) => State::read_deferred_body(&bytes[range.clone()])
150 -
                .expect("a state index checked by its hash decodes"),
151 -
            None => DeferredBody::default(),
152 -
        })
251 +
    /// The history, read and checked on first use.
252 +
    fn body(&self) -> Result<&DeferredBody, ObjectError> {
253 +
        self.body
254 +
            .get_or_init(|| {
255 +
                let mut b = match &self.raw {
256 +
                    // Matching its hash, it is what `write_index` wrote: it
257 +
                    // decodes.
258 +
                    Some(raw) => raw
259 +
                        .bytes()
260 +
                        .and_then(|b| State::read_deferred_body(&b))
261 +
                        .map_err(|e| e.to_string())?,
262 +
                    None => DeferredBody::default(),
263 +
                };
264 +
                b.edits.extend(self.more.iter().cloned());
265 +
                b.texts.extend(self.more_texts.iter().cloned());
266 +
                Ok(b)
267 +
            })
268 +
            .as_ref()
269 +
            .map_err(|e| ObjectError::DamagedIndex(e.clone()))
270 +
    }
271 +
272 +
    /// Add a change's edits to the history: beside the index's encoding
273 +
    /// while it is unread, so that nothing is read or decoded to add them.
274 +
    fn add(&mut self, h: ChangeHash, edits: Vec<Edit>, texts: Vec<TextRef>) {
275 +
        if self.raw.is_some() && !matches!(self.body.get(), Some(Ok(_))) {
276 +
            self.more.push((h, edits));
277 +
            self.more_texts.extend(texts);
278 +
            // A body found damaged stays so: what was added cannot mend it.
279 +
            return;
280 +
        }
281 +
        // No encoding to read, or decoded already: this cannot fail.
282 +
        let b = self.body_mut().expect("a history in memory");
283 +
        b.edits.push((h, edits));
284 +
        b.texts.extend(texts);
153 285
    }
154 286
155 -
    fn body_mut(&mut self) -> &mut DeferredBody {
156 -
        self.body();
287 +
    fn body_mut(&mut self) -> Result<&mut DeferredBody, ObjectError> {
288 +
        self.body()?;
157 289
        self.raw = None;
158 -
        self.body.get_mut().expect("decoded above")
290 +
        self.more.clear();
291 +
        self.more_texts.clear();
292 +
        Ok(self
293 +
            .body
294 +
            .get_mut()
295 +
            .and_then(|b| b.as_mut().ok())
296 +
            .expect("decoded above"))
159 297
    }
160 298
161 -
    fn into_body(mut self) -> DeferredBody {
162 -
        self.body();
163 -
        self.body.take().expect("decoded above")
299 +
    fn into_body(mut self) -> Result<DeferredBody, ObjectError> {
300 +
        self.body()?;
301 +
        Ok(self
302 +
            .body
303 +
            .take()
304 +
            .and_then(|b| b.ok())
305 +
            .expect("decoded above"))
164 306
    }
165 307
}
166 308
…
9 unchanged lines
176 318
/// command reads it rather than replaying history. Derived only: written
177 319
/// from a state loaded the long way, and read back as the same state.
178 320
impl State {
179 -
    const INDEX_MAGIC: &'static [u8] = b"oxy state index 3\n";
321 +
    const INDEX_MAGIC: &'static [u8] = b"oxy state index 4\n";
180 322
181 323
    /// The index of this state, or `None` when it has anything an index
182 324
    /// does not hold: a built graph, or text in memory (a change added
…
4 unchanged lines
187 329
        if !self.files.is_empty() {
188 330
            return None;
189 331
        }
190 -
        let atom = |w: &mut Writer, a: &AtomRef| match a {
191 -
            AtomRef::Start => w.u8(0),
192 -
            AtomRef::End => w.u8(1),
193 -
            AtomRef::Atom { change, index } => {
194 -
                w.u8(2);
195 -
                w.object(change);
196 -
                w.u32(*index);
197 -
            }
198 -
        };
199 332
        // The changes first, as a block a reader can take on its own (see
200 333
        // [`State::read_index_changes`]).
201 334
        let mut w = Writer::default();
202 335
        w.fixed(Self::INDEX_MAGIC);
336 +
        // In log order, newest first (oxygen#94): the log takes them as
337 +
        // they come, with no map and no sort.
203 338
        let mut changes: Vec<(&ChangeHash, &Arc<ChangeMeta>)> = self.changes.iter().collect();
204 -
        changes.sort_by_key(|(h, _)| **h);
339 +
        changes.sort_unstable_by(|a, b| log_order((a.0, &a.1.hlc), (b.0, &b.1.hlc)));
205 340
        w.count(changes.len());
206 341
        for (h, m) in changes {
207 342
            w.object(h);
…
21 unchanged lines
229 364
                w.opt_object(new);
230 365
            }
231 366
        }
232 -
        let outer = w;
233 -
        let mut w = Writer::default();
367 +
        let mut front = w;
368 +
        // The table: per file its key and where its history is, with the
369 +
        // history's hash, which is checked when it is read (oxygen#94).
370 +
        let mut table = Writer::default();
371 +
        let mut histories: Vec<u8> = Vec::new();
234 372
        let mut nodes: Vec<(&NodeId, &Arc<Deferred>)> = deferred.iter().collect();
235 373
        nodes.sort_by_key(|(n, _)| **n);
236 -
        w.count(nodes.len());
374 +
        table.count(nodes.len());
237 375
        for (n, d) in nodes {
238 -
            w.fixed(&n.0);
239 -
            let mut h = blake3::Hasher::new();
240 -
            h.update(b"oxy graph key 1\0");
241 -
            h.update(&n.0);
242 -
            w.fixed(d.key.get_or_init(|| d.graph_key(h)));
243 -
            // Unchanged since read (a change clears `raw`): copied as it is.
244 -
            if let Some((bytes, range)) = &d.raw {
245 -
                w.bytes(&bytes[range.clone()]);
246 -
                continue;
247 -
            }
248 -
            let d = d.body();
249 -
            let outer_w = std::mem::take(&mut w);
250 -
            match &d.base {
251 -
                None => w.u8(0),
252 -
                Some(DeferredBase::Blob(r)) => {
253 -
                    w.u8(1);
254 -
                    w.object(r);
376 +
            let key = match d.key.get() {
377 +
                Some(k) => *k,
378 +
                None => {
379 +
                    let mut h = blake3::Hasher::new();
380 +
                    h.update(b"oxy graph key 1\0");
381 +
                    h.update(&n.0);
382 +
                    let k = d.graph_key(h).ok()?;
383 +
                    let _ = d.key.set(k);
384 +
                    k
255 385
                }
256 -
                Some(DeferredBase::Inline(v)) => {
257 -
                    w.u8(2);
258 -
                    w.bytes(&cbor::encode(v));
259 -
                }
386 +
            };
387 +
            // Unchanged since read: its bytes as they are, checked.
388 +
            let bytes = match &d.raw {
389 +
                Some(raw) if d.more.is_empty() => raw.bytes().ok()?,
390 +
                _ => Self::write_deferred_body(d.body().ok()?)?,
391 +
            };
392 +
            table.fixed(&n.0);
393 +
            table.fixed(&key);
394 +
            table.u64(histories.len() as u64);
395 +
            table.u32(u32::try_from(bytes.len()).ok()?);
396 +
            table.fixed(blake3::hash(&bytes).as_bytes());
397 +
            histories.extend_from_slice(&bytes);
398 +
        }
399 +
        front.bytes(&table.0);
400 +
        let mut out = Writer::default();
401 +
        out.u64(front.0.len() as u64);
402 +
        out.fixed(&front.0);
403 +
        out.fixed(&histories);
404 +
        Some(out.0)
405 +
    }
406 +
407 +
    /// One file's history as an index encodes it; `None` when it holds
408 +
    /// text in memory, which an index does not.
409 +
    fn write_deferred_body(d: &DeferredBody) -> Option<Vec<u8>> {
410 +
        use crate::flat::Writer;
411 +
        let atom = |w: &mut Writer, a: &AtomRef| match a {
412 +
            AtomRef::Start => w.u8(0),
413 +
            AtomRef::End => w.u8(1),
414 +
            AtomRef::Atom { change, index } => {
415 +
                w.u8(2);
416 +
                w.object(change);
417 +
                w.u32(*index);
260 418
            }
261 -
            // Each change's edits with the texts of its inserts, sorted by
262 -
            // change: a graph does not depend on the order changes applied
263 -
            // in, and the index is then the same however history was read.
264 -
            let mut texts = d.texts.iter();
265 -
            let mut per_change: Vec<(&ChangeHash, &Vec<Edit>, Vec<&TextRef>)> = Vec::new();
266 -
            for (c, edits) in &d.edits {
267 -
                let n = edits
268 -
                    .iter()
269 -
                    .filter(|e| matches!(e, Edit::Insert { lines, .. } if lines.is_empty()))
270 -
                    .count();
271 -
                per_change.push((c, edits, texts.by_ref().take(n).collect()));
419 +
        };
420 +
        let mut w = Writer::default();
421 +
        match &d.base {
422 +
            None => w.u8(0),
423 +
            Some(DeferredBase::Blob(r)) => {
424 +
                w.u8(1);
425 +
                w.object(r);
272 426
            }
273 -
            per_change.sort_by_key(|(c, _, _)| **c);
274 -
            w.count(per_change.len());
275 -
            for (c, edits, _) in &per_change {
276 -
                w.object(c);
277 -
                w.count(edits.len());
278 -
                for e in edits.iter() {
279 -
                    match e {
280 -
                        Edit::Insert {
281 -
                            node,
282 -
                            after,
283 -
                            before,
284 -
                            lines,
285 -
                        } => {
286 -
                            if !lines.is_empty() {
287 -
                                return None;
288 -
                            }
289 -
                            w.u8(0);
290 -
                            w.fixed(&node.0);
291 -
                            atom(&mut w, after);
292 -
                            atom(&mut w, before);
293 -
                        }
294 -
                        Edit::Delete { node, atoms } => {
295 -
                            w.u8(1);
296 -
                            w.fixed(&node.0);
297 -
                            w.count(atoms.len());
298 -
                            for r in atoms {
299 -
                                w.object(&r.change);
300 -
                                w.u32(r.from);
301 -
                                w.u32(r.to);
302 -
                            }
303 -
                        }
304 -
                        Edit::Replace { .. } => return None,
305 -
                    }
306 -
                }
427 +
            Some(DeferredBase::Inline(v)) => {
428 +
                w.u8(2);
429 +
                w.bytes(&cbor::encode(v));
307 430
            }
308 -
            let texts: Vec<&TextRef> = per_change
431 +
        }
432 +
        // Each change's edits with the texts of its inserts, sorted by
433 +
        // change: a graph does not depend on the order changes applied
434 +
        // in, and the index is then the same however history was read.
435 +
        let mut texts = d.texts.iter();
436 +
        let mut per_change: Vec<(&ChangeHash, &Vec<Edit>, Vec<&TextRef>)> = Vec::new();
437 +
        for (c, edits) in &d.edits {
438 +
            let n = edits
309 439
                .iter()
310 -
                .flat_map(|(_, _, t)| t.iter().copied())
311 -
                .collect();
312 -
            w.count(texts.len());
313 -
            for t in texts {
314 -
                match t {
315 -
                    TextRef::Blob(r, count, form) => {
440 +
                .filter(|e| matches!(e, Edit::Insert { lines, .. } if lines.is_empty()))
441 +
                .count();
442 +
            per_change.push((c, edits, texts.by_ref().take(n).collect()));
443 +
        }
444 +
        per_change.sort_by_key(|(c, _, _)| **c);
445 +
        w.count(per_change.len());
446 +
        for (c, edits, _) in &per_change {
447 +
            w.object(c);
448 +
            w.count(edits.len());
449 +
            for e in edits.iter() {
450 +
                match e {
451 +
                    Edit::Insert {
452 +
                        node,
453 +
                        after,
454 +
                        before,
455 +
                        lines,
456 +
                    } => {
457 +
                        if !lines.is_empty() {
458 +
                            return None;
459 +
                        }
316 460
                        w.u8(0);
317 -
                        w.object(r);
318 -
                        w.u32(*count);
319 -
                        w.u8(match form {
320 -
                            crate::objects::TextForm::Cbor => 0,
321 -
                            crate::objects::TextForm::Raw => 1,
322 -
                            crate::objects::TextForm::CborFallback => 2,
323 -
                        });
461 +
                        w.fixed(&node.0);
462 +
                        atom(&mut w, after);
463 +
                        atom(&mut w, before);
324 464
                    }
325 -
                    TextRef::Inline { change, start, end } => {
465 +
                    Edit::Delete { node, atoms } => {
326 466
                        w.u8(1);
327 -
                        w.object(change);
328 -
                        w.u32(*start);
329 -
                        w.u32(*end);
467 +
                        w.fixed(&node.0);
468 +
                        w.count(atoms.len());
469 +
                        for r in atoms {
470 +
                            w.object(&r.change);
471 +
                            w.u32(r.from);
472 +
                            w.u32(r.to);
473 +
                        }
330 474
                    }
475 +
                    Edit::Replace { .. } => return None,
476 +
                }
477 +
            }
478 +
        }
479 +
        let texts: Vec<&TextRef> = per_change
480 +
            .iter()
481 +
            .flat_map(|(_, _, t)| t.iter().copied())
482 +
            .collect();
483 +
        w.count(texts.len());
484 +
        for t in texts {
485 +
            match t {
486 +
                TextRef::Blob(r, count, form) => {
487 +
                    w.u8(0);
488 +
                    w.object(r);
489 +
                    w.u32(*count);
490 +
                    w.u8(match form {
491 +
                        crate::objects::TextForm::Cbor => 0,
492 +
                        crate::objects::TextForm::Raw => 1,
493 +
                        crate::objects::TextForm::CborFallback => 2,
494 +
                    });
495 +
                }
496 +
                TextRef::Inline { change, start, end } => {
497 +
                    w.u8(1);
498 +
                    w.object(change);
499 +
                    w.u32(*start);
500 +
                    w.u32(*end);
331 501
                }
332 502
            }
333 -
            let body = std::mem::replace(&mut w, outer_w);
334 -
            w.bytes(&body.0);
335 503
        }
336 -
        let mut outer = outer;
337 -
        outer.bytes(&w.0);
338 -
        Some(outer.0)
504 +
        Some(w.0)
339 505
    }
340 506
341 507
    /// The state an index holds (see [`State::write_index`]).
342 508
    pub fn read_index(bytes: &[u8]) -> Result<State, ObjectError> {
509 +
        let bad = || ObjectError::Schema("state index: truncated".into());
510 +
        let front_len =
511 +
            usize::try_from(crate::flat::Reader::new(bytes).u64()?).map_err(|_| bad())?;
512 +
        let front = bytes.get(8..8 + front_len).ok_or_else(bad)?;
513 +
        let histories = Histories::Bytes(Arc::new(bytes[8 + front_len..].to_vec()));
514 +
        Self::read_index_front(front, histories)
515 +
    }
516 +
517 +
    /// An index's front, everything but the files' histories, with where
518 +
    /// those are: each is read, and checked against the hash the front
519 +
    /// holds, only when its graph is built (oxygen#94).
520 +
    pub fn read_index_front(bytes: &[u8], histories: Histories) -> Result<State, ObjectError> {
343 521
        use crate::flat::Reader;
344 522
        let bad = |what: &str| ObjectError::Schema(format!("state index: {what}"));
345 523
        let mut r = Reader::new(bytes);
…
17 unchanged lines
363 541
        }
364 542
        let (tree, deferred) = rayon::join(
365 543
            || TreeGraph::read_flat(&mut Reader::new(tree_bytes)),
366 -
            || Self::read_deferred(deferred_bytes),
544 +
            || Self::read_deferred(deferred_bytes, &histories),
367 545
        );
368 546
        st.tree = tree?;
369 547
        st.deferred = Some(deferred?);
…
1 unchanged lines
371 549
    }
372 550
373 551
    /// The changes of a state index: its first block, which a reader can
374 -
    /// take without the rest (the log needs no more).
552 +
    /// take without the rest.
375 553
    pub fn read_index_changes(
376 554
        bytes: &[u8],
377 -
    ) -> Result<HashMap<ChangeHash, Arc<ChangeMeta>>, ObjectError> {
555 +
    ) -> Result<crate::fasthash::FastMap<ChangeHash, Arc<ChangeMeta>>, ObjectError> {
556 +
        Ok(Self::read_index_changes_listed(bytes)?
557 +
            .into_iter()
558 +
            .map(|(h, m)| (h, Arc::new(m)))
559 +
            .collect())
560 +
    }
561 +
562 +
    /// The same changes as a list in the order the index keeps them, which
563 +
    /// a current index keeps in log order ([`log_order`]): what the log
564 +
    /// needs (oxygen#94).
565 +
    pub fn read_index_changes_listed(
566 +
        bytes: &[u8],
567 +
    ) -> Result<Vec<(ChangeHash, ChangeMeta)>, ObjectError> {
378 568
        use crate::flat::Reader;
379 569
        let mut r = Reader::new(bytes);
380 570
        if r.fixed::<18>()?.as_slice() != Self::INDEX_MAGIC {
381 571
            return Err(ObjectError::Schema("state index: not a state index".into()));
382 572
        }
383 573
        let n = r.count()?;
384 -
        let mut changes = HashMap::with_capacity(n);
574 +
        let mut changes = Vec::with_capacity(n);
385 575
        for _ in 0..n {
386 576
            let h = r.object()?;
387 -
            changes.insert(h, Arc::new(ChangeMeta::read_flat(&mut r)?));
577 +
            changes.push((h, ChangeMeta::read_flat(&mut r)?));
388 578
        }
389 579
        if !r.at_end() {
390 580
            return Err(ObjectError::Schema("state index: trailing bytes".into()));
…
4 unchanged lines
395 585
    /// The deferred block: each node with its graph key and its body,
396 586
    /// left encoded (see [`Deferred`]).
397 587
    fn read_deferred(
398 -
        bytes: &[u8],
588 +
        table: &[u8],
589 +
        histories: &Histories,
399 590
    ) -> Result<crate::fasthash::FastMap<NodeId, Arc<Deferred>>, ObjectError> {
400 591
        use crate::flat::Reader;
401 592
        let _t = crate::timing::phase("index: deferred");
402 -
        let all: Arc<[u8]> = Arc::from(bytes);
403 -
        let mut r = Reader::new(bytes);
593 +
        let mut r = Reader::new(table);
404 594
        let mut deferred = crate::fasthash::FastMap::default();
405 595
        let n = r.count()?;
406 596
        deferred.reserve(n);
407 597
        for _ in 0..n {
408 598
            let id = NodeId(r.fixed::<16>()?);
409 599
            let key = r.fixed::<32>()?;
410 -
            let body = r.bytes()?;
411 -
            let start = body.as_ptr() as usize - bytes.as_ptr() as usize;
600 +
            // The length is of bytes in the histories, not in this table:
601 +
            // checked against the histories' size below.
602 +
            let (at, len, sum) = (r.u64()?, r.u32()? as usize, r.fixed::<32>()?);
603 +
            if at
604 +
                .checked_add(len as u64)
605 +
                .is_none_or(|end| end > histories.len())
606 +
            {
607 +
                return Err(ObjectError::Schema(
608 +
                    "state index: a file's history lies outside it".into(),
609 +
                ));
610 +
            }
412 611
            let d = Deferred {
413 -
                raw: Some((all.clone(), start..start + body.len())),
612 +
                raw: Some(RawHistory {
613 +
                    from: histories.clone(),
614 +
                    at,
615 +
                    len,
616 +
                    sum,
617 +
                }),
414 618
                key: std::sync::OnceLock::from(key),
415 619
                ..Deferred::default()
416 620
            };
…
90 unchanged lines
507 711
            files: self.files.clone(),
508 712
            replaces: self.replaces.clone(),
509 713
            // A cache: the clone builds its own.
510 -
            closure: Mutex::new(HashMap::new()),
714 +
            closure: Default::default(),
715 +
            levels: Default::default(),
511 716
            deferred: self.deferred.clone(),
717 +
            unified_view: self.unified_view.clone(),
512 718
        }
513 719
    }
514 720
}
…
29 unchanged lines
544 750
    }
545 751
}
546 752
753 +
/// How one conflict settles without a person ([`State::settle_plans`]).
754 +
#[derive(Clone, Debug, PartialEq, Eq)]
755 +
pub struct Settle {
756 +
    /// Its sides' text, as its markers show them.
757 +
    pub sides: Vec<Vec<u8>>,
758 +
    /// The text that takes the markers' place, side by side, in order.
759 +
    pub kept: Vec<Vec<u8>>,
760 +
    /// Its sides are identical, and one is kept.
761 +
    pub identical: bool,
762 +
}
763 +
547 764
/// See `State::conflict_regions`.
548 765
#[derive(Clone, Debug, Default, PartialEq, Eq)]
549 766
pub struct ConflictRegions {
…
99 unchanged lines
649 866
    load: impl FnOnce(&[ObjectRef]) -> Result<Vec<Vec<u8>>, ObjectError>,
650 867
    read_change: impl Fn(&ChangeHash) -> Result<Arc<Vec<u8>>, ObjectError>,
651 868
) -> Result<Vec<(NodeId, FileGraph)>, ObjectError> {
652 -
    let mut taken: Vec<(NodeId, DeferredBody)> =
653 -
        taken.into_iter().map(|(n, d)| (n, d.into_body())).collect();
869 +
    let mut taken: Vec<(NodeId, DeferredBody)> = taken
870 +
        .into_iter()
871 +
        .map(|(n, d)| Ok((n, d.into_body()?)))
872 +
        .collect::<Result<_, ObjectError>>()?;
654 873
    let mut refs: Vec<ObjectRef> = taken
655 874
        .iter()
656 875
        .filter_map(|(_, d)| match &d.base {
…
213 unchanged lines
870 1089
        let Some(deferred) = self.deferred.as_ref() else {
871 1090
            return Ok(Vec::new());
872 1091
        };
873 -
        let copies: Vec<(NodeId, Deferred)> = nodes
874 -
            .iter()
875 -
            .map(|n| {
876 -
                (
877 -
                    *n,
878 -
                    deferred.get(n).map(|d| (**d).clone()).unwrap_or_default(),
879 -
                )
880 -
            })
881 -
            .collect();
882 -
        build_deferred(copies, load, read_change)
1092 +
        // A file this state has built has no deferred history: its graph
1093 +
        // is the built one, copied. Built from an empty history instead, it
1094 +
        // rendered as an empty file, and a merge taking this path recorded
1095 +
        // the whole file as inserted again, conflicting with itself (#71).
1096 +
        let mut built: Vec<(NodeId, FileGraph)> = Vec::new();
1097 +
        let mut copies: Vec<(NodeId, Deferred)> = Vec::new();
1098 +
        for n in nodes {
1099 +
            match (self.files.get(n), deferred.get(n)) {
1100 +
                (Some(g), _) => built.push((*n, (**g).clone())),
1101 +
                (None, d) => copies.push((*n, d.map(|d| (**d).clone()).unwrap_or_default())),
1102 +
            }
1103 +
        }
1104 +
        let mut out = build_deferred(copies, load, read_change)?;
1105 +
        out.extend(built);
1106 +
        Ok(out)
883 1107
    }
884 1108
885 1109
    /// Render a graph built for this state (see
…
45 unchanged lines
931 1155
        if self.files.contains_key(node) {
932 1156
            return None;
933 1157
        }
934 -
        let d = self.deferred.as_ref()?.get(node)?.body();
1158 +
        // A history that cannot be read is no single text: building its
1159 +
        // graph reports why.
1160 +
        let d = self.deferred.as_ref()?.get(node)?.body().ok()?;
935 1161
        let [(_, edits)] = d.edits.as_slice() else {
936 1162
            return None;
937 1163
        };
…
33 unchanged lines
971 1197
            h.update(&[0]);
972 1198
            return Some(*h.finalize().as_bytes());
973 1199
        };
974 -
        Some(*d.key.get_or_init(|| d.graph_key(h)))
1200 +
        // A history that cannot be read has no key: the file is read.
1201 +
        if let Some(k) = d.key.get() {
1202 +
            return Some(*k);
1203 +
        }
1204 +
        let k = d.graph_key(h).ok()?;
1205 +
        Some(*d.key.get_or_init(|| k))
975 1206
    }
976 1207
977 1208
    pub fn len(&self) -> usize {
…
39 unchanged lines
1017 1248
    pub fn built_count(&self) -> usize {
1018 1249
        self.files.len()
1019 1250
    }
1251 +
1252 +
    /// Atoms in this state's built graphs, live and deleted: what their
1253 +
    /// memory follows, since a graph keeps every line its file ever had.
1254 +
    pub fn built_atoms(&self) -> usize {
1255 +
        self.files.values().map(|g| g.atom_count()).sum()
1256 +
    }
1257 +
1258 +
    /// Edits held in the deferred histories of files not built, and the
1259 +
    /// most any one file holds (for measuring, #25).
1260 +
    pub fn deferred_edit_count(&self) -> (usize, usize) {
1261 +
        let Some(d) = self.deferred.as_ref() else {
1262 +
            return (0, 0);
1263 +
        };
1264 +
        let mut total = 0;
1265 +
        let mut most = 0;
1266 +
        for v in d.values() {
1267 +
            let n: usize = v
1268 +
                .body()
1269 +
                .map_or(0, |b| b.edits.iter().map(|(_, e)| e.len()).sum());
1270 +
            total += n;
1271 +
            most = most.max(n);
1272 +
        }
1273 +
        (total, most)
1274 +
    }
1020 1275
}
1021 1276
1022 1277
impl Deferred {
1023 1278
    /// See [`State::graph_key`]: `h` has taken the node already.
1024 -
    fn graph_key(&self, mut h: blake3::Hasher) -> [u8; 32] {
1025 -
        let d = self.body();
1279 +
    fn graph_key(&self, mut h: blake3::Hasher) -> Result<[u8; 32], ObjectError> {
1280 +
        let d = self.body()?;
1026 1281
        match &d.base {
1027 1282
            None => h.update(&[0]),
1028 1283
            Some(DeferredBase::Blob(r)) => h.update(&[1]).update(&r.0),
…
6 unchanged lines
1035 1290
        for c in changes {
1036 1291
            h.update(&c.0);
1037 1292
        }
1038 -
        *h.finalize().as_bytes()
1293 +
        Ok(*h.finalize().as_bytes())
1039 1294
    }
1040 1295
}
1041 1296
…
13 unchanged lines
1055 1310
    pub fn tree(&self) -> &TreeGraph {
1056 1311
        &self.tree
1057 1312
    }
1313 +
1314 +
    /// The tree as every reader sees it: the tree graph's view, with the
1315 +
    /// same path added on several lines of history with identical content
1316 +
    /// shown as one entry rather than a name conflict (spec §9.2). The
1317 +
    /// text files among such candidates must be built in a history-only
1318 +
    /// state ([`State::unify_candidates`]); until they are, they are shown
1319 +
    /// as the conflict.
1320 +
    pub fn view(&self) -> TreeView {
1321 +
        if let Some(v) = self.unified_view.get() {
1322 +
            return v.clone();
1323 +
        }
1324 +
        let raw = self.tree.view();
1325 +
        if !raw
1326 +
            .conflicts
1327 +
            .iter()
1328 +
            .any(|c| matches!(c, crate::tree::TreeConflict::Name { .. }))
1329 +
        {
1330 +
            return raw;
1331 +
        }
1332 +
        let (view, decided) = self.tree.unify(raw, |es| self.same_content(es));
1333 +
        if decided {
1334 +
            let _ = self.unified_view.set(view.clone());
1335 +
        }
1336 +
        view
1337 +
    }
1338 +
1339 +
    /// The text files [`State::view`] must compare and a history-only
1340 +
    /// state has not built: candidates of the tree's name conflicts.
1341 +
    pub fn unify_candidates(&self) -> Vec<NodeId> {
1342 +
        let Some(deferred) = &self.deferred else {
1343 +
            return Vec::new();
1344 +
        };
1345 +
        if self.unified_view.get().is_some() {
1346 +
            return Vec::new();
1347 +
        }
1348 +
        let raw = self.tree.view();
1349 +
        let mut out = Vec::new();
1350 +
        for c in &raw.conflicts {
1351 +
            if let crate::tree::TreeConflict::Name { nodes, .. } = c {
1352 +
                out.extend(nodes.iter().filter(|n| deferred.contains_key(*n)));
1353 +
            }
1354 +
        }
1355 +
        out
1356 +
    }
1357 +
1358 +
    /// Whether the entries' contents are identical, unconflicted: the same
1359 +
    /// rendering for text files, the same single blob for opaque ones,
1360 +
    /// symlinks and subrepos. `None` for a text file not built yet.
1361 +
    fn same_content(&self, entries: &[&crate::tree::TreeEntry]) -> Option<bool> {
1362 +
        let text = |e: &crate::tree::TreeEntry| e.kind == NodeKind::File && !e.mode.opaque;
1363 +
        let mut first: Option<Vec<u8>> = None;
1364 +
        let mut first_blob: Option<ObjectRef> = None;
1365 +
        for e in entries {
1366 +
            if text(e) {
1367 +
                if self
1368 +
                    .deferred
1369 +
                    .as_ref()
1370 +
                    .is_some_and(|d| d.contains_key(&e.node))
1371 +
                {
1372 +
                    return None;
1373 +
                }
1374 +
                let (bytes, m) = self.render_file(&e.node);
1375 +
                if m.is_conflicted() {
1376 +
                    return Some(false);
1377 +
                }
1378 +
                match &first {
1379 +
                    None => first = Some(bytes),
1380 +
                    Some(b) if *b == bytes => {}
1381 +
                    Some(_) => return Some(false),
1382 +
                }
1383 +
            } else {
1384 +
                let [blob] = self.live_blobs(&e.node)[..] else {
1385 +
                    return Some(false);
1386 +
                };
1387 +
                match first_blob {
1388 +
                    None => first_blob = Some(blob),
1389 +
                    Some(b) if b == blob => {}
1390 +
                    Some(_) => return Some(false),
1391 +
                }
1392 +
            }
1393 +
        }
1394 +
        Some(true)
1395 +
    }
1058 1396
    pub fn file(&self, node: &NodeId) -> Option<&FileGraph> {
1059 1397
        self.graph(node)
1060 1398
    }
…
93 unchanged lines
1154 1492
            return;
1155 1493
        }
1156 1494
        let meta = ChangeMeta::of(&change);
1495 +
        self.unified_view = Default::default();
1157 1496
        self.apply_with(h, change, texts);
1158 1497
        self.changes.insert(h, Arc::new(meta));
1159 -
        self.closure.lock().unwrap().clear();
1498 +
        {
1499 +
            let mut m = self.closure.write().unwrap();
1500 +
            m.sets.clear();
1501 +
            m.held = 0;
1502 +
        }
1160 1503
    }
1161 1504
1162 1505
    // ------------------------------------------------------------ checkpoints (spec §11)
…
333 unchanged lines
1496 1839
                    }
1497 1840
                    None => {
1498 1841
                        let d = Arc::make_mut(deferred.entry(node).or_default());
1499 -
                        let b = d.body_mut();
1500 -
                        b.edits.push((h, edits));
1501 -
                        b.texts.extend(node_texts);
1842 +
                        d.add(h, edits, node_texts);
1502 1843
                        d.key = Default::default();
1503 1844
                    }
1504 1845
                }
…
39 unchanged lines
1544 1885
1545 1886
    /// Transitive dependencies of `h`, excluding `h` itself.
1546 1887
    pub fn closure(&self, h: &ChangeHash) -> Arc<HashSet<ChangeHash>> {
1547 -
        if let Some(c) = self.closure.lock().unwrap().get(h) {
1888 +
        if let Some(c) = self.closure.read().unwrap().sets.get(h) {
1548 1889
            return c.clone();
1549 1890
        }
1550 1891
        let mut out = HashSet::new();
…
10 unchanged lines
1561 1902
            }
1562 1903
        }
1563 1904
        let rc = Arc::new(out);
1564 -
        self.closure.lock().unwrap().insert(*h, rc.clone());
1905 +
        let mut memo = self.closure.write().unwrap();
1906 +
        if memo.held + rc.len() > CLOSURE_MEMO_HASHES {
1907 +
            memo.sets.clear();
1908 +
            memo.held = 0;
1909 +
        }
1910 +
        memo.held += rc.len();
1911 +
        memo.sets.insert(*h, rc.clone());
1565 1912
        rc
1566 1913
    }
1567 1914
1915 +
    /// `h`'s level (see `levels`), worked out for it and its ancestors not
1916 +
    /// yet known, without recursion: histories are deeper than a stack. An
1917 +
    /// unknown change has level 0.
1918 +
    fn level(&self, h: &ChangeHash) -> u32 {
1919 +
        // Nearly always known: read without excluding the other threads
1920 +
        // rendering beside this one (oxygen#96).
1921 +
        if let Some(l) = self.levels.read().unwrap().get(h) {
1922 +
            return *l;
1923 +
        }
1924 +
        let mut levels = self.levels.write().unwrap();
1925 +
        if let Some(l) = levels.get(h) {
1926 +
            return *l;
1927 +
        }
1928 +
        let mut stack: Vec<(ChangeHash, bool)> = vec![(*h, false)];
1929 +
        while let Some((c, expanded)) = stack.pop() {
1930 +
            if levels.contains_key(&c) {
1931 +
                continue;
1932 +
            }
1933 +
            let deps: &[ChangeHash] = self
1934 +
                .changes
1935 +
                .get(&c)
1936 +
                .map(|m| m.deps.as_slice())
1937 +
                .unwrap_or(&[]);
1938 +
            if expanded {
1939 +
                let l = deps
1940 +
                    .iter()
1941 +
                    .map(|d| levels.get(d).map_or(0, |l| l + 1))
1942 +
                    .max()
1943 +
                    .unwrap_or(0);
1944 +
                levels.insert(c, l);
1945 +
            } else {
1946 +
                stack.push((c, true));
1947 +
                for d in deps {
1948 +
                    if !levels.contains_key(d) {
1949 +
                        stack.push((*d, false));
1950 +
                    }
1951 +
                }
1952 +
            }
1953 +
        }
1954 +
        levels[h]
1955 +
    }
1956 +
1568 1957
    /// Changes that no other change depends on directly, sorted.
1569 1958
    pub fn heads(&self) -> Vec<ChangeHash> {
1570 -
        let depended: HashSet<&ChangeHash> =
1959 +
        let depended: crate::fasthash::FastSet<&ChangeHash> =
1571 1960
            self.changes.values().flat_map(|c| c.deps.iter()).collect();
1572 1961
        let mut heads: Vec<ChangeHash> = self
1573 1962
            .changes
…
12 unchanged lines
1586 1975
    /// of heads, which a bookmark of independent changes has many of.
1587 1976
    pub fn maximal(&self, hashes: &[ChangeHash]) -> Vec<ChangeHash> {
1588 1977
        let set: BTreeSet<ChangeHash> = hashes.iter().copied().collect();
1589 -
        let mut below: HashSet<ChangeHash> = HashSet::new();
1978 +
        let mut below: crate::fasthash::FastSet<ChangeHash> = Default::default();
1590 1979
        let mut stack: Vec<ChangeHash> = set
1591 1980
            .iter()
1592 1981
            .filter_map(|h| self.changes.get(h))
…
27 unchanged lines
1620 2009
            .collect()
1621 2010
    }
1622 2011
2012 +
    /// The nodes with replace records: opaque files, as their blobs go.
2013 +
    pub fn opaque_nodes(&self) -> Vec<NodeId> {
2014 +
        let mut nodes: Vec<NodeId> = self.replaces.keys().copied().collect();
2015 +
        nodes.sort();
2016 +
        nodes
2017 +
    }
2018 +
1623 2019
    /// Live blobs of an opaque node: every `new` that no later `Replace` retired.
1624 2020
    pub fn live_blobs(&self, node: &NodeId) -> Vec<ObjectRef> {
1625 2021
        let mut live: Vec<ObjectRef> = self
…
18 unchanged lines
1644 2040
        let Some(rs) = self.replaces.get(node) else {
1645 2041
            return vec![];
1646 2042
        };
2043 +
        // Dependence by the bounded walk rather than whole closures: on a
2044 +
        // long history a closure is a hundred thousand changes, and the
2045 +
        // memo cannot keep them all (#107).
2046 +
        let dep =
2047 +
            |a: &ChangeHash, b: &ChangeHash| a != b && <Self as DepOracle>::depends_on(self, a, b);
1647 2048
        let knows_its_old = |s: &ChangeHash, old: &ObjectRef| {
1648 2049
            rs.iter()
1649 -
                .any(|(q, _, n)| n.as_ref() == Some(old) && q != s && self.closure(s).contains(q))
2050 +
                .any(|(q, _, n)| n.as_ref() == Some(old) && q != s && dep(s, q))
1650 2051
        };
1651 2052
        rs.iter()
1652 2053
            .filter_map(|(c, _, n)| n.map(|n| (*c, n)))
1653 2054
            .filter(|(c, n)| {
1654 2055
                !rs.iter().any(|(s, o, _)| {
1655 -
                    s != c
1656 -
                        && *o == Some(*n)
1657 -
                        && (self.closure(s).contains(c)
1658 -
                            || (!knows_its_old(s, n) && !self.closure(c).contains(s)))
2056 +
                    s != c && *o == Some(*n) && (dep(s, c) || (!knows_its_old(s, n) && !dep(c, s)))
1659 2057
                })
1660 2058
            })
1661 2059
            .collect()
…
66 unchanged lines
1728 2126
        out
1729 2127
    }
1730 2128
2129 +
    /// A file graph materialized with each conflict's sides earliest change
2130 +
    /// first (oxygen#121), by the clock of the oldest change on a side, so
2131 +
    /// markers read in the order the work was done and `oxy resolve --both`
2132 +
    /// keeps that order.
2133 +
    fn materialize_ordered(&self, g: &FileGraph) -> Materialized {
2134 +
        let mut m = g.materialize(self);
2135 +
        let when = |h: &ChangeHash| match self.changes.get(h) {
2136 +
            Some(c) => (false, c.hlc, *h),
2137 +
            None => (true, Default::default(), *h),
2138 +
        };
2139 +
        for seg in &mut m.segments {
2140 +
            if let crate::graph::Segment::Conflict(sides) = seg {
2141 +
                sides.sort_by_cached_key(|s| s.iter().map(|a| when(&a.0)).min());
2142 +
            }
2143 +
        }
2144 +
        m
2145 +
    }
2146 +
2147 +
    /// For each conflict of a text node, in marker order (spec §9.5): the
2148 +
    /// text of its sides, earliest change first, when it is an adjacent
2149 +
    /// insertion that keeping both sides settles (oxygen#121); `None` for
2150 +
    /// the others.
2151 +
    pub fn both_sides(&self, node: &NodeId) -> Vec<Option<Vec<Vec<u8>>>> {
2152 +
        let Some(g) = self.graph(node) else {
2153 +
            return Vec::new();
2154 +
        };
2155 +
        let m = self.materialize_ordered(g);
2156 +
        let mut out = Vec::new();
2157 +
        for (i, seg) in m.segments.iter().enumerate() {
2158 +
            match seg {
2159 +
                crate::graph::Segment::Lines(_) => {}
2160 +
                crate::graph::Segment::Conflict(sides) if g.adjacent_insertion(&m, i, self) => {
2161 +
                    out.push(Some(
2162 +
                        sides
2163 +
                            .iter()
2164 +
                            .map(|s| {
2165 +
                                s.iter()
2166 +
                                    .flat_map(|a| g.content(a).unwrap_or_default().to_vec())
2167 +
                                    .collect()
2168 +
                            })
2169 +
                            .collect(),
2170 +
                    ));
2171 +
                }
2172 +
                _ => out.push(None),
2173 +
            }
2174 +
        }
2175 +
        out
2176 +
    }
2177 +
2178 +
    /// How each conflict of a text node settles without a person, in
2179 +
    /// marker order (spec §9.5); `None` for one that needs a person. A
2180 +
    /// conflict whose sides are identical keeps one of them, in any file
2181 +
    /// (oxygen#146): two bookmarks that resolved the same conflict the same
2182 +
    /// way agree. With `union` (the file is `merge=union`, oxygen#145), an
2183 +
    /// adjacent insertion keeps every side, earliest change first.
2184 +
    pub fn settle_plans(&self, node: &NodeId, union: bool) -> Vec<Option<Settle>> {
2185 +
        let Some(g) = self.graph(node) else {
2186 +
            return Vec::new();
2187 +
        };
2188 +
        let m = self.materialize_ordered(g);
2189 +
        let text = |s: &[AtomKey]| -> Vec<u8> {
2190 +
            s.iter()
2191 +
                .flat_map(|a| g.content(a).unwrap_or_default().to_vec())
2192 +
                .collect()
2193 +
        };
2194 +
        let mut out = Vec::new();
2195 +
        for (i, seg) in m.segments.iter().enumerate() {
2196 +
            match seg {
2197 +
                crate::graph::Segment::Lines(_) => {}
2198 +
                crate::graph::Segment::Conflict(sides) => {
2199 +
                    let texts: Vec<Vec<u8>> = sides.iter().map(|s| text(s)).collect();
2200 +
                    out.push(if texts.len() > 1 && texts.iter().all(|t| *t == texts[0]) {
2201 +
                        Some(Settle {
2202 +
                            kept: vec![texts[0].clone()],
2203 +
                            sides: texts,
2204 +
                            identical: true,
2205 +
                        })
2206 +
                    } else if union && g.adjacent_insertion(&m, i, self) {
2207 +
                        Some(Settle {
2208 +
                            kept: texts.clone(),
2209 +
                            sides: texts,
2210 +
                            identical: false,
2211 +
                        })
2212 +
                    } else {
2213 +
                        None
2214 +
                    });
2215 +
                }
2216 +
                _ => out.push(None),
2217 +
            }
2218 +
        }
2219 +
        out
2220 +
    }
2221 +
1731 2222
    pub fn render_file(&self, node: &NodeId) -> (Vec<u8>, Materialized) {
1732 2223
        let Some(g) = self.graph(node) else {
1733 2224
            return (Vec::new(), Materialized::default());
1734 2225
        };
1735 -
        let m = g.materialize(self);
2226 +
        let m = self.materialize_ordered(g);
1736 2227
        let bytes = g.render(&m, &|h| self.label(h));
1737 2228
        (bytes, m)
1738 2229
    }
…
3 unchanged lines
1742 2233
    pub fn materialize(&self) -> Snapshot {
1743 2234
        use rayon::prelude::*;
1744 2235
        self.require_full("materializing a state");
1745 -
        let tree = self.tree.view();
2236 +
        let tree = self.view();
1746 2237
        let entries: Vec<(&String, &crate::tree::TreeEntry)> = tree.entries.iter().collect();
1747 2238
        let rendered: Vec<(String, FileState)> = entries
1748 2239
            .par_iter()
…
10 unchanged lines
1759 2250
    /// Like [`State::materialize`], leaving the text files of nodes in
1760 2251
    /// `skip` unrendered (listed in [`Snapshot::unrendered`]); every other
1761 2252
    /// file's graph must be loaded. For a caller that already knows those
1762 -
    /// files are unchanged and not conflicted.
2253 +
    /// files are unchanged and not conflicted. Its tree is the tree
2254 +
    /// graph's own view, as a Git import builds against it: same-path
2255 +
    /// additions are not unified ([`State::view`]), since the import's
2256 +
    /// merge records Git's tree for them itself.
1763 2257
    pub fn materialize_except(&self, skip: &HashSet<NodeId>) -> Snapshot {
1764 -
        self.materialize_leaving(|_, e| skip.contains(&e.node))
2258 +
        self.materialize_leaving(self.tree.view(), |_, e| skip.contains(&e.node))
1765 2259
    }
1766 2260
1767 2261
    /// As [`State::materialize_except`], rendering only the text files at
1768 -
    /// `render` (a few, where the caller knows the rest unchanged).
2262 +
    /// `render` (a few, where the caller knows the rest unchanged), with
2263 +
    /// the tree as [`State::view`] has it.
1769 2264
    pub fn materialize_only(&self, render: &std::collections::HashSet<&str>) -> Snapshot {
1770 -
        self.materialize_leaving(|p, _| !render.contains(p))
2265 +
        self.materialize_leaving(self.view(), |p, _| !render.contains(p))
1771 2266
    }
1772 2267
1773 -
    /// As [`State::materialize_except`], rendering only the text files of
2268 +
    /// As [`State::materialize_only`], rendering only the text files of
1774 2269
    /// the nodes in `render` (a change's edited files), without a set of
1775 2270
    /// every other file to leave out.
1776 2271
    pub fn materialize_nodes(&self, render: &HashSet<NodeId>) -> Snapshot {
1777 -
        self.materialize_leaving(|_, e| !render.contains(&e.node))
2272 +
        self.materialize_leaving(self.view(), |_, e| !render.contains(&e.node))
1778 2273
    }
1779 2274
1780 2275
    fn materialize_leaving(
1781 2276
        &self,
2277 +
        tree: TreeView,
1782 2278
        leave: impl Fn(&str, &crate::tree::TreeEntry) -> bool + Sync,
1783 2279
    ) -> Snapshot {
1784 2280
        use rayon::prelude::*;
1785 -
        let tree = self.tree.view();
1786 2281
        let entries: Vec<(&String, &crate::tree::TreeEntry)> = tree.entries.iter().collect();
1787 2282
        let rendered: Vec<(String, FileState, bool)> = entries
1788 2283
            .par_iter()
…
37 unchanged lines
1826 2321
        view: crate::tree::TreeView,
1827 2322
        paths: impl IntoIterator<Item = &'p str>,
1828 2323
    ) -> Snapshot {
1829 -
        let files = paths
2324 +
        use rayon::prelude::*;
2325 +
        // On every core: a merge may render thousands (oxygen#96).
2326 +
        let entries: Vec<(&str, &crate::tree::TreeEntry)> = paths
1830 2327
            .into_iter()
1831 -
            .filter_map(|p| Some((p.to_string(), self.file_state(view.entries.get(p)?))))
2328 +
            .filter_map(|p| Some((p, view.entries.get(p)?)))
1832 2329
            .collect();
2330 +
        let files = entries
2331 +
            .par_iter()
2332 +
            .map(|(p, e)| (p.to_string(), self.file_state(e)))
2333 +
            .collect::<Vec<_>>()
2334 +
            .into_iter()
2335 +
            .collect();
1833 2336
        Snapshot {
1834 2337
            tree: view,
1835 2338
            files,
…
36 unchanged lines
1872 2375
}
1873 2376
1874 2377
impl DepOracle for State {
2378 +
    /// Whether `a` depends on `b`, by a walk down from `a` that stops at
2379 +
    /// changes no higher than `b`: they cannot have `b` below them. Nothing
2380 +
    /// of the walk is kept, so asking about many pairs costs no more memory
2381 +
    /// than one walk (#25); a closure already memoised answers at once.
1875 2382
    fn depends_on(&self, a: &ChangeHash, b: &ChangeHash) -> bool {
1876 -
        a == b || self.closure(a).contains(b)
2383 +
        if a == b {
2384 +
            return true;
2385 +
        }
2386 +
        if let Some(c) = self.closure.read().unwrap().sets.get(a) {
2387 +
            return c.contains(b);
2388 +
        }
2389 +
        if !self.changes.contains_key(b) {
2390 +
            return false;
2391 +
        }
2392 +
        let lb = self.level(b);
2393 +
        if self.level(a) <= lb {
2394 +
            return false;
2395 +
        }
2396 +
        let mut seen: crate::fasthash::FastSet<ChangeHash> = Default::default();
2397 +
        // A dependency is looked for as it is found, before the walk goes
2398 +
        // below it: most questions are about a direct one (spec §8.1 makes
2399 +
        // a deletion depend on the inserter of the line after it, which is
2400 +
        // what a render asks about), and a depth-first walk that only
2401 +
        // looked when it came back up went through thousands of changes
2402 +
        // first (oxygen#137: 4.4 billion steps rendering the VS Code tree).
2403 +
        let mut stack: Vec<ChangeHash> = self
2404 +
            .changes
2405 +
            .get(a)
2406 +
            .map(|m| m.deps.clone())
2407 +
            .unwrap_or_default();
2408 +
        if stack.contains(b) {
2409 +
            return true;
2410 +
        }
2411 +
        while let Some(c) = stack.pop() {
2412 +
            if c == *b {
2413 +
                return true;
2414 +
            }
2415 +
            if !seen.insert(c) || self.level(&c) <= lb {
2416 +
                continue;
2417 +
            }
2418 +
            if let Some(m) = self.changes.get(&c) {
2419 +
                if m.deps.contains(b) {
2420 +
                    return true;
2421 +
                }
2422 +
                stack.extend(m.deps.iter().copied());
2423 +
            }
2424 +
        }
2425 +
        false
1877 2426
    }
1878 2427
}
1879 2428
1880 2429
#[cfg(test)]
1881 2430
mod tests {
1882 2431
    use super::*;
2432 +
2433 +
    /// A changes block of thousands comes back whole and in order, and a
2434 +
    /// truncated one is refused (oxygen#94).
2435 +
    #[test]
2436 +
    fn a_long_changes_block_decodes_in_order() {
2437 +
        use crate::flat::Writer;
2438 +
        let metas: Vec<(ChangeHash, ChangeMeta)> = (0..5000u32)
2439 +
            .map(|i| {
2440 +
                let mut h = [0u8; 32];
2441 +
                h[..4].copy_from_slice(&i.to_le_bytes());
2442 +
                let meta = ChangeMeta {
2443 +
                    change_id: crate::ids::ChangeId([i as u8; 16]),
2444 +
                    deps: (0..i % 3).map(|d| ObjectRef([d as u8; 32])).collect(),
2445 +
                    author: ObjectRef([7; 32]),
2446 +
                    hlc: Hlc {
2447 +
                        wall_ms: 1_000_000 - i as u64,
2448 +
                        logical: i,
2449 +
                    },
2450 +
                    message: "x".repeat((i % 50) as usize),
2451 +
                    kind: ChangeKind::Normal,
2452 +
                };
2453 +
                (ObjectRef(h), meta)
2454 +
            })
2455 +
            .collect();
2456 +
        let mut w = Writer::default();
2457 +
        w.fixed(State::INDEX_MAGIC);
2458 +
        w.count(metas.len());
2459 +
        for (h, m) in &metas {
2460 +
            w.object(h);
2461 +
            m.write_flat(&mut w);
2462 +
        }
2463 +
        assert_eq!(State::read_index_changes_listed(&w.0).unwrap(), metas);
2464 +
        let mut cut = w.0.clone();
2465 +
        cut.pop();
2466 +
        assert!(
2467 +
            State::read_index_changes_listed(&cut).is_err(),
2468 +
            "a truncated block is refused"
2469 +
        );
2470 +
    }
1883 2471
    use crate::ids::ChangeId;
1884 2472
    use crate::objects::{AtomRange, ChangeKind, Hlc, OpRef};
1885 2473
…
680 unchanged lines
2566 3154
        assert_eq!(st.heads(), want);
2567 3155
    }
2568 3156
3157 +
    /// The bounded walk of `depends_on` agrees with the closure on random
3158 +
    /// graphs, and the memo stays within its bound.
3159 +
    #[test]
3160 +
    fn depends_on_matches_the_closure() {
3161 +
        use rand::{Rng, SeedableRng};
3162 +
        let mut rng = rand::rngs::StdRng::seed_from_u64(11);
3163 +
        for _ in 0..10 {
3164 +
            let mut st = State::new();
3165 +
            let mut all: Vec<ChangeHash> = Vec::new();
3166 +
            for _ in 0..80 {
3167 +
                let deps: Vec<ChangeHash> =
3168 +
                    all.iter().copied().filter(|_| rng.gen_bool(0.06)).collect();
3169 +
                all.push(st.add(change(deps, vec![], vec![])).unwrap());
3170 +
            }
3171 +
            let unknown = ObjectRef([0xee; 32]);
3172 +
            for a in &all {
3173 +
                let want = st.closure(a).clone();
3174 +
                st.closure.write().unwrap().sets.clear();
3175 +
                st.closure.write().unwrap().held = 0;
3176 +
                for b in all.iter().chain([&unknown]) {
3177 +
                    assert_eq!(
3178 +
                        st.depends_on(a, b),
3179 +
                        a == b || want.contains(b),
3180 +
                        "{a:?} on {b:?}"
3181 +
                    );
3182 +
                }
3183 +
            }
3184 +
            assert!(st.closure.write().unwrap().held <= CLOSURE_MEMO_HASHES);
3185 +
        }
3186 +
    }
3187 +
2569 3188
    /// The one-walk `maximal` agrees with the definition (not in the
2570 3189
    /// closure of another element) on random graphs, with repeated and
2571 3190
    /// unknown hashes among the input.
…
27 unchanged lines
modified crates/oxy-core/src/store.rs +2329 −108
…
23 unchanged lines
24 24
//! `receipt/<hh>/<rest>/<attest>` index the seals covering a change and
25 25
//! the receipts on a state. Older stores hold these as one list per
26 26
//! object (`sigs/`, `seals/`, `receipts/`), which are still read.
27 +
//! `pack-retired` records the packs and loose objects a repack replaced,
28 +
//! which stay for readers until a collection an hour later (oxygenhub#249).
29 +
//!
30 +
//! A store that is not this disk's directory, where finding an object's
31 +
//! entries is a LIST request, also keeps every entry key in the
32 +
//! `pack-entries` journal and, once a collection folds them, in entry
33 +
//! lists (`pack-<id>.entries`), read once per open (spec §13.7, #116).
27 34
28 35
use crate::attest::Attest;
29 36
use crate::hash::{hash_object, header, ObjectRef, ObjectType};
…
28 unchanged lines
58 65
/// Largest object a pack entry may claim; anything bigger is corruption,
59 66
/// and refusing it keeps a bad index from driving huge allocations.
60 67
const MAX_OBJECT_LEN: u32 = 1 << 30;
68 +
/// What reading an entry of a pack whose dictionary is gone says. A pack
69 +
/// being deleted can lose its dictionary first, so a read that meets this
70 +
/// looks at the packs again before it is believed.
71 +
const NO_DICT: &str = "packed with a dictionary the pack lacks";
61 72
73 +
/// The record of what repacks replaced (oxygenhub#249): packs and loose
74 +
/// objects whose contents a newer pack holds, kept for readers that
75 +
/// still use them until garbage collection deletes them, at least
76 +
/// [`RETIRED_FOR`] later. One line per key, `<retired at, wall ms> <key>`.
77 +
/// It is a reference record (updated by compare-and-swap, so concurrent
78 +
/// repacks and collections never lose each other's lines), named under
79 +
/// `pack-` so the pack listing every open makes shows whether it holds
80 +
/// anything.
81 +
const RETIRED_KEY: &str = "pack-retired";
82 +
/// How long a retired pack or loose object stays in the store: a reader
83 +
/// that loaded its list of packs before a repack reads on from the old
84 +
/// packs this long, and after that finds the objects in the new pack by
85 +
/// looking again (see [`Store::get`]).
86 +
pub const RETIRED_FOR: std::time::Duration = std::time::Duration::from_secs(60 * 60);
87 +
/// Compare-and-swap rounds for the retired record before giving up.
88 +
const RETIRED_TRIES: usize = 32;
89 +
/// After a look at the pack listing found nothing new, how long a missed
90 +
/// [`Store::has`] answers without looking again: a writer probing for
91 +
/// thousands of objects it is about to store would otherwise list the
92 +
/// packs for each.
93 +
const HAS_RELIST_AFTER: std::time::Duration = std::time::Duration::from_secs(1);
94 +
95 +
/// The journal of signatures and index entries (spec §13.7): every entry
96 +
/// key a writer on a store that is not this disk's directory wrote since
97 +
/// the last fold, one line each, `<folded at, wall ms; 0 until folded>
98 +
/// <key>`, as the retired record's lines are. A reference record updated
99 +
/// by compare-and-swap, so concurrent writers never lose each other's
100 +
/// lines; read once per open, instead of a listing per lookup.
101 +
pub const JOURNAL_KEY: &str = "pack-entries";
102 +
/// A complete entry list (spec §13.7): every entry a fold found.
103 +
const LIST_SUFFIX: &str = ".entries";
104 +
/// A partial entry list: journal lines a writer folded out.
105 +
const PART_SUFFIX: &str = ".entries-part";
106 +
/// The first line of an entry list.
107 +
const LIST_MAGIC: &str = "oxy-entries 1";
108 +
/// Written once a collection has copied the older one-list keys into
109 +
/// per-object entries: readers stop looking for them.
110 +
const LEGACY_FOLDED_KEY: &str = "pack-legacy-folded";
111 +
/// Unfolded journal lines a writer lets the journal hold before it folds
112 +
/// them into a partial list.
113 +
const JOURNAL_FOLD_AT: usize = 1000;
114 +
/// How long a handle answers lookups from the entries it read before it
115 +
/// reads the pack listing and the journal again (spec §13.7): the
116 +
/// interval a cloud client keeps the pack listing for, so another
117 +
/// writer's entry is seen as soon as its pack would be.
118 +
pub const ENTRIES_FRESH: std::time::Duration = std::time::Duration::from_secs(30);
119 +
/// The per-object entry indexes, each with the one-list name older
120 +
/// stores kept it under.
121 +
const ENTRY_INDEXES: [(&str, &str); 6] = [
122 +
    ("sig", "sigs"),
123 +
    ("seal", "seals"),
124 +
    ("receipt", "receipts"),
125 +
    ("approval", "approvals"),
126 +
    ("cert", "certs"),
127 +
    ("accepted", "accepted-lists"),
128 +
];
129 +
130 +
/// A pack listing as compared: each key with its size and time.
131 +
type Listed = Vec<(String, u64, Option<u64>)>;
132 +
133 +
/// The retired record's lines: each key with the time it was retired.
134 +
fn parse_retired(bytes: &[u8]) -> std::collections::BTreeMap<String, u64> {
135 +
    String::from_utf8_lossy(bytes)
136 +
        .lines()
137 +
        .filter_map(|l| {
138 +
            let (t, k) = l.split_once(' ')?;
139 +
            Some((k.to_string(), t.parse().ok()?))
140 +
        })
141 +
        .collect()
142 +
}
143 +
144 +
fn encode_retired(m: &std::collections::BTreeMap<String, u64>) -> Vec<u8> {
145 +
    m.iter()
146 +
        .map(|(k, t)| format!("{t} {k}\n"))
147 +
        .collect::<String>()
148 +
        .into_bytes()
149 +
}
150 +
151 +
/// What deleting retired packs and loose objects did.
152 +
#[derive(Debug, Clone, Default, PartialEq, Eq)]
153 +
pub struct RetiredReport {
154 +
    /// Retired packs deleted, each with its index and dictionary.
155 +
    pub packs: usize,
156 +
    /// Retired loose objects deleted.
157 +
    pub loose: usize,
158 +
    /// Retired entries younger than [`RETIRED_FOR`], left for the next
159 +
    /// collection.
160 +
    pub kept: usize,
161 +
}
162 +
62 163
/// An object read from the store: its type and body.
63 164
pub type Object = (ObjectType, Vec<u8>);
64 165
…
60 unchanged lines
125 226
    format!("{index}/{}/{}/", &h[..2], &h[2..])
126 227
}
127 228
229 +
/// Whether a key names an entry list (spec §13.7), complete or partial.
230 +
fn is_entry_list(key: &str) -> bool {
231 +
    key.starts_with("pack-") && (key.ends_with(LIST_SUFFIX) || key.ends_with(PART_SUFFIX))
232 +
}
233 +
234 +
/// An entry list's bytes: zstd-compressed text, [`LIST_MAGIC`] and then
235 +
/// one key per line, sorted.
236 +
fn encode_entry_list<'a>(keys: impl Iterator<Item = &'a String>) -> Result<Vec<u8>> {
237 +
    let mut text = String::from(LIST_MAGIC);
238 +
    text.push('\n');
239 +
    for k in keys {
240 +
        text.push_str(k);
241 +
        text.push('\n');
242 +
    }
243 +
    Ok(zstd::bulk::compress(text.as_bytes(), ZSTD_LEVEL)?)
244 +
}
245 +
246 +
/// The keys an entry list holds.
247 +
fn decode_entry_list(key: &str, bytes: &[u8]) -> Result<Vec<String>> {
248 +
    let bad = |why: &str| StoreError::Io(io::Error::other(format!("{key}: {why}")));
249 +
    let text = zstd::stream::decode_all(bytes).map_err(|_| bad("not zstd"))?;
250 +
    let text = String::from_utf8(text).map_err(|_| bad("not UTF-8"))?;
251 +
    let mut lines = text.lines();
252 +
    if lines.next() != Some(LIST_MAGIC) {
253 +
        return Err(bad("not an entry list"));
254 +
    }
255 +
    Ok(lines
256 +
        .filter(|l| !l.is_empty())
257 +
        .map(str::to_string)
258 +
        .collect())
259 +
}
260 +
261 +
/// Drop journal lines folded at least [`RETIRED_FOR`] before `now`:
262 +
/// every reader that could need them has a list holding them by then.
263 +
fn expire_folded(m: &mut std::collections::BTreeMap<String, u64>, now: u64) {
264 +
    let hour = RETIRED_FOR.as_millis() as u64;
265 +
    m.retain(|_, t| *t == 0 || now.saturating_sub(*t) < hour);
266 +
}
267 +
128 268
/// `<index>/<hh>/<rest>`: the one-file-per-object layout of older stores.
129 269
fn legacy_key(index: &str, r: &ObjectRef) -> String {
130 270
    let h = r.to_hex();
…
44 unchanged lines
175 315
    dict: Option<&Arc<zstd::dict::DecoderDictionary<'static>>>,
176 316
) -> Result<Object> {
177 317
    let body = if e.flags & FLAG_DICT != 0 {
178 -
        let dict = dict.ok_or_else(|| {
179 -
            StoreError::Corrupt(*r, "packed with a dictionary the pack lacks".into())
180 -
        })?;
318 +
        let dict = dict.ok_or_else(|| StoreError::Corrupt(*r, NO_DICT.into()))?;
181 319
        zstd::bulk::Decompressor::with_prepared_dictionary(dict)
182 320
            .and_then(|mut d| d.decompress(&stored, e.raw_len as usize))
183 321
            .map_err(|err| StoreError::Corrupt(*r, format!("zstd: {err}")))?
…
13 unchanged lines
197 335
}
198 336
199 337
impl PackIndex {
200 -
    fn load(backend: &Backend, idx_key: &str, pack_key: &str) -> Result<PackIndex> {
201 -
        let data = backend.get(idx_key).map_err(backend_err)?;
338 +
    /// Load a pack's index; `pack_len` is the pack's size, from the listing
339 +
    /// that found it (a stat of it would be a request of its own). `None`
340 +
    /// when the index is gone since the listing: the pack was deleted.
341 +
    fn load(
342 +
        backend: &Backend,
343 +
        idx_key: &str,
344 +
        pack_key: &str,
345 +
        pack_len: u64,
346 +
    ) -> Result<Option<PackIndex>> {
347 +
        let data = match backend.get(idx_key) {
348 +
            Ok(d) => d,
349 +
            Err(BackendError::NotFound(_)) => return Ok(None),
350 +
            Err(e) => return Err(backend_err(e)),
351 +
        };
202 352
        let bad = |m: &str| StoreError::Corrupt(ObjectRef::ZERO, format!("{idx_key}: {m}"));
203 353
        if data.len() < 12 || &data[..8] != IDX_MAGIC {
204 354
            return Err(bad("bad index header"));
…
2 unchanged lines
207 357
        if data.len() != 12 + count * IDX_ENTRY {
208 358
            return Err(bad("index length mismatch"));
209 359
        }
210 -
        let pack_len = backend
211 -
            .stat(pack_key)
212 -
            .map_err(backend_err)?
213 -
            .ok_or_else(|| bad("the index names a pack that is not there"))?
214 -
            .size;
215 360
        let mut refs = Vec::with_capacity(count);
216 361
        let mut entries = Vec::with_capacity(count);
217 362
        for i in 0..count {
…
32 unchanged lines
250 395
        let dict_key = format!("{}.dict", pack_key.trim_end_matches(".pack"));
251 396
        let dict = get_optional(backend, &dict_key)?
252 397
            .map(|d| Arc::new(zstd::dict::DecoderDictionary::copy(&d)));
253 -
        Ok(PackIndex {
398 +
        Ok(Some(PackIndex {
254 399
            key: pack_key.to_string(),
255 400
            refs,
256 401
            entries,
257 402
            dict,
258 -
        })
403 +
        }))
259 404
    }
260 405
261 406
    fn find(&self, r: &ObjectRef) -> Option<PackEntry> {
…
65 unchanged lines
327 472
    scratch: PathBuf,
328 473
    /// The backend is that directory, so the old file paths still hold.
329 474
    local: bool,
330 -
    packs: RefCell<Vec<PackIndex>>,
475 +
    packs: RefCell<Vec<Arc<PackIndex>>>,
476 +
    /// The pack listing `packs` was loaded from (key, size, time), to tell
477 +
    /// whether a look at the store would find anything new.
478 +
    packs_listed: RefCell<Listed>,
479 +
    /// When a look at the pack listing for a missed [`Store::has`] last
480 +
    /// found nothing new.
481 +
    has_relisted: std::cell::Cell<Option<std::time::Instant>>,
331 482
    /// An open write batch (spec §13.4): objects go here instead of to loose
332 483
    /// files, and are written out with one `fsync` when the batch finishes.
333 484
    pending: RefCell<Option<Pending>>,
334 485
    batch_depth: std::cell::Cell<u32>,
335 486
    /// Change metadata already read, from the `metas` cache file.
336 487
    metas: RefCell<Option<crate::fasthash::FastMap<ObjectRef, crate::state::ChangeMeta>>>,
488 +
    /// The live entry lists (spec §13.7) the pack listing showed: listed
489 +
    /// and not retired.
490 +
    entry_lists: RefCell<Vec<String>>,
491 +
    /// Whether the store is folded: a live entry list is complete.
492 +
    folded: std::cell::Cell<bool>,
493 +
    /// Whether the pack listing showed `pack-legacy-folded`.
494 +
    legacy_folded: std::cell::Cell<bool>,
495 +
    /// The entry keys of a folded store, read from its lists and journal
496 +
    /// on the first lookup after the packs were loaded.
497 +
    entries: RefCell<Option<std::collections::BTreeSet<String>>>,
498 +
    /// When `entries` was read; past [`ENTRIES_FRESH`] it is read again.
499 +
    entries_read: std::cell::Cell<Option<std::time::Instant>>,
500 +
    /// Entry lists already read, by key: written once, so never read
501 +
    /// twice; those no longer live are let go.
502 +
    lists_read: RefCell<std::collections::HashMap<String, Arc<Vec<String>>>>,
503 +
    /// Entry keys written in an open batch, added to the journal when the
504 +
    /// batch is flushed.
505 +
    journal_queue: RefCell<Vec<String>>,
337 506
}
338 507
339 508
impl std::fmt::Debug for Store {
…
107 unchanged lines
447 616
        return None;
448 617
    }
449 618
    static STORES: OnceLock<Mutex<HashMap<PathBuf, Arc<MemoryStore>>>> = OnceLock::new();
450 -
    let key = dir.canonicalize().unwrap_or_else(|_| dir.to_path_buf());
619 +
    let key = crate::portable::canonical(dir).unwrap_or_else(|_| dir.to_path_buf());
451 620
    let mut map = STORES
452 621
        .get_or_init(Default::default)
453 622
        .lock()
…
5 unchanged lines
459 628
    Some(store)
460 629
}
461 630
631 +
#[cfg(test)]
632 +
thread_local! {
633 +
    /// A backend every store opened on this thread uses instead of its
634 +
    /// directory: tests of a repository kept on a store that is not this
635 +
    /// disk's directory, with several handles on one store.
636 +
    pub(crate) static TEST_BACKEND: RefCell<Option<Arc<Backend>>> = const { RefCell::new(None) };
637 +
}
638 +
462 639
impl Store {
463 640
    /// Open the store in `dir`, which is also its scratch directory.
464 641
    pub fn open(dir: impl Into<PathBuf>) -> io::Result<Store> {
465 642
        let dir = dir.into();
466 643
        fs::create_dir_all(dir.join("loose"))?;
644 +
        #[cfg(test)]
645 +
        if let Some(backend) = TEST_BACKEND.with(|t| t.borrow().clone()) {
646 +
            return Store::open_on(backend, dir);
647 +
        }
467 648
        if let Some(backend) = memory_backend_for(&dir) {
468 649
            return Store::open_on(backend, dir);
469 650
        }
…
14 unchanged lines
484 665
            scratch,
485 666
            local: false,
486 667
            packs: RefCell::new(Vec::new()),
668 +
            packs_listed: RefCell::new(Vec::new()),
669 +
            has_relisted: std::cell::Cell::new(None),
487 670
            pending: RefCell::new(None),
488 671
            batch_depth: std::cell::Cell::new(0),
489 672
            metas: RefCell::new(None),
673 +
            entry_lists: RefCell::new(Vec::new()),
674 +
            folded: std::cell::Cell::new(false),
675 +
            legacy_folded: std::cell::Cell::new(false),
676 +
            entries: RefCell::new(None),
677 +
            entries_read: std::cell::Cell::new(None),
678 +
            lists_read: RefCell::new(Default::default()),
679 +
            journal_queue: RefCell::new(Vec::new()),
490 680
        };
491 681
        store
492 682
            .reload_packs()
…
12 unchanged lines
505 695
        self.local
506 696
    }
507 697
698 +
    /// Load the list of packs again, from a new listing: what other
699 +
    /// writers packed since becomes readable, and packs since retired or
700 +
    /// deleted are let go. Packs already loaded are kept, not read again.
701 +
    pub fn refresh(&self) -> Result<()> {
702 +
        *self.entries.borrow_mut() = None;
703 +
        self.reload_packs()
704 +
    }
705 +
706 +
    /// The pack listing, with how `packs_listed` records it.
707 +
    fn list_packs(&self) -> Result<(Vec<strata::Entry>, Listed)> {
708 +
        let listed = self.backend.list("pack-").map_err(backend_err)?;
709 +
        let sig = listed
710 +
            .iter()
711 +
            .map(|e| (e.key.clone(), e.size, e.modified_ms))
712 +
            .collect();
713 +
        Ok((listed, sig))
714 +
    }
715 +
716 +
    /// Load every pack the listing names that is not retired. An index
717 +
    /// that is gone by the time it is read (its pack deleted after the
718 +
    /// listing) is passed over, and the store listed once more, since
719 +
    /// what replaced it may be new since the first listing.
508 720
    fn reload_packs(&self) -> Result<()> {
509 -
        let listed = self.backend.list("pack-").map_err(backend_err)?;
510 -
        let mut idx_keys: Vec<&str> = listed
721 +
        let (listed, sig) = self.list_packs()?;
722 +
        self.load_listed(listed, sig)
723 +
    }
724 +
725 +
    fn load_listed(&self, mut listed: Vec<strata::Entry>, mut sig: Listed) -> Result<()> {
726 +
        let had: std::collections::HashMap<String, Arc<PackIndex>> = self
727 +
            .packs
728 +
            .borrow()
729 +
            .iter()
730 +
            .map(|p| (p.key.clone(), p.clone()))
731 +
            .collect();
732 +
        let mut loaded: std::collections::HashMap<String, Arc<PackIndex>> = Default::default();
733 +
        for attempt in 0..2 {
734 +
            let retired = self.retired_in(&listed)?;
735 +
            let mut idx_keys: Vec<&str> = listed
736 +
                .iter()
737 +
                .map(|e| e.key.as_str())
738 +
                .filter(|k| k.ends_with(".idx"))
739 +
                .collect();
740 +
            idx_keys.sort();
741 +
            let mut packs = Vec::new();
742 +
            let mut vanished = false;
743 +
            for idx in idx_keys {
744 +
                let pack = format!("{}.pack", idx.trim_end_matches(".idx"));
745 +
                if retired.contains_key(&pack) {
746 +
                    continue;
747 +
                }
748 +
                let Some(p) = listed.iter().find(|e| e.key == pack) else {
749 +
                    continue;
750 +
                };
751 +
                if let Some(kept) = had.get(&pack).or_else(|| loaded.get(&pack)) {
752 +
                    packs.push(kept.clone());
753 +
                    continue;
754 +
                }
755 +
                match PackIndex::load(self.backend.as_ref(), idx, &pack, p.size)? {
756 +
                    Some(index) => {
757 +
                        let index = Arc::new(index);
758 +
                        loaded.insert(pack, index.clone());
759 +
                        packs.push(index);
760 +
                    }
761 +
                    None => vanished = true,
762 +
                }
763 +
            }
764 +
            if vanished && attempt == 0 {
765 +
                (listed, sig) = self.list_packs()?;
766 +
                continue;
767 +
            }
768 +
            let lists: Vec<String> = listed
769 +
                .iter()
770 +
                .map(|e| e.key.as_str())
771 +
                .filter(|k| is_entry_list(k) && !retired.contains_key(*k))
772 +
                .map(str::to_string)
773 +
                .collect();
774 +
            self.folded
775 +
                .set(lists.iter().any(|k| k.ends_with(LIST_SUFFIX)));
776 +
            self.legacy_folded
777 +
                .set(listed.iter().any(|e| e.key == LEGACY_FOLDED_KEY));
778 +
            *self.entry_lists.borrow_mut() = lists;
779 +
            if *self.packs_listed.borrow() != sig {
780 +
                *self.entries.borrow_mut() = None;
781 +
            }
782 +
            *self.packs.borrow_mut() = packs;
783 +
            *self.packs_listed.borrow_mut() = sig;
784 +
            return Ok(());
785 +
        }
786 +
        unreachable!("the second round always returns")
787 +
    }
788 +
789 +
    /// Load the packs again when the pack listing differs from the one
790 +
    /// they were loaded from; whether it did. What a miss calls before it
791 +
    /// is believed (oxygenhub#249): another writer may have packed the
792 +
    /// object, or a repack moved it, since this store loaded its packs.
793 +
    fn reload_if_changed(&self) -> Result<bool> {
794 +
        let (listed, sig) = self.list_packs()?;
795 +
        if *self.packs_listed.borrow() == sig {
796 +
            return Ok(false);
797 +
        }
798 +
        self.load_listed(listed, sig)?;
799 +
        Ok(true)
800 +
    }
801 +
802 +
    /// The retired record, read when `listed` (a pack listing) shows it
803 +
    /// holds anything.
804 +
    fn retired_in(
805 +
        &self,
806 +
        listed: &[strata::Entry],
807 +
    ) -> Result<std::collections::BTreeMap<String, u64>> {
808 +
        match listed.iter().find(|e| e.key == RETIRED_KEY) {
809 +
            Some(e) if e.size > 0 => Ok(self.read_retired()?.0),
810 +
            _ => Ok(Default::default()),
811 +
        }
812 +
    }
813 +
814 +
    /// The retired record and its version (`None`: never written).
815 +
    fn read_retired(
816 +
        &self,
817 +
    ) -> Result<(
818 +
        std::collections::BTreeMap<String, u64>,
819 +
        Option<strata::Version>,
820 +
    )> {
821 +
        self.read_record(RETIRED_KEY)
822 +
    }
823 +
824 +
    /// A record of `<wall ms> <key>` lines (the retired record, the
825 +
    /// journal) and its version (`None`: never written).
826 +
    fn read_record(
827 +
        &self,
828 +
        key: &str,
829 +
    ) -> Result<(
830 +
        std::collections::BTreeMap<String, u64>,
831 +
        Option<strata::Version>,
832 +
    )> {
833 +
        match self.backend.read_ref(key).map_err(backend_err)? {
834 +
            Some((bytes, v)) => Ok((parse_retired(&bytes), Some(v))),
835 +
            None => Ok((Default::default(), None)),
836 +
        }
837 +
    }
838 +
839 +
    /// Change the retired record by compare-and-swap, reading it again
840 +
    /// whenever another writer changed it first. `edit` must give the
841 +
    /// same result however often it runs.
842 +
    fn update_retired(
843 +
        &self,
844 +
        edit: impl FnMut(&mut std::collections::BTreeMap<String, u64>),
845 +
    ) -> Result<()> {
846 +
        self.update_record(RETIRED_KEY, "the record of retired packs", edit)
847 +
    }
848 +
849 +
    /// Change a record of `<wall ms> <key>` lines by compare-and-swap, as
850 +
    /// [`Store::update_retired`] does; `what` names it in the error when
851 +
    /// other writers kept winning.
852 +
    fn update_record(
853 +
        &self,
854 +
        key: &str,
855 +
        what: &str,
856 +
        mut edit: impl FnMut(&mut std::collections::BTreeMap<String, u64>),
857 +
    ) -> Result<()> {
858 +
        for _ in 0..RETIRED_TRIES {
859 +
            let (mut m, version) = self.read_record(key)?;
860 +
            let before = m.clone();
861 +
            edit(&mut m);
862 +
            // Unchanged; a record never written stays unwritten.
863 +
            if m == before && (version.is_some() || m.is_empty()) {
864 +
                return Ok(());
865 +
            }
866 +
            match self
867 +
                .backend
868 +
                .write_ref(key, &encode_retired(&m), version.as_ref())
869 +
            {
870 +
                Ok(_) => return Ok(()),
871 +
                Err(BackendError::Conflict(_)) => continue,
872 +
                Err(e) => return Err(backend_err(e)),
873 +
            }
874 +
        }
875 +
        Err(StoreError::Io(io::Error::other(format!(
876 +
            "other writers kept changing {what}; run the command again"
877 +
        ))))
878 +
    }
879 +
880 +
    /// Record `keys` (packs, by their `.pack` key, and loose objects) as
881 +
    /// retired now: kept for the readers still using them, and deleted by
882 +
    /// a collection at least [`RETIRED_FOR`] later. A key already retired
883 +
    /// keeps its first time.
884 +
    fn retire(&self, keys: &[String]) -> Result<()> {
885 +
        if keys.is_empty() {
886 +
            return Ok(());
887 +
        }
888 +
        let now = now_ms();
889 +
        self.update_retired(|m| {
890 +
            for k in keys {
891 +
                m.entry(k.clone()).or_insert(now);
892 +
            }
893 +
        })
894 +
    }
895 +
896 +
    /// Delete what was retired at least `older_than` ago (a collection
897 +
    /// does, with [`RETIRED_FOR`]): each pack with its index and
898 +
    /// dictionary, and each loose object a pack now holds. A retired loose
899 +
    /// object no pack holds any more (the pack that had it was pruned, and
900 +
    /// the object stored again since) is kept, as an ordinary loose object.
901 +
    pub fn expire_retired(
902 +
        &self,
903 +
        older_than: std::time::Duration,
904 +
        progress: Progress,
905 +
    ) -> Result<RetiredReport> {
906 +
        let (listed, _) = self.list_packs()?;
907 +
        let retired = self.retired_in(&listed)?;
908 +
        let mut report = RetiredReport::default();
909 +
        if retired.is_empty() {
910 +
            return Ok(report);
911 +
        }
912 +
        let now = now_ms();
913 +
        let older_than = older_than.as_millis() as u64;
914 +
        let due: Vec<(String, u64)> = retired
511 915
            .iter()
512 -
            .map(|e| e.key.as_str())
513 -
            .filter(|k| k.ends_with(".idx"))
916 +
            .filter(|(_, t)| now.saturating_sub(**t) >= older_than)
917 +
            .map(|(k, t)| (k.clone(), *t))
514 918
            .collect();
515 -
        idx_keys.sort();
516 -
        let mut packs = Vec::new();
517 -
        for idx in idx_keys {
518 -
            let pack = format!("{}.pack", idx.trim_end_matches(".idx"));
519 -
            if listed.iter().any(|e| e.key == pack) {
520 -
                packs.push(PackIndex::load(self.backend.as_ref(), idx, &pack)?);
919 +
        report.kept = retired.len() - due.len();
920 +
        if due.is_empty() {
921 +
            return Ok(report);
922 +
        }
923 +
        progress(&format!("deleting {} retired files", due.len()));
924 +
        // Which objects the live packs hold.
925 +
        self.reload_packs()?;
926 +
        for (key, _) in &due {
927 +
            if key.ends_with(".pack") {
928 +
                self.remove_pack(key)?;
929 +
                report.packs += 1;
930 +
            } else if is_entry_list(key) {
931 +
                // An entry list a fold replaced (spec §13.7).
932 +
                delete(self.backend.as_ref(), key)?;
933 +
            } else if let Some(r) = loose_ref(key) {
934 +
                if self.find_packed(&r).is_some() {
935 +
                    delete(self.backend.as_ref(), key)?;
936 +
                    report.loose += 1;
937 +
                }
521 938
            }
522 939
        }
523 -
        *self.packs.borrow_mut() = packs;
524 -
        Ok(())
940 +
        self.update_retired(|m| {
941 +
            for (k, t) in &due {
942 +
                if m.get(k) == Some(t) {
943 +
                    m.remove(k);
944 +
                }
945 +
            }
946 +
        })?;
947 +
        self.reload_packs()?;
948 +
        Ok(report)
949 +
    }
950 +
951 +
    /// The loose objects retired and not yet deleted, by key.
952 +
    fn retired_loose(&self) -> Result<std::collections::HashSet<String>> {
953 +
        let (listed, _) = self.list_packs()?;
954 +
        Ok(self
955 +
            .retired_in(&listed)?
956 +
            .into_keys()
957 +
            .filter(|k| k.starts_with("loose/"))
958 +
            .collect())
525 959
    }
526 960
527 961
    pub fn pack_count(&self) -> usize {
…
8 unchanged lines
536 970
            .find_map(|(i, p)| p.find(r).map(|e| (i, e)))
537 971
    }
538 972
539 -
    /// Loose objects, with their keys and ages.
973 +
    /// Loose objects, with their keys and ages; not those retired, whose
974 +
    /// objects a pack holds.
540 975
    fn loose_objects(&self) -> Result<Vec<(ObjectRef, strata::Entry)>> {
976 +
        let retired = self.retired_loose()?;
977 +
        let mut out = self.loose_objects_all()?;
978 +
        out.retain(|(_, e)| !retired.contains(&e.key));
979 +
        Ok(out)
980 +
    }
981 +
982 +
    /// Every loose object, retired or not.
983 +
    fn loose_objects_all(&self) -> Result<Vec<(ObjectRef, strata::Entry)>> {
541 984
        let mut out: Vec<(ObjectRef, strata::Entry)> = self
542 985
            .backend
543 986
            .list("loose/")
…
34 unchanged lines
578 1021
    /// [`Store::repack_all`], reporting progress.
579 1022
    pub fn repack_all_with(&self, level: i32, progress: Progress) -> Result<(PackReport, usize)> {
580 1023
        let loose = self.loose_objects()?;
581 -
        let (mut refs, keys): (Vec<ObjectRef>, Vec<String>) = {
1024 +
        let (mut refs, keys, trained): (Vec<ObjectRef>, Vec<String>, bool) = {
582 1025
            let packs = self.packs.borrow();
583 1026
            (
584 1027
                packs.iter().flat_map(|p| p.refs.iter().copied()).collect(),
585 1028
                packs.iter().map(|p| p.key.clone()).collect(),
1029 +
                packs.first().is_some_and(|p| p.dict.is_some()),
586 1030
            )
587 1031
        };
588 -
        if loose.is_empty() && keys.len() <= 1 && level == ZSTD_LEVEL {
1032 +
        // Nothing loose and one pack already made by a full repack (only
1033 +
        // such a pack has a trained dictionary): rewriting it would give
1034 +
        // the same pack. A hub's nightly gc rewrote andymccall/oxygen's
1035 +
        // 48 000 objects (37 s under its write lock) every time.
1036 +
        if loose.is_empty() && keys.len() <= 1 && (level == ZSTD_LEVEL || trained) {
589 1037
            return Ok((PackReport::default(), 0));
590 1038
        }
591 1039
        refs.extend(loose.iter().map(|(r, _)| *r));
…
48 unchanged lines
640 1088
        let old_enough =
641 1089
            |modified: Option<u64>| modified.is_none_or(|t| now.saturating_sub(t) >= grace_ms);
642 1090
        let mut report = PruneReport::default();
643 -
        // Loose objects.
644 -
        for (r, entry) in self.loose_objects()? {
1091 +
        let mut gone: Vec<String> = Vec::new();
1092 +
        // Loose objects. A retired one (its object packed) whose object is
1093 +
        // unreachable goes with it, uncounted: the pack's copy is counted.
1094 +
        let retired = self.retired_loose()?;
1095 +
        let mut unretired: Vec<String> = Vec::new();
1096 +
        for (r, entry) in self.loose_objects_all()? {
645 1097
            if keep.contains(&r) {
646 1098
                continue;
647 1099
            }
1100 +
            if retired.contains(&entry.key) {
1101 +
                if old_enough(entry.modified_ms) {
1102 +
                    delete(self.backend.as_ref(), &entry.key)?;
1103 +
                    unretired.push(entry.key);
1104 +
                }
1105 +
                continue;
1106 +
            }
648 1107
            if !old_enough(entry.modified_ms) {
649 1108
                report.deferred += 1;
650 1109
                continue;
…
1 unchanged lines
652 1111
            report.bytes += entry.size;
653 1112
            report.objects += 1;
654 1113
            delete(self.backend.as_ref(), &entry.key)?;
655 -
            self.remove_sidecars(&r)?;
1114 +
            self.remove_sidecars(&r, &mut gone)?;
656 1115
        }
1116 +
        if !unretired.is_empty() {
1117 +
            self.update_retired(|m| {
1118 +
                for k in &unretired {
1119 +
                    m.remove(k);
1120 +
                }
1121 +
            })?;
1122 +
        }
657 1123
        // Packs: rewrite those that hold unreachable objects.
658 1124
        let mut keep_refs: Vec<ObjectRef> = Vec::new();
659 1125
        let mut old_keys: Vec<String> = Vec::new();
…
36 unchanged lines
696 1162
            };
697 1163
            self.write_pack(&keep_refs, &[], &old_keys, ZSTD_LEVEL, progress)?;
698 1164
            for r in dropped {
699 -
                self.remove_sidecars(&r)?;
1165 +
                self.remove_sidecars(&r, &mut gone)?;
700 1166
            }
701 1167
        }
1168 +
        self.journal_forget(&gone)?;
702 1169
        Ok(report)
703 1170
    }
704 1171
705 1172
    /// Everything kept beside an object: its signatures and the seal and
706 1173
    /// receipt entries under it, in both layouts.
707 -
    fn remove_sidecars(&self, r: &ObjectRef) -> Result<()> {
1174 +
    /// The entry keys deleted are added to `gone`, for the journal.
1175 +
    fn remove_sidecars(&self, r: &ObjectRef, gone: &mut Vec<String>) -> Result<()> {
708 1176
        for index in ["sigs", "seals", "receipts"] {
709 1177
            delete(self.backend.as_ref(), &legacy_key(index, r))?;
710 1178
        }
…
4 unchanged lines
715 1183
                .map_err(backend_err)?
716 1184
            {
717 1185
                delete(self.backend.as_ref(), &e.key)?;
1186 +
                gone.push(e.key);
718 1187
            }
719 1188
        }
720 1189
        Ok(())
721 1190
    }
722 1191
1192 +
    /// Take entry keys a collection deleted out of the journal, so the
1193 +
    /// next complete fold leaves them out (spec §13.7).
1194 +
    fn journal_forget(&self, gone: &[String]) -> Result<()> {
1195 +
        if self.local || gone.is_empty() {
1196 +
            return Ok(());
1197 +
        }
1198 +
        if let Some(set) = self.entries.borrow_mut().as_mut() {
1199 +
            for k in gone {
1200 +
                set.remove(k);
1201 +
            }
1202 +
        }
1203 +
        self.update_record(JOURNAL_KEY, "the journal of entries", |m| {
1204 +
            for k in gone {
1205 +
                m.remove(k);
1206 +
            }
1207 +
        })
1208 +
    }
1209 +
723 1210
    /// Put a complete, synced temporary pack file and its index into the
724 1211
    /// store; `index` must be sorted by ObjectRef. The pack goes first: an
725 1212
    /// index never names a pack that is not there.
…
27 unchanged lines
753 1240
        self.reload_packs()
754 1241
    }
755 1242
756 -
    /// Remove a pack with its index and dictionary.
1243 +
    /// Remove a pack with its index and dictionary: the index first, so
1244 +
    /// a store opened meanwhile does not load a pack that is going.
757 1245
    fn remove_pack(&self, pack_key: &str) -> Result<()> {
758 1246
        let stem = pack_key.trim_end_matches(".pack");
1247 +
        delete(self.backend.as_ref(), &format!("{stem}.idx"))?;
759 1248
        delete(self.backend.as_ref(), pack_key)?;
760 -
        delete(self.backend.as_ref(), &format!("{stem}.idx"))?;
761 1249
        delete(self.backend.as_ref(), &format!("{stem}.dict"))
762 1250
    }
763 1251
764 -
    /// Write `refs` into a new pack, then delete `loose_keys` and `old_packs`.
1252 +
    /// Write `refs` into a new pack, then retire `loose_keys` and
1253 +
    /// `old_packs`: they stay in the store, unused by stores opened from
1254 +
    /// now on, until a collection [`RETIRED_FOR`] later deletes them, so a
1255 +
    /// reader that loaded its packs before goes on reading (oxygenhub#249).
765 1256
    fn write_pack(
766 1257
        &self,
767 1258
        refs: &[ObjectRef],
…
3 unchanged lines
771 1262
        progress: Progress,
772 1263
    ) -> Result<PackReport> {
773 1264
        if refs.is_empty() {
774 -
            for p in old_packs {
775 -
                self.remove_pack(p)?;
776 -
            }
1265 +
            self.retire(old_packs)?;
777 1266
            self.reload_packs()?;
778 1267
            return Ok(PackReport::default());
779 1268
        }
…
115 unchanged lines
895 1384
        }
896 1385
        progress(&format!("writing the index of {} objects", index.len()));
897 1386
        self.finish_pack_files(&id, &tmp_pack, &index)?;
898 -
        for p in old_packs {
899 -
            self.remove_pack(p)?;
900 -
        }
1387 +
        let mut retired = old_packs.to_vec();
1388 +
        retired.extend_from_slice(loose_keys);
1389 +
        self.retire(&retired)?;
901 1390
        self.reload_packs()?;
902 -
        for key in loose_keys {
903 -
            delete(self.backend.as_ref(), key)?;
904 -
        }
905 1391
        Ok(report)
906 1392
    }
907 1393
…
42 unchanged lines
950 1436
        delete(self.backend.as_ref(), &loose_key(r))
951 1437
    }
952 1438
1439 +
    /// Whether the store holds `r`. A miss looks at the pack listing
1440 +
    /// again before it is believed, as [`Store::get`]'s does; a writer
1441 +
    /// asking about many objects it is about to store looks at most once
1442 +
    /// in [`HAS_RELIST_AFTER`] while nothing new turns up.
953 1443
    pub fn has(&self, r: &ObjectRef) -> bool {
954 -
        self.backend.has(&loose_key(r)).unwrap_or(false)
955 -
            || self.find_packed(r).is_some()
1444 +
        if self.has_now(r) {
1445 +
            return true;
1446 +
        }
1447 +
        let recent = self
1448 +
            .has_relisted
1449 +
            .get()
1450 +
            .is_some_and(|t| t.elapsed() < HAS_RELIST_AFTER);
1451 +
        if recent {
1452 +
            return false;
1453 +
        }
1454 +
        match self.reload_if_changed() {
1455 +
            Ok(true) => {
1456 +
                self.has_relisted.set(None);
1457 +
                self.has_now(r)
1458 +
            }
1459 +
            _ => {
1460 +
                self.has_relisted.set(Some(std::time::Instant::now()));
1461 +
                false
1462 +
            }
1463 +
        }
1464 +
    }
1465 +
1466 +
    /// [`Store::has`] against the packs as loaded.
1467 +
    fn has_now(&self, r: &ObjectRef) -> bool {
1468 +
        self.find_packed(r).is_some()
956 1469
            || self
957 1470
                .pending
958 1471
                .borrow()
959 1472
                .as_ref()
960 1473
                .is_some_and(|p| p.lookup.contains_key(r))
1474 +
            || self.backend.has(&loose_key(r)).unwrap_or(false)
961 1475
    }
962 1476
963 1477
    /// Start a write batch: until the matching `finish`, `put` collects
…
28 unchanged lines
992 1506
            return Ok(());
993 1507
        }
994 1508
        let Some(pending) = self.pending.borrow_mut().take() else {
995 -
            return Ok(());
1509 +
            return self.flush_journal();
996 1510
        };
997 -
        self.write_pending(pending)
1511 +
        self.write_pending(pending)?;
1512 +
        self.flush_journal()
998 1513
    }
999 1514
1000 1515
    /// Make every object written so far durable, even inside an open
…
1 unchanged lines
1002 1517
    /// is moved (spec §13.6): the operation head, a bookmark pointer.
1003 1518
    pub fn flush(&self) -> Result<()> {
1004 1519
        let Some(pending) = self.pending.borrow_mut().take() else {
1005 -
            return Ok(());
1520 +
            return self.flush_journal();
1006 1521
        };
1007 1522
        self.write_pending(pending)?;
1523 +
        self.flush_journal()?;
1008 1524
        *self.pending.borrow_mut() = Some(Pending {
1009 1525
            buffered: vec![],
1010 1526
            buffered_bytes: 0,
…
259 unchanged lines
1270 1786
        self.backend.put_many_new(&items).map_err(backend_err)
1271 1787
    }
1272 1788
1273 -
    /// Read and verify an object, loose, packed, or pending in a batch.
1789 +
    /// Read and verify an object, packed, loose, or pending in a batch.
1790 +
    /// The packs' indexes are in memory, so they are asked first: asking
1791 +
    /// for a loose copy of a packed object costs a request that finds
1792 +
    /// nothing on a store in a bucket, every time (oxygenhub#228).
1793 +
    ///
1794 +
    /// A miss is not believed at once (oxygenhub#249): when the pack it
1795 +
    /// was in is gone (deleted after a repack retired it) the packs are
1796 +
    /// loaded again, and when the object is nowhere the pack listing is
1797 +
    /// looked at, and the packs loaded again if it changed (another
1798 +
    /// writer packed it, or a repack moved it); then it is read once more.
1274 1799
    pub fn get(&self, r: &ObjectRef) -> Result<Object> {
1800 +
        match self.get_now(r) {
1801 +
            Err(e) if is_miss(&e) => {
1802 +
                // Read from a pack this store loaded, which is gone.
1803 +
                let packed_miss =
1804 +
                    !matches!(e, StoreError::NotFound(_)) || self.find_packed(r).is_some();
1805 +
                let changed = if packed_miss {
1806 +
                    self.reload_packs()?;
1807 +
                    true
1808 +
                } else {
1809 +
                    self.reload_if_changed()?
1810 +
                };
1811 +
                if changed {
1812 +
                    self.get_now(r)
1813 +
                } else {
1814 +
                    Err(e)
1815 +
                }
1816 +
            }
1817 +
            other => other,
1818 +
        }
1819 +
    }
1820 +
1821 +
    /// [`Store::get`] against the packs as loaded.
1822 +
    fn get_now(&self, r: &ObjectRef) -> Result<Object> {
1823 +
        if let Some((i, e)) = self.find_packed(r) {
1824 +
            let pack = self.packs.borrow()[i].clone();
1825 +
            return pack.fetch(self.backend.as_ref(), r, e);
1826 +
        }
1275 1827
        match get_optional(self.backend.as_ref(), &loose_key(r))? {
1276 1828
            Some(bytes) => parse_loose(&bytes, r),
1277 -
            None => match self.find_packed(r) {
1278 -
                Some((i, e)) => self.packs.borrow()[i].fetch(self.backend.as_ref(), r, e),
1279 -
                None => self.pending_get(r).unwrap_or(Err(StoreError::NotFound(*r))),
1280 -
            },
1829 +
            None => self.pending_get(r).unwrap_or(Err(StoreError::NotFound(*r))),
1281 1830
        }
1282 1831
    }
1283 1832
1284 -
    /// Many objects at once: loose ones are read and verified in parallel,
1285 -
    /// then packed ones in parallel, then what is pending in a batch.
1286 -
    /// Results are in the order of `refs`.
1833 +
    /// Many objects at once, read and verified in parallel: packed ones
1834 +
    /// from their packs, the others loose (as [`Store::get`], packs first),
1835 +
    /// then what is pending in a batch. Results are in the order of `refs`.
1836 +
    ///
1837 +
    /// Misses are looked for again as [`Store::get`]'s are, together.
1287 1838
    pub fn get_many(&self, refs: &[ObjectRef]) -> Vec<Result<Object>> {
1839 +
        let mut out = self.get_many_now(refs);
1840 +
        let missed: Vec<usize> = out
1841 +
            .iter()
1842 +
            .enumerate()
1843 +
            .filter(|(_, g)| g.as_ref().is_err_and(is_miss))
1844 +
            .map(|(i, _)| i)
1845 +
            .collect();
1846 +
        if missed.is_empty() {
1847 +
            return out;
1848 +
        }
1849 +
        let packed_miss = missed.iter().any(|i| {
1850 +
            !matches!(out[*i], Err(StoreError::NotFound(_)))
1851 +
                || self.find_packed(&refs[*i]).is_some()
1852 +
        });
1853 +
        let changed = if packed_miss {
1854 +
            self.reload_packs().map(|_| true)
1855 +
        } else {
1856 +
            self.reload_if_changed()
1857 +
        };
1858 +
        match changed {
1859 +
            Ok(true) => {
1860 +
                let again: Vec<ObjectRef> = missed.iter().map(|i| refs[*i]).collect();
1861 +
                for (i, g) in missed.into_iter().zip(self.get_many_now(&again)) {
1862 +
                    out[i] = g;
1863 +
                }
1864 +
            }
1865 +
            Ok(false) => {}
1866 +
            Err(e) => {
1867 +
                let why = e.to_string();
1868 +
                for i in missed {
1869 +
                    out[i] = Err(StoreError::Io(io::Error::other(why.clone())));
1870 +
                }
1871 +
            }
1872 +
        }
1873 +
        out
1874 +
    }
1875 +
1876 +
    /// [`Store::get_many`] against the packs as loaded.
1877 +
    fn get_many_now(&self, refs: &[ObjectRef]) -> Vec<Result<Object>> {
1288 1878
        use rayon::prelude::*;
1289 1879
        let backend = self.backend.as_ref();
1290 -
        let mut found: Vec<Option<Result<Object>>> = refs
1291 -
            .par_iter()
1292 -
            .map(|r| match get_optional(backend, &loose_key(r)) {
1293 -
                Ok(Some(bytes)) => Some(parse_loose(&bytes, r)),
1294 -
                Ok(None) => None,
1295 -
                Err(e) => Some(Err(e)),
1296 -
            })
1297 -
            .collect();
1298 -
        // Packed ones: look them up in order, then read them in parallel.
1299 1880
        let packs = self.packs.borrow();
1300 -
        let packed: Vec<(usize, usize, PackEntry)> = found
1881 +
        let place: Vec<Option<(usize, PackEntry)>> = refs
1301 1882
            .iter()
1302 -
            .enumerate()
1303 -
            .filter(|(_, f)| f.is_none())
1304 -
            .filter_map(|(i, _)| {
1883 +
            .map(|r| {
1305 1884
                packs
1306 1885
                    .iter()
1307 1886
                    .enumerate()
1308 -
                    .find_map(|(p, pack)| pack.find(&refs[i]).map(|e| (i, p, e)))
1887 +
                    .find_map(|(p, pack)| pack.find(r).map(|e| (p, e)))
1309 1888
            })
1310 1889
            .collect();
1311 -
        let indexes: &[PackIndex] = &packs;
1312 -
        let read: Vec<(usize, Result<Object>)> = packed
1890 +
        let indexes: &[Arc<PackIndex>] = &packs;
1891 +
        // Read in (pack, offset) order, each thread a run of it, so a
1892 +
        // spinning disk sweeps rather than seeks (oxygen#137: a clone's
1893 +
        // checkpoint reads tens of thousands of objects across ~95 packs).
1894 +
        let order = read_order(&place);
1895 +
        let fetched: Vec<(usize, Option<Result<Object>>)> = order
1313 1896
            .par_iter()
1314 -
            .map(|(i, p, e)| (*i, indexes[*p].fetch(backend, &refs[*i], *e)))
1897 +
            .map(|&i| {
1898 +
                let r = &refs[i];
1899 +
                let got = match &place[i] {
1900 +
                    Some((p, e)) => Some(indexes[*p].fetch(backend, r, *e)),
1901 +
                    None => match get_optional(backend, &loose_key(r)) {
1902 +
                        Ok(Some(bytes)) => Some(parse_loose(&bytes, r)),
1903 +
                        Ok(None) => None,
1904 +
                        Err(e) => Some(Err(e)),
1905 +
                    },
1906 +
                };
1907 +
                (i, got)
1908 +
            })
1315 1909
            .collect();
1316 1910
        drop(packs);
1317 -
        for (i, r) in read {
1318 -
            found[i] = Some(r);
1911 +
        let mut found: Vec<Option<Result<Object>>> = (0..refs.len()).map(|_| None).collect();
1912 +
        for (i, got) in fetched {
1913 +
            found[i] = got;
1319 1914
        }
1320 1915
        found
1321 1916
            .into_iter()
1322 1917
            .zip(refs)
1323 -
            .map(|(f, r)| match f {
1324 -
                Some(f) => f,
1325 -
                None => self.get(r),
1918 +
            .map(|(f, r)| {
1919 +
                f.unwrap_or_else(|| self.pending_get(r).unwrap_or(Err(StoreError::NotFound(*r))))
1326 1920
            })
1327 1921
            .collect()
1328 1922
    }
…
244 unchanged lines
1573 2167
    /// Record `value` under `key` in `index`: one write-once entry, so two
1574 2168
    /// writers of a shared store never lose each other's.
1575 2169
    fn index_add(&self, index: &str, key: &ObjectRef, value: &ObjectRef) -> Result<()> {
2170 +
        debug_assert!(ENTRY_INDEXES.iter().any(|(i, _)| *i == index));
1576 2171
        let entry = format!("{}{}", fanout_prefix(index, key), value.to_hex());
1577 -
        put_new_or_existing(self.backend.as_ref(), &entry, b"")
2172 +
        put_new_or_existing(self.backend.as_ref(), &entry, b"")?;
2173 +
        self.journal_add(entry)
1578 2174
    }
1579 2175
1580 2176
    /// The values recorded under `key` in `index`, with those an older
1581 2177
    /// store kept as one list in `legacy`, sorted.
1582 2178
    fn index_get(&self, index: &str, legacy: &str, key: &ObjectRef) -> Result<Vec<ObjectRef>> {
1583 2179
        let mut values: Vec<ObjectRef> = self
1584 -
            .backend
1585 -
            .list(&fanout_prefix(index, key))
1586 -
            .map_err(backend_err)?
2180 +
            .entry_keys(&fanout_prefix(index, key))?
1587 2181
            .into_iter()
1588 -
            .filter_map(|e| e.key.rsplit('/').next().and_then(ObjectRef::from_hex))
2182 +
            .filter_map(|k| k.rsplit('/').next().and_then(ObjectRef::from_hex))
1589 2183
            .collect();
1590 -
        if let Some(bytes) = get_optional(self.backend.as_ref(), &legacy_key(legacy, key))? {
2184 +
        if !self.reads_legacy() {
2185 +
            // Older lists folded into entries (spec §13.7).
2186 +
        } else if let Some(bytes) = get_optional(self.backend.as_ref(), &legacy_key(legacy, key))? {
1591 2187
            values.extend(
1592 2188
                String::from_utf8_lossy(&bytes)
1593 2189
                    .lines()
…
16 unchanged lines
1610 2206
    }
1611 2207
1612 2208
    /// Record receipt attestation `attest` for `state` (spec §14.6).
2209 +
    /// Keep a short-lived key's certificate (spec §14.7), with its issuer's
2210 +
    /// signature, indexed by the key it certifies.
2211 +
    pub fn put_certificate(&self, c: &crate::cert::SignedCertificate) -> Result<ObjectRef> {
2212 +
        let r = self.put(ObjectType::Certificate, &c.cert.encode())?;
2213 +
        self.put_signature(&r, &c.sig)?;
2214 +
        self.index_add("cert", &c.cert.key_id(), &r)?;
2215 +
        Ok(r)
2216 +
    }
2217 +
2218 +
    /// A kept certificate with its issuer's signature.
2219 +
    pub fn get_certificate(&self, r: &ObjectRef) -> Result<crate::cert::SignedCertificate> {
2220 +
        let cert = crate::cert::Certificate::decode(&self.get_typed(r, ObjectType::Certificate)?)?;
2221 +
        let sig = self
2222 +
            .signatures(r)?
2223 +
            .into_iter()
2224 +
            .find(|s| s.key == cert.issuer)
2225 +
            .ok_or(StoreError::NotFound(*r))?;
2226 +
        Ok(crate::cert::SignedCertificate { cert, sig })
2227 +
    }
2228 +
2229 +
    /// The certificate kept for short-lived key `key`, if any.
2230 +
    pub fn certificate_of(&self, key: &ObjectRef) -> Result<Option<ObjectRef>> {
2231 +
        Ok(self
2232 +
            .index_get("cert", "certs", key)?
2233 +
            .into_iter()
2234 +
            .find(|r| self.has(r)))
2235 +
    }
2236 +
2237 +
    /// Record `value` under `key` in index `index` (for repository code).
2238 +
    pub(crate) fn index_add_pub(
2239 +
        &self,
2240 +
        index: &str,
2241 +
        key: &ObjectRef,
2242 +
        value: &ObjectRef,
2243 +
    ) -> Result<()> {
2244 +
        self.index_add(index, key, value)
2245 +
    }
2246 +
2247 +
    /// The values recorded under `key` in index `index` (which has no
2248 +
    /// older one-list form: the legacy name only has to differ from it).
2249 +
    pub(crate) fn index_get_pub(&self, index: &str, key: &ObjectRef) -> Result<Vec<ObjectRef>> {
2250 +
        self.index_get(index, &format!("{index}-lists"), key)
2251 +
    }
2252 +
1613 2253
    pub fn index_receipt(&self, state: &ObjectRef, attest: &ObjectRef) -> Result<()> {
1614 2254
        self.index_add("receipt", state, attest)
1615 2255
    }
…
3 unchanged lines
1619 2259
        self.index_get("receipt", "receipts", state)
1620 2260
    }
1621 2261
2262 +
    pub fn index_approval(&self, state: &ObjectRef, attest: &ObjectRef) -> Result<()> {
2263 +
        self.index_add("approval", state, attest)
2264 +
    }
2265 +
2266 +
    /// Approval attestations recorded for `state` (OxygenFlow approval
2267 +
    /// gates, oxygenhub#235).
2268 +
    pub fn approvals_for(&self, state: &ObjectRef) -> Result<Vec<ObjectRef>> {
2269 +
        self.index_get("approval", "approvals", state)
2270 +
    }
2271 +
2272 +
    /// Every state with an approval, for garbage collection.
2273 +
    pub fn approval_states(&self) -> Result<Vec<ObjectRef>> {
2274 +
        self.indexed_states("approval", "approvals")
2275 +
    }
2276 +
1622 2277
    /// Every state with a receipt, for garbage collection.
1623 2278
    pub fn receipt_states(&self) -> Result<Vec<ObjectRef>> {
2279 +
        self.indexed_states("receipt", "receipts")
2280 +
    }
2281 +
2282 +
    /// The states an index (`one/<hh>/<rest>/<attest>` entries, and the
2283 +
    /// packed `many/<hh>/<rest>` lists) holds attestations for.
2284 +
    fn indexed_states(&self, one: &str, many: &str) -> Result<Vec<ObjectRef>> {
1624 2285
        let mut out: Vec<ObjectRef> = Vec::new();
1625 -
        for e in self.backend.list("receipt/").map_err(backend_err)? {
2286 +
        for key in self.entry_keys(&format!("{one}/"))? {
1626 2287
            // receipt/<hh>/<rest>/<attest>
1627 -
            let mut parts = e.key.splitn(4, '/');
2288 +
            let mut parts = key.splitn(4, '/');
1628 2289
            let (Some(_), Some(hh), Some(rest), Some(_)) =
1629 2290
                (parts.next(), parts.next(), parts.next(), parts.next())
1630 2291
            else {
…
1 unchanged lines
1632 2293
            };
1633 2294
            out.extend(ObjectRef::from_hex(&format!("{hh}{rest}")));
1634 2295
        }
1635 -
        for e in self.backend.list("receipts/").map_err(backend_err)? {
2296 +
        let legacy = if self.reads_legacy() {
2297 +
            self.backend
2298 +
                .list(&format!("{many}/"))
2299 +
                .map_err(backend_err)?
2300 +
        } else {
2301 +
            Vec::new()
2302 +
        };
2303 +
        for e in legacy {
1636 2304
            let mut parts = e.key.splitn(3, '/');
1637 2305
            let (Some(_), Some(hh), Some(rest)) = (parts.next(), parts.next(), parts.next()) else {
1638 2306
                continue;
…
17 unchanged lines
1656 2324
    /// key, sorted by key. Empty if none.
1657 2325
    pub fn signatures(&self, r: &ObjectRef) -> Result<Vec<Signature>> {
1658 2326
        let mut sigs: Vec<Signature> = Vec::new();
1659 -
        for e in self
1660 -
            .backend
1661 -
            .list(&fanout_prefix("sig", r))
1662 -
            .map_err(backend_err)?
1663 -
        {
1664 -
            let bytes = self.backend.get(&e.key).map_err(backend_err)?;
2327 +
        for key in self.entry_keys(&fanout_prefix("sig", r))? {
2328 +
            // Gone since it was listed or journalled: a collection deleted
2329 +
            // it with its object, or a new signature by the same signer is
2330 +
            // replacing it.
2331 +
            let Some(bytes) = get_optional(self.backend.as_ref(), &key)? else {
2332 +
                continue;
2333 +
            };
1665 2334
            sigs.extend(Signature::decode_list(&bytes)?);
1666 2335
        }
1667 2336
        // An older store's one list per object; a signer kept there and
1668 2337
        // here counts once, as written here.
1669 -
        if let Some(bytes) = get_optional(self.backend.as_ref(), &legacy_key("sigs", r))? {
2338 +
        let legacy = if self.reads_legacy() {
2339 +
            get_optional(self.backend.as_ref(), &legacy_key("sigs", r))?
2340 +
        } else {
2341 +
            None
2342 +
        };
2343 +
        if let Some(bytes) = legacy {
1670 2344
            for sig in Signature::decode_list(&bytes)? {
1671 2345
                if !sigs.iter().any(|s| s.key == sig.key) {
1672 2346
                    sigs.push(sig);
…
11 unchanged lines
1684 2358
    pub fn put_signature(&self, r: &ObjectRef, sig: &Signature) -> Result<()> {
1685 2359
        let key = format!("{}{}", fanout_prefix("sig", r), sig.key.to_hex());
1686 2360
        let bytes = Signature::encode_list(std::slice::from_ref(sig));
1687 -
        if get_optional(self.backend.as_ref(), &key)?.as_deref() == Some(bytes.as_slice()) {
2361 +
        if get_optional(self.backend.as_ref(), &key)?.as_deref() != Some(bytes.as_slice()) {
2362 +
            delete(self.backend.as_ref(), &key)?;
2363 +
            put_new_or_existing(self.backend.as_ref(), &key, &bytes)?;
2364 +
        }
2365 +
        // Journalled even when the key was there: an older writer's
2366 +
        // entry is not in the journal or any list.
2367 +
        self.journal_add(key)
2368 +
    }
2369 +
2370 +
    // ---- the journal and entry lists (spec §13.7, #116)
2371 +
2372 +
    /// Whether lookups still read the one-list keys of older stores: until
2373 +
    /// a complete fold has copied them into per-object entries and the
2374 +
    /// pack listing shows its marker (spec §13.7).
2375 +
    fn reads_legacy(&self) -> bool {
2376 +
        !self.legacy_folded.get()
2377 +
    }
2378 +
2379 +
    /// Copy every older one-list key's contents into per-object entries,
2380 +
    /// keeping an entry already there, then write the marker that tells
2381 +
    /// readers to stop reading those keys. The keys themselves are left
2382 +
    /// for older readers. A list that cannot be read is passed over, and
2383 +
    /// said.
2384 +
    fn fold_legacy(&self, progress: Progress) -> Result<()> {
2385 +
        let mut copied = 0usize;
2386 +
        for (index, legacy) in ENTRY_INDEXES {
2387 +
            for e in self
2388 +
                .backend
2389 +
                .list(&format!("{legacy}/"))
2390 +
                .map_err(backend_err)?
2391 +
            {
2392 +
                // <legacy>/<hh>/<rest>
2393 +
                let mut parts = e.key.splitn(3, '/');
2394 +
                let (Some(_), Some(hh), Some(rest)) = (parts.next(), parts.next(), parts.next())
2395 +
                else {
2396 +
                    continue;
2397 +
                };
2398 +
                let Some(r) = ObjectRef::from_hex(&format!("{hh}{rest}")) else {
2399 +
                    continue;
2400 +
                };
2401 +
                let Some(bytes) = get_optional(self.backend.as_ref(), &e.key)? else {
2402 +
                    continue;
2403 +
                };
2404 +
                if index == "sig" {
2405 +
                    let Ok(sigs) = Signature::decode_list(&bytes) else {
2406 +
                        progress(&format!("{}: not a list of signatures, passed over", e.key));
2407 +
                        continue;
2408 +
                    };
2409 +
                    for sig in sigs {
2410 +
                        let key = format!("{}{}", fanout_prefix("sig", &r), sig.key.to_hex());
2411 +
                        let one = Signature::encode_list(std::slice::from_ref(&sig));
2412 +
                        put_new_or_existing(self.backend.as_ref(), &key, &one)?;
2413 +
                        copied += 1;
2414 +
                    }
2415 +
                } else {
2416 +
                    for v in String::from_utf8_lossy(&bytes)
2417 +
                        .lines()
2418 +
                        .filter_map(ObjectRef::from_hex)
2419 +
                    {
2420 +
                        let key = format!("{}{}", fanout_prefix(index, &r), v.to_hex());
2421 +
                        put_new_or_existing(self.backend.as_ref(), &key, b"")?;
2422 +
                        copied += 1;
2423 +
                    }
2424 +
                }
2425 +
            }
2426 +
        }
2427 +
        if copied > 0 {
2428 +
            progress(&format!("{copied} entries copied out of older lists"));
2429 +
        }
2430 +
        put_new_or_existing(
2431 +
            self.backend.as_ref(),
2432 +
            LEGACY_FOLDED_KEY,
2433 +
            format!("{}\n", now_ms()).as_bytes(),
2434 +
        )?;
2435 +
        self.legacy_folded.set(true);
2436 +
        Ok(())
2437 +
    }
2438 +
2439 +
    /// The entry keys under `prefix` (an index's `<index>/` or one
2440 +
    /// object's `<index>/<hh>/<rest>/`), sorted: on a folded store from
2441 +
    /// its lists and journal, with no request; otherwise by listing.
2442 +
    fn entry_keys(&self, prefix: &str) -> Result<Vec<String>> {
2443 +
        if let Some(keys) = self.with_entries(|set| {
2444 +
            set.range(prefix.to_string()..)
2445 +
                .take_while(|k| k.starts_with(prefix))
2446 +
                .cloned()
2447 +
                .collect::<Vec<String>>()
2448 +
        })? {
2449 +
            return Ok(keys);
2450 +
        }
2451 +
        Ok(self
2452 +
            .backend
2453 +
            .list(prefix)
2454 +
            .map_err(backend_err)?
2455 +
            .into_iter()
2456 +
            .map(|e| e.key)
2457 +
            .collect())
2458 +
    }
2459 +
2460 +
    /// `f` over a folded store's entry keys, read on first use; `None`
2461 +
    /// when the store is not folded (or is this disk's directory), and
2462 +
    /// lookups list.
2463 +
    fn with_entries<T>(
2464 +
        &self,
2465 +
        f: impl FnOnce(&std::collections::BTreeSet<String>) -> T,
2466 +
    ) -> Result<Option<T>> {
2467 +
        if self.local || !self.folded.get() {
2468 +
            return Ok(None);
2469 +
        }
2470 +
        // Read over the interval ago: list the packs again (a cloud
2471 +
        // client keeps that listing as long), loading what changed, and
2472 +
        // read the journal again, so another writer's entry is seen as
2473 +
        // its pack would be.
2474 +
        if self
2475 +
            .entries_read
2476 +
            .get()
2477 +
            .is_some_and(|t| t.elapsed() >= ENTRIES_FRESH)
2478 +
        {
2479 +
            *self.entries.borrow_mut() = None;
2480 +
            self.reload_if_changed()?;
2481 +
            if !self.folded.get() {
2482 +
                return Ok(None);
2483 +
            }
2484 +
        }
2485 +
        if self.entries.borrow().is_none() {
2486 +
            let Some(set) = self.read_entries()? else {
2487 +
                return Ok(None);
2488 +
            };
2489 +
            *self.entries.borrow_mut() = Some(set);
2490 +
        }
2491 +
        Ok(self.entries.borrow().as_ref().map(f))
2492 +
    }
2493 +
2494 +
    /// Every live entry list's keys and every journal line's, folded or
2495 +
    /// not, with what this handle has queued. A list deleted since the
2496 +
    /// packs were listed (retired over an hour ago) has the packs listed
2497 +
    /// again; `None` when one is still missing then, or the store is no
2498 +
    /// longer folded: lookups list.
2499 +
    fn read_entries(&self) -> Result<Option<std::collections::BTreeSet<String>>> {
2500 +
        'attempt: for attempt in 0..2 {
2501 +
            if !self.folded.get() {
2502 +
                return Ok(None);
2503 +
            }
2504 +
            let mut set = std::collections::BTreeSet::new();
2505 +
            let lists = self.entry_lists.borrow().clone();
2506 +
            self.lists_read
2507 +
                .borrow_mut()
2508 +
                .retain(|k, _| lists.contains(k));
2509 +
            for key in &lists {
2510 +
                let known = self.lists_read.borrow().get(key).cloned();
2511 +
                let keys = match known {
2512 +
                    Some(keys) => keys,
2513 +
                    None => match get_optional(self.backend.as_ref(), key)? {
2514 +
                        Some(bytes) => {
2515 +
                            let keys = Arc::new(decode_entry_list(key, &bytes)?);
2516 +
                            self.lists_read
2517 +
                                .borrow_mut()
2518 +
                                .insert(key.clone(), keys.clone());
2519 +
                            keys
2520 +
                        }
2521 +
                        None if attempt == 0 => {
2522 +
                            self.reload_packs()?;
2523 +
                            continue 'attempt;
2524 +
                        }
2525 +
                        None => return Ok(None),
2526 +
                    },
2527 +
                };
2528 +
                set.extend(keys.iter().cloned());
2529 +
            }
2530 +
            set.extend(self.read_record(JOURNAL_KEY)?.0.into_keys());
2531 +
            set.extend(self.journal_queue.borrow().iter().cloned());
2532 +
            self.entries_read.set(Some(std::time::Instant::now()));
2533 +
            return Ok(Some(set));
2534 +
        }
2535 +
        Ok(None)
2536 +
    }
2537 +
2538 +
    /// Add an entry key this handle wrote to the journal: at once, or
2539 +
    /// when the open batch is flushed. One the handle already knows from
2540 +
    /// the lists or the journal is not added again. Nothing on this
2541 +
    /// disk's directory.
2542 +
    fn journal_add(&self, key: String) -> Result<()> {
2543 +
        if self.local {
2544 +
            return Ok(());
2545 +
        }
2546 +
        if let Some(set) = self.entries.borrow_mut().as_mut() {
2547 +
            if !set.insert(key.clone()) {
2548 +
                return Ok(());
2549 +
            }
2550 +
        }
2551 +
        if self.batch_depth.get() > 0 {
2552 +
            self.journal_queue.borrow_mut().push(key);
2553 +
            return Ok(());
2554 +
        }
2555 +
        self.journal_append(std::slice::from_ref(&key))
2556 +
    }
2557 +
2558 +
    /// Add what the open batch queued to the journal.
2559 +
    fn flush_journal(&self) -> Result<()> {
2560 +
        let queued = std::mem::take(&mut *self.journal_queue.borrow_mut());
2561 +
        self.journal_append(&queued)
2562 +
    }
2563 +
2564 +
    /// Add `keys` to the journal, unfolded, in one compare-and-swap
2565 +
    /// update, dropping lines folded over an hour ago; past
2566 +
    /// [`JOURNAL_FOLD_AT`] unfolded lines, fold them into a partial list.
2567 +
    fn journal_append(&self, keys: &[String]) -> Result<()> {
2568 +
        if keys.is_empty() {
2569 +
            return Ok(());
2570 +
        }
2571 +
        let now = now_ms();
2572 +
        let mut unfolded = 0;
2573 +
        self.update_record(JOURNAL_KEY, "the journal of entries", |m| {
2574 +
            expire_folded(m, now);
2575 +
            for k in keys {
2576 +
                m.entry(k.clone()).or_insert(0);
2577 +
            }
2578 +
            unfolded = m.values().filter(|t| **t == 0).count();
2579 +
        })?;
2580 +
        if unfolded > JOURNAL_FOLD_AT {
2581 +
            self.fold_journal()?;
2582 +
        }
2583 +
        Ok(())
2584 +
    }
2585 +
2586 +
    /// Fold the journal's unfolded lines into a partial entry list, and
2587 +
    /// mark them folded. Lines added meanwhile stay unfolded.
2588 +
    fn fold_journal(&self) -> Result<()> {
2589 +
        let (journal, _) = self.read_record(JOURNAL_KEY)?;
2590 +
        let keys: Vec<String> = journal
2591 +
            .into_iter()
2592 +
            .filter(|(_, t)| *t == 0)
2593 +
            .map(|(k, _)| k)
2594 +
            .collect();
2595 +
        if keys.is_empty() {
1688 2596
            return Ok(());
1689 2597
        }
1690 -
        delete(self.backend.as_ref(), &key)?;
1691 -
        put_new_or_existing(self.backend.as_ref(), &key, &bytes)
2598 +
        self.put_entry_list(PART_SUFFIX, keys.iter())?;
2599 +
        self.mark_folded(&keys)
2600 +
    }
2601 +
2602 +
    /// Mark these journal lines folded now, if they are not yet; drop
2603 +
    /// those folded over an hour ago.
2604 +
    fn mark_folded(&self, keys: &[String]) -> Result<()> {
2605 +
        let now = now_ms();
2606 +
        self.update_record(JOURNAL_KEY, "the journal of entries", |m| {
2607 +
            expire_folded(m, now);
2608 +
            for k in keys {
2609 +
                if let Some(t) = m.get_mut(k) {
2610 +
                    if *t == 0 {
2611 +
                        *t = now;
2612 +
                    }
2613 +
                }
2614 +
            }
2615 +
        })
2616 +
    }
2617 +
2618 +
    /// Write a new entry list of `keys` (sorted), complete or partial by
2619 +
    /// `suffix`; its key.
2620 +
    fn put_entry_list<'a>(
2621 +
        &self,
2622 +
        suffix: &str,
2623 +
        keys: impl Iterator<Item = &'a String>,
2624 +
    ) -> Result<String> {
2625 +
        let key = format!("pack-{}{suffix}", fresh_id());
2626 +
        let bytes = encode_entry_list(keys)?;
2627 +
        self.backend.put_new(&key, &bytes).map_err(backend_err)?;
2628 +
        Ok(key)
2629 +
    }
2630 +
2631 +
    /// Fold every signature and index entry into one complete entry list
2632 +
    /// (spec §13.7), what a collection does last: first the older
2633 +
    /// one-list keys into per-object entries, once; then the journal's lines and
2634 +
    /// a listing of every entry prefix, written as a new list, every list
2635 +
    /// the packs' listing showed retired, and the journal lines read
2636 +
    /// marked folded. Nothing when the store is this disk's directory, or
2637 +
    /// when the one live list already holds exactly these entries and the
2638 +
    /// journal has nothing unfolded. Whether a list was written.
2639 +
    pub fn fold_entries(&self, progress: Progress) -> Result<bool> {
2640 +
        if self.local {
2641 +
            return Ok(false);
2642 +
        }
2643 +
        self.reload_packs()?;
2644 +
        if !self.legacy_folded.get() {
2645 +
            progress("folding older lists of signatures and index entries");
2646 +
            self.fold_legacy(progress)?;
2647 +
        }
2648 +
        let (journal, _) = self.read_record(JOURNAL_KEY)?;
2649 +
        progress("folding signatures and index entries");
2650 +
        let mut all: std::collections::BTreeSet<String> = journal.keys().cloned().collect();
2651 +
        for (index, _) in ENTRY_INDEXES {
2652 +
            for e in self
2653 +
                .backend
2654 +
                .list(&format!("{index}/"))
2655 +
                .map_err(backend_err)?
2656 +
            {
2657 +
                all.insert(e.key);
2658 +
            }
2659 +
        }
2660 +
        let lists = self.entry_lists.borrow().clone();
2661 +
        let unfolded: Vec<String> = journal
2662 +
            .iter()
2663 +
            .filter(|(_, t)| **t == 0)
2664 +
            .map(|(k, _)| k.clone())
2665 +
            .collect();
2666 +
        if unfolded.is_empty() && lists.len() == 1 && lists[0].ends_with(LIST_SUFFIX) {
2667 +
            if let Some(bytes) = get_optional(self.backend.as_ref(), &lists[0])? {
2668 +
                let held: std::collections::BTreeSet<String> =
2669 +
                    decode_entry_list(&lists[0], &bytes)?.into_iter().collect();
2670 +
                if held == all {
2671 +
                    return Ok(false);
2672 +
                }
2673 +
            }
2674 +
        }
2675 +
        self.put_entry_list(LIST_SUFFIX, all.iter())?;
2676 +
        self.retire(&lists)?;
2677 +
        if !unfolded.is_empty() {
2678 +
            self.mark_folded(&unfolded)?;
2679 +
        }
2680 +
        *self.entries.borrow_mut() = None;
2681 +
        self.reload_packs()?;
2682 +
        Ok(true)
1692 2683
    }
1693 2684
2685 +
    /// Finish packs a writer left unfinished: a batch writes its objects to
2686 +
    /// `.tmp-ID.pack` and only indexes it at the end, so a process that
2687 +
    /// stopped part way (an import killed after hours) leaves every object
2688 +
    /// in a pack nothing can read. Each such pack untouched for `min_age`
2689 +
    /// is read through: every whole entry checked against its reference,
2690 +
    /// a torn last one cut off, then indexed as a pack of the store.
2691 +
    /// Returns the packs and objects adopted.
2692 +
    pub fn adopt_unfinished_packs(
2693 +
        &self,
2694 +
        min_age: std::time::Duration,
2695 +
        progress: &dyn Fn(&str),
2696 +
    ) -> Result<(usize, usize)> {
2697 +
        use std::io::{BufReader, Read};
2698 +
        let mut packs = 0;
2699 +
        let mut objects = 0;
2700 +
        let Ok(dir) = fs::read_dir(&self.scratch) else {
2701 +
            return Ok((0, 0));
2702 +
        };
2703 +
        for entry in dir.flatten() {
2704 +
            let name = entry.file_name().to_string_lossy().to_string();
2705 +
            let Some(id) = name
2706 +
                .strip_prefix(".tmp-")
2707 +
                .and_then(|n| n.strip_suffix(".pack"))
2708 +
            else {
2709 +
                continue;
2710 +
            };
2711 +
            let path = entry.path();
2712 +
            let meta = fs::metadata(&path)?;
2713 +
            let age = meta
2714 +
                .modified()
2715 +
                .ok()
2716 +
                .and_then(|m| m.elapsed().ok())
2717 +
                .unwrap_or_default();
2718 +
            if age < min_age {
2719 +
                progress(&format!(
2720 +
                    "{name}: written to {}s ago, left alone",
2721 +
                    age.as_secs()
2722 +
                ));
2723 +
                continue;
2724 +
            }
2725 +
            let mut f = BufReader::with_capacity(1 << 20, fs::File::open(&path)?);
2726 +
            let mut magic = [0u8; 8];
2727 +
            if f.read_exact(&mut magic).is_err() || &magic != PACK_MAGIC {
2728 +
                progress(&format!("{name}: not a pack, left alone"));
2729 +
                continue;
2730 +
            }
2731 +
            let mut offset = PACK_MAGIC.len() as u64;
2732 +
            let mut index: Vec<(ObjectRef, PackEntry)> = Vec::new();
2733 +
            loop {
2734 +
                let mut head = [0u8; 42];
2735 +
                if f.read_exact(&mut head).is_err() {
2736 +
                    break;
2737 +
                }
2738 +
                let Some(ty) = ObjectType::ALL.get(head[0] as usize).copied() else {
2739 +
                    break;
2740 +
                };
2741 +
                let flags = head[1];
2742 +
                let stored_len = u32::from_le_bytes(head[2..6].try_into().unwrap());
2743 +
                let raw_len = u32::from_le_bytes(head[6..10].try_into().unwrap());
2744 +
                let r = ObjectRef(head[10..42].try_into().unwrap());
2745 +
                let mut stored = vec![0u8; stored_len as usize];
2746 +
                if flags & FLAG_DICT != 0 || f.read_exact(&mut stored).is_err() {
2747 +
                    break;
2748 +
                }
2749 +
                let e = PackEntry {
2750 +
                    offset: offset + 42,
2751 +
                    stored_len,
2752 +
                    raw_len,
2753 +
                    ty,
2754 +
                    flags,
2755 +
                };
2756 +
                if decode_entry(&r, e, stored, None).is_err() {
2757 +
                    break;
2758 +
                }
2759 +
                index.push((r, e));
2760 +
                offset += 42 + stored_len as u64;
2761 +
                if index.len().is_multiple_of(200_000) {
2762 +
                    progress(&format!(
2763 +
                        "{name}: {} objects checked, {} MiB",
2764 +
                        index.len(),
2765 +
                        offset >> 20
2766 +
                    ));
2767 +
                }
2768 +
            }
2769 +
            if index.is_empty() {
2770 +
                progress(&format!("{name}: no whole object, left alone"));
2771 +
                continue;
2772 +
            }
2773 +
            if offset < meta.len() {
2774 +
                progress(&format!(
2775 +
                    "{name}: {} torn bytes at the end cut off",
2776 +
                    meta.len() - offset
2777 +
                ));
2778 +
                fs::OpenOptions::new()
2779 +
                    .write(true)
2780 +
                    .open(&path)?
2781 +
                    .set_len(offset)?;
2782 +
            }
2783 +
            index.sort_by_key(|(r, _)| *r);
2784 +
            index.dedup_by_key(|(r, _)| *r);
2785 +
            objects += index.len();
2786 +
            packs += 1;
2787 +
            progress(&format!("{name}: {} objects indexed", index.len()));
2788 +
            self.finish_pack_files(id, &path, &index)?;
2789 +
        }
2790 +
        Ok((packs, objects))
2791 +
    }
2792 +
1694 2793
    /// Every object reference in the store, loose and packed (slow; for gc and tests).
1695 2794
    pub fn list(&self) -> io::Result<Vec<ObjectRef>> {
1696 2795
        let mut out: Vec<ObjectRef> = self
…
11 unchanged lines
1708 2807
    }
1709 2808
}
1710 2809
2810 +
/// Whether a read failed in a way a newer list of packs may cure: the
2811 +
/// object was not found, or its pack's dictionary is gone.
2812 +
fn is_miss(e: &StoreError) -> bool {
2813 +
    match e {
2814 +
        StoreError::NotFound(_) => true,
2815 +
        StoreError::Corrupt(_, why) => why == NO_DICT,
2816 +
        _ => false,
2817 +
    }
2818 +
}
2819 +
1711 2820
/// A loose object's bytes, verified against `r`.
2821 +
/// The order to read objects in: packed ones by pack then offset, loose
2822 +
/// ones after, each group keeping the asked order otherwise.
2823 +
fn read_order(place: &[Option<(usize, PackEntry)>]) -> Vec<usize> {
2824 +
    let mut order: Vec<usize> = (0..place.len()).collect();
2825 +
    order.sort_by_key(|&i| match &place[i] {
2826 +
        Some((p, e)) => (0, *p, e.offset),
2827 +
        None => (1, 0, 0),
2828 +
    });
2829 +
    order
2830 +
}
2831 +
1712 2832
fn parse_loose(data: &[u8], r: &ObjectRef) -> Result<Object> {
1713 2833
    let nul = data
1714 2834
        .iter()
…
180 unchanged lines
1895 3015
        for (r, b) in refs.iter().zip(&bodies) {
1896 3016
            assert_eq!(&again.get(r).unwrap().1, b);
1897 3017
        }
1898 -
        // A repack after that replaces the pack and its dictionary.
3018 +
        let packs = |dir: &Path| {
3019 +
            let mut v: Vec<String> = fs::read_dir(dir)
3020 +
                .unwrap()
3021 +
                .flatten()
3022 +
                .map(|e| e.file_name().to_string_lossy().into_owned())
3023 +
                .filter(|n| n.starts_with("pack-"))
3024 +
                .collect();
3025 +
            v.sort();
3026 +
            v
3027 +
        };
3028 +
        // With nothing new, a repack leaves that pack as it is: it would
3029 +
        // only write the same again.
3030 +
        let before = packs(dir.path());
3031 +
        let (report, merged) = again.repack_all(REPACK_ZSTD_LEVEL).unwrap();
3032 +
        assert_eq!((report.objects, merged), (0, 0));
3033 +
        assert_eq!(packs(dir.path()), before);
3034 +
        // One new object: a repack replaces the pack and its dictionary,
3035 +
        // retiring the old ones, which a collection an hour on deletes.
3036 +
        let extra = again.put(ObjectType::Chunk, b"one more\n").unwrap();
1899 3037
        again.repack_all(REPACK_ZSTD_LEVEL).unwrap();
1900 -
        let files: Vec<String> = fs::read_dir(dir.path())
1901 -
            .unwrap()
1902 -
            .flatten()
1903 -
            .map(|e| e.file_name().to_string_lossy().into_owned())
1904 -
            .filter(|n| n.starts_with("pack-"))
1905 -
            .collect();
1906 -
        assert_eq!(files.len(), 3, "one pack, index and dictionary: {files:?}");
3038 +
        let files = packs(dir.path());
3039 +
        assert_eq!(
3040 +
            files.len(),
3041 +
            7,
3042 +
            "two of pack, index and dictionary, and the retired record: {files:?}"
3043 +
        );
3044 +
        assert_eq!(again.pack_count(), 1, "the old pack is not read");
1907 3045
        assert_eq!(&again.get(&refs[7]).unwrap().1, &bodies[7]);
3046 +
        assert_eq!(again.get(&extra).unwrap().1, b"one more\n");
3047 +
        again
3048 +
            .expire_retired(std::time::Duration::ZERO, &mut silent())
3049 +
            .unwrap();
3050 +
        let files = packs(dir.path());
3051 +
        assert_eq!(files.len(), 4, "one pack, index, dictionary: {files:?}");
3052 +
        assert!(before
3053 +
            .iter()
3054 +
            .filter(|f| *f != RETIRED_KEY)
3055 +
            .all(|f| !files.contains(f)));
3056 +
        assert_eq!(&again.get(&refs[8]).unwrap().1, &bodies[8]);
1908 3057
    }
1909 3058
1910 3059
    #[test]
…
381 unchanged lines
2292 3441
    }
2293 3442
2294 3443
    #[test]
3444 +
    fn packed_objects_are_read_by_pack_then_offset() {
3445 +
        let at = |p: usize, offset: u64| {
3446 +
            Some((
3447 +
                p,
3448 +
                PackEntry {
3449 +
                    offset,
3450 +
                    stored_len: 1,
3451 +
                    raw_len: 1,
3452 +
                    ty: ObjectType::Chunk,
3453 +
                    flags: 0,
3454 +
                },
3455 +
            ))
3456 +
        };
3457 +
        let place = vec![at(1, 50), None, at(0, 900), at(1, 10), None, at(0, 5)];
3458 +
        assert_eq!(read_order(&place), [5, 2, 3, 0, 1, 4]);
3459 +
    }
3460 +
3461 +
    #[test]
2295 3462
    fn get_many_reads_loose_and_packed_objects_in_order() {
2296 3463
        let dir = tempfile::tempdir().unwrap();
2297 3464
        each_store(dir.path(), |which, s| {
…
35 unchanged lines
2333 3500
            assert!(matches!(got[40], Err(StoreError::NotFound(_))));
2334 3501
        });
2335 3502
    }
3503 +
3504 +
    /// A packed object is read from its pack without first asking the
3505 +
    /// backend for a loose copy (oxygenhub#228: on a bucket each such ask
3506 +
    /// was a request that found nothing, every time).
3507 +
    #[test]
3508 +
    fn packed_objects_cost_no_loose_lookups() {
3509 +
        use std::sync::atomic::{AtomicUsize, Ordering};
3510 +
        struct Counting(MemoryStore, AtomicUsize);
3511 +
        impl crate::strata::Store for Counting {
3512 +
            fn describe(&self) -> String {
3513 +
                "counting".into()
3514 +
            }
3515 +
            fn has(&self, k: &str) -> crate::strata::Result<bool> {
3516 +
                self.0.has(k)
3517 +
            }
3518 +
            fn get(&self, k: &str) -> crate::strata::Result<Vec<u8>> {
3519 +
                if k.starts_with("loose/") {
3520 +
                    self.1.fetch_add(1, Ordering::Relaxed);
3521 +
                }
3522 +
                self.0.get(k)
3523 +
            }
3524 +
            fn get_range(
3525 +
                &self,
3526 +
                k: &str,
3527 +
                r: std::ops::Range<u64>,
3528 +
            ) -> crate::strata::Result<Vec<u8>> {
3529 +
                self.0.get_range(k, r)
3530 +
            }
3531 +
            fn put_new(&self, k: &str, b: &[u8]) -> crate::strata::Result<()> {
3532 +
                self.0.put_new(k, b)
3533 +
            }
3534 +
            fn list(&self, p: &str) -> crate::strata::Result<Vec<crate::strata::Entry>> {
3535 +
                self.0.list(p)
3536 +
            }
3537 +
            fn delete(&self, k: &str) -> crate::strata::Result<()> {
3538 +
                self.0.delete(k)
3539 +
            }
3540 +
            fn read_ref(
3541 +
                &self,
3542 +
                k: &str,
3543 +
            ) -> crate::strata::Result<Option<(Vec<u8>, crate::strata::Version)>> {
3544 +
                self.0.read_ref(k)
3545 +
            }
3546 +
            fn write_ref(
3547 +
                &self,
3548 +
                k: &str,
3549 +
                b: &[u8],
3550 +
                e: Option<&crate::strata::Version>,
3551 +
            ) -> crate::strata::Result<crate::strata::Version> {
3552 +
                self.0.write_ref(k, b, e)
3553 +
            }
3554 +
        }
3555 +
        let dir = tempfile::tempdir().unwrap();
3556 +
        let backend = Arc::new(Counting(MemoryStore::new(), AtomicUsize::new(0)));
3557 +
        let s = Store::open_on(backend.clone(), dir.path().join("scratch")).unwrap();
3558 +
        let refs: Vec<ObjectRef> = (0..10)
3559 +
            .map(|i| {
3560 +
                s.put(ObjectType::Chunk, format!("p {i}").as_bytes())
3561 +
                    .unwrap()
3562 +
            })
3563 +
            .collect();
3564 +
        s.pack_loose().unwrap();
3565 +
        let s = Store::open_on(backend.clone(), dir.path().join("scratch")).unwrap();
3566 +
        backend.1.store(0, Ordering::Relaxed);
3567 +
        for r in &refs {
3568 +
            assert!(s.get(r).is_ok());
3569 +
        }
3570 +
        assert!(s.get_many(&refs).iter().all(|g| g.is_ok()));
3571 +
        assert_eq!(backend.1.load(Ordering::Relaxed), 0, "no loose lookups");
3572 +
        // An object no pack holds is still looked for loose.
3573 +
        let missing = hash_object(ObjectType::Chunk, b"never stored");
3574 +
        assert!(matches!(s.get(&missing), Err(StoreError::NotFound(_))));
3575 +
        assert!(backend.1.load(Ordering::Relaxed) > 0);
3576 +
    }
2336 3577
}
2337 3578
3579 +
/// Readers never wait (oxygenhub#249): a store keeps reading while other
3580 +
/// handles repack, collect and push into the store under it.
2338 3581
#[cfg(test)]
3582 +
pub(crate) mod reader_tests {
3583 +
    use super::*;
3584 +
3585 +
    fn chunk(i: usize) -> Vec<u8> {
3586 +
        format!("object {i} ").repeat(1 + i % 5).into_bytes()
3587 +
    }
3588 +
3589 +
    /// Two handles on one store: a writer, and a reader opened at once.
3590 +
    fn each_pair(f: impl Fn(&str, Store, Store)) {
3591 +
        let dir = tempfile::tempdir().unwrap();
3592 +
        let s = Store::open(dir.path().join("objects")).unwrap();
3593 +
        let r = Store::open(dir.path().join("objects")).unwrap();
3594 +
        f("directory", s, r);
3595 +
        let m: Arc<Backend> = Arc::new(MemoryStore::new());
3596 +
        let s = Store::open_on(m.clone(), dir.path().join("scratch")).unwrap();
3597 +
        let r = Store::open_on(m, dir.path().join("scratch-r")).unwrap();
3598 +
        f("memory", s, r);
3599 +
    }
3600 +
3601 +
    /// Set every retired entry's time back by `by`, as if retired then.
3602 +
    pub(crate) fn age_retired(s: &Store, by: std::time::Duration) {
3603 +
        s.update_retired(|m| {
3604 +
            for t in m.values_mut() {
3605 +
                *t = t.saturating_sub(by.as_millis() as u64);
3606 +
            }
3607 +
        })
3608 +
        .unwrap();
3609 +
    }
3610 +
3611 +
    fn retired_keys(s: &Store) -> Vec<String> {
3612 +
        s.read_retired().unwrap().0.into_keys().collect()
3613 +
    }
3614 +
3615 +
    #[test]
3616 +
    fn a_reader_opened_before_a_repack_reads_every_object_after_it() {
3617 +
        each_pair(|which, writer, _| {
3618 +
            let mut refs = Vec::new();
3619 +
            for i in 0..40 {
3620 +
                refs.push(writer.put(ObjectType::Chunk, &chunk(i)).unwrap());
3621 +
            }
3622 +
            writer.pack_loose().unwrap();
3623 +
            for i in 40..80 {
3624 +
                refs.push(writer.put(ObjectType::Chunk, &chunk(i)).unwrap());
3625 +
            }
3626 +
            // Opened now: one pack and forty loose objects.
3627 +
            let reader = Store::open_on(writer.backend().clone(), writer.dir()).unwrap();
3628 +
            assert_eq!(reader.pack_count(), 1, "{which}");
3629 +
            let (_, merged) = writer.repack_all(REPACK_ZSTD_LEVEL).unwrap();
3630 +
            assert_eq!(merged, 1);
3631 +
            // The old pack and the loose objects are retired, not deleted.
3632 +
            let retired = retired_keys(&writer);
3633 +
            assert_eq!(retired.len(), 81, "{which}: {retired:?}");
3634 +
            assert_eq!(writer.loose_count().unwrap(), 0, "retired is not loose");
3635 +
            // The reader reads on from what it loaded.
3636 +
            for (i, r) in refs.iter().enumerate() {
3637 +
                assert_eq!(reader.get(r).unwrap().1, chunk(i), "{which}: {i}");
3638 +
            }
3639 +
            assert_eq!(reader.pack_count(), 1);
3640 +
            // A store opened now reads the new pack only.
3641 +
            let fresh = Store::open_on(writer.backend().clone(), writer.dir()).unwrap();
3642 +
            assert_eq!(fresh.pack_count(), 1);
3643 +
            assert_ne!(fresh.packs.borrow()[0].key, reader.packs.borrow()[0].key);
3644 +
            // Once the retired files are deleted, the reader finds every
3645 +
            // object in the new pack, by get, get_many and has alike.
3646 +
            let report = writer
3647 +
                .expire_retired(std::time::Duration::ZERO, &mut silent())
3648 +
                .unwrap();
3649 +
            assert_eq!((report.packs, report.loose), (1, 80), "{which}");
3650 +
            assert!(retired_keys(&writer).is_empty());
3651 +
            for (i, r) in refs.iter().enumerate() {
3652 +
                assert_eq!(reader.get(r).unwrap().1, chunk(i), "{which}: {i}");
3653 +
            }
3654 +
            let other = Store::open_on(writer.backend().clone(), writer.dir()).unwrap();
3655 +
            let before = other.packs.borrow()[0].key.clone();
3656 +
            writer.put(ObjectType::Chunk, b"new").unwrap();
3657 +
            writer.repack_all(REPACK_ZSTD_LEVEL).unwrap();
3658 +
            writer
3659 +
                .expire_retired(std::time::Duration::ZERO, &mut silent())
3660 +
                .unwrap();
3661 +
            assert!(other.get_many(&refs).iter().all(|g| g.is_ok()), "{which}");
3662 +
            assert_ne!(other.packs.borrow()[0].key, before);
3663 +
            let third = Store::open_on(writer.backend().clone(), writer.dir()).unwrap();
3664 +
            writer.put(ObjectType::Chunk, b"newer").unwrap();
3665 +
            writer.repack_all(REPACK_ZSTD_LEVEL).unwrap();
3666 +
            writer
3667 +
                .expire_retired(std::time::Duration::ZERO, &mut silent())
3668 +
                .unwrap();
3669 +
            assert!(refs.iter().all(|r| third.has(r)), "{which}");
3670 +
            assert!(!third.has(&hash_object(ObjectType::Chunk, b"never")));
3671 +
        });
3672 +
    }
3673 +
3674 +
    #[test]
3675 +
    fn a_reader_opened_before_a_spilled_batch_reads_its_objects() {
3676 +
        each_pair(|which, writer, reader| {
3677 +
            assert_eq!(reader.pack_count(), 0);
3678 +
            let mut refs = Vec::new();
3679 +
            {
3680 +
                let batch = writer.begin_batch().unwrap();
3681 +
                for i in 0..(SPILL_OBJECTS + 50) {
3682 +
                    refs.push(writer.put(ObjectType::Chunk, &chunk(i)).unwrap());
3683 +
                }
3684 +
                batch.finish().unwrap();
3685 +
            }
3686 +
            assert_eq!(writer.pack_count(), 1, "{which}: the batch spilled");
3687 +
            assert_eq!(reader.get(&refs[0]).unwrap().1, chunk(0), "{which}");
3688 +
            assert_eq!(reader.pack_count(), 1, "the miss loaded the new pack");
3689 +
            let got = reader.get_many(&refs);
3690 +
            assert!(got.iter().all(|g| g.is_ok()), "{which}");
3691 +
            // has, too, from another reader.
3692 +
            let late = Store::open_on(writer.backend().clone(), writer.dir()).unwrap();
3693 +
            {
3694 +
                let batch = writer.begin_batch().unwrap();
3695 +
                for i in 1000..(1000 + SPILL_OBJECTS + 1) {
3696 +
                    writer.put(ObjectType::Chunk, &chunk(i)).unwrap();
3697 +
                }
3698 +
                batch.finish().unwrap();
3699 +
            }
3700 +
            assert!(late.has(&hash_object(ObjectType::Chunk, &chunk(1000))));
3701 +
        });
3702 +
    }
3703 +
3704 +
    #[test]
3705 +
    fn retired_entries_expire_after_the_hour_and_are_never_loaded() {
3706 +
        each_pair(|which, s, _| {
3707 +
            for i in 0..10 {
3708 +
                s.put(ObjectType::Chunk, &chunk(i)).unwrap();
3709 +
            }
3710 +
            s.pack_loose().unwrap();
3711 +
            s.put(ObjectType::Chunk, &chunk(10)).unwrap();
3712 +
            s.repack_all(ZSTD_LEVEL).unwrap();
3713 +
            // Retired over an hour ago: the first pack and 10 loose objects.
3714 +
            age_retired(&s, RETIRED_FOR + std::time::Duration::from_secs(60));
3715 +
            let old = retired_keys(&s);
3716 +
            assert_eq!(old.len(), 12, "{which}: {old:?}");
3717 +
            // Retired just now: the second pack.
3718 +
            s.put(ObjectType::Chunk, &chunk(11)).unwrap();
3719 +
            s.repack_all(ZSTD_LEVEL).unwrap();
3720 +
            assert_eq!(retired_keys(&s).len(), 14);
3721 +
            let listed = s
3722 +
                .backend()
3723 +
                .list("pack-")
3724 +
                .unwrap()
3725 +
                .into_iter()
3726 +
                .filter(|e| e.key.ends_with(".idx"))
3727 +
                .count();
3728 +
            assert_eq!(listed, 3, "{which}");
3729 +
            let fresh = Store::open_on(s.backend().clone(), s.dir()).unwrap();
3730 +
            assert_eq!(
3731 +
                fresh.pack_count(),
3732 +
                1,
3733 +
                "{which}: retired packs are not loaded"
3734 +
            );
3735 +
            let report = s.expire_retired(RETIRED_FOR, &mut silent()).unwrap();
3736 +
            assert_eq!(
3737 +
                report,
3738 +
                RetiredReport {
3739 +
                    packs: 1,
3740 +
                    loose: 11,
3741 +
                    kept: 2
3742 +
                },
3743 +
                "{which}"
3744 +
            );
3745 +
            let young = retired_keys(&s);
3746 +
            assert_eq!(young.len(), 2, "{which}: {young:?}");
3747 +
            assert!(young.iter().all(|k| !old.contains(k)));
3748 +
            for k in old.iter().filter(|k| k.ends_with(".pack")) {
3749 +
                assert!(!s.backend().has(k).unwrap(), "{which}: {k}");
3750 +
            }
3751 +
            for k in &young {
3752 +
                assert!(s.backend().has(k).unwrap(), "{which}: {k} kept");
3753 +
            }
3754 +
            for i in 0..12 {
3755 +
                let r = hash_object(ObjectType::Chunk, &chunk(i));
3756 +
                assert_eq!(fresh.get(&r).unwrap().1, chunk(i), "{which}");
3757 +
            }
3758 +
        });
3759 +
    }
3760 +
3761 +
    /// A retired loose object whose pack copy was pruned since is not
3762 +
    /// lost when the retired files go: it is loose again.
3763 +
    #[test]
3764 +
    fn a_retired_loose_object_no_pack_holds_is_kept() {
3765 +
        each_pair(|which, s, _| {
3766 +
            let a = s.put(ObjectType::Chunk, b"a").unwrap();
3767 +
            let b = s.put(ObjectType::Chunk, b"b").unwrap();
3768 +
            s.pack_loose().unwrap();
3769 +
            assert_eq!(s.loose_count().unwrap(), 0);
3770 +
            // The pack rewritten without b, as pruning does.
3771 +
            let first = s.packs.borrow()[0].key.clone();
3772 +
            s.write_pack(&[a], &[], &[first], ZSTD_LEVEL, &mut silent())
3773 +
                .unwrap();
3774 +
            let report = s
3775 +
                .expire_retired(std::time::Duration::ZERO, &mut silent())
3776 +
                .unwrap();
3777 +
            assert_eq!((report.packs, report.loose), (1, 1), "{which}: a's copy");
3778 +
            assert!(retired_keys(&s).is_empty());
3779 +
            assert_eq!(s.get(&b).unwrap().1, b"b", "{which}");
3780 +
            assert_eq!(s.get(&a).unwrap().1, b"a");
3781 +
            assert_eq!(s.loose_count().unwrap(), 1, "b is loose again");
3782 +
            // And pruning it, unreachable, deletes it.
3783 +
            let keep = [a].into_iter().collect();
3784 +
            let r = s.prune(&keep, std::time::Duration::ZERO).unwrap();
3785 +
            assert_eq!(r.objects, 1);
3786 +
            assert!(!s.has(&b));
3787 +
        });
3788 +
    }
3789 +
3790 +
    /// A backend whose first pack listing names an index that is deleted
3791 +
    /// before it is read, as a collection running meanwhile would.
3792 +
    pub(crate) struct Racing {
3793 +
        inner: MemoryStore,
3794 +
        lists: std::sync::atomic::AtomicUsize,
3795 +
        /// Deleted right after the first listing returns.
3796 +
        pub(crate) doomed: std::sync::Mutex<Option<String>>,
3797 +
        /// Named in every listing, never stored.
3798 +
        ghost: bool,
3799 +
    }
3800 +
3801 +
    impl crate::strata::Store for Racing {
3802 +
        fn describe(&self) -> String {
3803 +
            "racing".into()
3804 +
        }
3805 +
        fn has(&self, k: &str) -> crate::strata::Result<bool> {
3806 +
            self.inner.has(k)
3807 +
        }
3808 +
        fn get(&self, k: &str) -> crate::strata::Result<Vec<u8>> {
3809 +
            self.inner.get(k)
3810 +
        }
3811 +
        fn get_range(&self, k: &str, r: std::ops::Range<u64>) -> crate::strata::Result<Vec<u8>> {
3812 +
            self.inner.get_range(k, r)
3813 +
        }
3814 +
        fn put_new(&self, k: &str, b: &[u8]) -> crate::strata::Result<()> {
3815 +
            self.inner.put_new(k, b)
3816 +
        }
3817 +
        fn list(&self, p: &str) -> crate::strata::Result<Vec<crate::strata::Entry>> {
3818 +
            let mut out = self.inner.list(p)?;
3819 +
            if p == "pack-" {
3820 +
                self.lists
3821 +
                    .fetch_add(1, std::sync::atomic::Ordering::Relaxed);
3822 +
                if let Some(k) = self.doomed.lock().unwrap().take() {
3823 +
                    self.inner.delete(&k)?;
3824 +
                }
3825 +
                if self.ghost {
3826 +
                    for k in ["pack-0000.idx", "pack-0000.pack"] {
3827 +
                        out.push(crate::strata::Entry {
3828 +
                            key: k.into(),
3829 +
                            size: 100,
3830 +
                            modified_ms: None,
3831 +
                        });
3832 +
                    }
3833 +
                    out.sort_by(|a, b| a.key.cmp(&b.key));
3834 +
                }
3835 +
            }
3836 +
            Ok(out)
3837 +
        }
3838 +
        fn delete(&self, k: &str) -> crate::strata::Result<()> {
3839 +
            self.inner.delete(k)
3840 +
        }
3841 +
        fn read_ref(
3842 +
            &self,
3843 +
            k: &str,
3844 +
        ) -> crate::strata::Result<Option<(Vec<u8>, crate::strata::Version)>> {
3845 +
            self.inner.read_ref(k)
3846 +
        }
3847 +
        fn write_ref(
3848 +
            &self,
3849 +
            k: &str,
3850 +
            b: &[u8],
3851 +
            e: Option<&crate::strata::Version>,
3852 +
        ) -> crate::strata::Result<crate::strata::Version> {
3853 +
            self.inner.write_ref(k, b, e)
3854 +
        }
3855 +
    }
3856 +
3857 +
    pub(crate) fn racing(ghost: bool) -> Arc<Racing> {
3858 +
        Arc::new(Racing {
3859 +
            inner: MemoryStore::new(),
3860 +
            lists: Default::default(),
3861 +
            doomed: Default::default(),
3862 +
            ghost,
3863 +
        })
3864 +
    }
3865 +
3866 +
    #[test]
3867 +
    fn an_index_deleted_between_listing_and_loading_is_passed_over() {
3868 +
        let dir = tempfile::tempdir().unwrap();
3869 +
        let backend = racing(false);
3870 +
        let s = Store::open_on(backend.clone(), dir.path()).unwrap();
3871 +
        let a = s.put(ObjectType::Chunk, b"a").unwrap();
3872 +
        s.pack_loose().unwrap();
3873 +
        let b = s.put(ObjectType::Chunk, b"b").unwrap();
3874 +
        s.pack_loose().unwrap();
3875 +
        let idx = {
3876 +
            let packs = s.packs.borrow();
3877 +
            let (i, _) = s.find_packed(&a).unwrap();
3878 +
            format!("{}.idx", packs[i].key.trim_end_matches(".pack"))
3879 +
        };
3880 +
        *backend.doomed.lock().unwrap() = Some(idx);
3881 +
        let before = backend.lists.load(std::sync::atomic::Ordering::Relaxed);
3882 +
        let again = Store::open_on(backend.clone(), dir.path()).unwrap();
3883 +
        assert_eq!(again.pack_count(), 1, "the deleted one is passed over");
3884 +
        assert_eq!(
3885 +
            backend.lists.load(std::sync::atomic::Ordering::Relaxed) - before,
3886 +
            2,
3887 +
            "and the store listed again"
3888 +
        );
3889 +
        assert_eq!(again.get(&b).unwrap().1, b"b");
3890 +
        // A listing that names an index never there opens too.
3891 +
        let ghosted = racing(true);
3892 +
        let s = Store::open_on(ghosted.clone(), dir.path().join("g")).unwrap();
3893 +
        let c = s.put(ObjectType::Chunk, b"c").unwrap();
3894 +
        s.pack_loose().unwrap();
3895 +
        let again = Store::open_on(ghosted, dir.path().join("g")).unwrap();
3896 +
        assert_eq!(again.pack_count(), 1);
3897 +
        assert_eq!(again.get(&c).unwrap().1, b"c");
3898 +
    }
3899 +
}
3900 +
3901 +
#[cfg(test)]
2339 3902
mod batch_tests {
2340 3903
    use super::*;
2341 3904
…
17 unchanged lines
2359 3922
                .unwrap_or_else(|e| panic!("{}: {e}", r.short(12)));
2360 3923
        }
2361 3924
    }
3925 +
3926 +
    #[test]
3927 +
    fn a_pack_a_stopped_batch_left_is_adopted_and_its_torn_end_cut() {
3928 +
        // oxygen#79: a batch never finished (a process killed part way)
3929 +
        // leaves its objects in an unindexed .tmp pack.
3930 +
        let dir = tempfile::tempdir().unwrap();
3931 +
        let mut refs = Vec::new();
3932 +
        {
3933 +
            let store = Store::open(dir.path()).unwrap();
3934 +
            let batch = store.begin_batch().unwrap();
3935 +
            for i in 0..300u32 {
3936 +
                let body = format!("object {i} {}", "y".repeat(20_000 + i as usize)).into_bytes();
3937 +
                refs.push(store.put(ObjectType::Blob, &body).unwrap());
3938 +
            }
3939 +
            std::mem::forget(batch);
3940 +
        }
3941 +
        let tmp: Vec<_> = fs::read_dir(dir.path())
3942 +
            .unwrap()
3943 +
            .flatten()
3944 +
            .filter(|e| e.file_name().to_string_lossy().starts_with(".tmp-"))
3945 +
            .map(|e| e.path())
3946 +
            .collect();
3947 +
        assert_eq!(tmp.len(), 1, "one unfinished pack");
3948 +
        // A torn last entry: a header and part of its body.
3949 +
        let mut f = fs::OpenOptions::new().append(true).open(&tmp[0]).unwrap();
3950 +
        f.write_all(&[0, 1, 0xff, 0, 0, 0, 9, 9, 0, 0]).unwrap();
3951 +
        f.write_all(&[7; 40]).unwrap();
3952 +
        drop(f);
3953 +
        let store = Store::open(dir.path()).unwrap();
3954 +
        assert!(store.get(&refs[0]).is_err(), "unreadable before");
3955 +
        let young = store
3956 +
            .adopt_unfinished_packs(std::time::Duration::from_secs(3600), &|_| {})
3957 +
            .unwrap();
3958 +
        assert_eq!(young, (0, 0), "a pack written to lately is left alone");
3959 +
        let (packs, objects) = store
3960 +
            .adopt_unfinished_packs(std::time::Duration::ZERO, &|_| {})
3961 +
            .unwrap();
3962 +
        assert_eq!((packs, objects), (1, 300));
3963 +
        for r in &refs {
3964 +
            store
3965 +
                .get(r)
3966 +
                .unwrap_or_else(|e| panic!("{}: {e}", r.short(12)));
3967 +
        }
3968 +
        let again = Store::open(dir.path()).unwrap();
3969 +
        assert!(again.get(&refs[299]).is_ok(), "indexed for good");
3970 +
    }
2362 3971
}
2363 3972
2364 3973
#[cfg(test)]
…
119 unchanged lines
2484 4093
        }
2485 4094
    }
2486 4095
}
4096 +
4097 +
/// Signatures and index entries on a store that is not this disk's
4098 +
/// directory: the journal and the entry lists (spec §13.7, #116).
4099 +
#[cfg(test)]
4100 +
mod entry_tests {
4101 +
    use super::*;
4102 +
    use crate::identity::{Domain, SecretKey};
4103 +
    use crate::strata::Store as _;
4104 +
    use std::collections::HashMap;
4105 +
    use std::sync::atomic::{AtomicBool, Ordering};
4106 +
    use std::sync::Mutex;
4107 +
4108 +
    /// A store in memory that counts its requests by kind, as a bucket
4109 +
    /// store's /health counters do; it can also slip a competing journal
4110 +
    /// line in just before a write of the journal, as another process
4111 +
    /// would.
4112 +
    #[derive(Default)]
4113 +
    struct Counted {
4114 +
        inner: MemoryStore,
4115 +
        counts: Mutex<HashMap<&'static str, usize>>,
4116 +
        interfere: AtomicBool,
4117 +
    }
4118 +
4119 +
    impl Counted {
4120 +
        fn count(&self, kind: &'static str) {
4121 +
            *self.counts.lock().unwrap().entry(kind).or_default() += 1;
4122 +
        }
4123 +
        fn n(&self, kind: &str) -> usize {
4124 +
            self.counts.lock().unwrap().get(kind).copied().unwrap_or(0)
4125 +
        }
4126 +
        fn reset(&self) {
4127 +
            self.counts.lock().unwrap().clear();
4128 +
        }
4129 +
        fn total(&self) -> usize {
4130 +
            self.counts.lock().unwrap().values().sum()
4131 +
        }
4132 +
        fn summary(&self) -> String {
4133 +
            let c = self.counts.lock().unwrap();
4134 +
            let mut v: Vec<String> = c.iter().map(|(k, n)| format!("{k} {n}")).collect();
4135 +
            v.sort();
4136 +
            v.join(", ")
4137 +
        }
4138 +
    }
4139 +
4140 +
    impl crate::strata::Store for Counted {
4141 +
        fn describe(&self) -> String {
4142 +
            "counted".into()
4143 +
        }
4144 +
        fn has(&self, k: &str) -> crate::strata::Result<bool> {
4145 +
            self.count("has");
4146 +
            self.inner.has(k)
4147 +
        }
4148 +
        fn get(&self, k: &str) -> crate::strata::Result<Vec<u8>> {
4149 +
            self.count(if is_legacy(k) { "get_legacy" } else { "get" });
4150 +
            self.inner.get(k)
4151 +
        }
4152 +
        fn get_range(&self, k: &str, r: std::ops::Range<u64>) -> crate::strata::Result<Vec<u8>> {
4153 +
            self.count("range");
4154 +
            self.inner.get_range(k, r)
4155 +
        }
4156 +
        fn put_new(&self, k: &str, b: &[u8]) -> crate::strata::Result<()> {
4157 +
            self.count("put");
4158 +
            self.inner.put_new(k, b)
4159 +
        }
4160 +
        fn list(&self, p: &str) -> crate::strata::Result<Vec<crate::strata::Entry>> {
4161 +
            self.count(if p == "pack-" {
4162 +
                "list_packs"
4163 +
            } else if is_legacy(p) {
4164 +
                "list_legacy"
4165 +
            } else {
4166 +
                "list"
4167 +
            });
4168 +
            self.inner.list(p)
4169 +
        }
4170 +
        fn delete(&self, k: &str) -> crate::strata::Result<()> {
4171 +
            self.count("delete");
4172 +
            self.inner.delete(k)
4173 +
        }
4174 +
        fn read_ref(
4175 +
            &self,
4176 +
            k: &str,
4177 +
        ) -> crate::strata::Result<Option<(Vec<u8>, crate::strata::Version)>> {
4178 +
            self.count(if k == JOURNAL_KEY {
4179 +
                "read_journal"
4180 +
            } else {
4181 +
                "read_ref"
4182 +
            });
4183 +
            self.inner.read_ref(k)
4184 +
        }
4185 +
        fn write_ref(
4186 +
            &self,
4187 +
            k: &str,
4188 +
            b: &[u8],
4189 +
            e: Option<&crate::strata::Version>,
4190 +
        ) -> crate::strata::Result<crate::strata::Version> {
4191 +
            self.count("write_ref");
4192 +
            if k == JOURNAL_KEY && self.interfere.swap(false, Ordering::SeqCst) {
4193 +
                let (mut m, v) = match self.inner.read_ref(k)? {
4194 +
                    Some((bytes, v)) => (parse_retired(&bytes), Some(v)),
4195 +
                    None => (Default::default(), None),
4196 +
                };
4197 +
                m.insert(
4198 +
                    format!("seal/{}", "c".repeat(64)) + "/" + &"d".repeat(64),
4199 +
                    0,
4200 +
                );
4201 +
                self.inner.write_ref(k, &encode_retired(&m), v.as_ref())?;
4202 +
            }
4203 +
            self.inner.write_ref(k, b, e)
4204 +
        }
4205 +
    }
4206 +
4207 +
    /// Whether a key or prefix is under an older one-list layout.
4208 +
    fn is_legacy(k: &str) -> bool {
4209 +
        ENTRY_INDEXES
4210 +
            .iter()
4211 +
            .any(|(_, legacy)| k.starts_with(&format!("{legacy}/")))
4212 +
    }
4213 +
4214 +
    struct Fixture {
4215 +
        backend: Arc<Counted>,
4216 +
        dir: tempfile::TempDir,
4217 +
        handles: std::cell::Cell<usize>,
4218 +
    }
4219 +
4220 +
    impl Fixture {
4221 +
        fn new() -> Fixture {
4222 +
            Fixture {
4223 +
                backend: Arc::new(Counted::default()),
4224 +
                dir: tempfile::tempdir().unwrap(),
4225 +
                handles: Default::default(),
4226 +
            }
4227 +
        }
4228 +
        /// Another handle on the store, as another process opens it.
4229 +
        fn open(&self) -> Store {
4230 +
            let n = self.handles.get() + 1;
4231 +
            self.handles.set(n);
4232 +
            Store::open_on(self.backend.clone(), self.dir.path().join(format!("s{n}"))).unwrap()
4233 +
        }
4234 +
    }
4235 +
4236 +
    fn signer(n: u8) -> SecretKey {
4237 +
        SecretKey {
4238 +
            ed_seed: [n; 32],
4239 +
            ml_seed: [n + 100; 32],
4240 +
        }
4241 +
    }
4242 +
4243 +
    fn sign(s: &Store, r: &ObjectRef, n: u8) -> Signature {
4244 +
        let sk = signer(n);
4245 +
        let key = s.put_identity(&sk.identity("a", "b", 0)).unwrap();
4246 +
        let sig = sk.sign(Domain::Change, &key, r);
4247 +
        s.put_signature(r, &sig).unwrap();
4248 +
        sig
4249 +
    }
4250 +
4251 +
    fn journal(s: &Store) -> std::collections::BTreeMap<String, u64> {
4252 +
        s.read_record(JOURNAL_KEY).unwrap().0
4253 +
    }
4254 +
4255 +
    fn fold(s: &Store) -> bool {
4256 +
        s.fold_entries(&mut silent()).unwrap()
4257 +
    }
4258 +
4259 +
    /// What 0.4.6 writes for a signature: its own key, and nothing else.
4260 +
    fn old_writer_signs(f: &Fixture, r: &ObjectRef, n: u8) -> Signature {
4261 +
        let sk = signer(n);
4262 +
        let sig = sk.sign(Domain::Change, &ObjectRef([n; 32]), r);
4263 +
        f.backend
4264 +
            .put_new(
4265 +
                &format!("{}{}", fanout_prefix("sig", r), sig.key.to_hex()),
4266 +
                &Signature::encode_list(std::slice::from_ref(&sig)),
4267 +
            )
4268 +
            .unwrap();
4269 +
        sig
4270 +
    }
4271 +
4272 +
    /// After a fold, a signature or seal another handle writes is found
4273 +
    /// by a store opened later through the journal, with no listing; a
4274 +
    /// handle that already read the entries sees it once refreshed.
4275 +
    #[test]
4276 +
    fn an_entry_written_after_a_fold_is_found_through_the_journal() {
4277 +
        let f = Fixture::new();
4278 +
        let w = f.open();
4279 +
        let r1 = w.put(ObjectType::Chunk, b"one").unwrap();
4280 +
        let s1 = sign(&w, &r1, 1);
4281 +
        assert!(fold(&w));
4282 +
        assert!(!fold(&w), "nothing new: no second list");
4283 +
4284 +
        let early = f.open();
4285 +
        assert_eq!(early.signatures(&r1).unwrap(), vec![s1.clone()]);
4286 +
4287 +
        let w2 = f.open();
4288 +
        let r2 = w2.put(ObjectType::Chunk, b"two").unwrap();
4289 +
        let s2 = sign(&w2, &r2, 2);
4290 +
        w2.index_seal(&r2, &ObjectRef([9; 32])).unwrap();
4291 +
        w2.index_receipt(&r2, &ObjectRef([8; 32])).unwrap();
4292 +
        // Its own writes, at once.
4293 +
        assert_eq!(w2.signatures(&r2).unwrap(), vec![s2.clone()]);
4294 +
4295 +
        let later = f.open();
4296 +
        f.backend.reset();
4297 +
        assert_eq!(later.signatures(&r1).unwrap(), vec![s1]);
4298 +
        assert_eq!(later.signatures(&r2).unwrap(), vec![s2.clone()]);
4299 +
        assert_eq!(later.seals_for(&r2).unwrap(), vec![ObjectRef([9; 32])]);
4300 +
        assert_eq!(later.receipts_for(&r2).unwrap(), vec![ObjectRef([8; 32])]);
4301 +
        assert_eq!(later.receipt_states().unwrap(), vec![r2]);
4302 +
        assert!(later.seals_for(&r1).unwrap().is_empty());
4303 +
        assert_eq!(f.backend.n("list"), 0, "{}", f.backend.summary());
4304 +
        assert_eq!(f.backend.n("read_journal"), 1, "{}", f.backend.summary());
4305 +
4306 +
        // The handle that read the entries before keeps its view until
4307 +
        // it is refreshed, as it keeps its operation head.
4308 +
        assert!(early.signatures(&r2).unwrap().is_empty());
4309 +
        early.refresh().unwrap();
4310 +
        assert_eq!(early.signatures(&r2).unwrap(), vec![s2]);
4311 +
    }
4312 +
4313 +
    /// A fold marks the journal's lines rather than dropping them, so a
4314 +
    /// reader whose listing predates it still finds every entry; an hour
4315 +
    /// on, the old list is deleted and the lines dropped, and readers
4316 +
    /// find everything in the new list.
4317 +
    #[test]
4318 +
    fn a_fold_keeps_the_journal_for_the_hour_readers_may_need_it() {
4319 +
        let f = Fixture::new();
4320 +
        let w = f.open();
4321 +
        let r1 = w.put(ObjectType::Chunk, b"one").unwrap();
4322 +
        let s1 = sign(&w, &r1, 1);
4323 +
        assert!(fold(&w));
4324 +
        let r2 = w.put(ObjectType::Chunk, b"two").unwrap();
4325 +
        let s2 = sign(&w, &r2, 2);
4326 +
        assert!(journal(&w).values().any(|t| *t == 0));
4327 +
4328 +
        // Opened before the second fold: lists the first list only.
4329 +
        let stale = f.open();
4330 +
        let stale_after_expiry = f.open();
4331 +
        assert!(fold(&w));
4332 +
        assert!(
4333 +
            journal(&w).values().all(|t| *t != 0),
4334 +
            "every line is marked folded"
4335 +
        );
4336 +
        assert!(journal(&w).len() >= 2, "and kept");
4337 +
        assert_eq!(stale.signatures(&r1).unwrap(), vec![s1.clone()]);
4338 +
        assert_eq!(stale.signatures(&r2).unwrap(), vec![s2.clone()]);
4339 +
4340 +
        // An hour later: the first list goes with the next collection,
4341 +
        // and the next journal update drops the folded lines.
4342 +
        reader_tests::age_retired(&w, RETIRED_FOR * 2);
4343 +
        w.update_record(JOURNAL_KEY, "test", |m| {
4344 +
            for t in m.values_mut() {
4345 +
                *t = t.saturating_sub(2 * RETIRED_FOR.as_millis() as u64);
4346 +
            }
4347 +
        })
4348 +
        .unwrap();
4349 +
        w.expire_retired(RETIRED_FOR, &mut silent()).unwrap();
4350 +
        let lists: Vec<String> = f
4351 +
            .backend
4352 +
            .inner
4353 +
            .list("pack-")
4354 +
            .unwrap()
4355 +
            .into_iter()
4356 +
            .map(|e| e.key)
4357 +
            .filter(|k| is_entry_list(k))
4358 +
            .collect();
4359 +
        assert_eq!(lists.len(), 1, "the replaced list is deleted: {lists:?}");
4360 +
        let r3 = w.put(ObjectType::Chunk, b"three").unwrap();
4361 +
        let s3 = sign(&w, &r3, 3);
4362 +
        let j = journal(&w);
4363 +
        assert_eq!(j.len(), 1, "only the new line is left: {j:?}");
4364 +
        assert!(j.keys().all(|k| k.starts_with(&fanout_prefix("sig", &r3))));
4365 +
4366 +
        // A handle whose listing named the deleted list lists again.
4367 +
        assert_eq!(
4368 +
            stale_after_expiry.signatures(&r1).unwrap(),
4369 +
            vec![s1.clone()]
4370 +
        );
4371 +
        assert_eq!(
4372 +
            stale_after_expiry.signatures(&r3).unwrap(),
4373 +
            vec![s3.clone()]
4374 +
        );
4375 +
        let fresh = f.open();
4376 +
        f.backend.reset();
4377 +
        assert_eq!(fresh.signatures(&r1).unwrap(), vec![s1]);
4378 +
        assert_eq!(fresh.signatures(&r2).unwrap(), vec![s2]);
4379 +
        assert_eq!(fresh.signatures(&r3).unwrap(), vec![s3]);
4380 +
        assert_eq!(f.backend.n("list"), 0);
4381 +
    }
4382 +
4383 +
    /// A store no fold has touched (as 0.4.6 left it: per-object keys and
4384 +
    /// older one-list keys, no journal, no lists) reads by listing, and
4385 +
    /// reads the same once folded. An older writer's entry after the
4386 +
    /// fold is missed until the next fold: the trade-off of §13.7.
4387 +
    #[test]
4388 +
    fn an_old_layout_store_reads_before_and_after_its_first_fold() {
4389 +
        let f = Fixture::new();
4390 +
        let w = f.open();
4391 +
        let r1 = w.put(ObjectType::Chunk, b"one").unwrap();
4392 +
        let r2 = w.put(ObjectType::Chunk, b"two").unwrap();
4393 +
        let s1 = old_writer_signs(&f, &r1, 1);
4394 +
        let s2 = SecretKey {
4395 +
            ed_seed: [2; 32],
4396 +
            ml_seed: [3; 32],
4397 +
        }
4398 +
        .sign(Domain::Change, &ObjectRef([2; 32]), &r2);
4399 +
        f.backend
4400 +
            .put_new(
4401 +
                &legacy_key("sigs", &r2),
4402 +
                &Signature::encode_list(std::slice::from_ref(&s2)),
4403 +
            )
4404 +
            .unwrap();
4405 +
        f.backend
4406 +
            .put_new(
4407 +
                &format!("{}{}", fanout_prefix("seal", &r1), "ab".repeat(32)),
4408 +
                b"",
4409 +
            )
4410 +
            .unwrap();
4411 +
        f.backend
4412 +
            .put_new(
4413 +
                &legacy_key("receipts", &r2),
4414 +
                format!("{}\n", ObjectRef([7; 32]).to_hex()).as_bytes(),
4415 +
            )
4416 +
            .unwrap();
4417 +
        let check = |s: &Store| {
4418 +
            assert_eq!(s.signatures(&r1).unwrap(), vec![s1.clone()]);
4419 +
            assert_eq!(s.signatures(&r2).unwrap(), vec![s2.clone()]);
4420 +
            assert_eq!(s.seals_for(&r1).unwrap(), vec![ObjectRef([0xab; 32])]);
4421 +
            assert_eq!(s.receipts_for(&r2).unwrap(), vec![ObjectRef([7; 32])]);
4422 +
            assert_eq!(s.receipt_states().unwrap(), vec![r2]);
4423 +
        };
4424 +
        let reader = f.open();
4425 +
        assert!(!reader.folded.get());
4426 +
        check(&reader);
4427 +
        assert!(f.backend.inner.read_ref(JOURNAL_KEY).unwrap().is_none());
4428 +
4429 +
        assert!(fold(&w));
4430 +
        let reader = f.open();
4431 +
        assert!(reader.folded.get());
4432 +
        check(&reader);
4433 +
4434 +
        // The older lists' entries are per-object entries now, and the
4435 +
        // marker is there.
4436 +
        assert!(f.backend.inner.has(LEGACY_FOLDED_KEY).unwrap());
4437 +
        assert!(f
4438 +
            .backend
4439 +
            .inner
4440 +
            .has(&format!("{}{}", fanout_prefix("sig", &r2), s2.key.to_hex()))
4441 +
            .unwrap());
4442 +
        assert!(f
4443 +
            .backend
4444 +
            .inner
4445 +
            .has(&format!(
4446 +
                "{}{}",
4447 +
                fanout_prefix("receipt", &r2),
4448 +
                ObjectRef([7; 32]).to_hex()
4449 +
            ))
4450 +
            .unwrap());
4451 +
4452 +
        // An older writer after the fold: missed until the next one.
4453 +
        let r3 = w.put(ObjectType::Chunk, b"three").unwrap();
4454 +
        let s3 = old_writer_signs(&f, &r3, 3);
4455 +
        assert!(f.open().signatures(&r3).unwrap().is_empty());
4456 +
        assert!(fold(&w));
4457 +
        assert_eq!(f.open().signatures(&r3).unwrap(), vec![s3]);
4458 +
    }
4459 +
4460 +
    /// Writers racing on the journal never lose a line: a lost
4461 +
    /// compare-and-swap reads the journal again and retries.
4462 +
    #[test]
4463 +
    fn concurrent_journal_appends_lose_nothing() {
4464 +
        let f = Fixture::new();
4465 +
        // One race, made to happen.
4466 +
        let w = f.open();
4467 +
        f.backend.interfere.store(true, Ordering::SeqCst);
4468 +
        w.index_seal(&ObjectRef([1; 32]), &ObjectRef([2; 32]))
4469 +
            .unwrap();
4470 +
        let j = journal(&w);
4471 +
        assert_eq!(j.len(), 2, "the competitor's line and ours: {j:?}");
4472 +
        // Many, from threads, each its own handle.
4473 +
        let backend: Arc<Backend> = f.backend.clone();
4474 +
        std::thread::scope(|scope| {
4475 +
            for t in 0..8u8 {
4476 +
                let backend = backend.clone();
4477 +
                let dir = f.dir.path().join(format!("t{t}"));
4478 +
                scope.spawn(move || {
4479 +
                    let s = Store::open_on(backend, dir).unwrap();
4480 +
                    for i in 0..25u8 {
4481 +
                        s.index_seal(&ObjectRef([t + 10; 32]), &ObjectRef([i; 32]))
4482 +
                            .unwrap();
4483 +
                    }
4484 +
                });
4485 +
            }
4486 +
        });
4487 +
        assert_eq!(journal(&w).len(), 2 + 8 * 25);
4488 +
        assert!(fold(&w));
4489 +
        let r = f.open();
4490 +
        for t in 0..8u8 {
4491 +
            assert_eq!(r.seals_for(&ObjectRef([t + 10; 32])).unwrap().len(), 25);
4492 +
        }
4493 +
    }
4494 +
4495 +
    /// A batch adds its entries to the journal in one update when it is
4496 +
    /// flushed; past the limit the writer folds them into a partial list.
4497 +
    #[test]
4498 +
    fn a_long_journal_is_folded_into_a_partial_list() {
4499 +
        let f = Fixture::new();
4500 +
        let w = f.open();
4501 +
        assert!(fold(&w), "folded, with nothing yet");
4502 +
        let key = ObjectRef([5; 32]);
4503 +
        f.backend.reset();
4504 +
        let batch = w.begin_batch().unwrap();
4505 +
        for i in 0..=JOURNAL_FOLD_AT as u32 {
4506 +
            let mut v = [0u8; 32];
4507 +
            v[..4].copy_from_slice(&i.to_le_bytes());
4508 +
            w.index_seal(&key, &ObjectRef(v)).unwrap();
4509 +
        }
4510 +
        assert_eq!(f.backend.n("write_ref"), 0, "nothing until the flush");
4511 +
        batch.finish().unwrap();
4512 +
        let parts = f
4513 +
            .backend
4514 +
            .inner
4515 +
            .list("pack-")
4516 +
            .unwrap()
4517 +
            .into_iter()
4518 +
            .filter(|e| e.key.ends_with(PART_SUFFIX))
4519 +
            .count();
4520 +
        assert_eq!(parts, 1);
4521 +
        assert!(journal(&w).values().all(|t| *t != 0), "all folded");
4522 +
        let r = f.open();
4523 +
        assert_eq!(r.seals_for(&key).unwrap().len(), JOURNAL_FOLD_AT + 1);
4524 +
        // The next complete fold takes the partial list in.
4525 +
        assert!(fold(&w));
4526 +
        let lists: Vec<String> = f.open().entry_lists.borrow().clone();
4527 +
        assert_eq!(lists.len(), 1);
4528 +
        assert!(lists[0].ends_with(LIST_SUFFIX));
4529 +
        assert_eq!(f.open().seals_for(&key).unwrap().len(), JOURNAL_FOLD_AT + 1);
4530 +
    }
4531 +
4532 +
    /// A collection that deletes an object takes its entries out of the
4533 +
    /// journal too, so the next fold leaves them out.
4534 +
    #[test]
4535 +
    fn pruned_objects_leave_the_journal() {
4536 +
        let f = Fixture::new();
4537 +
        let w = f.open();
4538 +
        let gone = w.put(ObjectType::Chunk, b"gone").unwrap();
4539 +
        let kept = w.put(ObjectType::Chunk, b"kept").unwrap();
4540 +
        sign(&w, &gone, 1);
4541 +
        let s = sign(&w, &kept, 2);
4542 +
        assert!(fold(&w));
4543 +
        let keep: std::collections::HashSet<ObjectRef> = w
4544 +
            .list()
4545 +
            .unwrap()
4546 +
            .into_iter()
4547 +
            .filter(|r| *r != gone)
4548 +
            .collect();
4549 +
        w.prune(&keep, std::time::Duration::ZERO).unwrap();
4550 +
        assert!(journal(&w).keys().all(|k| !k.contains(&gone.to_hex()[2..])));
4551 +
        fold(&w);
4552 +
        let r = f.open();
4553 +
        assert!(r.signatures(&gone).unwrap().is_empty());
4554 +
        assert_eq!(r.signatures(&kept).unwrap(), vec![s]);
4555 +
        let all = r.with_entries(|e| e.clone()).unwrap().unwrap();
4556 +
        assert!(
4557 +
            all.iter().all(|k| !k.contains(&gone.to_hex()[2..])),
4558 +
            "{all:?}"
4559 +
        );
4560 +
    }
4561 +
4562 +
    /// Requests per lookup, before and after the fold (#116): twenty
4563 +
    /// objects, five of them signed, each looked up for its signatures
4564 +
    /// and its seals, as a fetch does.
4565 +
    #[test]
4566 +
    fn a_folded_store_looks_up_entries_without_listing() {
4567 +
        let f = Fixture::new();
4568 +
        let w = f.open();
4569 +
        let refs: Vec<ObjectRef> = (0..20)
4570 +
            .map(|i| {
4571 +
                w.put(ObjectType::Chunk, format!("o {i}").as_bytes())
4572 +
                    .unwrap()
4573 +
            })
4574 +
            .collect();
4575 +
        for r in &refs[..5] {
4576 +
            sign(&w, r, 1);
4577 +
        }
4578 +
        let lookups = |s: &Store| {
4579 +
            for r in &refs {
4580 +
                s.signatures(r).unwrap();
4581 +
                s.seals_for(r).unwrap();
4582 +
            }
4583 +
        };
4584 +
        let before = f.open();
4585 +
        f.backend.reset();
4586 +
        lookups(&before);
4587 +
        let (before_total, before_list) = (f.backend.total(), f.backend.n("list"));
4588 +
        eprintln!("40 lookups before the fold: {}", f.backend.summary());
4589 +
4590 +
        assert!(fold(&w));
4591 +
        let after = f.open();
4592 +
        f.backend.reset();
4593 +
        lookups(&after);
4594 +
        eprintln!("40 lookups after the fold: {}", f.backend.summary());
4595 +
        assert_eq!(before_list, 40);
4596 +
        assert_eq!(f.backend.n("list"), 0);
4597 +
        assert_eq!(f.backend.n("get_legacy"), 0, "the older lists are folded");
4598 +
        assert_eq!(f.backend.n("read_journal"), 1);
4599 +
        assert!(
4600 +
            f.backend.total() < before_total / 10,
4601 +
            "{}",
4602 +
            f.backend.summary()
4603 +
        );
4604 +
    }
4605 +
4606 +
    /// Once a fold has copied the older lists into per-object entries and
4607 +
    /// written its marker, readers stop reading them, folded or not; the
4608 +
    /// answers are the same.
4609 +
    #[test]
4610 +
    fn the_legacy_marker_stops_reads_of_the_older_lists() {
4611 +
        let f = Fixture::new();
4612 +
        let w = f.open();
4613 +
        let r = w.put(ObjectType::Chunk, b"old").unwrap();
4614 +
        let sig = SecretKey {
4615 +
            ed_seed: [4; 32],
4616 +
            ml_seed: [5; 32],
4617 +
        }
4618 +
        .sign(Domain::Change, &ObjectRef([4; 32]), &r);
4619 +
        f.backend
4620 +
            .put_new(
4621 +
                &legacy_key("sigs", &r),
4622 +
                &Signature::encode_list(std::slice::from_ref(&sig)),
4623 +
            )
4624 +
            .unwrap();
4625 +
        for (legacy, v) in [("seals", 9u8), ("certs", 8), ("accepted-lists", 6)] {
4626 +
            f.backend
4627 +
                .put_new(
4628 +
                    &legacy_key(legacy, &r),
4629 +
                    format!("{}\n", ObjectRef([v; 32]).to_hex()).as_bytes(),
4630 +
                )
4631 +
                .unwrap();
4632 +
        }
4633 +
        let check = |s: &Store| {
4634 +
            assert_eq!(s.signatures(&r).unwrap(), vec![sig.clone()]);
4635 +
            assert_eq!(s.seals_for(&r).unwrap(), vec![ObjectRef([9; 32])]);
4636 +
            assert_eq!(
4637 +
                s.index_get("cert", "certs", &r).unwrap(),
4638 +
                vec![ObjectRef([8; 32])]
4639 +
            );
4640 +
            assert_eq!(
4641 +
                s.index_get_pub("accepted", &r).unwrap(),
4642 +
                vec![ObjectRef([6; 32])]
4643 +
            );
4644 +
        };
4645 +
        let before = f.open();
4646 +
        f.backend.reset();
4647 +
        check(&before);
4648 +
        assert_eq!(f.backend.n("get_legacy"), 4, "{}", f.backend.summary());
4649 +
4650 +
        assert!(fold(&w));
4651 +
        let folded = f.open();
4652 +
        f.backend.reset();
4653 +
        check(&folded);
4654 +
        assert_eq!(f.backend.n("get_legacy") + f.backend.n("list"), 0);
4655 +
4656 +
        // Not folded (its lists gone), but marked: per-object listings,
4657 +
        // still no older lists.
4658 +
        for e in f.backend.inner.list("pack-").unwrap() {
4659 +
            if is_entry_list(&e.key) {
4660 +
                f.backend.inner.delete(&e.key).unwrap();
4661 +
            }
4662 +
        }
4663 +
        let marked = f.open();
4664 +
        assert!(!marked.folded.get() && marked.legacy_folded.get());
4665 +
        f.backend.reset();
4666 +
        check(&marked);
4667 +
        assert_eq!(f.backend.n("get_legacy"), 0, "{}", f.backend.summary());
4668 +
        assert_eq!(f.backend.n("list"), 4);
4669 +
    }
4670 +
4671 +
    /// A handle that has read the entries finds another process's new
4672 +
    /// signature once the pack listing's interval has passed, with no
4673 +
    /// refresh, at the cost of one pack listing and one journal read.
4674 +
    #[test]
4675 +
    fn a_long_lived_handle_sees_new_entries_within_the_interval() {
4676 +
        let f = Fixture::new();
4677 +
        let w = f.open();
4678 +
        let r1 = w.put(ObjectType::Chunk, b"one").unwrap();
4679 +
        let s1 = sign(&w, &r1, 1);
4680 +
        assert!(fold(&w));
4681 +
        let a = f.open();
4682 +
        assert_eq!(a.signatures(&r1).unwrap(), vec![s1]);
4683 +
4684 +
        let b = f.open();
4685 +
        let r2 = b.put(ObjectType::Chunk, b"two").unwrap();
4686 +
        let s2 = sign(&b, &r2, 2);
4687 +
        b.index_approval(&r2, &ObjectRef([3; 32])).unwrap();
4688 +
        // Within the interval: answered from what was read.
4689 +
        assert!(a.signatures(&r2).unwrap().is_empty());
4690 +
4691 +
        // The interval passes, as the pack listing's does.
4692 +
        let t = a.entries_read.get().unwrap();
4693 +
        a.entries_read.set(t.checked_sub(ENTRIES_FRESH));
4694 +
        f.backend.reset();
4695 +
        assert_eq!(a.signatures(&r2).unwrap(), vec![s2]);
4696 +
        assert_eq!(a.approvals_for(&r2).unwrap(), vec![ObjectRef([3; 32])]);
4697 +
        assert_eq!(f.backend.n("read_journal"), 1, "{}", f.backend.summary());
4698 +
        assert_eq!(f.backend.n("list_packs"), 1, "{}", f.backend.summary());
4699 +
        assert_eq!(f.backend.n("list"), 0, "{}", f.backend.summary());
4700 +
        assert_eq!(
4701 +
            f.backend.n("get"),
4702 +
            1,
4703 +
            "the signature; the list already read is not read again: {}",
4704 +
            f.backend.summary()
4705 +
        );
4706 +
    }
4707 +
}
modified crates/oxy-core/src/strata.rs +467 −3
…
293 unchanged lines
294 294
        .map(|d| d.as_millis() as u64)
295 295
}
296 296
297 +
/// Put `tmp` at `path` only if `path` is free, as `fs::hard_link` does
298 +
/// (`tmp` may remain). Android forbids apps hard links in their own data
299 +
/// (oxygen#141): there a create-only rename does the same, atomically.
300 +
fn place_new(tmp: &Path, path: &Path) -> io::Result<()> {
301 +
    match fs::hard_link(tmp, path) {
302 +
        Err(e) if e.kind() == io::ErrorKind::PermissionDenied => rename_new(tmp, path),
303 +
        r => r,
304 +
    }
305 +
}
306 +
307 +
#[cfg(any(target_os = "linux", target_os = "android"))]
308 +
fn rename_new(tmp: &Path, path: &Path) -> io::Result<()> {
309 +
    use std::ffi::CString;
310 +
    use std::os::unix::ffi::OsStrExt;
311 +
    let from = CString::new(tmp.as_os_str().as_bytes())?;
312 +
    let to = CString::new(path.as_os_str().as_bytes())?;
313 +
    let r = unsafe {
314 +
        libc::renameat2(
315 +
            libc::AT_FDCWD,
316 +
            from.as_ptr(),
317 +
            libc::AT_FDCWD,
318 +
            to.as_ptr(),
319 +
            libc::RENAME_NOREPLACE as _,
320 +
        )
321 +
    };
322 +
    if r == 0 {
323 +
        Ok(())
324 +
    } else {
325 +
        Err(io::Error::last_os_error())
326 +
    }
327 +
}
328 +
329 +
#[cfg(not(any(target_os = "linux", target_os = "android")))]
330 +
fn rename_new(tmp: &Path, path: &Path) -> io::Result<()> {
331 +
    if path.exists() {
332 +
        return Err(io::ErrorKind::AlreadyExists.into());
333 +
    }
334 +
    fs::rename(tmp, path)
335 +
}
336 +
297 337
impl LocalStore {
298 338
    /// Link `tmp` into `path`, create-only; the directory is synced and
299 339
    /// `tmp` removed either way. `Exists` when the name is taken.
300 340
    fn link_into_place(&self, key: &str, tmp: &Path, path: &Path) -> Result<()> {
301 -
        let linked = fs::hard_link(tmp, path);
341 +
        let linked = place_new(tmp, path);
302 342
        let _ = fs::remove_file(tmp);
303 343
        match linked {
304 344
            Ok(()) => {
…
78 unchanged lines
383 423
        crate::durable::step("rename loose objects")?;
384 424
        let mut dirs: std::collections::BTreeSet<PathBuf> = std::collections::BTreeSet::new();
385 425
        for (path, tmp, _) in &todo {
386 -
            let linked = fs::hard_link(tmp, path);
426 +
            let linked = place_new(tmp, path);
387 427
            let _ = fs::remove_file(tmp);
388 428
            match linked {
389 429
                Ok(()) => {
…
19 unchanged lines
409 449
            return Err(Error::Exists(key.to_string()));
410 450
        }
411 451
        self.ensure_dir(&path)?;
412 -
        match fs::hard_link(file, &path) {
452 +
        match place_new(file, &path) {
413 453
            Ok(()) => {
414 454
                let _ = fs::remove_file(file);
415 455
                if let Some(dir) = path.parent() {
…
242 unchanged lines
658 698
    }
659 699
}
660 700
701 +
// ---------------------------------------------------------------- caching
702 +
703 +
/// A store in front of another (oxygen#68): objects are read through a
704 +
/// local directory and written through it, to the remote first, so the
705 +
/// remote always holds everything and the cache can be dropped at any
706 +
/// time. Objects never change, so a cached one is always right (and every
707 +
/// read is hashed again above this layer). The cache is bounded: past its
708 +
/// limit the objects used least recently are removed. An object larger
709 +
/// than an eighth of the limit is not cached; it is read by range from
710 +
/// the remote. References are never cached: they are read from and
711 +
/// written to the remote every time, so compare-and-swap is the remote's.
712 +
pub struct CachingStore {
713 +
    cache: LocalStore,
714 +
    remote: std::sync::Arc<dyn Store>,
715 +
    limit: u64,
716 +
    /// Bytes in the cache, counted on first need and kept up to date.
717 +
    used: Mutex<Option<u64>>,
718 +
}
719 +
720 +
/// Where a caching store keeps the ranges it read of objects too large to
721 +
/// keep whole, and those objects' sizes: apart from every object key.
722 +
const RANGES: &str = "cached-range";
723 +
const SIZES: &str = "cached-size";
724 +
725 +
/// How stale a cached object's use time may get before a read renews it.
726 +
const TOUCH_AFTER: Duration = Duration::from_secs(60);
727 +
728 +
impl CachingStore {
729 +
    /// `remote` behind a cache in the directory `dir` of at most `limit`
730 +
    /// bytes.
731 +
    pub fn open(
732 +
        remote: std::sync::Arc<dyn Store>,
733 +
        dir: impl Into<PathBuf>,
734 +
        limit: u64,
735 +
    ) -> Result<CachingStore> {
736 +
        Ok(CachingStore {
737 +
            cache: LocalStore::open(dir)?,
738 +
            remote,
739 +
            limit,
740 +
            used: Mutex::new(None),
741 +
        })
742 +
    }
743 +
744 +
    pub fn remote(&self) -> &std::sync::Arc<dyn Store> {
745 +
        &self.remote
746 +
    }
747 +
748 +
    /// Bytes the cache holds now.
749 +
    pub fn cached_bytes(&self) -> Result<u64> {
750 +
        let mut used = self.used.lock().unwrap_or_else(|p| p.into_inner());
751 +
        if used.is_none() {
752 +
            *used = Some(self.cache.list("")?.iter().map(|e| e.size).sum());
753 +
        }
754 +
        Ok(used.unwrap_or(0))
755 +
    }
756 +
757 +
    /// Whether an object of `size` bytes is kept in the cache at all.
758 +
    fn cacheable(&self, size: u64) -> bool {
759 +
        size <= self.limit / 8
760 +
    }
761 +
762 +
    /// Keep a copy of an object the remote holds.
763 +
    fn keep(&self, key: &str, bytes: &[u8]) -> Result<()> {
764 +
        if !self.cacheable(bytes.len() as u64) {
765 +
            return Ok(());
766 +
        }
767 +
        match self.cache.put_new(key, bytes) {
768 +
            Ok(()) => self.added(bytes.len() as u64),
769 +
            Err(Error::Exists(_)) => Ok(()),
770 +
            Err(e) => Err(e),
771 +
        }
772 +
    }
773 +
774 +
    /// Count `n` new bytes, and remove the least recently used objects
775 +
    /// once the cache is past its limit, down to nine tenths of it.
776 +
    fn added(&self, n: u64) -> Result<()> {
777 +
        let total = {
778 +
            let mut used = self.used.lock().unwrap_or_else(|p| p.into_inner());
779 +
            let total = match *used {
780 +
                Some(u) => u + n,
781 +
                // Counted now, with the new object already in place.
782 +
                None => self.cache.list("")?.iter().map(|e| e.size).sum(),
783 +
            };
784 +
            *used = Some(total);
785 +
            total
786 +
        };
787 +
        if total <= self.limit {
788 +
            return Ok(());
789 +
        }
790 +
        let mut entries = self.cache.list("")?;
791 +
        entries.sort_by_key(|e| e.modified_ms.unwrap_or(0));
792 +
        let mut left: u64 = entries.iter().map(|e| e.size).sum();
793 +
        let target = self.limit / 10 * 9;
794 +
        for e in entries {
795 +
            if left <= target {
796 +
                break;
797 +
            }
798 +
            self.cache.delete(&e.key)?;
799 +
            left -= e.size;
800 +
        }
801 +
        *self.used.lock().unwrap_or_else(|p| p.into_inner()) = Some(left);
802 +
        Ok(())
803 +
    }
804 +
805 +
    /// An object's size: kept on disk once the remote has said it (objects
806 +
    /// never change), so it is asked once. `None` when there is no object.
807 +
    fn size_of(&self, key: &str) -> Result<Option<u64>> {
808 +
        let skey = format!("{SIZES}/{key}");
809 +
        if let Some(n) = self
810 +
            .cache
811 +
            .get(&skey)
812 +
            .ok()
813 +
            .and_then(|b| String::from_utf8(b).ok())
814 +
            .and_then(|t| t.parse().ok())
815 +
        {
816 +
            return Ok(Some(n));
817 +
        }
818 +
        let size = self.remote.stat(key)?.map(|e| e.size);
819 +
        if let Some(n) = size {
820 +
            self.keep(&skey, n.to_string().as_bytes())?;
821 +
        }
822 +
        Ok(size)
823 +
    }
824 +
825 +
    /// Mark a cached object used now, at most once a minute.
826 +
    fn touch(&self, key: &str) {
827 +
        let Ok(path) = self.cache.path(key) else {
828 +
            return;
829 +
        };
830 +
        let stale = fs::metadata(&path)
831 +
            .and_then(|m| m.modified())
832 +
            .ok()
833 +
            .and_then(|t| t.elapsed().ok())
834 +
            .is_some_and(|age| age > TOUCH_AFTER);
835 +
        if stale {
836 +
            if let Ok(f) = fs::File::options().append(true).open(&path) {
837 +
                let _ = f.set_modified(std::time::SystemTime::now());
838 +
            }
839 +
        }
840 +
    }
841 +
}
842 +
843 +
impl Store for CachingStore {
844 +
    fn describe(&self) -> String {
845 +
        format!(
846 +
            "{} (cached in {})",
847 +
            self.remote.describe(),
848 +
            self.cache.root().display()
849 +
        )
850 +
    }
851 +
852 +
    fn has(&self, key: &str) -> Result<bool> {
853 +
        Ok(self.cache.has(key)? || self.remote.has(key)?)
854 +
    }
855 +
856 +
    fn get(&self, key: &str) -> Result<Vec<u8>> {
857 +
        match self.cache.get(key) {
858 +
            Ok(bytes) => {
859 +
                self.touch(key);
860 +
                return Ok(bytes);
861 +
            }
862 +
            // Not cached, or removed by another process's eviction meanwhile.
863 +
            Err(Error::NotFound(_)) => {}
864 +
            Err(e) => return Err(e),
865 +
        }
866 +
        let bytes = self.remote.get(key)?;
867 +
        self.keep(key, &bytes)?;
868 +
        Ok(bytes)
869 +
    }
870 +
871 +
    fn get_range(&self, key: &str, range: Range<u64>) -> Result<Vec<u8>> {
872 +
        match self.cache.get_range(key, range.clone()) {
873 +
            Ok(bytes) => {
874 +
                self.touch(key);
875 +
                return Ok(bytes);
876 +
            }
877 +
            Err(Error::NotFound(_)) => {}
878 +
            Err(e) if self.cache.has(key)? => return Err(e),
879 +
            Err(_) => {}
880 +
        }
881 +
        // A large object's ranges are kept as they are read (packs never
882 +
        // change, oxygenhub#234): a server opens the store again for every
883 +
        // request, so only what is on disk carries over.
884 +
        let rkey = format!("{RANGES}/{key}/{}-{}", range.start, range.end);
885 +
        if let Ok(bytes) = self.cache.get(&rkey) {
886 +
            self.touch(&rkey);
887 +
            return Ok(bytes);
888 +
        }
889 +
        // A small object is fetched whole and kept; a large one is read by
890 +
        // range and the range kept. Its size is kept too, so the remote is
891 +
        // asked it once.
892 +
        match self.size_of(key)? {
893 +
            Some(size) if self.cacheable(size) => {
894 +
                let bytes = self.get(key)?;
895 +
                let len = bytes.len() as u64;
896 +
                if range.end > len || range.start > range.end {
897 +
                    return Err(Error::Backend(format!(
898 +
                        "{key}: bytes {}..{} asked of an object of {len} bytes",
899 +
                        range.start, range.end
900 +
                    )));
901 +
                }
902 +
                Ok(bytes[range.start as usize..range.end as usize].to_vec())
903 +
            }
904 +
            Some(_) => {
905 +
                let bytes = self.remote.get_range(key, range)?;
906 +
                self.keep(&rkey, &bytes)?;
907 +
                Ok(bytes)
908 +
            }
909 +
            None => Err(Error::NotFound(key.to_string())),
910 +
        }
911 +
    }
912 +
913 +
    fn put_new(&self, key: &str, bytes: &[u8]) -> Result<()> {
914 +
        self.remote.put_new(key, bytes)?;
915 +
        self.keep(key, bytes)
916 +
    }
917 +
918 +
    fn put_many_new(&self, objects: &[(String, Vec<u8>)]) -> Result<()> {
919 +
        self.remote.put_many_new(objects)?;
920 +
        for (key, bytes) in objects {
921 +
            self.keep(key, bytes)?;
922 +
        }
923 +
        Ok(())
924 +
    }
925 +
926 +
    fn put_new_file(&self, key: &str, path: &Path) -> Result<()> {
927 +
        let size = fs::metadata(path)?.len();
928 +
        let copy = if self.cacheable(size) {
929 +
            Some(fs::read(path)?)
930 +
        } else {
931 +
            None
932 +
        };
933 +
        self.remote.put_new_file(key, path)?;
934 +
        match copy {
935 +
            Some(bytes) => self.keep(key, &bytes),
936 +
            None => Ok(()),
937 +
        }
938 +
    }
939 +
940 +
    fn list(&self, prefix: &str) -> Result<Vec<Entry>> {
941 +
        self.remote.list(prefix)
942 +
    }
943 +
944 +
    fn stat(&self, key: &str) -> Result<Option<Entry>> {
945 +
        self.remote.stat(key)
946 +
    }
947 +
948 +
    fn delete(&self, key: &str) -> Result<()> {
949 +
        self.remote.delete(key)?;
950 +
        // Its kept ranges and size go with it.
951 +
        let mut extra = self.cache.list(&format!("{RANGES}/{key}/"))?;
952 +
        extra.extend(self.cache.stat(&format!("{SIZES}/{key}"))?);
953 +
        for e in extra {
954 +
            self.cache.delete(&e.key)?;
955 +
            let mut used = self.used.lock().unwrap_or_else(|p| p.into_inner());
956 +
            if let Some(u) = used.as_mut() {
957 +
                *u = u.saturating_sub(e.size);
958 +
            }
959 +
        }
960 +
        let size = self.cache.stat(key)?.map(|e| e.size);
961 +
        self.cache.delete(key)?;
962 +
        if let Some(n) = size {
963 +
            let mut used = self.used.lock().unwrap_or_else(|p| p.into_inner());
964 +
            if let Some(u) = used.as_mut() {
965 +
                *u = u.saturating_sub(n);
966 +
            }
967 +
        }
968 +
        Ok(())
969 +
    }
970 +
971 +
    fn read_ref(&self, key: &str) -> Result<Option<(Vec<u8>, Version)>> {
972 +
        self.remote.read_ref(key)
973 +
    }
974 +
975 +
    fn write_ref(&self, key: &str, bytes: &[u8], expected: Option<&Version>) -> Result<Version> {
976 +
        self.remote.write_ref(key, bytes, expected)
977 +
    }
978 +
}
979 +
661 980
// ---------------------------------------------------------------- conformance
662 981
663 982
/// Exercise a store under `prefix` (which must be free, and is left
…
144 unchanged lines
808 1127
    use super::*;
809 1128
    use std::sync::Arc;
810 1129
1130 +
    /// Where hard links are refused (Android, oxygen#141), objects are
1131 +
    /// renamed into place, still never over a name that is taken.
811 1132
    #[test]
1133 +
    fn a_create_only_rename_never_replaces() {
1134 +
        let dir = tempfile::tempdir().unwrap();
1135 +
        let (a, b, to) = (
1136 +
            dir.path().join("a"),
1137 +
            dir.path().join("b"),
1138 +
            dir.path().join("to"),
1139 +
        );
1140 +
        fs::write(&a, b"first").unwrap();
1141 +
        fs::write(&b, b"second").unwrap();
1142 +
        rename_new(&a, &to).unwrap();
1143 +
        assert_eq!(
1144 +
            rename_new(&b, &to).unwrap_err().kind(),
1145 +
            io::ErrorKind::AlreadyExists
1146 +
        );
1147 +
        assert_eq!(fs::read(&to).unwrap(), b"first");
1148 +
        assert!(!a.exists() && b.exists());
1149 +
    }
1150 +
1151 +
    #[test]
812 1152
    fn keys_are_plain_relative_paths() {
813 1153
        for ok in ["a", "packs/ab.pack", "a-b_c.d/e"] {
814 1154
            check_key(ok).unwrap();
…
7 unchanged lines
822 1162
    }
823 1163
824 1164
    #[test]
1165 +
    fn a_caching_store_conforms_and_reads_and_writes_through() {
1166 +
        let dir = tempfile::tempdir().unwrap();
1167 +
        let remote: Arc<dyn Store> = Arc::new(MemoryStore::new());
1168 +
        let store = CachingStore::open(remote.clone(), dir.path().join("cache"), 8 << 20).unwrap();
1169 +
        assert_eq!(conformance(&store, "probe").unwrap().len(), 8);
1170 +
        assert_eq!(
1171 +
            store.cached_bytes().unwrap(),
1172 +
            0,
1173 +
            "the probe left nothing cached"
1174 +
        );
1175 +
1176 +
        // Written through: the remote has it, and so does the cache.
1177 +
        store.put_new("loose/aa", b"written").unwrap();
1178 +
        assert_eq!(remote.get("loose/aa").unwrap(), b"written");
1179 +
        assert!(store.cache.has("loose/aa").unwrap());
1180 +
        // Read through: an object only the remote has is kept on first read.
1181 +
        remote.put_new("loose/bb", b"remote only").unwrap();
1182 +
        assert!(!store.cache.has("loose/bb").unwrap());
1183 +
        assert_eq!(store.get_range("loose/bb", 0..6).unwrap(), b"remote");
1184 +
        assert!(store.cache.has("loose/bb").unwrap());
1185 +
        // A cached object read after the remote lost it is still served.
1186 +
        remote.delete("loose/bb").unwrap();
1187 +
        assert_eq!(store.get("loose/bb").unwrap(), b"remote only");
1188 +
        // References are always the remote's.
1189 +
        let v = remote.write_ref("refs", b"one", None).unwrap();
1190 +
        assert_eq!(
1191 +
            store.read_ref("refs").unwrap(),
1192 +
            Some((b"one".to_vec(), v.clone()))
1193 +
        );
1194 +
        remote.write_ref("refs", b"two", Some(&v)).unwrap();
1195 +
        assert_eq!(store.read_ref("refs").unwrap().unwrap().0, b"two");
1196 +
        assert!(matches!(
1197 +
            store.write_ref("refs", b"three", Some(&v)),
1198 +
            Err(Error::Conflict(_))
1199 +
        ));
1200 +
        assert!(!store.cache.has("refs").unwrap());
1201 +
    }
1202 +
1203 +
    /// A remote that answers nothing: what is read through it was cached.
1204 +
    struct Unreachable;
1205 +
1206 +
    impl Store for Unreachable {
1207 +
        fn describe(&self) -> String {
1208 +
            "unreachable".into()
1209 +
        }
1210 +
        fn has(&self, _: &str) -> Result<bool> {
1211 +
            Err(Error::Backend("asked the remote".into()))
1212 +
        }
1213 +
        fn get(&self, _: &str) -> Result<Vec<u8>> {
1214 +
            Err(Error::Backend("asked the remote".into()))
1215 +
        }
1216 +
        fn get_range(&self, _: &str, _: Range<u64>) -> Result<Vec<u8>> {
1217 +
            Err(Error::Backend("asked the remote".into()))
1218 +
        }
1219 +
        fn put_new(&self, _: &str, _: &[u8]) -> Result<()> {
1220 +
            Err(Error::Backend("asked the remote".into()))
1221 +
        }
1222 +
        fn list(&self, _: &str) -> Result<Vec<Entry>> {
1223 +
            Err(Error::Backend("asked the remote".into()))
1224 +
        }
1225 +
        fn delete(&self, _: &str) -> Result<()> {
1226 +
            Err(Error::Backend("asked the remote".into()))
1227 +
        }
1228 +
        fn stat(&self, _: &str) -> Result<Option<Entry>> {
1229 +
            Err(Error::Backend("asked the remote".into()))
1230 +
        }
1231 +
        fn read_ref(&self, _: &str) -> Result<Option<(Vec<u8>, Version)>> {
1232 +
            Err(Error::Backend("asked the remote".into()))
1233 +
        }
1234 +
        fn write_ref(&self, _: &str, _: &[u8], _: Option<&Version>) -> Result<Version> {
1235 +
            Err(Error::Backend("asked the remote".into()))
1236 +
        }
1237 +
    }
1238 +
1239 +
    #[test]
1240 +
    fn a_caching_store_stays_within_its_limit() {
1241 +
        let dir = tempfile::tempdir().unwrap();
1242 +
        let remote: Arc<dyn Store> = Arc::new(MemoryStore::new());
1243 +
        let store = CachingStore::open(remote.clone(), dir.path().join("cache"), 8000).unwrap();
1244 +
        for i in 0..20 {
1245 +
            store
1246 +
                .put_new(&format!("loose/{i:02}"), &[i as u8; 900])
1247 +
                .unwrap();
1248 +
            assert!(store.cached_bytes().unwrap() <= 8000);
1249 +
        }
1250 +
        assert_eq!(
1251 +
            remote.list("loose/").unwrap().len(),
1252 +
            20,
1253 +
            "the remote holds everything"
1254 +
        );
1255 +
        let cached = store.cache.list("loose/").unwrap();
1256 +
        assert!(cached.len() < 20 && cached.len() >= 7, "{}", cached.len());
1257 +
        assert!(
1258 +
            cached.iter().any(|e| e.key == "loose/19"),
1259 +
            "the newest is kept"
1260 +
        );
1261 +
        for i in 0..20 {
1262 +
            assert_eq!(
1263 +
                store.get(&format!("loose/{i:02}")).unwrap(),
1264 +
                vec![i as u8; 900]
1265 +
            );
1266 +
        }
1267 +
        // An object over an eighth of the limit is read by range, not kept.
1268 +
        remote.put_new("pack-big", &[7u8; 2000]).unwrap();
1269 +
        assert_eq!(store.get_range("pack-big", 10..20).unwrap(), vec![7u8; 10]);
1270 +
        assert_eq!(store.get("pack-big").unwrap().len(), 2000);
1271 +
        assert!(!store.cache.has("pack-big").unwrap());
1272 +
        // Its ranges are kept as read, and its size, so a store opened
1273 +
        // again (a server's next request) asks the remote nothing.
1274 +
        let again =
1275 +
            CachingStore::open(Arc::new(Unreachable), dir.path().join("cache"), 8000).unwrap();
1276 +
        assert_eq!(again.get_range("pack-big", 10..20).unwrap(), vec![7u8; 10]);
1277 +
        // Counting from scratch agrees with what was kept track of.
1278 +
        let fresh = CachingStore::open(remote, dir.path().join("cache"), 8000).unwrap();
1279 +
        assert_eq!(fresh.cached_bytes().unwrap(), store.cached_bytes().unwrap());
1280 +
        // Deleting it takes its ranges and size too.
1281 +
        store.delete("pack-big").unwrap();
1282 +
        assert!(
1283 +
            store.cache.list("cached-range/").unwrap().is_empty()
1284 +
                && store.cache.list("cached-size/").unwrap().is_empty()
1285 +
        );
1286 +
    }
1287 +
1288 +
    #[test]
825 1289
    fn the_memory_store_conforms() {
826 1290
        let store = MemoryStore::new();
827 1291
        let passed = conformance(&store, "probe").unwrap();
…
106 unchanged lines
modified crates/oxy-core/src/subrepo.rs +30 −0
…
25 unchanged lines
26 26
}
27 27
28 28
impl Pin {
29 +
    /// Whether two pins name the same state: the same repository at the
30 +
    /// same heads, in any order, or the same Git commit. The bookmark or
31 +
    /// branch each names does not count (#147): a subrepo at its pinned
32 +
    /// state is at it whatever bookmark it sits on.
33 +
    pub fn same_state(&self, other: &Pin) -> bool {
34 +
        match (self, other) {
35 +
            (
36 +
                Pin::Oxy {
37 +
                    repo: r1,
38 +
                    heads: h1,
39 +
                    ..
40 +
                },
41 +
                Pin::Oxy {
42 +
                    repo: r2,
43 +
                    heads: h2,
44 +
                    ..
45 +
                },
46 +
            ) => {
47 +
                let (mut a, mut b) = (h1.clone(), h2.clone());
48 +
                a.sort();
49 +
                a.dedup();
50 +
                b.sort();
51 +
                b.dedup();
52 +
                r1 == r2 && a == b
53 +
            }
54 +
            (Pin::Git { sha: a, .. }, Pin::Git { sha: b, .. }) => a == b,
55 +
            _ => false,
56 +
        }
57 +
    }
58 +
29 59
    pub fn encode(&self) -> Vec<u8> {
30 60
        let v = match self {
31 61
            Pin::Oxy {
…
278 unchanged lines
added crates/oxy-core/src/tlog.rs +1646 −0
1 +
//! The transparency log (spec §14.8, oxygenhub#238, oxygenhub
2 +
//! docs/transparency.md): an append-only Merkle tree of what a hub
3 +
//! accepts, served as C2SP tlog-tiles with C2SP signed-note checkpoints,
4 +
//! so a hub cannot show one history to some and another to others.
5 +
//!
6 +
//! The tree is RFC 6962's, with BLAKE3 for its hashes, domain-separated
7 +
//! for leaves and nodes. Hashes are kept in *stored order*: every hash of
8 +
//! the tree, each written as soon as it is known, so the store is
9 +
//! append-only (the arithmetic is that of Go's `sumdb/tlog`). Tiles are
10 +
//! views of it: level L tile N holds the hashes of tree level 8·L for
11 +
//! leaves N·256 onwards.
12 +
//!
13 +
//! A checkpoint is a signed note: the origin, the tree size and the root,
14 +
//! signed with the hub's Ed25519 key (what note tools read) and with its
15 +
//! composite key in an extra line, so the log stays post-quantum.
16 +
17 +
use crate::cbor::{self, Value};
18 +
use crate::hash::{KeyId, ObjectRef};
19 +
use crate::identity::{verify, Domain, Identity, SecretKey, Signature};
20 +
use crate::objects::{array, bytes, check_version, field, map, oref, text, uint, ObjectError};
21 +
use sha2::{Digest, Sha256};
22 +
23 +
/// A hash in the tree.
24 +
pub type Hash = [u8; 32];
25 +
26 +
/// Tile height: a tile holds up to 2⁸ hashes (C2SP tlog-tiles).
27 +
pub const TILE_HEIGHT: u32 = 8;
28 +
/// Hashes in a full tile.
29 +
pub const TILE_WIDTH: u64 = 1 << TILE_HEIGHT;
30 +
31 +
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
32 +
pub enum TlogError {
33 +
    #[error("the proof does not verify")]
34 +
    ProofFailed,
35 +
    #[error("{0}")]
36 +
    BadIndex(String),
37 +
    #[error("reading the log's hashes: {0}")]
38 +
    Read(String),
39 +
    #[error("not a valid note: {0}")]
40 +
    BadNote(String),
41 +
    #[error("{0}")]
42 +
    BadSignature(String),
43 +
    #[error("{0}")]
44 +
    Schema(String),
45 +
}
46 +
47 +
impl From<ObjectError> for TlogError {
48 +
    fn from(e: ObjectError) -> Self {
49 +
        TlogError::Schema(e.to_string())
50 +
    }
51 +
}
52 +
53 +
type Result<T> = std::result::Result<T, TlogError>;
54 +
55 +
// ------------------------------------------------------------- hashing
56 +
57 +
/// A leaf's hash: `BLAKE3("oxygen-tlog-leaf\0" || entry bytes)`.
58 +
pub fn leaf_hash(entry: &[u8]) -> Hash {
59 +
    let mut h = blake3::Hasher::new();
60 +
    h.update(b"oxygen-tlog-leaf\0");
61 +
    h.update(entry);
62 +
    *h.finalize().as_bytes()
63 +
}
64 +
65 +
/// An interior node's hash: `BLAKE3("oxygen-tlog-node\0" || left || right)`.
66 +
pub fn node_hash(left: &Hash, right: &Hash) -> Hash {
67 +
    let mut h = blake3::Hasher::new();
68 +
    h.update(b"oxygen-tlog-node\0");
69 +
    h.update(left);
70 +
    h.update(right);
71 +
    *h.finalize().as_bytes()
72 +
}
73 +
74 +
/// The root of the empty tree: the hash of nothing, as RFC 6962 has it.
75 +
pub fn empty_root() -> Hash {
76 +
    *blake3::hash(b"").as_bytes()
77 +
}
78 +
79 +
// --------------------------------------------------------------- entries
80 +
81 +
/// One accepted object (spec §14.8): small and fixed-shape, never the
82 +
/// object itself, so a private repository's entries reveal only ids and
83 +
/// hashes.
84 +
#[derive(Clone, Debug, PartialEq, Eq)]
85 +
pub struct Entry {
86 +
    /// `ref`, `seal`, `tag`, `rotate`, `revoke`, `receipt`, `approval`,
87 +
    /// `policy`, `identity`, `certificate`, `accepted`.
88 +
    pub kind: String,
89 +
    /// The repository's id; zero for identities.
90 +
    pub repo: [u8; 16],
91 +
    /// The attestation, policy, identity or certificate.
92 +
    pub object: ObjectRef,
93 +
    /// The keys whose signatures were checked.
94 +
    pub signers: Vec<KeyId>,
95 +
    /// The hub's clock when it accepted it, in milliseconds (its claim).
96 +
    pub received: u64,
97 +
}
98 +
99 +
impl Entry {
100 +
    pub fn to_value(&self) -> Value {
101 +
        Value::map([
102 +
            (0, Value::Uint(1)),
103 +
            (1, Value::Text(self.kind.clone())),
104 +
            (2, Value::Bytes(self.repo.to_vec())),
105 +
            (3, Value::Bytes(self.object.0.to_vec())),
106 +
            (
107 +
                4,
108 +
                Value::Array(
109 +
                    self.signers
110 +
                        .iter()
111 +
                        .map(|k| Value::Bytes(k.0.to_vec()))
112 +
                        .collect(),
113 +
                ),
114 +
            ),
115 +
            (5, Value::Uint(self.received)),
116 +
        ])
117 +
    }
118 +
119 +
    pub fn encode(&self) -> Vec<u8> {
120 +
        cbor::encode(&self.to_value())
121 +
    }
122 +
123 +
    pub fn decode(b: &[u8]) -> Result<Entry> {
124 +
        let v = cbor::decode(b).map_err(ObjectError::from)?;
125 +
        let m = map(&v, "log entry")?;
126 +
        check_version(m)?;
127 +
        let repo = bytes(field(m, 2, "repo")?, "repo")?;
128 +
        let repo: [u8; 16] = repo
129 +
            .try_into()
130 +
            .map_err(|_| TlogError::Schema("log entry: repo is not 16 bytes".into()))?;
131 +
        let mut signers = Vec::new();
132 +
        for s in array(field(m, 4, "signers")?, "signers")? {
133 +
            signers.push(oref(s, "signer")?);
134 +
        }
135 +
        Ok(Entry {
136 +
            kind: text(field(m, 1, "kind")?, "kind")?.to_owned(),
137 +
            repo,
138 +
            object: oref(field(m, 3, "object")?, "object")?,
139 +
            signers,
140 +
            received: uint(field(m, 5, "received")?, "received")?,
141 +
        })
142 +
    }
143 +
144 +
    /// Its leaf hash in the tree.
145 +
    pub fn leaf_hash(&self) -> Hash {
146 +
        leaf_hash(&self.encode())
147 +
    }
148 +
}
149 +
150 +
/// An entry bundle (C2SP tlog-tiles): each entry as a big-endian u16
151 +
/// length and its bytes.
152 +
pub fn encode_bundle(entries: &[Vec<u8>]) -> Result<Vec<u8>> {
153 +
    let mut out = Vec::new();
154 +
    for e in entries {
155 +
        let n = u16::try_from(e.len())
156 +
            .map_err(|_| TlogError::Schema("a log entry over 65535 bytes".into()))?;
157 +
        out.extend_from_slice(&n.to_be_bytes());
158 +
        out.extend_from_slice(e);
159 +
    }
160 +
    Ok(out)
161 +
}
162 +
163 +
/// The entries of a bundle.
164 +
pub fn decode_bundle(mut b: &[u8]) -> Result<Vec<Vec<u8>>> {
165 +
    let mut out = Vec::new();
166 +
    while !b.is_empty() {
167 +
        if b.len() < 2 {
168 +
            return Err(TlogError::Schema("entry bundle: a truncated length".into()));
169 +
        }
170 +
        let n = u16::from_be_bytes([b[0], b[1]]) as usize;
171 +
        if b.len() < 2 + n {
172 +
            return Err(TlogError::Schema("entry bundle: a truncated entry".into()));
173 +
        }
174 +
        out.push(b[2..2 + n].to_vec());
175 +
        b = &b[2 + n..];
176 +
    }
177 +
    Ok(out)
178 +
}
179 +
180 +
// -------------------------------------------------- the tree's arithmetic
181 +
182 +
/// Reads stored hashes by their stored index.
183 +
pub trait HashReader {
184 +
    /// The hashes at `indexes`, in that order.
185 +
    fn read_hashes(&self, indexes: &[u64]) -> Result<Vec<Hash>>;
186 +
}
187 +
188 +
/// Every stored hash in order, as a log in memory holds them.
189 +
impl HashReader for [Hash] {
190 +
    fn read_hashes(&self, indexes: &[u64]) -> Result<Vec<Hash>> {
191 +
        indexes
192 +
            .iter()
193 +
            .map(|&i| {
194 +
                self.get(i as usize)
195 +
                    .copied()
196 +
                    .ok_or_else(|| TlogError::Read(format!("no stored hash {i}")))
197 +
            })
198 +
            .collect()
199 +
    }
200 +
}
201 +
202 +
impl HashReader for Vec<Hash> {
203 +
    fn read_hashes(&self, indexes: &[u64]) -> Result<Vec<Hash>> {
204 +
        self.as_slice().read_hashes(indexes)
205 +
    }
206 +
}
207 +
208 +
/// The largest power of two below `n` (n ≥ 2), and its log.
209 +
fn maxpow2(n: u64) -> (u64, u32) {
210 +
    let mut l = 0;
211 +
    while (1u64 << (l + 1)) < n {
212 +
        l += 1;
213 +
    }
214 +
    (1 << l, l)
215 +
}
216 +
217 +
/// Where the `n`th hash of tree level `level` is stored: level L's nth
218 +
/// hash is written right after level L+1's (2n+1)th, and level 0's nth
219 +
/// at n + n/2 + n/4 + ….
220 +
pub fn stored_hash_index(level: u32, mut n: u64) -> u64 {
221 +
    for _ in 0..level {
222 +
        n = 2 * n + 1;
223 +
    }
224 +
    let mut i = 0;
225 +
    let mut m = n;
226 +
    while m > 0 {
227 +
        i += m;
228 +
        m >>= 1;
229 +
    }
230 +
    i + level as u64
231 +
}
232 +
233 +
/// The level and position of the hash at a stored index.
234 +
pub fn split_stored_hash_index(index: u64) -> (u32, u64) {
235 +
    let mut n = index / 2;
236 +
    let mut index_n = stored_hash_index(0, n);
237 +
    debug_assert!(index_n <= index);
238 +
    loop {
239 +
        // Each new leaf n adds 1 + trailing_zeros(n+1) hashes.
240 +
        let x = index_n + 1 + (n + 1).trailing_zeros() as u64;
241 +
        if x > index {
242 +
            break;
243 +
        }
244 +
        n += 1;
245 +
        index_n = x;
246 +
    }
247 +
    let level = (index - index_n) as u32;
248 +
    (level, n >> level)
249 +
}
250 +
251 +
/// How many hashes a tree of `n` leaves stores.
252 +
pub fn stored_hash_count(n: u64) -> u64 {
253 +
    if n == 0 {
254 +
        return 0;
255 +
    }
256 +
    let mut count = stored_hash_index(0, n - 1) + 1;
257 +
    let mut i = n - 1;
258 +
    while i & 1 != 0 {
259 +
        count += 1;
260 +
        i >>= 1;
261 +
    }
262 +
    count
263 +
}
264 +
265 +
/// The hashes to append when leaf `n` (with hash `leaf`) joins a tree of
266 +
/// `n` leaves: the leaf's, then those of the subtrees it completes.
267 +
pub fn stored_hashes_for_leaf(
268 +
    n: u64,
269 +
    leaf: Hash,
270 +
    r: &(impl HashReader + ?Sized),
271 +
) -> Result<Vec<Hash>> {
272 +
    let m = (n + 1).trailing_zeros();
273 +
    let indexes: Vec<u64> = (0..m)
274 +
        .rev()
275 +
        .map(|i| stored_hash_index(i, (n >> i) - 1))
276 +
        .collect();
277 +
    let old = r.read_hashes(&indexes)?;
278 +
    let mut h = leaf;
279 +
    let mut out = vec![h];
280 +
    for i in 0..m as usize {
281 +
        h = node_hash(&old[m as usize - 1 - i], &h);
282 +
        out.push(h);
283 +
    }
284 +
    Ok(out)
285 +
}
286 +
287 +
fn sub_tree_index(mut lo: u64, hi: u64, need: &mut Vec<u64>) {
288 +
    while lo < hi {
289 +
        let (k, level) = maxpow2(hi - lo + 1);
290 +
        debug_assert_eq!(lo & (k - 1), 0);
291 +
        need.push(stored_hash_index(level, lo >> level));
292 +
        lo += k;
293 +
    }
294 +
}
295 +
296 +
fn sub_tree_hash(mut lo: u64, hi: u64, hashes: &[Hash]) -> (Hash, &[Hash]) {
297 +
    let mut count = 0;
298 +
    while lo < hi {
299 +
        let (k, _) = maxpow2(hi - lo + 1);
300 +
        count += 1;
301 +
        lo += k;
302 +
    }
303 +
    let mut h = hashes[count - 1];
304 +
    for i in (0..count - 1).rev() {
305 +
        h = node_hash(&hashes[i], &h);
306 +
    }
307 +
    (h, &hashes[count..])
308 +
}
309 +
310 +
/// The root of the first `n` leaves.
311 +
pub fn tree_hash(n: u64, r: &(impl HashReader + ?Sized)) -> Result<Hash> {
312 +
    if n == 0 {
313 +
        return Ok(empty_root());
314 +
    }
315 +
    let mut need = Vec::new();
316 +
    sub_tree_index(0, n, &mut need);
317 +
    let hashes = r.read_hashes(&need)?;
318 +
    Ok(sub_tree_hash(0, n, &hashes).0)
319 +
}
320 +
321 +
fn leaf_proof_index(lo: u64, hi: u64, n: u64, need: &mut Vec<u64>) {
322 +
    if lo + 1 == hi {
323 +
        return;
324 +
    }
325 +
    let (k, _) = maxpow2(hi - lo);
326 +
    if n < lo + k {
327 +
        leaf_proof_index(lo, lo + k, n, need);
328 +
        sub_tree_index(lo + k, hi, need);
329 +
    } else {
330 +
        sub_tree_index(lo, lo + k, need);
331 +
        leaf_proof_index(lo + k, hi, n, need);
332 +
    }
333 +
}
334 +
335 +
fn leaf_proof(lo: u64, hi: u64, n: u64, hashes: &[Hash]) -> (Vec<Hash>, &[Hash]) {
336 +
    if lo + 1 == hi {
337 +
        // The verifier has the leaf's hash.
338 +
        return (Vec::new(), hashes);
339 +
    }
340 +
    let (k, _) = maxpow2(hi - lo);
341 +
    let (mut p, th, rest);
342 +
    if n < lo + k {
343 +
        let (pp, h1) = leaf_proof(lo, lo + k, n, hashes);
344 +
        let (t, h2) = sub_tree_hash(lo + k, hi, h1);
345 +
        (p, th, rest) = (pp, t, h2);
346 +
    } else {
347 +
        let (t, h1) = sub_tree_hash(lo, lo + k, hashes);
348 +
        let (pp, h2) = leaf_proof(lo + k, hi, n, h1);
349 +
        (p, th, rest) = (pp, t, h2);
350 +
    }
351 +
    p.push(th);
352 +
    (p, rest)
353 +
}
354 +
355 +
/// The inclusion proof of leaf `n` in the tree of the first `t` leaves.
356 +
pub fn prove_inclusion(t: u64, n: u64, r: &(impl HashReader + ?Sized)) -> Result<Vec<Hash>> {
357 +
    if n >= t {
358 +
        return Err(TlogError::BadIndex(format!(
359 +
            "leaf {n} is not in a tree of {t}"
360 +
        )));
361 +
    }
362 +
    let mut need = Vec::new();
363 +
    leaf_proof_index(0, t, n, &mut need);
364 +
    if need.is_empty() {
365 +
        return Ok(Vec::new());
366 +
    }
367 +
    let hashes = r.read_hashes(&need)?;
368 +
    Ok(leaf_proof(0, t, n, &hashes).0)
369 +
}
370 +
371 +
fn run_inclusion(p: &[Hash], lo: u64, hi: u64, n: u64, leaf: Hash) -> Result<Hash> {
372 +
    if lo + 1 == hi {
373 +
        return if p.is_empty() {
374 +
            Ok(leaf)
375 +
        } else {
376 +
            Err(TlogError::ProofFailed)
377 +
        };
378 +
    }
379 +
    let Some((last, rest)) = p.split_last() else {
380 +
        return Err(TlogError::ProofFailed);
381 +
    };
382 +
    let (k, _) = maxpow2(hi - lo);
383 +
    if n < lo + k {
384 +
        Ok(node_hash(&run_inclusion(rest, lo, lo + k, n, leaf)?, last))
385 +
    } else {
386 +
        Ok(node_hash(last, &run_inclusion(rest, lo + k, hi, n, leaf)?))
387 +
    }
388 +
}
389 +
390 +
/// Check that leaf `n`, with hash `leaf`, is in the tree of `t` leaves
391 +
/// whose root is `root`.
392 +
pub fn check_inclusion(proof: &[Hash], t: u64, root: &Hash, n: u64, leaf: &Hash) -> Result<()> {
393 +
    if n >= t {
394 +
        return Err(TlogError::ProofFailed);
395 +
    }
396 +
    if run_inclusion(proof, 0, t, n, *leaf)? == *root {
397 +
        Ok(())
398 +
    } else {
399 +
        Err(TlogError::ProofFailed)
400 +
    }
401 +
}
402 +
403 +
fn tree_proof_index(lo: u64, hi: u64, n: u64, need: &mut Vec<u64>) {
404 +
    if n == hi {
405 +
        if lo != 0 {
406 +
            sub_tree_index(lo, hi, need);
407 +
        }
408 +
        return;
409 +
    }
410 +
    let (k, _) = maxpow2(hi - lo);
411 +
    if n <= lo + k {
412 +
        tree_proof_index(lo, lo + k, n, need);
413 +
        sub_tree_index(lo + k, hi, need);
414 +
    } else {
415 +
        sub_tree_index(lo, lo + k, need);
416 +
        tree_proof_index(lo + k, hi, n, need);
417 +
    }
418 +
}
419 +
420 +
fn tree_proof(lo: u64, hi: u64, n: u64, hashes: &[Hash]) -> (Vec<Hash>, &[Hash]) {
421 +
    if n == hi {
422 +
        if lo == 0 {
423 +
            // Exactly the old tree, whose root the verifier has.
424 +
            return (Vec::new(), hashes);
425 +
        }
426 +
        let (th, rest) = sub_tree_hash(lo, hi, hashes);
427 +
        return (vec![th], rest);
428 +
    }
429 +
    let (k, _) = maxpow2(hi - lo);
430 +
    let (mut p, th, rest);
431 +
    if n <= lo + k {
432 +
        let (pp, h1) = tree_proof(lo, lo + k, n, hashes);
433 +
        let (t, h2) = sub_tree_hash(lo + k, hi, h1);
434 +
        (p, th, rest) = (pp, t, h2);
435 +
    } else {
436 +
        let (t, h1) = sub_tree_hash(lo, lo + k, hashes);
437 +
        let (pp, h2) = tree_proof(lo + k, hi, n, h1);
438 +
        (p, th, rest) = (pp, t, h2);
439 +
    }
440 +
    p.push(th);
441 +
    (p, rest)
442 +
}
443 +
444 +
/// The consistency proof that the tree of `n` leaves is a prefix of the
445 +
/// tree of `t`.
446 +
pub fn prove_consistency(t: u64, n: u64, r: &(impl HashReader + ?Sized)) -> Result<Vec<Hash>> {
447 +
    if n < 1 || n > t {
448 +
        return Err(TlogError::BadIndex(format!(
449 +
            "no consistency proof from {n} to {t}"
450 +
        )));
451 +
    }
452 +
    let mut need = Vec::new();
453 +
    tree_proof_index(0, t, n, &mut need);
454 +
    if need.is_empty() {
455 +
        return Ok(Vec::new());
456 +
    }
457 +
    let hashes = r.read_hashes(&need)?;
458 +
    Ok(tree_proof(0, t, n, &hashes).0)
459 +
}
460 +
461 +
fn run_consistency(p: &[Hash], lo: u64, hi: u64, n: u64, old: Hash) -> Result<(Hash, Hash)> {
462 +
    if n == hi {
463 +
        if lo == 0 {
464 +
            return if p.is_empty() {
465 +
                Ok((old, old))
466 +
            } else {
467 +
                Err(TlogError::ProofFailed)
468 +
            };
469 +
        }
470 +
        return match p {
471 +
            [h] => Ok((*h, *h)),
472 +
            _ => Err(TlogError::ProofFailed),
473 +
        };
474 +
    }
475 +
    let Some((last, rest)) = p.split_last() else {
476 +
        return Err(TlogError::ProofFailed);
477 +
    };
478 +
    let (k, _) = maxpow2(hi - lo);
479 +
    if n <= lo + k {
480 +
        let (oh, th) = run_consistency(rest, lo, lo + k, n, old)?;
481 +
        Ok((oh, node_hash(&th, last)))
482 +
    } else {
483 +
        let (oh, th) = run_consistency(rest, lo + k, hi, n, old)?;
484 +
        Ok((node_hash(last, &oh), node_hash(last, &th)))
485 +
    }
486 +
}
487 +
488 +
/// Check that the tree of `n` leaves with root `old_root` is a prefix of
489 +
/// the tree of `t` leaves with root `root`.
490 +
pub fn check_consistency(
491 +
    proof: &[Hash],
492 +
    t: u64,
493 +
    root: &Hash,
494 +
    n: u64,
495 +
    old_root: &Hash,
496 +
) -> Result<()> {
497 +
    if n < 1 || n > t {
498 +
        return Err(TlogError::ProofFailed);
499 +
    }
500 +
    let (oh, th) = run_consistency(proof, 0, t, n, *old_root)?;
501 +
    if oh == *old_root && th == *root {
502 +
        Ok(())
503 +
    } else {
504 +
        Err(TlogError::ProofFailed)
505 +
    }
506 +
}
507 +
508 +
// ----------------------------------------------------------------- tiles
509 +
510 +
/// A tile (C2SP tlog-tiles, height 8): level `level`'s hashes for leaves
511 +
/// from `n`·256, `width` of them (256 when full).
512 +
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
513 +
pub struct Tile {
514 +
    pub level: u32,
515 +
    pub n: u64,
516 +
    pub width: u32,
517 +
}
518 +
519 +
/// A tile index as a path: three-digit groups, all but the last with an
520 +
/// `x`, as `x001/x234/067` for 1234067.
521 +
pub fn index_path(n: u64) -> String {
522 +
    let mut groups = Vec::new();
523 +
    let mut m = n;
524 +
    loop {
525 +
        groups.push(m % 1000);
526 +
        m /= 1000;
527 +
        if m == 0 {
528 +
            break;
529 +
        }
530 +
    }
531 +
    groups.reverse();
532 +
    let last = groups.len() - 1;
533 +
    groups
534 +
        .iter()
535 +
        .enumerate()
536 +
        .map(|(i, g)| {
537 +
            if i < last {
538 +
                format!("x{g:03}")
539 +
            } else {
540 +
                format!("{g:03}")
541 +
            }
542 +
        })
543 +
        .collect::<Vec<_>>()
544 +
        .join("/")
545 +
}
546 +
547 +
/// The index a path names (the inverse of [`index_path`]).
548 +
pub fn parse_index_path(path: &str) -> Option<u64> {
549 +
    let parts: Vec<&str> = path.split('/').collect();
550 +
    let mut n: u64 = 0;
551 +
    for (i, p) in parts.iter().enumerate() {
552 +
        let digits = if i + 1 < parts.len() {
553 +
            p.strip_prefix('x')?
554 +
        } else {
555 +
            p
556 +
        };
557 +
        if digits.len() != 3 || !digits.bytes().all(|c| c.is_ascii_digit()) {
558 +
            return None;
559 +
        }
560 +
        n = n.checked_mul(1000)?.checked_add(digits.parse().ok()?)?;
561 +
    }
562 +
    Some(n)
563 +
}
564 +
565 +
impl Tile {
566 +
    /// `tile/<L>/<N>[.p/<W>]`.
567 +
    pub fn path(&self) -> String {
568 +
        format!(
569 +
            "tile/{}/{}{}",
570 +
            self.level,
571 +
            index_path(self.n),
572 +
            self.partial()
573 +
        )
574 +
    }
575 +
576 +
    /// The entry bundle beside a level-0 tile: `tile/entries/<N>[.p/<W>]`.
577 +
    pub fn entries_path(&self) -> String {
578 +
        format!("tile/entries/{}{}", index_path(self.n), self.partial())
579 +
    }
580 +
581 +
    fn partial(&self) -> String {
582 +
        if (self.width as u64) < TILE_WIDTH {
583 +
            format!(".p/{}", self.width)
584 +
        } else {
585 +
            String::new()
586 +
        }
587 +
    }
588 +
589 +
    /// The tile a path names (`<L>/<N>[.p/<W>]` after `tile/`).
590 +
    pub fn parse(path: &str) -> Option<Tile> {
591 +
        let (level, rest) = path.split_once('/')?;
592 +
        let level: u32 = level.parse().ok()?;
593 +
        let (index, width) = match rest.split_once(".p/") {
594 +
            Some((i, w)) => {
595 +
                let w: u32 = w.parse().ok()?;
596 +
                if w == 0 || w as u64 >= TILE_WIDTH {
597 +
                    return None;
598 +
                }
599 +
                (i, w)
600 +
            }
601 +
            None => (rest, TILE_WIDTH as u32),
602 +
        };
603 +
        Some(Tile {
604 +
            level,
605 +
            n: parse_index_path(index)?,
606 +
            width,
607 +
        })
608 +
    }
609 +
610 +
    /// The tile holding the stored hash at `index`, and the byte range its
611 +
    /// hash comes from in the tile's data.
612 +
    pub fn for_index(index: u64) -> (Tile, usize, usize) {
613 +
        let (level, mut n) = split_stored_hash_index(index);
614 +
        let tl = level / TILE_HEIGHT;
615 +
        let within = level - tl * TILE_HEIGHT;
616 +
        let tn = (n << within) >> TILE_HEIGHT;
617 +
        n -= (tn << TILE_HEIGHT) >> within;
618 +
        let width = ((n + 1) << within) as u32;
619 +
        let tile = Tile {
620 +
            level: tl,
621 +
            n: tn,
622 +
            width,
623 +
        };
624 +
        (
625 +
            tile,
626 +
            ((n << within) * 32) as usize,
627 +
            (((n + 1) << within) * 32) as usize,
628 +
        )
629 +
    }
630 +
631 +
    /// The tile's data: its hashes, read from stored order.
632 +
    pub fn read_data(&self, r: &(impl HashReader + ?Sized)) -> Result<Vec<u8>> {
633 +
        let start = self.n << TILE_HEIGHT;
634 +
        let indexes: Vec<u64> = (0..self.width as u64)
635 +
            .map(|i| stored_hash_index(TILE_HEIGHT * self.level, start + i))
636 +
            .collect();
637 +
        Ok(r.read_hashes(&indexes)?.concat())
638 +
    }
639 +
640 +
    /// The stored hash at `index` from this tile's data.
641 +
    pub fn hash_at(&self, data: &[u8], index: u64) -> Result<Hash> {
642 +
        if self.width == 0
643 +
            || self.width as u64 > TILE_WIDTH
644 +
            || data.len() < self.width as usize * 32
645 +
        {
646 +
            return Err(TlogError::Read(format!("tile {} is short", self.path())));
647 +
        }
648 +
        let (t, start, end) = Tile::for_index(index);
649 +
        if t.level != self.level || t.n != self.n || t.width > self.width {
650 +
            return Err(TlogError::Read(format!(
651 +
                "stored hash {index} is not in tile {}",
652 +
                self.path()
653 +
            )));
654 +
        }
655 +
        Ok(tile_hash(&data[start..end]))
656 +
    }
657 +
658 +
    /// The tiles a tree that grows from `old` to `new` leaves writes, full
659 +
    /// ones and the new partial ones.
660 +
    pub fn new_tiles(old: u64, new: u64) -> Vec<Tile> {
661 +
        let mut tiles = Vec::new();
662 +
        let mut level = 0u32;
663 +
        while level < 8 && (new >> (TILE_HEIGHT * level)) > 0 {
664 +
            let old_n = old >> (TILE_HEIGHT * level);
665 +
            let new_n = new >> (TILE_HEIGHT * level);
666 +
            if old_n != new_n {
667 +
                for n in (old_n >> TILE_HEIGHT)..(new_n >> TILE_HEIGHT) {
668 +
                    tiles.push(Tile {
669 +
                        level,
670 +
                        n,
671 +
                        width: TILE_WIDTH as u32,
672 +
                    });
673 +
                }
674 +
                let n = new_n >> TILE_HEIGHT;
675 +
                let w = new_n - (n << TILE_HEIGHT);
676 +
                if w > 0 {
677 +
                    tiles.push(Tile {
678 +
                        level,
679 +
                        n,
680 +
                        width: w as u32,
681 +
                    });
682 +
                }
683 +
            }
684 +
            level += 1;
685 +
        }
686 +
        tiles
687 +
    }
688 +
}
689 +
690 +
fn tile_hash(data: &[u8]) -> Hash {
691 +
    if data.len() == 32 {
692 +
        let mut h = [0u8; 32];
693 +
        h.copy_from_slice(data);
694 +
        return h;
695 +
    }
696 +
    let half = data.len() / 2;
697 +
    node_hash(&tile_hash(&data[..half]), &tile_hash(&data[half..]))
698 +
}
699 +
700 +
// ----------------------------------------------------- notes, checkpoints
701 +
702 +
/// One signature line of a note: `— <name> <base64(key hash || sig)>`.
703 +
#[derive(Clone, Debug, PartialEq, Eq)]
704 +
pub struct NoteSignature {
705 +
    pub name: String,
706 +
    pub key_hash: [u8; 4],
707 +
    pub sig: Vec<u8>,
708 +
}
709 +
710 +
/// A C2SP signed note: its text and its signature lines.
711 +
#[derive(Clone, Debug, PartialEq, Eq)]
712 +
pub struct Note {
713 +
    /// The text, ending with a newline.
714 +
    pub text: String,
715 +
    pub sigs: Vec<NoteSignature>,
716 +
}
717 +
718 +
/// Signature types in a note's key hash (C2SP signed-note).
719 +
const SIG_ED25519: u8 = 0x01;
720 +
/// Oxygen's composite Ed25519 + ML-DSA-65 signature (spec §14.8).
721 +
const SIG_COMPOSITE: u8 = 0x4f;
722 +
723 +
fn key_hash(name: &str, sig_type: u8, key: &[u8]) -> [u8; 4] {
724 +
    let mut h = Sha256::new();
725 +
    h.update(name.as_bytes());
726 +
    h.update(b"\n");
727 +
    h.update([sig_type]);
728 +
    h.update(key);
729 +
    let d = h.finalize();
730 +
    [d[0], d[1], d[2], d[3]]
731 +
}
732 +
733 +
/// What a composite signature of a note signs (domain `checkpoint`).
734 +
fn note_target(text: &str) -> ObjectRef {
735 +
    let mut h = blake3::Hasher::new();
736 +
    h.update(b"oxygen-tlog-note\0");
737 +
    h.update(text.as_bytes());
738 +
    ObjectRef(*h.finalize().as_bytes())
739 +
}
740 +
741 +
fn valid_name(name: &str) -> bool {
742 +
    !name.is_empty() && !name.contains(|c: char| c.is_whitespace() || c == '+')
743 +
}
744 +
745 +
impl Note {
746 +
    /// The note as served: the text, a blank line, the signature lines.
747 +
    pub fn encode(&self) -> String {
748 +
        let mut out = self.text.clone();
749 +
        out.push('\n');
750 +
        for s in &self.sigs {
751 +
            let mut b = s.key_hash.to_vec();
752 +
            b.extend_from_slice(&s.sig);
753 +
            out.push_str(&format!(
754 +
                "\u{2014} {} {}\n",
755 +
                s.name,
756 +
                crate::dsse::base64(&b)
757 +
            ));
758 +
        }
759 +
        out
760 +
    }
761 +
762 +
    pub fn parse(s: &str) -> Result<Note> {
763 +
        let bad = |w: &str| TlogError::BadNote(w.to_string());
764 +
        let split = s
765 +
            .rfind("\n\n")
766 +
            .ok_or_else(|| bad("no blank line before the signatures"))?;
767 +
        let text = &s[..split + 1];
768 +
        if text.chars().any(|c| c.is_control() && c != '\n') {
769 +
            return Err(bad("control characters in the text"));
770 +
        }
771 +
        let mut sigs = Vec::new();
772 +
        for line in s[split + 2..].lines() {
773 +
            let rest = line
774 +
                .strip_prefix("\u{2014} ")
775 +
                .ok_or_else(|| bad("a signature line without its dash"))?;
776 +
            let (name, b64) = rest
777 +
                .split_once(' ')
778 +
                .ok_or_else(|| bad("a signature line without a key"))?;
779 +
            let b =
780 +
                crate::dsse::base64_decode(b64).ok_or_else(|| bad("a signature is not base64"))?;
781 +
            if b.len() < 5 || !valid_name(name) {
782 +
                return Err(bad("a malformed signature line"));
783 +
            }
784 +
            sigs.push(NoteSignature {
785 +
                name: name.to_string(),
786 +
                key_hash: [b[0], b[1], b[2], b[3]],
787 +
                sig: b[4..].to_vec(),
788 +
            });
789 +
        }
790 +
        if sigs.is_empty() {
791 +
            return Err(bad("no signatures"));
792 +
        }
793 +
        Ok(Note {
794 +
            text: text.to_string(),
795 +
            sigs,
796 +
        })
797 +
    }
798 +
799 +
    /// Sign as `name` with `identity`'s key: the Ed25519 line note tools
800 +
    /// read, and the composite line that keeps it post-quantum.
801 +
    pub fn sign(text: &str, name: &str, identity: &Identity, sk: &SecretKey) -> Result<Note> {
802 +
        if !valid_name(name) {
803 +
            return Err(TlogError::BadNote(format!(
804 +
                "{name:?} cannot name a note's key"
805 +
            )));
806 +
        }
807 +
        if !text.ends_with('\n') || text.contains("\n\n") {
808 +
            return Err(TlogError::BadNote(
809 +
                "the text must end with one newline and hold no blank line".into(),
810 +
            ));
811 +
        }
812 +
        let ed = NoteSignature {
813 +
            name: name.to_string(),
814 +
            key_hash: key_hash(name, SIG_ED25519, &identity.ed25519_pk),
815 +
            sig: sk.sign_ed25519(text.as_bytes()).to_vec(),
816 +
        };
817 +
        let composite = NoteSignature {
818 +
            name: name.to_string(),
819 +
            key_hash: key_hash(name, SIG_COMPOSITE, &identity.encode()),
820 +
            sig: cbor::encode(
821 +
                &sk.sign(Domain::Checkpoint, &identity.hash(), &note_target(text))
822 +
                    .to_value(),
823 +
            ),
824 +
        };
825 +
        Ok(Note {
826 +
            text: text.to_string(),
827 +
            sigs: vec![ed, composite],
828 +
        })
829 +
    }
830 +
831 +
    /// Both of `name`'s signatures by `identity` verify (a note signed
832 +
    /// with Ed25519 only is not enough).
833 +
    pub fn verify(&self, name: &str, identity: &Identity) -> Result<()> {
834 +
        let find = |t: u8, key: &[u8]| {
835 +
            let h = key_hash(name, t, key);
836 +
            self.sigs.iter().find(|s| s.name == name && s.key_hash == h)
837 +
        };
838 +
        let ed = find(SIG_ED25519, &identity.ed25519_pk)
839 +
            .ok_or_else(|| TlogError::BadSignature(format!("no Ed25519 signature by {name}")))?;
840 +
        identity
841 +
            .verify_ed25519(self.text.as_bytes(), &ed.sig)
842 +
            .map_err(|_| {
843 +
                TlogError::BadSignature(format!("{name}'s Ed25519 signature does not verify"))
844 +
            })?;
845 +
        let c = find(SIG_COMPOSITE, &identity.encode())
846 +
            .ok_or_else(|| TlogError::BadSignature(format!("no composite signature by {name}")))?;
847 +
        let sig = cbor::decode(&c.sig)
848 +
            .map_err(|e| e.to_string())
849 +
            .and_then(|v| Signature::from_value(&v).map_err(|e| e.to_string()))
850 +
            .map_err(|e| TlogError::BadSignature(format!("{name}'s composite signature: {e}")))?;
851 +
        verify(identity, &sig, Domain::Checkpoint, &note_target(&self.text)).map_err(|e| {
852 +
            TlogError::BadSignature(format!("{name}'s composite signature does not verify: {e}"))
853 +
        })
854 +
    }
855 +
}
856 +
857 +
/// A checkpoint (C2SP tlog-checkpoint): the log's origin, size and root.
858 +
#[derive(Clone, Debug, PartialEq, Eq)]
859 +
pub struct Checkpoint {
860 +
    /// The log's name: the hub's host and `/log`.
861 +
    pub origin: String,
862 +
    pub size: u64,
863 +
    pub root: Hash,
864 +
    /// Further lines, each without its newline.
865 +
    pub extensions: Vec<String>,
866 +
}
867 +
868 +
impl Checkpoint {
869 +
    pub fn text(&self) -> String {
870 +
        let mut t = format!(
871 +
            "{}\n{}\n{}\n",
872 +
            self.origin,
873 +
            self.size,
874 +
            crate::dsse::base64(&self.root)
875 +
        );
876 +
        for e in &self.extensions {
877 +
            t.push_str(e);
878 +
            t.push('\n');
879 +
        }
880 +
        t
881 +
    }
882 +
883 +
    pub fn parse(text: &str) -> Result<Checkpoint> {
884 +
        let bad = |w: &str| TlogError::BadNote(format!("checkpoint: {w}"));
885 +
        if !text.ends_with('\n') {
886 +
            return Err(bad("no final newline"));
887 +
        }
888 +
        let mut lines = text[..text.len() - 1].split('\n');
889 +
        let origin = lines
890 +
            .next()
891 +
            .filter(|o| !o.is_empty())
892 +
            .ok_or_else(|| bad("no origin"))?;
893 +
        let size = lines
894 +
            .next()
895 +
            .and_then(|s| {
896 +
                (s == "0" || !s.starts_with('0'))
897 +
                    .then(|| s.parse().ok())
898 +
                    .flatten()
899 +
            })
900 +
            .ok_or_else(|| bad("no tree size"))?;
901 +
        let root = lines
902 +
            .next()
903 +
            .and_then(crate::dsse::base64_decode)
904 +
            .and_then(|b| <Hash>::try_from(b).ok())
905 +
            .ok_or_else(|| bad("no root hash"))?;
906 +
        Ok(Checkpoint {
907 +
            origin: origin.to_string(),
908 +
            size,
909 +
            root,
910 +
            extensions: lines.map(String::from).collect(),
911 +
        })
912 +
    }
913 +
914 +
    /// Signed by the log's key, named by its origin.
915 +
    pub fn sign(&self, identity: &Identity, sk: &SecretKey) -> Result<Note> {
916 +
        Note::sign(&self.text(), &self.origin, identity, sk)
917 +
    }
918 +
919 +
    /// The checkpoint a note carries, once the log's key (`identity`)
920 +
    /// signed it for `origin`.
921 +
    pub fn verify(note: &Note, origin: &str, identity: &Identity) -> Result<Checkpoint> {
922 +
        note.verify(origin, identity)?;
923 +
        let c = Checkpoint::parse(&note.text)?;
924 +
        if c.origin != origin {
925 +
            return Err(TlogError::BadNote(format!(
926 +
                "checkpoint of {}, not of {origin}",
927 +
                c.origin
928 +
            )));
929 +
        }
930 +
        Ok(c)
931 +
    }
932 +
}
933 +
934 +
// ------------------------------------------------------------- witnesses
935 +
936 +
/// C2SP tlog-cosignature: a witness's Ed25519 cosignature.
937 +
const SIG_COSIGNATURE: u8 = 0x04;
938 +
/// Oxygen's composite cosignature beside it (spec §14.8).
939 +
const SIG_COSIG_COMPOSITE: u8 = 0x4e;
940 +
941 +
/// What a cosignature signs: `cosignature/v1`, the witness's time in
942 +
/// seconds, then the checkpoint's text.
943 +
fn cosigned_message(text: &str, time: u64) -> String {
944 +
    format!("cosignature/v1\ntime {time}\n{text}")
945 +
}
946 +
947 +
fn cosig_target(message: &str) -> ObjectRef {
948 +
    let mut h = blake3::Hasher::new();
949 +
    h.update(b"oxygen-tlog-cosig\0");
950 +
    h.update(message.as_bytes());
951 +
    ObjectRef(*h.finalize().as_bytes())
952 +
}
953 +
954 +
/// A witness's cosignature of a checkpoint's text at `time` (seconds):
955 +
/// the C2SP cosignature/v1 line, and a composite line that keeps it
956 +
/// post-quantum. Each carries the time before the signature.
957 +
pub fn cosign(
958 +
    text: &str,
959 +
    name: &str,
960 +
    witness: &Identity,
961 +
    sk: &SecretKey,
962 +
    time: u64,
963 +
) -> Result<Vec<NoteSignature>> {
964 +
    if !valid_name(name) {
965 +
        return Err(TlogError::BadNote(format!(
966 +
            "{name:?} cannot name a witness"
967 +
        )));
968 +
    }
969 +
    let message = cosigned_message(text, time);
970 +
    let mut ed = time.to_be_bytes().to_vec();
971 +
    ed.extend_from_slice(&sk.sign_ed25519(message.as_bytes()));
972 +
    let mut composite = time.to_be_bytes().to_vec();
973 +
    composite.extend_from_slice(&cbor::encode(
974 +
        &sk.sign(Domain::Checkpoint, &witness.hash(), &cosig_target(&message))
975 +
            .to_value(),
976 +
    ));
977 +
    Ok(vec![
978 +
        NoteSignature {
979 +
            name: name.to_string(),
980 +
            key_hash: key_hash(name, SIG_COSIGNATURE, &witness.ed25519_pk),
981 +
            sig: ed,
982 +
        },
983 +
        NoteSignature {
984 +
            name: name.to_string(),
985 +
            key_hash: key_hash(name, SIG_COSIG_COMPOSITE, &witness.encode()),
986 +
            sig: composite,
987 +
        },
988 +
    ])
989 +
}
990 +
991 +
impl Note {
992 +
    /// The time `witness` (named `name`) cosigned this note, once both its
993 +
    /// lines verify and agree on the time.
994 +
    pub fn cosigned_by(&self, name: &str, witness: &Identity) -> Result<u64> {
995 +
        let find = |t: u8, key: &[u8]| {
996 +
            let h = key_hash(name, t, key);
997 +
            self.sigs.iter().find(|s| s.name == name && s.key_hash == h)
998 +
        };
999 +
        let time_of = |sig: &[u8]| -> Result<(u64, Vec<u8>)> {
1000 +
            if sig.len() < 9 {
1001 +
                return Err(TlogError::BadSignature(format!(
1002 +
                    "{name}'s cosignature is short"
1003 +
                )));
1004 +
            }
1005 +
            let mut t = [0u8; 8];
1006 +
            t.copy_from_slice(&sig[..8]);
1007 +
            Ok((u64::from_be_bytes(t), sig[8..].to_vec()))
1008 +
        };
1009 +
        let ed = find(SIG_COSIGNATURE, &witness.ed25519_pk)
1010 +
            .ok_or_else(|| TlogError::BadSignature(format!("no cosignature by {name}")))?;
1011 +
        let (time, sig) = time_of(&ed.sig)?;
1012 +
        let message = cosigned_message(&self.text, time);
1013 +
        witness
1014 +
            .verify_ed25519(message.as_bytes(), &sig)
1015 +
            .map_err(|_| {
1016 +
                TlogError::BadSignature(format!("{name}'s cosignature does not verify"))
1017 +
            })?;
1018 +
        let c = find(SIG_COSIG_COMPOSITE, &witness.encode()).ok_or_else(|| {
1019 +
            TlogError::BadSignature(format!("no composite cosignature by {name}"))
1020 +
        })?;
1021 +
        let (ctime, csig) = time_of(&c.sig)?;
1022 +
        if ctime != time {
1023 +
            return Err(TlogError::BadSignature(format!(
1024 +
                "{name}'s two cosignatures disagree on the time"
1025 +
            )));
1026 +
        }
1027 +
        let sig = cbor::decode(&csig)
1028 +
            .map_err(|e| e.to_string())
1029 +
            .and_then(|v| Signature::from_value(&v).map_err(|e| e.to_string()))
1030 +
            .map_err(|e| TlogError::BadSignature(format!("{name}'s composite cosignature: {e}")))?;
1031 +
        verify(witness, &sig, Domain::Checkpoint, &cosig_target(&message)).map_err(|e| {
1032 +
            TlogError::BadSignature(format!(
1033 +
                "{name}'s composite cosignature does not verify: {e}"
1034 +
            ))
1035 +
        })?;
1036 +
        Ok(time)
1037 +
    }
1038 +
}
1039 +
1040 +
/// A C2SP tlog-witness `add-checkpoint` request: the size of the last
1041 +
/// checkpoint the witness cosigned for this log, the consistency proof
1042 +
/// from it, and the new checkpoint note.
1043 +
#[derive(Clone, Debug, PartialEq, Eq)]
1044 +
pub struct AddCheckpoint {
1045 +
    pub old_size: u64,
1046 +
    pub proof: Vec<Hash>,
1047 +
    pub note: String,
1048 +
}
1049 +
1050 +
impl AddCheckpoint {
1051 +
    pub fn encode(&self) -> String {
1052 +
        let mut out = format!("old {}\n", self.old_size);
1053 +
        for h in &self.proof {
1054 +
            out.push_str(&crate::dsse::base64(h));
1055 +
            out.push('\n');
1056 +
        }
1057 +
        out.push('\n');
1058 +
        out.push_str(&self.note);
1059 +
        out
1060 +
    }
1061 +
1062 +
    pub fn parse(body: &str) -> Result<AddCheckpoint> {
1063 +
        let bad = |w: &str| TlogError::BadNote(format!("add-checkpoint: {w}"));
1064 +
        let (head, note) = body
1065 +
            .split_once("\n\n")
1066 +
            .ok_or_else(|| bad("no blank line before the checkpoint"))?;
1067 +
        let mut lines = head.split('\n');
1068 +
        let old_size = lines
1069 +
            .next()
1070 +
            .and_then(|l| l.strip_prefix("old "))
1071 +
            .and_then(|n| n.parse().ok())
1072 +
            .ok_or_else(|| bad("no old size"))?;
1073 +
        let mut proof = Vec::new();
1074 +
        for l in lines {
1075 +
            let h = crate::dsse::base64_decode(l)
1076 +
                .and_then(|b| <Hash>::try_from(b).ok())
1077 +
                .ok_or_else(|| bad("a proof line is not a hash"))?;
1078 +
            proof.push(h);
1079 +
        }
1080 +
        Ok(AddCheckpoint {
1081 +
            old_size,
1082 +
            proof,
1083 +
            note: note.to_string(),
1084 +
        })
1085 +
    }
1086 +
}
1087 +
1088 +
/// What a witness answers.
1089 +
#[derive(Clone, Debug, PartialEq, Eq)]
1090 +
pub enum Witnessed {
1091 +
    /// Cosigned: the checkpoint, and the cosignature lines to return.
1092 +
    Cosigned {
1093 +
        checkpoint: Checkpoint,
1094 +
        lines: Vec<NoteSignature>,
1095 +
    },
1096 +
    /// The request's old size is not the witness's: it answers with its own
1097 +
    /// (409, so the log retries with a proof from there).
1098 +
    Conflict(u64),
1099 +
}
1100 +
1101 +
/// Why a witness refused.
1102 +
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
1103 +
pub enum WitnessError {
1104 +
    /// Not signed by the log's key (403).
1105 +
    #[error("{0}")]
1106 +
    Unsigned(String),
1107 +
    /// The checkpoint does not extend what the witness cosigned (422).
1108 +
    #[error("{0}")]
1109 +
    Inconsistent(String),
1110 +
    /// Malformed (400).
1111 +
    #[error("{0}")]
1112 +
    Bad(String),
1113 +
}
1114 +
1115 +
/// A witness: its name in cosignatures, and its key.
1116 +
pub struct WitnessKey<'a> {
1117 +
    pub name: &'a str,
1118 +
    pub identity: &'a Identity,
1119 +
    pub sk: &'a SecretKey,
1120 +
}
1121 +
1122 +
/// A witness's decision (C2SP tlog-witness) on `req` for the log named
1123 +
/// `origin` with key `log_key`, whose last cosigned checkpoint was
1124 +
/// `latest` (size and root): cosign when the new checkpoint verifies and
1125 +
/// extends it, as `me` at `now` (seconds).
1126 +
pub fn witness(
1127 +
    req: &AddCheckpoint,
1128 +
    origin: &str,
1129 +
    log_key: &Identity,
1130 +
    latest: Option<(u64, Hash)>,
1131 +
    me: &WitnessKey,
1132 +
    now: u64,
1133 +
) -> std::result::Result<Witnessed, WitnessError> {
1134 +
    let note = Note::parse(&req.note).map_err(|e| WitnessError::Bad(e.to_string()))?;
1135 +
    let cp = Checkpoint::verify(&note, origin, log_key)
1136 +
        .map_err(|e| WitnessError::Unsigned(e.to_string()))?;
1137 +
    let (old, old_root) = latest.unwrap_or((0, empty_root()));
1138 +
    if req.old_size != old {
1139 +
        return Ok(Witnessed::Conflict(old));
1140 +
    }
1141 +
    if cp.size < old {
1142 +
        return Err(WitnessError::Bad(format!(
1143 +
            "a checkpoint of {} entries is older than the {old} cosigned",
1144 +
            cp.size
1145 +
        )));
1146 +
    }
1147 +
    let consistent = match (old, cp.size == old) {
1148 +
        (0, _) => req.proof.is_empty(),
1149 +
        (_, true) => req.proof.is_empty() && cp.root == old_root,
1150 +
        _ => check_consistency(&req.proof, cp.size, &cp.root, old, &old_root).is_ok(),
1151 +
    };
1152 +
    if !consistent {
1153 +
        return Err(WitnessError::Inconsistent(format!(
1154 +
            "the checkpoint of {} entries does not extend the one of {old} cosigned",
1155 +
            cp.size
1156 +
        )));
1157 +
    }
1158 +
    let lines = cosign(&note.text, me.name, me.identity, me.sk, now)
1159 +
        .map_err(|e| WitnessError::Bad(e.to_string()))?;
1160 +
    Ok(Witnessed::Cosigned {
1161 +
        checkpoint: cp,
1162 +
        lines,
1163 +
    })
1164 +
}
1165 +
1166 +
// -------------------------------------------------------- integrated time
1167 +
1168 +
/// How far a claimed time may stray from an integrated time before it is
1169 +
/// noted (spec §14.8, *Integrated time*): ten minutes, in milliseconds.
1170 +
pub const TIME_TOLERANCE_MS: u64 = 10 * 60 * 1000;
1171 +
1172 +
impl Note {
1173 +
    /// The witnesses among `trusted` (name and identity) whose
1174 +
    /// cosignatures of this note verify, with their times in seconds.
1175 +
    /// Lines by anyone else are ignored.
1176 +
    pub fn cosignatures(&self, trusted: &[(String, Identity)]) -> Vec<(String, u64)> {
1177 +
        trusted
1178 +
            .iter()
1179 +
            .filter_map(|(n, id)| self.cosigned_by(n, id).ok().map(|t| (n.clone(), t)))
1180 +
            .collect()
1181 +
    }
1182 +
}
1183 +
1184 +
/// When witnesses first saw a log entry (spec §14.8): an upper bound on
1185 +
/// when what it logs was signed, by a clock neither its signer nor the
1186 +
/// log chose.
1187 +
#[derive(Clone, Debug, PartialEq, Eq)]
1188 +
pub struct Integrated {
1189 +
    /// The earliest trusted cosignature's time, in seconds.
1190 +
    pub time: u64,
1191 +
    /// The witness that made it.
1192 +
    pub witness: String,
1193 +
    /// The size of the checkpoint it cosigned.
1194 +
    pub size: u64,
1195 +
}
1196 +
1197 +
impl Integrated {
1198 +
    /// The time in milliseconds, as Oxygen's other times are.
1199 +
    pub fn ms(&self) -> u64 {
1200 +
        self.time.saturating_mul(1000)
1201 +
    }
1202 +
}
1203 +
1204 +
/// Entry `index` (`entry`, its bytes)'s integrated time: the earliest time
1205 +
/// a witness in `trusted` cosigned one of `cosigned` (checkpoint notes,
1206 +
/// each with the entry's inclusion proof against it) that `log_key`
1207 +
/// signed for `origin` and that includes the entry. A note that does not
1208 +
/// verify, or does not include the entry, counts for nothing; `Err` says
1209 +
/// why no time could be read.
1210 +
pub fn integrated_time(
1211 +
    entry: &[u8],
1212 +
    index: u64,
1213 +
    cosigned: &[(&str, &[Hash])],
1214 +
    origin: &str,
1215 +
    log_key: &Identity,
1216 +
    trusted: &[(String, Identity)],
1217 +
) -> std::result::Result<Integrated, String> {
1218 +
    if cosigned.is_empty() {
1219 +
        return Err("no witness has cosigned a checkpoint holding it yet".into());
1220 +
    }
1221 +
    let leaf = leaf_hash(entry);
1222 +
    let mut best: Option<Integrated> = None;
1223 +
    let mut why = String::new();
1224 +
    for (text, proof) in cosigned {
1225 +
        let checked = Note::parse(text)
1226 +
            .and_then(|n| Checkpoint::verify(&n, origin, log_key).map(|c| (n, c)))
1227 +
            .and_then(|(n, c)| {
1228 +
                check_inclusion(proof, c.size, &c.root, index, &leaf)?;
1229 +
                Ok((n, c))
1230 +
            });
1231 +
        let (note, cp) = match checked {
1232 +
            Ok(x) => x,
1233 +
            Err(e) => {
1234 +
                why = format!("a cosigned checkpoint served for it does not hold it ({e})");
1235 +
                continue;
1236 +
            }
1237 +
        };
1238 +
        let times = note.cosignatures(trusted);
1239 +
        if times.is_empty() {
1240 +
            why = format!(
1241 +
                "the checkpoint of {} entries holding it is cosigned by no witness this user trusts",
1242 +
                cp.size
1243 +
            );
1244 +
            continue;
1245 +
        }
1246 +
        for (witness, time) in times {
1247 +
            if best.as_ref().is_none_or(|b| time < b.time) {
1248 +
                best = Some(Integrated {
1249 +
                    time,
1250 +
                    witness,
1251 +
                    size: cp.size,
1252 +
                });
1253 +
            }
1254 +
        }
1255 +
    }
1256 +
    best.ok_or(why)
1257 +
}
1258 +
1259 +
#[cfg(test)]
1260 +
mod tests {
1261 +
    use super::*;
1262 +
1263 +
    /// A log in memory: its stored hashes, grown leaf by leaf.
1264 +
    fn grow(leaves: u64) -> (Vec<Hash>, Vec<Hash>) {
1265 +
        let mut stored: Vec<Hash> = Vec::new();
1266 +
        let mut leaf_hashes = Vec::new();
1267 +
        for n in 0..leaves {
1268 +
            let h = leaf_hash(format!("entry {n}").as_bytes());
1269 +
            leaf_hashes.push(h);
1270 +
            let add = stored_hashes_for_leaf(n, h, &stored).unwrap();
1271 +
            assert_eq!(stored.len() as u64, stored_hash_index(0, n));
1272 +
            stored.extend(add);
1273 +
            assert_eq!(stored.len() as u64, stored_hash_count(n + 1));
1274 +
        }
1275 +
        (stored, leaf_hashes)
1276 +
    }
1277 +
1278 +
    /// RFC 6962's MTH, straight from its definition.
1279 +
    fn mth(leaves: &[Hash]) -> Hash {
1280 +
        match leaves.len() {
1281 +
            0 => empty_root(),
1282 +
            1 => leaves[0],
1283 +
            n => {
1284 +
                let (k, _) = maxpow2(n as u64);
1285 +
                node_hash(&mth(&leaves[..k as usize]), &mth(&leaves[k as usize..]))
1286 +
            }
1287 +
        }
1288 +
    }
1289 +
1290 +
    #[test]
1291 +
    fn stored_indexes_round_trip() {
1292 +
        for level in 0..6 {
1293 +
            for n in 0..300 {
1294 +
                let i = stored_hash_index(level, n);
1295 +
                assert_eq!(split_stored_hash_index(i), (level, n));
1296 +
            }
1297 +
        }
1298 +
    }
1299 +
1300 +
    #[test]
1301 +
    fn roots_match_rfc_6962_and_every_proof_verifies() {
1302 +
        let (stored, leaves) = grow(140);
1303 +
        for t in 1..=140u64 {
1304 +
            let root = tree_hash(t, &stored).unwrap();
1305 +
            assert_eq!(root, mth(&leaves[..t as usize]), "root of {t}");
1306 +
            for n in 0..t {
1307 +
                let p = prove_inclusion(t, n, &stored).unwrap();
1308 +
                check_inclusion(&p, t, &root, n, &leaves[n as usize]).unwrap();
1309 +
                // Another leaf, another tree, a tampered path: refused.
1310 +
                let other = (n + 1) % t;
1311 +
                if other != n {
1312 +
                    assert!(check_inclusion(&p, t, &root, n, &leaves[other as usize]).is_err());
1313 +
                }
1314 +
                if let Some(first) = p.first() {
1315 +
                    let mut bad = p.clone();
1316 +
                    bad[0] = node_hash(first, first);
1317 +
                    assert!(check_inclusion(&bad, t, &root, n, &leaves[n as usize]).is_err());
1318 +
                }
1319 +
                let old = n + 1;
1320 +
                let old_root = tree_hash(old, &stored).unwrap();
1321 +
                let c = prove_consistency(t, old, &stored).unwrap();
1322 +
                check_consistency(&c, t, &root, old, &old_root).unwrap();
1323 +
                if old < t {
1324 +
                    let fork = node_hash(&old_root, &old_root);
1325 +
                    assert!(
1326 +
                        check_consistency(&c, t, &root, old, &fork).is_err(),
1327 +
                        "a forked history"
1328 +
                    );
1329 +
                }
1330 +
            }
1331 +
        }
1332 +
        assert_eq!(tree_hash(0, &stored).unwrap(), empty_root());
1333 +
    }
1334 +
1335 +
    #[test]
1336 +
    fn tiles_hold_the_stored_hashes() {
1337 +
        let n = 70_000u64;
1338 +
        let (stored, _) = grow(n);
1339 +
        let tiles = Tile::new_tiles(0, n);
1340 +
        // 273 full tiles of level 0 and a partial one, one full and one
1341 +
        // partial at level 1, a partial at level 2.
1342 +
        assert_eq!(tiles.iter().filter(|t| t.level == 0).count(), 274);
1343 +
        assert_eq!(tiles.iter().filter(|t| t.level == 2).count(), 1);
1344 +
        let datas: Vec<(Tile, Vec<u8>)> = tiles
1345 +
            .iter()
1346 +
            .map(|t| (*t, t.read_data(&stored).unwrap()))
1347 +
            .collect();
1348 +
        for (i, want) in stored.iter().enumerate().step_by(7) {
1349 +
            let (t, ..) = Tile::for_index(i as u64);
1350 +
            let (tile, data) = datas
1351 +
                .iter()
1352 +
                .find(|(d, _)| d.level == t.level && d.n == t.n && d.width >= t.width)
1353 +
                .expect("the tile holding it was written");
1354 +
            assert_eq!(
1355 +
                tile.hash_at(data, i as u64).unwrap(),
1356 +
                *want,
1357 +
                "stored hash {i}"
1358 +
            );
1359 +
        }
1360 +
        // Growing writes only what changed.
1361 +
        let grown = Tile::new_tiles(n, n + 300);
1362 +
        assert!(grown.iter().all(|t| t.level > 0 || t.n >= n / 256));
1363 +
    }
1364 +
1365 +
    #[test]
1366 +
    fn tile_paths() {
1367 +
        assert_eq!(index_path(1_234_067), "x001/x234/067");
1368 +
        assert_eq!(index_path(5), "005");
1369 +
        assert_eq!(index_path(1000), "x001/000");
1370 +
        assert_eq!(parse_index_path("x001/x234/067"), Some(1_234_067));
1371 +
        assert_eq!(parse_index_path("x001/234/067"), None);
1372 +
        let t = Tile {
1373 +
            level: 1,
1374 +
            n: 1234,
1375 +
            width: 17,
1376 +
        };
1377 +
        assert_eq!(t.path(), "tile/1/x001/234.p/17");
1378 +
        assert_eq!(Tile::parse("1/x001/234.p/17"), Some(t));
1379 +
        assert_eq!(
1380 +
            Tile {
1381 +
                level: 0,
1382 +
                n: 3,
1383 +
                width: 256
1384 +
            }
1385 +
            .entries_path(),
1386 +
            "tile/entries/003"
1387 +
        );
1388 +
        assert_eq!(Tile::parse("0/003.p/256"), None, "a full tile has no .p");
1389 +
    }
1390 +
1391 +
    #[test]
1392 +
    fn entries_and_bundles_round_trip() {
1393 +
        let e = Entry {
1394 +
            kind: "ref".into(),
1395 +
            repo: [7; 16],
1396 +
            object: ObjectRef([1; 32]),
1397 +
            signers: vec![ObjectRef([2; 32])],
1398 +
            received: 1_792_000_000_000,
1399 +
        };
1400 +
        assert_eq!(Entry::decode(&e.encode()).unwrap(), e);
1401 +
        let b = encode_bundle(&[e.encode(), b"x".to_vec()]).unwrap();
1402 +
        assert_eq!(decode_bundle(&b).unwrap(), vec![e.encode(), b"x".to_vec()]);
1403 +
        assert!(decode_bundle(&b[..b.len() - 1]).is_err());
1404 +
        assert_eq!(e.leaf_hash(), leaf_hash(&e.encode()));
1405 +
    }
1406 +
1407 +
    #[test]
1408 +
    fn a_witness_cosigns_what_extends_what_it_saw_and_nothing_else() {
1409 +
        let log_sk = SecretKey::generate();
1410 +
        let log = log_sk.identity("log", "[email protected]", 1);
1411 +
        let w_sk = SecretKey::generate();
1412 +
        let w = w_sk.identity("witness", "[email protected]", 1);
1413 +
        let origin = "hub.example/log";
1414 +
        let (stored, _) = grow(40);
1415 +
        let note_of = |n: u64| {
1416 +
            Checkpoint {
1417 +
                origin: origin.into(),
1418 +
                size: n,
1419 +
                root: tree_hash(n, &stored).unwrap(),
1420 +
                extensions: vec![],
1421 +
            }
1422 +
            .sign(&log, &log_sk)
1423 +
            .unwrap()
1424 +
            .encode()
1425 +
        };
1426 +
        let ask = |old: u64, n: u64, latest: Option<(u64, Hash)>| {
1427 +
            let proof = if old == 0 || old == n {
1428 +
                vec![]
1429 +
            } else {
1430 +
                prove_consistency(n, old, &stored).unwrap()
1431 +
            };
1432 +
            let req = AddCheckpoint {
1433 +
                old_size: old,
1434 +
                proof,
1435 +
                note: note_of(n),
1436 +
            };
1437 +
            let req = AddCheckpoint::parse(&req.encode()).unwrap();
1438 +
            let me = WitnessKey {
1439 +
                name: "w.example",
1440 +
                identity: &w,
1441 +
                sk: &w_sk,
1442 +
            };
1443 +
            witness(&req, origin, &log, latest, &me, 1_792_300_000)
1444 +
        };
1445 +
        // First sight, then a later checkpoint with its proof.
1446 +
        let Ok(Witnessed::Cosigned { checkpoint, lines }) = ask(0, 10, None) else {
1447 +
            panic!()
1448 +
        };
1449 +
        let mut note = Note::parse(&note_of(10)).unwrap();
1450 +
        note.sigs.extend(lines);
1451 +
        let back = Note::parse(&note.encode()).unwrap();
1452 +
        assert_eq!(back.cosigned_by("w.example", &w).unwrap(), 1_792_300_000);
1453 +
        assert!(
1454 +
            back.cosigned_by("w.example", &log).is_err(),
1455 +
            "another witness's key"
1456 +
        );
1457 +
        let latest = Some((10, checkpoint.root));
1458 +
        assert!(matches!(
1459 +
            ask(10, 40, latest),
1460 +
            Ok(Witnessed::Cosigned { .. })
1461 +
        ));
1462 +
        // The log's idea of what the witness saw is wrong: told the size.
1463 +
        assert_eq!(ask(5, 40, latest), Ok(Witnessed::Conflict(10)));
1464 +
        // A fork: what the witness saw is not a prefix.
1465 +
        let forked = Some((10, [9; 32]));
1466 +
        assert!(matches!(
1467 +
            ask(10, 40, forked),
1468 +
            Err(WitnessError::Inconsistent(_))
1469 +
        ));
1470 +
        // Signed by someone else.
1471 +
        let req = AddCheckpoint {
1472 +
            old_size: 0,
1473 +
            proof: vec![],
1474 +
            note: note_of(10),
1475 +
        };
1476 +
        assert!(matches!(
1477 +
            witness(
1478 +
                &req,
1479 +
                origin,
1480 +
                &w,
1481 +
                None,
1482 +
                &WitnessKey {
1483 +
                    name: "w.example",
1484 +
                    identity: &w,
1485 +
                    sk: &w_sk
1486 +
                },
1487 +
                1
1488 +
            ),
1489 +
            Err(WitnessError::Unsigned(_))
1490 +
        ));
1491 +
    }
1492 +
1493 +
    #[test]
1494 +
    fn checkpoints_are_signed_twice_and_checked() {
1495 +
        let sk = SecretKey::generate();
1496 +
        let hub = sk.identity("oxygenhub.io", "[email protected]", 1);
1497 +
        let (stored, _) = grow(10);
1498 +
        let c = Checkpoint {
1499 +
            origin: "oxygenhub.io/log".into(),
1500 +
            size: 10,
1501 +
            root: tree_hash(10, &stored).unwrap(),
1502 +
            extensions: vec![],
1503 +
        };
1504 +
        let note = c.sign(&hub, &sk).unwrap();
1505 +
        let served = note.encode();
1506 +
        assert!(served.starts_with("oxygenhub.io/log\n10\n"));
1507 +
        assert_eq!(served.matches("\n\u{2014} oxygenhub.io/log ").count(), 2);
1508 +
        let back = Note::parse(&served).unwrap();
1509 +
        assert_eq!(
1510 +
            Checkpoint::verify(&back, "oxygenhub.io/log", &hub).unwrap(),
1511 +
            c
1512 +
        );
1513 +
        // Another key, another origin, a changed size, a lost line: refused.
1514 +
        let other_sk = SecretKey::generate();
1515 +
        let other = other_sk.identity("x", "x@x", 1);
1516 +
        assert!(Checkpoint::verify(&back, "oxygenhub.io/log", &other).is_err());
1517 +
        assert!(Checkpoint::verify(&back, "elsewhere/log", &hub).is_err());
1518 +
        let forged = Note {
1519 +
            text: back.text.replace("\n10\n", "\n11\n"),
1520 +
            ..back.clone()
1521 +
        };
1522 +
        assert!(Checkpoint::verify(&forged, "oxygenhub.io/log", &hub).is_err());
1523 +
        let ed_only = Note {
1524 +
            sigs: vec![back.sigs[0].clone()],
1525 +
            ..back.clone()
1526 +
        };
1527 +
        assert!(
1528 +
            matches!(Checkpoint::verify(&ed_only, "oxygenhub.io/log", &hub), Err(TlogError::BadSignature(m)) if m.contains("composite")),
1529 +
            "the post-quantum line is required"
1530 +
        );
1531 +
        // The Ed25519 line is a standard note signature over the text.
1532 +
        let h = key_hash("oxygenhub.io/log", SIG_ED25519, &hub.ed25519_pk);
1533 +
        assert_eq!(back.sigs[0].key_hash, h);
1534 +
        assert!(hub
1535 +
            .verify_ed25519(back.text.as_bytes(), &back.sigs[0].sig)
1536 +
            .is_ok());
1537 +
    }
1538 +
1539 +
    #[test]
1540 +
    fn an_entry_is_integrated_when_a_trusted_witness_first_cosigned_it() {
1541 +
        let log_sk = SecretKey::generate();
1542 +
        let log = log_sk.identity("log", "[email protected]", 1);
1543 +
        let origin = "hub.example/log";
1544 +
        let witness = |name: &str| {
1545 +
            let sk = SecretKey::generate();
1546 +
            (name.to_string(), sk.identity(name, "[email protected]", 1), sk)
1547 +
        };
1548 +
        let (a, b, c) = (
1549 +
            witness("a.example"),
1550 +
            witness("b.example"),
1551 +
            witness("c.example"),
1552 +
        );
1553 +
        let trusted = vec![(a.0.clone(), a.1.clone()), (b.0.clone(), b.1.clone())];
1554 +
        // A log of 40 real entries, so `integrated_time` hashes the leaf.
1555 +
        let leaf_bytes = |n: u64| {
1556 +
            Entry {
1557 +
                kind: "ref".into(),
1558 +
                repo: [n as u8; 16],
1559 +
                object: ObjectRef([n as u8; 32]),
1560 +
                signers: vec![],
1561 +
                received: n,
1562 +
            }
1563 +
            .encode()
1564 +
        };
1565 +
        let mut stored: Vec<Hash> = Vec::new();
1566 +
        for n in 0..40 {
1567 +
            let add = stored_hashes_for_leaf(n, leaf_hash(&leaf_bytes(n)), &stored).unwrap();
1568 +
            stored.extend(add);
1569 +
        }
1570 +
        // Checkpoint `n`, cosigned by each of `by` at its time.
1571 +
        type W = (String, Identity, SecretKey);
1572 +
        let cosigned = |n: u64, by: &[(&W, u64)]| {
1573 +
            let cp = Checkpoint {
1574 +
                origin: origin.into(),
1575 +
                size: n,
1576 +
                root: tree_hash(n, &stored).unwrap(),
1577 +
                extensions: vec![],
1578 +
            };
1579 +
            let mut note = cp.sign(&log, &log_sk).unwrap();
1580 +
            for ((name, id, sk), t) in by {
1581 +
                note.sigs
1582 +
                    .extend(cosign(&note.text, name, id, sk, *t).unwrap());
1583 +
            }
1584 +
            note.encode()
1585 +
        };
1586 +
        let proof = |n: u64| prove_inclusion(n, 7, &stored).unwrap();
1587 +
        let entry = leaf_bytes(7);
1588 +
        let time = |served: &[(String, Vec<Hash>)]| {
1589 +
            let s: Vec<(&str, &[Hash])> = served
1590 +
                .iter()
1591 +
                .map(|(n, p)| (n.as_str(), p.as_slice()))
1592 +
                .collect();
1593 +
            integrated_time(&entry, 7, &s, origin, &log, &trusted)
1594 +
        };
1595 +
1596 +
        // One checkpoint, two witnesses: the earlier cosignature.
1597 +
        let one = cosigned(10, &[(&a, 1_792_300_050), (&b, 1_792_300_020)]);
1598 +
        let got = time(&[(one.clone(), proof(10))]).unwrap();
1599 +
        assert_eq!(
1600 +
            (got.time, got.witness.as_str(), got.size),
1601 +
            (1_792_300_020, "b.example", 10)
1602 +
        );
1603 +
        assert_eq!(got.ms(), 1_792_300_020_000);
1604 +
        // Witnesses that first cosigned different checkpoints: the
1605 +
        // earliest of all.
1606 +
        let later = cosigned(20, &[(&a, 1_792_300_010)]);
1607 +
        let got = time(&[(later.clone(), proof(20)), (one.clone(), proof(10))]).unwrap();
1608 +
        assert_eq!(
1609 +
            (got.time, got.witness.as_str()),
1610 +
            (1_792_300_010, "a.example")
1611 +
        );
1612 +
1613 +
        // A witness this user does not trust is no witness, however early.
1614 +
        let stranger = cosigned(10, &[(&c, 1)]);
1615 +
        let err = time(&[(stranger, proof(10))]).unwrap_err();
1616 +
        assert!(err.contains("no witness this user trusts"), "{err}");
1617 +
        // A forged cosignature under a trusted name: c's key signed it.
1618 +
        let forged = cosigned(10, &[(&(a.0.clone(), c.1.clone(), c.2.clone()), 1)]);
1619 +
        assert!(time(&[(forged.clone(), proof(10))]).is_err());
1620 +
        // Beside a real one, it changes nothing.
1621 +
        let got = time(&[(forged, proof(10)), (one.clone(), proof(10))]).unwrap();
1622 +
        assert_eq!(got.time, 1_792_300_020);
1623 +
        // A time moved after cosigning no longer verifies.
1624 +
        let mut moved = Note::parse(&one).unwrap();
1625 +
        for s in moved.sigs.iter_mut().filter(|s| s.name == "b.example") {
1626 +
            s.sig[..8].copy_from_slice(&1u64.to_be_bytes());
1627 +
        }
1628 +
        let got = time(&[(moved.encode(), proof(10))]).unwrap();
1629 +
        assert_eq!(got.witness, "a.example", "b's lines no longer verify");
1630 +
1631 +
        // A checkpoint that does not hold the entry, a proof for another
1632 +
        // size, another log's key: no time.
1633 +
        let early = cosigned(5, &[(&a, 1)]);
1634 +
        assert!(time(&[(early, proof(10))]).is_err());
1635 +
        assert!(time(&[(one.clone(), proof(20))]).is_err());
1636 +
        let other_sk = SecretKey::generate();
1637 +
        let other = other_sk.identity("x", "x@x", 1);
1638 +
        let s = [(one.as_str(), proof(10))];
1639 +
        let s: Vec<(&str, &[Hash])> = s.iter().map(|(n, p)| (*n, p.as_slice())).collect();
1640 +
        assert!(integrated_time(&entry, 7, &s, origin, &other, &trusted).is_err());
1641 +
        assert!(integrated_time(&entry, 7, &s, "elsewhere/log", &log, &trusted).is_err());
1642 +
        assert!(integrated_time(&leaf_bytes(8), 7, &s, origin, &log, &trusted).is_err());
1643 +
        // Nothing served.
1644 +
        assert!(time(&[]).unwrap_err().contains("no witness has cosigned"));
1645 +
    }
1646 +
}
modified crates/oxy-core/src/tree.rs +176 −0
…
183 unchanged lines
184 184
    /// cheap to clone.
185 185
    pub entries: std::sync::Arc<BTreeMap<String, TreeEntry>>,
186 186
    pub conflicts: Vec<TreeConflict>,
187 +
    /// Nodes added at a place another node holds with identical content
188 +
    /// (spec §9.2), each with the node the view shows there in its stead:
189 +
    /// one file to every reader, and retired by the first change that
190 +
    /// touches the node shown ([`TreeGraph::unify`]).
191 +
    pub unified: BTreeMap<NodeId, NodeId>,
187 192
}
188 193
189 194
impl TreeView {
…
592 unchanged lines
782 787
}
783 788
784 789
impl TreeGraph {
790 +
    /// `view` (this graph's) with each name conflict between nodes added
791 +
    /// with identical content made one entry (spec §9.2): the node whose
792 +
    /// placement there has the least key is shown at the plain name, and
793 +
    /// the others are listed in [`TreeView::unified`]. Only a group of
794 +
    /// files, symlinks or subrepos of one kind and mode, none in another
795 +
    /// conflict, is offered to `same`, one entry per node, which says
796 +
    /// whether their contents are identical (`None`: it cannot tell yet).
797 +
    /// The key order is the same for every peer, so all show one node.
798 +
    /// Also returns whether `same` answered for every group.
799 +
    pub fn unify(
800 +
        &self,
801 +
        mut view: TreeView,
802 +
        same: impl Fn(&[&TreeEntry]) -> Option<bool>,
803 +
    ) -> (TreeView, bool) {
804 +
        let mut decided = true;
805 +
        if !view
806 +
            .conflicts
807 +
            .iter()
808 +
            .any(|c| matches!(c, TreeConflict::Name { .. }))
809 +
        {
810 +
            return (view, decided);
811 +
        }
812 +
        // Nodes in any other conflict stay as they are.
813 +
        let mut troubled: HashSet<NodeId> = HashSet::new();
814 +
        for c in &view.conflicts {
815 +
            match c {
816 +
                TreeConflict::Name { .. } => {}
817 +
                TreeConflict::MultiPlacement { node, .. }
818 +
                | TreeConflict::Orphan { node, .. }
819 +
                | TreeConflict::Cycle { node }
820 +
                | TreeConflict::Mode { node, .. } => {
821 +
                    troubled.insert(*node);
822 +
                }
823 +
            }
824 +
        }
825 +
        let mut of_node: HashMap<NodeId, Vec<&TreeEntry>> = HashMap::new();
826 +
        for e in view.entries.values() {
827 +
            of_node.entry(e.node).or_default().push(e);
828 +
        }
829 +
        // (paths to drop, entry to add, duplicates) per unified group.
830 +
        let mut merged: Vec<(Vec<String>, TreeEntry, Vec<NodeId>)> = Vec::new();
831 +
        let mut gone: HashSet<usize> = HashSet::new();
832 +
        for (i, c) in view.conflicts.iter().enumerate() {
833 +
            let TreeConflict::Name {
834 +
                parent,
835 +
                name,
836 +
                nodes,
837 +
            } = c
838 +
            else {
839 +
                continue;
840 +
            };
841 +
            if nodes.iter().any(|n| troubled.contains(n)) {
842 +
                continue;
843 +
            }
844 +
            // The group's entries: each node's placements at this place.
845 +
            let mut group: Vec<&TreeEntry> = Vec::new();
846 +
            for n in nodes {
847 +
                for e in of_node.get(n).into_iter().flatten() {
848 +
                    if self
849 +
                        .placement(&e.placement)
850 +
                        .is_some_and(|p| p.parent == *parent && p.name == *name)
851 +
                    {
852 +
                        group.push(e);
853 +
                    }
854 +
                }
855 +
            }
856 +
            let Some(first) = group.first() else {
857 +
                continue;
858 +
            };
859 +
            if first.kind == NodeKind::Dir
860 +
                || group
861 +
                    .iter()
862 +
                    .any(|e| e.kind != first.kind || e.mode != first.mode)
863 +
            {
864 +
                continue;
865 +
            }
866 +
            group.sort_by_key(|e| e.placement);
867 +
            let mut reps: Vec<&TreeEntry> = Vec::new();
868 +
            for e in &group {
869 +
                if !reps.iter().any(|r| r.node == e.node) {
870 +
                    reps.push(e);
871 +
                }
872 +
            }
873 +
            if reps.len() < 2 {
874 +
                continue;
875 +
            }
876 +
            match same(&reps) {
877 +
                Some(true) => {}
878 +
                Some(false) => continue,
879 +
                None => {
880 +
                    decided = false;
881 +
                    continue;
882 +
                }
883 +
            }
884 +
            let shown = reps[0];
885 +
            let path = match shown.path.rsplit_once('/') {
886 +
                Some((dir, _)) => format!("{dir}/{name}"),
887 +
                None => name.clone(),
888 +
            };
889 +
            merged.push((
890 +
                group.iter().map(|e| e.path.clone()).collect(),
891 +
                TreeEntry {
892 +
                    path,
893 +
                    ..shown.clone()
894 +
                },
895 +
                reps[1..].iter().map(|e| e.node).collect(),
896 +
            ));
897 +
            gone.insert(i);
898 +
        }
899 +
        drop(of_node);
900 +
        if merged.is_empty() {
901 +
            return (view, decided);
902 +
        }
903 +
        let entries = std::sync::Arc::make_mut(&mut view.entries);
904 +
        for (paths, shown, dups) in merged {
905 +
            for p in paths {
906 +
                entries.remove(&p);
907 +
            }
908 +
            for d in dups {
909 +
                view.unified.insert(d, shown.node);
910 +
            }
911 +
            entries.insert(shown.path.clone(), shown);
912 +
        }
913 +
        let mut i = 0;
914 +
        view.conflicts.retain(|_| {
915 +
            i += 1;
916 +
            !gone.contains(&(i - 1))
917 +
        });
918 +
        (view, decided)
919 +
    }
920 +
}
921 +
922 +
impl TreeGraph {
785 923
    /// Serialize for a checkpoint.
786 924
    pub fn to_value(&self) -> crate::cbor::Value {
787 925
        use crate::cbor::Value;
…
389 unchanged lines
1177 1315
        Ok(Some(TreeView {
1178 1316
            entries: std::sync::Arc::new(entries.into_iter().collect()),
1179 1317
            conflicts: Vec::new(),
1318 +
            unified: BTreeMap::new(),
1180 1319
        }))
1181 1320
    }
1182 1321
}
…
191 unchanged lines
1374 1513
        );
1375 1514
    }
1376 1515
1516 +
    /// Identical additions at one place are one entry, the node with the
1517 +
    /// least placement key shown; anything else stays a name conflict.
1518 +
    #[test]
1519 +
    fn identical_additions_unify() {
1520 +
        let mut g = TreeGraph::new();
1521 +
        add(&mut g, 1, 0, 1, NodeKind::Dir, NodeId::ROOT, "d");
1522 +
        add(&mut g, 3, 0, 3, NodeKind::File, nid(1), "a.txt");
1523 +
        add(&mut g, 2, 0, 2, NodeKind::File, nid(1), "a.txt");
1524 +
        let raw = g.view();
1525 +
        let (v, decided) = g.unify(raw.clone(), |es| {
1526 +
            assert_eq!(es.len(), 2, "one entry per node");
1527 +
            Some(true)
1528 +
        });
1529 +
        assert!(decided);
1530 +
        assert_eq!(paths(&v), ["d", "d/a.txt"]);
1531 +
        assert!(v.conflicts.is_empty());
1532 +
        assert_eq!(v.entries["d/a.txt"].node, nid(2), "the least placement key");
1533 +
        assert_eq!(v.entries["d/a.txt"].placement, (h(2), 0));
1534 +
        assert_eq!(v.unified, [(nid(3), nid(2))].into_iter().collect());
1535 +
        // Different contents, or not known yet: the conflict as it was.
1536 +
        let (v, decided) = g.unify(raw.clone(), |_| Some(false));
1537 +
        assert!(decided);
1538 +
        assert_eq!(v, raw);
1539 +
        let (v, decided) = g.unify(raw.clone(), |_| None);
1540 +
        assert!(!decided);
1541 +
        assert_eq!(v, raw);
1542 +
        // Directories, and kinds that differ, are never offered.
1543 +
        let mut g = TreeGraph::new();
1544 +
        add(&mut g, 1, 0, 1, NodeKind::Dir, NodeId::ROOT, "d");
1545 +
        add(&mut g, 2, 0, 2, NodeKind::Dir, NodeId::ROOT, "d");
1546 +
        add(&mut g, 3, 0, 3, NodeKind::File, NodeId::ROOT, "f");
1547 +
        add(&mut g, 4, 0, 4, NodeKind::Symlink, NodeId::ROOT, "f");
1548 +
        let raw = g.view();
1549 +
        let (v, _) = g.unify(raw.clone(), |_| panic!("not offered"));
1550 +
        assert_eq!(v, raw);
1551 +
    }
1552 +
1377 1553
    #[test]
1378 1554
    fn concurrent_renames_are_multi_placement() {
1379 1555
        let mut g = TreeGraph::new();
…
218 unchanged lines
modified crates/oxy-server/Cargo.toml +1 −0
…
14 unchanged lines
15 15
[dev-dependencies]
16 16
rcgen = "0.14"
17 17
tempfile = "3"
18 +
tower = { version = "0.5", features = ["util"] }
modified crates/oxy-server/src/lib.rs +151 −2
…
10 unchanged lines
11 11
//! and authorization) and routing to [`Server::handle`]; `router` is the
12 12
//! standalone server built on [`DirBackend`].
13 13
14 +
pub mod witness;
15 +
14 16
use axum::body::Bytes;
15 17
use axum::extract::{Path, State};
16 18
use axum::http::{header, HeaderMap, Method, StatusCode};
…
47 unchanged lines
64 66
    /// between, and the changes stored, for logs and cache invalidation.
65 67
    fn after_push(&self, _name: &str, _report: &oxy_core::repo::PushReport, _headers: &HeaderMap) {}
66 68
69 +
    /// Notes for the pusher on references this push moved: each
70 +
    /// `(reference, note)` is added to that accepted reference's result,
71 +
    /// which a client shows beside it (`Pushed origin/x - note`). For a
72 +
    /// push that did what the pusher may not expect, say moving the
73 +
    /// bookmark of a proposal that has landed (oxygenhub#329, #148). Called
74 +
    /// before [`Backend::after_push`], which sees the notes.
75 +
    fn push_notes(
76 +
        &self,
77 +
        _name: &str,
78 +
        _report: &oxy_core::repo::PushReport,
79 +
        _headers: &HeaderMap,
80 +
    ) -> Vec<(String, String)> {
81 +
        Vec::new()
82 +
    }
83 +
84 +
    /// Vet a push by what it adds before anything of it is stored or any
85 +
    /// reference moves (spec §16.1): `bundle` is the push as decoded, its
86 +
    /// changes with their lines, `updates` the references it would move.
87 +
    /// A host refuses the whole push by returning a [`Failure`], whose
88 +
    /// text reaches the client as the push's error, as it is, several
89 +
    /// lines included; it should answer neither 401, 403 nor 404, which a
90 +
    /// client takes for a credential it should change. Called for every
91 +
    /// part of a push sent in parts (each part's changes are its own),
92 +
    /// outside the repository's push lock, so a slow vet holds up no
93 +
    /// other push. The default accepts everything at no cost.
94 +
    fn vet_push(
95 +
        &self,
96 +
        _name: &str,
97 +
        _repo: &Repo,
98 +
        _bundle: &oxy_core::protocol::Bundle,
99 +
        _updates: &[oxy_core::protocol::RefUpdate],
100 +
        _headers: &HeaderMap,
101 +
    ) -> Result<(), Failure> {
102 +
        Ok(())
103 +
    }
104 +
105 +
    /// Add the host's transparency log to a refs answer (spec §14.8): its
106 +
    /// checkpoint, and each attestation's entry and inclusion proof. A host
107 +
    /// without a log leaves it as it is.
108 +
    fn add_log(&self, _name: &str, _refs: &mut oxy_core::protocol::RefsResponse) {}
109 +
110 +
    /// Tailor a refs answer to the client asking, by its request headers
111 +
    /// (its `User-Agent` and the features it declares in `Oxy-Features`),
112 +
    /// after [`Backend::add_log`]. The default leaves out the tags a client
113 +
    /// that does not declare `certified-tags` would refuse
114 +
    /// ([`Repo::refs_for_client`], oxygen#131); a host that overrides it
115 +
    /// should apply that too ([`tailor_refs`]).
116 +
    fn adjust_refs(
117 +
        &self,
118 +
        _name: &str,
119 +
        repo: &Repo,
120 +
        refs: &mut oxy_core::protocol::RefsResponse,
121 +
        headers: &HeaderMap,
122 +
    ) {
123 +
        tailor_refs(repo, refs, headers);
124 +
    }
125 +
126 +
    /// Whether tag `tag` of `name` may be withdrawn (oxygenhub#301): called
127 +
    /// for each withdrawal a push carries, before the repository sees it.
128 +
    /// A host refuses one it depends on (a forge's release names the tag)
129 +
    /// with the reason, which is that withdrawal's result. The default
130 +
    /// allows every one; the policy is checked after, as for any push.
131 +
    fn vet_tag_withdrawal(&self, _name: &str, _tag: &str) -> Result<(), String> {
132 +
        Ok(())
133 +
    }
134 +
67 135
    /// The bookmark of `name` that pushes may never delete (its default),
68 136
    /// when the host keeps one apart from the repository's own config.
69 137
    fn default_bookmark(&self, _name: &str) -> Option<String> {
…
39 unchanged lines
109 177
    Fetch,
110 178
    Objects,
111 179
    Checkpoint,
180 +
    CheckpointIndex,
112 181
    Shallow,
113 182
    Push,
114 183
}
…
7 unchanged lines
122 191
            "fetch" => Endpoint::Fetch,
123 192
            "objects" => Endpoint::Objects,
124 193
            "checkpoint" => Endpoint::Checkpoint,
194 +
            "checkpoint-index" => Endpoint::CheckpointIndex,
125 195
            "shallow" => Endpoint::Shallow,
126 196
            "push" => Endpoint::Push,
127 197
            _ => return None,
…
8 unchanged lines
136 206
            Endpoint::Fetch => "fetch",
137 207
            Endpoint::Objects => "objects",
138 208
            Endpoint::Checkpoint => "checkpoint",
209 +
            Endpoint::CheckpointIndex => "checkpoint-index",
139 210
            Endpoint::Shallow => "shallow",
140 211
            Endpoint::Push => "push",
141 212
        }
…
109 unchanged lines
251 322
                if let Some(d) = self.backend.default_bookmark(name) {
252 323
                    refs.default_bookmark = Some(d);
253 324
                }
325 +
                self.backend.add_log(name, &mut refs);
326 +
                self.backend.adjust_refs(name, &r, &mut refs, headers);
254 327
                ok(wire::encode_refs(r.id()?, &refs))
255 328
            }
256 329
            Endpoint::Negotiate => {
…
18 unchanged lines
275 348
                let heads = wire::decode_hashes(&body)?;
276 349
                let lock = self.lock_for(name);
277 350
                let _guard = lock.lock().unwrap();
278 -
                ok(wire::encode_bundle(&r.checkpoint_bundle_for(&heads)?))
351 +
                let bundle = r.checkpoint_bundle_for(&heads)?;
352 +
                let _t = oxy_core::timing::phase("serve: encode the checkpoint");
353 +
                ok(wire::encode_bundle(&bundle))
279 354
            }
355 +
            Endpoint::CheckpointIndex => {
356 +
                let heads = wire::decode_hashes(&body)?;
357 +
                let lock = self.lock_for(name);
358 +
                let _guard = lock.lock().unwrap();
359 +
                ok(wire::encode_bundle(&r.checkpoint_index_for(&heads)?))
360 +
            }
280 361
            Endpoint::Shallow => {
281 362
                let (heads, depth) = wire::decode_shallow(&body)?;
282 363
                ok(wire::encode_hashes(&r.shallow_changes(&heads, depth)?))
283 364
            }
284 365
            Endpoint::Push => {
285 366
                let (bundle, updates, policy, tags) = wire::decode_push(&body)?;
367 +
                // The host vets what the push adds before anything is stored
368 +
                // or bound: a refused push leaves the repository as it was.
369 +
                self.backend
370 +
                    .vet_push(name, &r, &bundle, &updates, headers)?;
286 371
                let lock = self.lock_for(name);
287 372
                let _guard = lock.lock().unwrap();
288 373
                if r.is_unbound() {
…
15 unchanged lines
304 389
                let (refused, updates): (Vec<_>, Vec<_>) = updates.into_iter().partition(|u| {
305 390
                    u.new_state.is_zero() && protected.as_deref() == Some(u.name.as_str())
306 391
                });
307 -
                let mut report = r.receive_push_report(bundle, updates, policy, tags)?;
392 +
                let mut kept_tags = Vec::new();
393 +
                let mut withdrawals_refused = Vec::new();
394 +
                for t in tags {
395 +
                    match t
396 +
                        .is_withdrawal()
397 +
                        .then(|| self.backend.vet_tag_withdrawal(name, &t.name))
398 +
                    {
399 +
                        Some(Err(reason)) => withdrawals_refused.push((t.name, reason)),
400 +
                        _ => kept_tags.push(t),
401 +
                    }
402 +
                }
403 +
                let mut report = r.receive_push_report(bundle, updates, policy, kept_tags)?;
404 +
                // The request named references, all refused here: it was no
405 +
                // push of objects alone, whatever the repository saw.
406 +
                if !refused.is_empty() || !withdrawals_refused.is_empty() {
407 +
                    report
408 +
                        .results
409 +
                        .retain(|r| r.name != oxy_core::repo::OBJECTS_RESULT);
410 +
                }
308 411
                for u in refused {
309 412
                    report.results.push(oxy_core::protocol::RefResult {
310 413
                        name: u.name.clone(),
…
4 unchanged lines
315 418
                        ),
316 419
                    });
317 420
                }
421 +
                for (tag, reason) in withdrawals_refused {
422 +
                    report.results.push(oxy_core::protocol::RefResult {
423 +
                        name: format!("tags/{tag}"),
424 +
                        ok: false,
425 +
                        reason,
426 +
                    });
427 +
                }
318 428
                for r in report.results.iter().filter(|r| !r.ok) {
319 429
                    eprintln!(
320 430
                        "oxy-server: push to {name}: {} refused: {}",
321 431
                        r.name, r.reason
322 432
                    );
323 433
                }
434 +
                for (reference, note) in self.backend.push_notes(name, &report, headers) {
435 +
                    if let Some(r) = report
436 +
                        .results
437 +
                        .iter_mut()
438 +
                        .find(|r| r.ok && r.name == reference)
439 +
                    {
440 +
                        r.reason = if r.reason.is_empty() {
441 +
                            note
442 +
                        } else {
443 +
                            format!("{}; {note}", r.reason)
444 +
                        };
445 +
                    }
446 +
                }
324 447
                self.backend.after_push(name, &report, headers);
325 448
                ok(wire::encode_push_response(&report.results))
326 449
            }
…
50 unchanged lines
377 500
    Some(out)
378 501
}
379 502
503 +
/// The features a request declares in its `Oxy-Features` header, as the
504 +
/// header's text; `None` when it sent none (a client of the 0.5.0 era).
505 +
pub fn client_features(headers: &HeaderMap) -> Option<&str> {
506 +
    headers
507 +
        .get(oxy_core::protocol::FEATURES_HEADER)
508 +
        .and_then(|v| v.to_str().ok())
509 +
}
510 +
511 +
/// Tailor `refs` of `repo` to the client whose request carried `headers`:
512 +
/// the shared rule of [`Repo::refs_for_client`], which every host applies.
513 +
pub fn tailor_refs(repo: &Repo, refs: &mut oxy_core::protocol::RefsResponse, headers: &HeaderMap) {
514 +
    repo.refs_for_client(refs, client_features(headers));
515 +
}
516 +
380 517
/// The token of an `Authorization: Bearer` header, if any.
381 518
pub fn bearer_token(headers: &HeaderMap) -> Option<&str> {
382 519
    headers
…
16 unchanged lines
399 536
    }
400 537
}
401 538
539 +
/// The largest request `oxy serve` takes: a push part is about
540 +
/// `push.batch-bytes` (64 MiB), but one change can weigh more; OxygenHub's
541 +
/// default `push_body_limit` is the same.
542 +
pub const BODY_LIMIT: usize = if usize::BITS > 32 {
543 +
    4 << 30
544 +
} else {
545 +
    usize::MAX
546 +
};
547 +
402 548
/// The router of `oxy serve`: `/{repo}/oxy/v1/{endpoint}` over a [`DirBackend`].
403 549
pub fn router(cfg: ServerConfig) -> Router {
404 550
    let server = Server::new(DirBackend {
405 551
        root: cfg.root,
406 552
        tokens: cfg.tokens,
407 553
    });
554 +
    // Without a limit of its own a request takes axum's 2 MB, and a larger
555 +
    // push was cut off with no word in the log ("Broken pipe" at the client).
408 556
    Router::new()
409 557
        .route("/{repo}/oxy/v1/{endpoint}", any(dispatch))
558 +
        .layer(axum::extract::DefaultBodyLimit::max(BODY_LIMIT))
410 559
        .with_state(server)
411 560
}
412 561
…
29 unchanged lines
added crates/oxy-server/src/witness.rs +272 −0
1 +
//! `oxy witness serve` (Oxygen spec §14.8, oxygenhub docs/transparency.md):
2 +
//! a small transparency-log witness anyone can run. It cosigns the
3 +
//! checkpoints of the logs it is told about (C2SP tlog-witness), each only
4 +
//! when it extends the last one this witness cosigned, with its own key
5 +
//! and clock, so a log cannot show two histories without the witness
6 +
//! seeing it.
7 +
//!
8 +
//! State is one file per log in the state directory: the size and root of
9 +
//! the last checkpoint cosigned, replaced atomically.
10 +
11 +
use axum::extract::State;
12 +
use axum::http::{header, StatusCode};
13 +
use axum::response::{IntoResponse, Response};
14 +
use axum::routing::post;
15 +
use axum::Router;
16 +
use oxy_core::identity::{Identity, SecretKey};
17 +
use oxy_core::tlog::{self, AddCheckpoint, Checkpoint, Note, WitnessError, WitnessKey, Witnessed};
18 +
use std::path::PathBuf;
19 +
use std::sync::{Arc, Mutex};
20 +
21 +
/// What a witness serves.
22 +
pub struct WitnessConfig {
23 +
    /// Its name in cosignatures (a host name, no spaces).
24 +
    pub name: String,
25 +
    pub identity: Identity,
26 +
    pub sk: SecretKey,
27 +
    /// The logs it witnesses: their origins and keys.
28 +
    pub logs: Vec<(String, Identity)>,
29 +
    /// Where it keeps what it cosigned.
30 +
    pub state: PathBuf,
31 +
    pub tls: Option<crate::TlsFiles>,
32 +
}
33 +
34 +
struct Witness {
35 +
    cfg: WitnessConfig,
36 +
    /// One add-checkpoint at a time: each reads and moves its log's state.
37 +
    lock: Mutex<()>,
38 +
}
39 +
40 +
fn state_file(w: &Witness, origin: &str) -> PathBuf {
41 +
    let safe: String = origin
42 +
        .chars()
43 +
        .map(|c| {
44 +
            if c.is_ascii_alphanumeric() || c == '.' || c == '-' {
45 +
                c
46 +
            } else {
47 +
                '_'
48 +
            }
49 +
        })
50 +
        .collect();
51 +
    w.cfg.state.join(safe)
52 +
}
53 +
54 +
fn read_state(w: &Witness, origin: &str) -> Option<(u64, tlog::Hash)> {
55 +
    let text = std::fs::read_to_string(state_file(w, origin)).ok()?;
56 +
    let (size, root) = text.trim().split_once(' ')?;
57 +
    let root = oxy_core::hash::ObjectRef::from_hex(root)?;
58 +
    Some((size.parse().ok()?, root.0))
59 +
}
60 +
61 +
fn plain(status: StatusCode, why: &str) -> Response {
62 +
    (
63 +
        status,
64 +
        [(header::CONTENT_TYPE, "text/plain; charset=utf-8")],
65 +
        format!("{why}\n"),
66 +
    )
67 +
        .into_response()
68 +
}
69 +
70 +
async fn add_checkpoint(State(w): State<Arc<Witness>>, body: String) -> Response {
71 +
    let Ok(req) = AddCheckpoint::parse(&body) else {
72 +
        return plain(StatusCode::BAD_REQUEST, "not an add-checkpoint request");
73 +
    };
74 +
    let Some(origin) = Note::parse(&req.note)
75 +
        .ok()
76 +
        .and_then(|n| Checkpoint::parse(&n.text).ok())
77 +
        .map(|c| c.origin)
78 +
    else {
79 +
        return plain(StatusCode::BAD_REQUEST, "the checkpoint does not parse");
80 +
    };
81 +
    let Some(key) = w
82 +
        .cfg
83 +
        .logs
84 +
        .iter()
85 +
        .find(|(o, _)| *o == origin)
86 +
        .map(|(_, k)| k.clone())
87 +
    else {
88 +
        return plain(
89 +
            StatusCode::NOT_FOUND,
90 +
            &format!("this witness does not witness {origin}"),
91 +
        );
92 +
    };
93 +
    let _held = w.lock.lock().unwrap_or_else(|p| p.into_inner());
94 +
    let latest = read_state(&w, &origin);
95 +
    let me = WitnessKey {
96 +
        name: &w.cfg.name,
97 +
        identity: &w.cfg.identity,
98 +
        sk: &w.cfg.sk,
99 +
    };
100 +
    let now = std::time::SystemTime::now()
101 +
        .duration_since(std::time::UNIX_EPOCH)
102 +
        .map(|d| d.as_secs())
103 +
        .unwrap_or(0);
104 +
    let conflict = |n: u64| {
105 +
        (
106 +
            StatusCode::CONFLICT,
107 +
            [(header::CONTENT_TYPE, "text/x.tlog.size")],
108 +
            format!("{n}\n"),
109 +
        )
110 +
            .into_response()
111 +
    };
112 +
    match tlog::witness(&req, &origin, &key, latest, &me, now) {
113 +
        Ok(Witnessed::Cosigned { checkpoint, lines }) => {
114 +
            let text = format!(
115 +
                "{} {}\n",
116 +
                checkpoint.size,
117 +
                oxy_core::hash::ObjectRef(checkpoint.root).to_hex()
118 +
            );
119 +
            if let Err(e) = std::fs::create_dir_all(&w.cfg.state).and_then(|_| {
120 +
                oxy_core::durable::write_atomic(&state_file(&w, &origin), text.as_bytes())
121 +
            }) {
122 +
                return plain(
123 +
                    StatusCode::INTERNAL_SERVER_ERROR,
124 +
                    &format!("keeping the witness's state: {e}"),
125 +
                );
126 +
            }
127 +
            let body = Note {
128 +
                text: String::new(),
129 +
                sigs: lines,
130 +
            }
131 +
            .encode();
132 +
            (
133 +
                [(header::CONTENT_TYPE, "text/plain; charset=utf-8")],
134 +
                body.trim_start_matches('\n').to_string(),
135 +
            )
136 +
                .into_response()
137 +
        }
138 +
        Ok(Witnessed::Conflict(n)) => conflict(n),
139 +
        Err(WitnessError::Unsigned(e)) => plain(StatusCode::FORBIDDEN, &e),
140 +
        Err(WitnessError::Inconsistent(e)) => plain(StatusCode::UNPROCESSABLE_ENTITY, &e),
141 +
        Err(WitnessError::Bad(e)) => plain(StatusCode::BAD_REQUEST, &e),
142 +
    }
143 +
}
144 +
145 +
/// The witness's routes: `POST /add-checkpoint`.
146 +
pub fn router(cfg: WitnessConfig) -> Router {
147 +
    Router::new()
148 +
        .route("/add-checkpoint", post(add_checkpoint))
149 +
        .with_state(Arc::new(Witness {
150 +
            cfg,
151 +
            lock: Mutex::new(()),
152 +
        }))
153 +
}
154 +
155 +
/// Serve a witness on `listener` until stopped.
156 +
pub fn run_blocking(
157 +
    listener: std::net::TcpListener,
158 +
    mut cfg: WitnessConfig,
159 +
) -> std::io::Result<()> {
160 +
    let rt = tokio::runtime::Builder::new_multi_thread()
161 +
        .enable_all()
162 +
        .build()?;
163 +
    let tls = cfg.tls.take();
164 +
    rt.block_on(async move {
165 +
        listener.set_nonblocking(true)?;
166 +
        match tls {
167 +
            Some(tls) => {
168 +
                let _ = rustls::crypto::ring::default_provider().install_default();
169 +
                let rustls_cfg =
170 +
                    axum_server::tls_rustls::RustlsConfig::from_pem_file(&tls.cert, &tls.key)
171 +
                        .await
172 +
                        .map_err(|e| std::io::Error::other(format!("loading TLS files: {e}")))?;
173 +
                axum_server::from_tcp_rustls(listener, rustls_cfg)?
174 +
                    .serve(router(cfg).into_make_service())
175 +
                    .await
176 +
            }
177 +
            None => axum::serve(tokio::net::TcpListener::from_std(listener)?, router(cfg)).await,
178 +
        }
179 +
    })
180 +
}
181 +
182 +
#[cfg(test)]
183 +
mod tests {
184 +
    use super::*;
185 +
    use axum::body::Body;
186 +
    use axum::http::Request;
187 +
    use tower::ServiceExt;
188 +
189 +
    #[tokio::test]
190 +
    async fn the_witness_cosigns_and_keeps_what_it_cosigned() {
191 +
        let dir = tempfile::tempdir().unwrap();
192 +
        let log_sk = SecretKey::generate();
193 +
        let log = log_sk.identity("log", "[email protected]", 1);
194 +
        let w_sk = SecretKey::generate();
195 +
        let w = w_sk.identity("witness", "[email protected]", 1);
196 +
        let cfg = || WitnessConfig {
197 +
            name: "witness.example".into(),
198 +
            identity: w.clone(),
199 +
            sk: w_sk.clone(),
200 +
            logs: vec![("hub.example/log".into(), log.clone())],
201 +
            state: dir.path().join("state"),
202 +
            tls: None,
203 +
        };
204 +
        let mut stored: Vec<tlog::Hash> = Vec::new();
205 +
        for n in 0..9u64 {
206 +
            stored.extend(
207 +
                tlog::stored_hashes_for_leaf(n, tlog::leaf_hash(&[n as u8]), &stored).unwrap(),
208 +
            );
209 +
        }
210 +
        let note = |n: u64| {
211 +
            Checkpoint {
212 +
                origin: "hub.example/log".into(),
213 +
                size: n,
214 +
                root: tlog::tree_hash(n, &stored).unwrap(),
215 +
                extensions: vec![],
216 +
            }
217 +
            .sign(&log, &log_sk)
218 +
            .unwrap()
219 +
            .encode()
220 +
        };
221 +
        let post = |body: String| async {
222 +
            let res = router(cfg())
223 +
                .oneshot(
224 +
                    Request::post("/add-checkpoint")
225 +
                        .body(Body::from(body))
226 +
                        .unwrap(),
227 +
                )
228 +
                .await
229 +
                .unwrap();
230 +
            let st = res.status();
231 +
            let b = axum::body::to_bytes(res.into_body(), 1 << 20)
232 +
                .await
233 +
                .unwrap();
234 +
            (st, String::from_utf8(b.to_vec()).unwrap())
235 +
        };
236 +
        let (st, lines) = post(
237 +
            AddCheckpoint {
238 +
                old_size: 0,
239 +
                proof: vec![],
240 +
                note: note(4),
241 +
            }
242 +
            .encode(),
243 +
        )
244 +
        .await;
245 +
        assert_eq!(st, StatusCode::OK, "{lines}");
246 +
        let cosigned =
247 +
            Note::parse(&format!("{}\n{lines}", Note::parse(&note(4)).unwrap().text)).unwrap();
248 +
        assert!(cosigned.cosigned_by("witness.example", &w).is_ok());
249 +
        // Kept across a restart (a new router): a wrong old size is told.
250 +
        let (st, size) = post(
251 +
            AddCheckpoint {
252 +
                old_size: 0,
253 +
                proof: vec![],
254 +
                note: note(9),
255 +
            }
256 +
            .encode(),
257 +
        )
258 +
        .await;
259 +
        assert_eq!((st, size.trim()), (StatusCode::CONFLICT, "4"));
260 +
        let proof = tlog::prove_consistency(9, 4, &stored).unwrap();
261 +
        let (st, _) = post(
262 +
            AddCheckpoint {
263 +
                old_size: 4,
264 +
                proof,
265 +
                note: note(9),
266 +
            }
267 +
            .encode(),
268 +
        )
269 +
        .await;
270 +
        assert_eq!(st, StatusCode::OK);
271 +
    }
272 +
}
modified crates/oxy-server/tests/http.rs +567 −0
…
113 unchanged lines
114 114
        .contains("404"));
115 115
}
116 116
117 +
/// A push of more than axum's default 2 MB body is taken: `oxy serve`
118 +
/// sets its own limit, as OxygenHub does.
119 +
#[test]
120 +
fn a_push_of_more_than_two_megabytes_is_taken() {
121 +
    hermetic();
122 +
    let dir = tempfile::tempdir().unwrap();
123 +
    let root = dir.path();
124 +
    let served = root.join("served");
125 +
    Repo::init(&served.join("origin")).unwrap();
126 +
    let url = format!("{}/origin", start(&served, vec![]));
127 +
    let a = Repo::clone(&url, &root.join("a")).unwrap();
128 +
    let mut x: u64 = 0x9e3779b97f4a7c15;
129 +
    let noise: Vec<u8> = (0..3 << 20)
130 +
        .map(|_| {
131 +
            x ^= x << 13;
132 +
            x ^= x >> 7;
133 +
            x ^= x << 17;
134 +
            (x >> 24) as u8
135 +
        })
136 +
        .collect();
137 +
    write(&root.join("a"), "big.bin", &noise);
138 +
    let h = a.commit("three megabytes", &[]).unwrap();
139 +
    let r = a.push("origin", false).unwrap();
140 +
    assert!(r[0].ok, "{}", r[0].reason);
141 +
    let b = Repo::clone(&url, &root.join("b")).unwrap();
142 +
    assert_eq!(b.bookmark_heads().unwrap(), vec![h]);
143 +
    assert_eq!(fs::read(root.join("b/big.bin")).unwrap(), noise);
144 +
}
145 +
117 146
#[test]
118 147
fn objects_endpoint_serves_lazy_blobs() {
119 148
    hermetic();
…
570 unchanged lines
690 719
        fs::set_permissions(&attest, fs::Permissions::from_mode(0o755)).unwrap();
691 720
    }
692 721
}
722 +
723 +
/// oxygen#112: a subrepo checkout whose stored login for the hub is stale
724 +
/// pushes with the token its parent checkout holds for the same hub, once
725 +
/// the stale one is refused; a token the subrepo sets itself is never
726 +
/// swapped for another.
727 +
#[test]
728 +
fn a_refused_stored_token_falls_back_to_the_parents() {
729 +
    hermetic();
730 +
    let dir = tempfile::tempdir().unwrap();
731 +
    let root = dir.path();
732 +
    let served = root.join("served");
733 +
    Repo::init(&served.join("parent")).unwrap();
734 +
    Repo::init(&served.join("child")).unwrap();
735 +
    let base = start(&served, vec!["good".into()]);
736 +
737 +
    let work = root.join("work");
738 +
    let mut parent = Repo::clone(&format!("{base}/parent"), &work).unwrap();
739 +
    parent.config.set("remote.origin.token", "good").unwrap();
740 +
    parent.save_config().unwrap();
741 +
    // The subrepo's own home holds a stale login for this hub.
742 +
    let home = root.join("home");
743 +
    oxy_core::credentials::Credentials::at(&home)
744 +
        .save(&base, "stale", Some("andy"), None)
745 +
        .unwrap();
746 +
    let nested = work.join("deps").join("child");
747 +
    let mut child = Repo::clone(&format!("{base}/child"), &nested)
748 +
        .unwrap()
749 +
        .with_home(&home);
750 +
    let options = child.options_for_url(&format!("{base}/child"));
751 +
    assert_eq!(
752 +
        options.token.as_deref(),
753 +
        Some("stale"),
754 +
        "the stored login comes first"
755 +
    );
756 +
    assert_eq!(
757 +
        options.fallback_tokens,
758 +
        vec!["good".to_string()],
759 +
        "the parent's, to fall back on"
760 +
    );
761 +
    assert!(!options.fallback_on_401_only);
762 +
    write(&nested, "c.txt", b"child\n");
763 +
    child.commit("child work", &[]).unwrap();
764 +
    let pushed = child.push("origin", false).unwrap();
765 +
    assert!(pushed[0].ok, "{}", pushed[0].reason);
766 +
    assert_eq!(
767 +
        Repo::open(&served.join("child"))
768 +
            .unwrap()
769 +
            .bookmark_heads()
770 +
            .unwrap()
771 +
            .len(),
772 +
        1
773 +
    );
774 +
775 +
    // Its own token, if refused, falls back to this computer's sign-in,
776 +
    // never to the parent's (#155): stale too here, so the push fails and
777 +
    // says what to do.
778 +
    child.config.set("remote.origin.token", "wrong").unwrap();
779 +
    child.save_config().unwrap();
780 +
    let options = child.options_for_url(&format!("{base}/child"));
781 +
    assert_eq!(
782 +
        options.fallback_tokens,
783 +
        vec!["stale".to_string()],
784 +
        "the sign-in only"
785 +
    );
786 +
    assert!(
787 +
        options.fallback_on_401_only,
788 +
        "and only for a token the hub does not know: one refused for permission is not swapped"
789 +
    );
790 +
    write(&nested, "d.txt", b"more\n");
791 +
    child.commit("more", &[]).unwrap();
792 +
    let err = child.push("origin", false).err().unwrap().to_string();
793 +
    assert!(err.contains("401"), "{err}");
794 +
    let note = child.stale_token_note("origin", &err).expect("a note");
795 +
    assert!(
796 +
        note.contains("oxy auth login") && note.contains("oxy config --unset remote.origin.token"),
797 +
        "{note}"
798 +
    );
799 +
    assert!(child.stale_token_note("origin", "HTTP 500").is_none());
800 +
    // With a good sign-in, an expired checkout token no longer blocks.
801 +
    oxy_core::credentials::Credentials::at(&home)
802 +
        .save(&base, "good", Some("andy"), None)
803 +
        .unwrap();
804 +
    let child = Repo::open(&nested).unwrap().with_home(&home);
805 +
    let pushed = child.push("origin", false).unwrap();
806 +
    assert!(pushed[0].ok, "{}", pushed[0].reason);
807 +
}
808 +
809 +
/// A host refuses a push by what it adds (`Backend::vet_push`): the client
810 +
/// sees the host's reason, every line of it, and nothing is stored, bound
811 +
/// or moved; a push without that content then lands.
812 +
#[test]
813 +
fn a_host_refuses_a_push_by_its_content() {
814 +
    use axum::extract::{Path as AxPath, State};
815 +
    use axum::http::{HeaderMap, Method, StatusCode};
816 +
    use axum::response::Response;
817 +
    use oxy_core::objects::Edit;
818 +
    use oxy_core::protocol::{Bundle, RefUpdate};
819 +
    use oxy_server::{Access, Backend, DirBackend, Endpoint, Failure, Server};
820 +
    use std::sync::Arc;
821 +
822 +
    struct Vetting(DirBackend);
823 +
    impl Backend for Vetting {
824 +
        fn open(&self, name: &str) -> Result<Repo, Failure> {
825 +
            self.0.open(name)
826 +
        }
827 +
        fn authorize(
828 +
            &self,
829 +
            name: &str,
830 +
            access: Access,
831 +
            headers: &HeaderMap,
832 +
        ) -> Result<(), Failure> {
833 +
            self.0.authorize(name, access, headers)
834 +
        }
835 +
        fn vet_push(
836 +
            &self,
837 +
            _name: &str,
838 +
            _repo: &Repo,
839 +
            bundle: &Bundle,
840 +
            updates: &[RefUpdate],
841 +
            _headers: &HeaderMap,
842 +
        ) -> Result<(), Failure> {
843 +
            let adds_it = bundle.changes.iter().any(|(c, _)| {
844 +
                c.edits.iter().any(|e| match e {
845 +
                    Edit::Insert { lines, .. } => lines
846 +
                        .iter()
847 +
                        .any(|l| l.windows(11).any(|w| w == b"DO NOT PUSH")),
848 +
                    _ => false,
849 +
                })
850 +
            });
851 +
            if adds_it {
852 +
                return Err(Failure(
853 +
                    StatusCode::UNPROCESSABLE_ENTITY,
854 +
                    format!(
855 +
                        "refused: a change adds DO NOT PUSH\n  a.txt:2\nremove it and push {} again",
856 +
                        updates.first().map(|u| u.name.as_str()).unwrap_or("")
857 +
                    ),
858 +
                ));
859 +
            }
860 +
            Ok(())
861 +
        }
862 +
    }
863 +
    async fn route(
864 +
        State(server): State<Arc<Server>>,
865 +
        AxPath((repo, endpoint)): AxPath<(String, String)>,
866 +
        method: Method,
867 +
        headers: HeaderMap,
868 +
        body: axum::body::Bytes,
869 +
    ) -> Response {
870 +
        let e = Endpoint::parse(&endpoint).unwrap();
871 +
        server.handle(repo, e, &method, &headers, body).await
872 +
    }
873 +
874 +
    hermetic();
875 +
    let dir = tempfile::tempdir().unwrap();
876 +
    let root = dir.path();
877 +
    let served = root.join("served");
878 +
    let hosted = Repo::init_bare_unbound(&served.join("lib")).unwrap();
879 +
    let server = Server::new(Vetting(DirBackend {
880 +
        root: served.clone(),
881 +
        tokens: vec![],
882 +
    }));
883 +
    let listener = TcpListener::bind("127.0.0.1:0").unwrap();
884 +
    let port = listener.local_addr().unwrap().port();
885 +
    std::thread::spawn(move || {
886 +
        let rt = tokio::runtime::Builder::new_multi_thread()
887 +
            .enable_all()
888 +
            .build()
889 +
            .unwrap();
890 +
        rt.block_on(async move {
891 +
            listener.set_nonblocking(true).unwrap();
892 +
            let app = axum::Router::new()
893 +
                .route("/{repo}/oxy/v1/{endpoint}", axum::routing::any(route))
894 +
                .with_state(server);
895 +
            axum::serve(tokio::net::TcpListener::from_std(listener).unwrap(), app)
896 +
                .await
897 +
                .unwrap();
898 +
        });
899 +
    });
900 +
    std::thread::sleep(std::time::Duration::from_millis(50));
901 +
    let url = format!("http://127.0.0.1:{port}/lib");
902 +
903 +
    let work = root.join("work");
904 +
    let mut local = Repo::init(&work).unwrap();
905 +
    local.config.set("remote.origin.url", &url).unwrap();
906 +
    local.save_config().unwrap();
907 +
    write(&work, "a.txt", b"one\nDO NOT PUSH\n");
908 +
    local.snapshot("test").unwrap();
909 +
    let bad = local.commit("a", &[]).unwrap();
910 +
    let err = local.push("origin", false).err().unwrap().to_string();
911 +
    assert!(
912 +
        err.contains("HTTP 422")
913 +
            && err.contains(
914 +
                "refused: a change adds DO NOT PUSH\n  a.txt:2\nremove it and push main again"
915 +
            )
916 +
            && !err.contains("hint:"),
917 +
        "{err}"
918 +
    );
919 +
    let hosted_now = Repo::open(&served.join("lib")).unwrap();
920 +
    assert!(hosted_now.is_unbound(), "a refused push binds nothing");
921 +
    assert!(hosted_now.bookmark_heads().unwrap().is_empty());
922 +
    assert!(
923 +
        hosted_now.store().get_change(&bad).is_err(),
924 +
        "nothing stored"
925 +
    );
926 +
    drop(hosted);
927 +
928 +
    // Without it, the push lands.
929 +
    write(&work, "a.txt", b"one\n");
930 +
    local.snapshot("test").unwrap();
931 +
    local.amend(None).unwrap();
932 +
    let results = local.push("origin", false).unwrap();
933 +
    assert!(results.iter().all(|r| r.ok), "{results:?}");
934 +
    assert_eq!(
935 +
        Repo::open(&served.join("lib"))
936 +
            .unwrap()
937 +
            .bookmark_heads()
938 +
            .unwrap()
939 +
            .len(),
940 +
        1
941 +
    );
942 +
}
943 +
944 +
/// A host adds a note to a reference a push moved (`Backend::push_notes`):
945 +
/// the client gets it as that reference's result, accepted as before,
946 +
/// and the host's `after_push` sees it too; a reference with no note
947 +
/// says nothing more.
948 +
#[test]
949 +
fn a_host_adds_a_note_to_an_accepted_reference() {
950 +
    use axum::extract::{Path as AxPath, State};
951 +
    use axum::http::{HeaderMap, Method};
952 +
    use axum::response::Response;
953 +
    use oxy_core::repo::PushReport;
954 +
    use oxy_server::{Access, Backend, DirBackend, Endpoint, Failure, Server};
955 +
    use std::sync::{Arc, Mutex};
956 +
957 +
    struct Noting(DirBackend, Arc<Mutex<Vec<String>>>);
958 +
    impl Backend for Noting {
959 +
        fn open(&self, name: &str) -> Result<Repo, Failure> {
960 +
            self.0.open(name)
961 +
        }
962 +
        fn authorize(
963 +
            &self,
964 +
            name: &str,
965 +
            access: Access,
966 +
            headers: &HeaderMap,
967 +
        ) -> Result<(), Failure> {
968 +
            self.0.authorize(name, access, headers)
969 +
        }
970 +
        fn push_notes(
971 +
            &self,
972 +
            _name: &str,
973 +
            report: &PushReport,
974 +
            _headers: &HeaderMap,
975 +
        ) -> Vec<(String, String)> {
976 +
            report
977 +
                .updated
978 +
                .iter()
979 +
                .filter(|u| u.name == "feature")
980 +
                .map(|u| (u.name.clone(), "its proposal #3 has landed".to_string()))
981 +
                .collect()
982 +
        }
983 +
        fn after_push(&self, _name: &str, report: &PushReport, _headers: &HeaderMap) {
984 +
            let mut seen = self.1.lock().unwrap();
985 +
            seen.extend(report.results.iter().map(|r| r.reason.clone()));
986 +
        }
987 +
    }
988 +
    async fn route(
989 +
        State(server): State<Arc<Server>>,
990 +
        AxPath((repo, endpoint)): AxPath<(String, String)>,
991 +
        method: Method,
992 +
        headers: HeaderMap,
993 +
        body: axum::body::Bytes,
994 +
    ) -> Response {
995 +
        let e = Endpoint::parse(&endpoint).unwrap();
996 +
        server.handle(repo, e, &method, &headers, body).await
997 +
    }
998 +
999 +
    hermetic();
1000 +
    let dir = tempfile::tempdir().unwrap();
1001 +
    let root = dir.path();
1002 +
    let served = root.join("served");
1003 +
    Repo::init_bare_unbound(&served.join("lib")).unwrap();
1004 +
    let seen = Arc::new(Mutex::new(Vec::new()));
1005 +
    let server = Server::new(Noting(
1006 +
        DirBackend {
1007 +
            root: served.clone(),
1008 +
            tokens: vec![],
1009 +
        },
1010 +
        seen.clone(),
1011 +
    ));
1012 +
    let listener = TcpListener::bind("127.0.0.1:0").unwrap();
1013 +
    let port = listener.local_addr().unwrap().port();
1014 +
    std::thread::spawn(move || {
1015 +
        let rt = tokio::runtime::Builder::new_multi_thread()
1016 +
            .enable_all()
1017 +
            .build()
1018 +
            .unwrap();
1019 +
        rt.block_on(async move {
1020 +
            listener.set_nonblocking(true).unwrap();
1021 +
            let app = axum::Router::new()
1022 +
                .route("/{repo}/oxy/v1/{endpoint}", axum::routing::any(route))
1023 +
                .with_state(server);
1024 +
            axum::serve(tokio::net::TcpListener::from_std(listener).unwrap(), app)
1025 +
                .await
1026 +
                .unwrap();
1027 +
        });
1028 +
    });
1029 +
    std::thread::sleep(std::time::Duration::from_millis(50));
1030 +
    let url = format!("http://127.0.0.1:{port}/lib");
1031 +
1032 +
    let work = root.join("work");
1033 +
    let mut local = Repo::init(&work).unwrap();
1034 +
    local.config.set("remote.origin.url", &url).unwrap();
1035 +
    local.save_config().unwrap();
1036 +
    write(&work, "a.txt", b"one\n");
1037 +
    local.snapshot("test").unwrap();
1038 +
    let base = local.commit("a", &[]).unwrap();
1039 +
    let results = local.push("origin", false).unwrap();
1040 +
    assert!(
1041 +
        results.iter().all(|r| r.ok && r.reason.is_empty()),
1042 +
        "{results:?}"
1043 +
    );
1044 +
    local.bookmark_create("feature", Some(&[base])).unwrap();
1045 +
    local.bookmark_switch("feature", false).unwrap();
1046 +
    write(&work, "a.txt", b"one\ntwo\n");
1047 +
    local.snapshot("test").unwrap();
1048 +
    local.commit("two", &[]).unwrap();
1049 +
    let results = local.push_bookmark("origin", "feature", false).unwrap();
1050 +
    let feature = results.iter().find(|r| r.name == "feature").unwrap();
1051 +
    assert!(feature.ok, "{results:?}");
1052 +
    assert_eq!(feature.reason, "its proposal #3 has landed");
1053 +
    assert!(seen
1054 +
        .lock()
1055 +
        .unwrap()
1056 +
        .iter()
1057 +
        .any(|r| r == "its proposal #3 has landed"));
1058 +
}
1059 +
1060 +
/// oxygen#131: oxy 0.5.0 and older authorize a tag only by a key the
1061 +
/// policy lists (or its delegate or predecessor) and fail the whole clone
1062 +
/// on any other, so a client that does not declare `certified-tags` is
1063 +
/// never sent a tag that only a short-lived key's certificate authorizes.
1064 +
/// It clones the rest without error; a current client still gets the tag;
1065 +
/// and an old client that holds such a tag pushes without trouble.
1066 +
#[test]
1067 +
fn a_client_without_certified_tags_never_sees_a_tag_only_a_certificate_authorizes() {
1068 +
    use axum::extract::{Path as AxPath, State};
1069 +
    use axum::http::{HeaderMap, Method, StatusCode};
1070 +
    use axum::response::{IntoResponse, Response};
1071 +
    use oxy_core::cert::{Certificate, SignedCertificate};
1072 +
    use oxy_core::identity::{Identity, Scope, SecretKey};
1073 +
    use oxy_core::protocol::{wire, RemoteOptions, FEATURES_HEADER};
1074 +
    use oxy_core::repo::IdentityScope;
1075 +
    use oxy_server::{Access, Backend, Endpoint, Failure, Server};
1076 +
    use std::sync::Arc;
1077 +
1078 +
    // A host that countersigns when it accepted a certified key's tag, as
1079 +
    // a hub does.
1080 +
    struct Host {
1081 +
        root: std::path::PathBuf,
1082 +
        acceptor: (Identity, SecretKey),
1083 +
    }
1084 +
    impl Backend for Host {
1085 +
        fn open(&self, name: &str) -> Result<Repo, Failure> {
1086 +
            Repo::open(&self.root.join(name))
1087 +
                .map(|r| r.with_acceptor(self.acceptor.0.clone(), self.acceptor.1.clone()))
1088 +
                .map_err(|e| Failure(StatusCode::NOT_FOUND, e.to_string()))
1089 +
        }
1090 +
        fn authorize(&self, _: &str, _: Access, _: &HeaderMap) -> Result<(), Failure> {
1091 +
            Ok(())
1092 +
        }
1093 +
    }
1094 +
    // `/new/...` as clients send it; `/old/...` without the features
1095 +
    // header, as oxy 0.5.0 sends it.
1096 +
    async fn route(
1097 +
        State(server): State<Arc<Server>>,
1098 +
        AxPath((era, repo, endpoint)): AxPath<(String, String, String)>,
1099 +
        method: Method,
1100 +
        mut headers: HeaderMap,
1101 +
        body: axum::body::Bytes,
1102 +
    ) -> Response {
1103 +
        if era == "old" {
1104 +
            headers.remove(FEATURES_HEADER);
1105 +
        }
1106 +
        match Endpoint::parse(&endpoint) {
1107 +
            Some(e) => server.handle(repo, e, &method, &headers, body).await,
1108 +
            None => Failure(StatusCode::NOT_FOUND, "no such endpoint".into()).into_response(),
1109 +
        }
1110 +
    }
1111 +
1112 +
    hermetic();
1113 +
    let dir = tempfile::tempdir().unwrap();
1114 +
    let root = dir.path();
1115 +
    let served = root.join("served");
1116 +
    Repo::init(&served.join("origin")).unwrap();
1117 +
    let hub_sk = SecretKey::generate();
1118 +
    let hub = hub_sk.identity("hub", "[email protected]", 1);
1119 +
    let server = Server::new(Host {
1120 +
        root: served.clone(),
1121 +
        acceptor: (hub.clone(), hub_sk.clone()),
1122 +
    });
1123 +
    let listener = TcpListener::bind("127.0.0.1:0").unwrap();
1124 +
    let port = listener.local_addr().unwrap().port();
1125 +
    std::thread::spawn(move || {
1126 +
        let rt = tokio::runtime::Builder::new_multi_thread()
1127 +
            .enable_all()
1128 +
            .build()
1129 +
            .unwrap();
1130 +
        rt.block_on(async move {
1131 +
            listener.set_nonblocking(true).unwrap();
1132 +
            let app = axum::Router::new()
1133 +
                .route("/{era}/{repo}/oxy/v1/{endpoint}", axum::routing::any(route))
1134 +
                .with_state(server);
1135 +
            axum::serve(tokio::net::TcpListener::from_std(listener).unwrap(), app)
1136 +
                .await
1137 +
                .unwrap();
1138 +
        });
1139 +
    });
1140 +
    std::thread::sleep(std::time::Duration::from_millis(50));
1141 +
    let new_url = format!("http://127.0.0.1:{port}/new/origin");
1142 +
    let old_url = format!("http://127.0.0.1:{port}/old/origin");
1143 +
1144 +
    // Ada's policy grants the hub's certified agents; her own tag v0 is
1145 +
    // by a key the policy lists.
1146 +
    let mut a = Repo::clone(&new_url, &root.join("a"))
1147 +
        .unwrap()
1148 +
        .with_home(root.join("home-a"));
1149 +
    a.create_identity("Ada", "[email protected]", IdentityScope::User)
1150 +
        .unwrap();
1151 +
    write(&root.join("a"), "a.txt", b"a\n");
1152 +
    a.commit("a", &[]).unwrap();
1153 +
    a.policy_init().unwrap();
1154 +
    assert!(a.push("origin", false).unwrap().iter().all(|r| r.ok));
1155 +
    a.policy_grant_subject(
1156 +
        "agent:claude@ada",
1157 +
        hub.clone(),
1158 +
        Scope::CHANGES | Scope::REFS,
1159 +
        Some("*"),
1160 +
    )
1161 +
    .unwrap();
1162 +
    let heads = a.bookmark_heads().unwrap();
1163 +
    a.tag_create("v0", &heads, "zero").unwrap();
1164 +
    let r = a.push("origin", false).unwrap();
1165 +
    assert!(r.iter().all(|x| x.ok), "{r:?}");
1166 +
1167 +
    // A keyless agent's tag v1, by a key the hub certified for ten minutes.
1168 +
    let now = oxy_core::objects::Hlc::now().wall_ms;
1169 +
    let mut c = Repo::clone(&new_url, &root.join("claude"))
1170 +
        .unwrap()
1171 +
        .with_home(root.join("home-claude"));
1172 +
    c.store().put_identity(&hub).unwrap();
1173 +
    let (_, id) = c
1174 +
        .create_identity("claude", "[email protected]", IdentityScope::User)
1175 +
        .unwrap();
1176 +
    let cert = SignedCertificate::sign(
1177 +
        Certificate {
1178 +
            key: id,
1179 +
            issuer: hub.hash(),
1180 +
            subject: "agent:claude@ada".into(),
1181 +
            claims: vec![],
1182 +
            scope: Scope::CHANGES | Scope::REFS,
1183 +
            not_before: now - 60_000,
1184 +
            expires: now + 600_000,
1185 +
            grant: None,
1186 +
        },
1187 +
        &hub,
1188 +
        &hub_sk,
1189 +
    );
1190 +
    c.store().put_certificate(&cert).unwrap();
1191 +
    c.tag_create("v1", &heads, "one").unwrap();
1192 +
    let r = c.push("origin", false).unwrap();
1193 +
    assert!(r.iter().all(|x| x.ok), "{r:?}");
1194 +
1195 +
    // The refs answer: without the header, v1 is left out; with it, there.
1196 +
    let tags = |bytes: Vec<u8>| -> Vec<String> {
1197 +
        let (_, refs) = wire::decode_refs(&bytes).unwrap();
1198 +
        refs.tags.into_iter().map(|t| t.name).collect()
1199 +
    };
1200 +
    let bare =
1201 +
        oxy_core::protocol::http_get(&format!("{new_url}/oxy/v1/refs"), &RemoteOptions::default())
1202 +
            .unwrap();
1203 +
    assert_eq!(tags(bare), ["v0"], "no Oxy-Features header");
1204 +
    let remote = oxy_core::protocol::open_remote(&new_url, &RemoteOptions::default()).unwrap();
1205 +
    let mut current: Vec<String> = remote
1206 +
        .refs()
1207 +
        .unwrap()
1208 +
        .tags
1209 +
        .into_iter()
1210 +
        .map(|t| t.name)
1211 +
        .collect();
1212 +
    current.sort();
1213 +
    assert_eq!(
1214 +
        current,
1215 +
        ["v0", "v1"],
1216 +
        "a current client declares certified-tags"
1217 +
    );
1218 +
1219 +
    // An old-era client clones without error and sees no v1.
1220 +
    let old = Repo::clone(&old_url, &root.join("old")).unwrap();
1221 +
    assert_eq!(old.tag_heads("v0").unwrap(), Some(heads.clone()));
1222 +
    assert_eq!(old.tag_heads("v1").unwrap(), None);
1223 +
    let rep = old.fetch("origin").unwrap();
1224 +
    assert!(rep.skipped.is_empty(), "{rep:?}");
1225 +
1226 +
    // A current client adopts v1 by its certificate at the hub's acceptance.
1227 +
    let b = Repo::clone(&new_url, &root.join("b")).unwrap();
1228 +
    assert_eq!(b.tag_heads("v1").unwrap(), Some(heads.clone()));
1229 +
1230 +
    // A client holding v1 that pushes where v1 is hidden from it sends v1
1231 +
    // again: the host already has it, so it is taken as it is, and the
1232 +
    // bookmark moves.
1233 +
    let r = a.fetch("origin").unwrap();
1234 +
    assert!(r.skipped.is_empty(), "{r:?}");
1235 +
    assert_eq!(a.tag_heads("v1").unwrap(), Some(heads.clone()));
1236 +
    a.config.set("remote.origin.url", &old_url).unwrap();
1237 +
    write(&root.join("a"), "b.txt", b"b\n");
1238 +
    a.commit("b", &[]).unwrap();
1239 +
    let r = a.push("origin", false).unwrap();
1240 +
    assert!(r.iter().all(|x| x.ok), "{r:?}");
1241 +
    assert!(r.iter().any(|x| x.name == "tags/v1"), "{r:?}");
1242 +
1243 +
    // `oxy serve` applies the same rule.
1244 +
    let base = start(&served, vec![]);
1245 +
    let bare = oxy_core::protocol::http_get(
1246 +
        &format!("{base}/origin/oxy/v1/refs"),
1247 +
        &RemoteOptions::default(),
1248 +
    )
1249 +
    .unwrap();
1250 +
    assert_eq!(tags(bare), ["v0"], "oxy serve, no header");
1251 +
    let remote =
1252 +
        oxy_core::protocol::open_remote(&format!("{base}/origin"), &RemoteOptions::default())
1253 +
            .unwrap();
1254 +
    assert_eq!(
1255 +
        remote.refs().unwrap().tags.len(),
1256 +
        2,
1257 +
        "oxy serve, current client"
1258 +
    );
1259 +
}
modified crates/oxy-strata/Cargo.toml +1 −1
…
18 unchanged lines
19 19
http = "1"
20 20
http-body-util = "0.1"
21 21
bytes = "1"
22 -
tokio = { version = "1", default-features = false, features = ["rt"] }
22 +
tokio = { version = "1", default-features = false, features = ["rt-multi-thread", "rt"] }
23 23
futures = "0.3"
24 24
serde_json = "1"
25 25
…
2 unchanged lines
modified crates/oxy-strata/src/aws.rs +33 −7
…
94 unchanged lines
95 95
    })
96 96
}
97 97
98 -
pub fn open(url: &StoreUrl) -> Result<AmazonS3> {
98 +
/// With `keys` given, only they are used: nothing from the environment
99 +
/// or a profile, and the region and endpoint come from the URL alone.
100 +
pub fn open(url: &StoreUrl, keys: Option<(&str, &str, Option<&str>)>) -> Result<AmazonS3> {
101 +
    if let Some((id, secret, token)) = keys {
102 +
        let mut b = AmazonS3Builder::new()
103 +
            .with_bucket_name(&url.host)
104 +
            .with_http_connector(crate::http::UreqConnector)
105 +
            .with_conditional_put(S3ConditionalPut::ETagMatch)
106 +
            .with_access_key_id(id)
107 +
            .with_secret_access_key(secret);
108 +
        if let Some(t) = token {
109 +
            b = b.with_token(t);
110 +
        }
111 +
        let endpoint = url.param("endpoint").map(str::to_string);
112 +
        return finish(b, url, endpoint, url.param("region").map(str::to_string));
113 +
    }
99 114
    let bucket = url.host.clone();
100 115
    let explicit_profile = url.param("profile").map(str::to_string);
101 116
    let profile = explicit_profile
…
61 unchanged lines
163 178
        .or_else(|| env("AWS_ENDPOINT_URL"))
164 179
        .or_else(|| env("AWS_ENDPOINT"))
165 180
        .or_else(|| section.and_then(|s| s.get("endpoint_url").cloned()));
181 +
    let region = url
182 +
        .param("region")
183 +
        .map(str::to_string)
184 +
        .or_else(|| env("AWS_REGION"))
185 +
        .or_else(|| env("AWS_DEFAULT_REGION"))
186 +
        .or_else(|| section.and_then(|s| s.get("region").cloned()));
187 +
    finish(b, url, endpoint, region)
188 +
}
189 +
190 +
/// The endpoint and region, then the store.
191 +
fn finish(
192 +
    mut b: AmazonS3Builder,
193 +
    url: &StoreUrl,
194 +
    endpoint: Option<String>,
195 +
    region: Option<String>,
196 +
) -> Result<AmazonS3> {
197 +
    let bucket = url.host.clone();
166 198
    if let Some(e) = &endpoint {
167 199
        b = b
168 200
            .with_endpoint(e.clone())
…
2 unchanged lines
171 203
        b = b.with_virtual_hosted_style_request(false);
172 204
    }
173 205
174 -
    let region = url
175 -
        .param("region")
176 -
        .map(str::to_string)
177 -
        .or_else(|| env("AWS_REGION"))
178 -
        .or_else(|| env("AWS_DEFAULT_REGION"))
179 -
        .or_else(|| section.and_then(|s| s.get("region").cloned()));
180 206
    let region = match region {
181 207
        Some(r) => r,
182 208
        None if endpoint.is_some() => "us-east-1".into(),
…
27 unchanged lines
modified crates/oxy-strata/src/azure.rs +22 −1
…
17 unchanged lines
18 18
}
19 19
20 20
/// The store and the prefix under the container.
21 -
pub fn open(url: &StoreUrl) -> Result<(MicrosoftAzure, String)> {
21 +
/// With `principal` (tenant, client id, client secret) given, only it
22 +
/// is used.
23 +
pub fn open(
24 +
    url: &StoreUrl,
25 +
    principal: Option<(&str, &str, &str)>,
26 +
) -> Result<(MicrosoftAzure, String)> {
22 27
    let (container, prefix) = match url.path.split_once('/') {
23 28
        Some((c, p)) => (c.to_string(), p.to_string()),
24 29
        None if !url.path.is_empty() => (url.path.clone(), String::new()),
…
4 unchanged lines
29 34
            )))
30 35
        }
31 36
    };
37 +
    if let Some((tenant, id, secret)) = principal {
38 +
        let mut b = MicrosoftAzureBuilder::new()
39 +
            .with_account(&url.host)
40 +
            .with_container_name(&container)
41 +
            .with_http_connector(crate::http::UreqConnector)
42 +
            .with_client_secret_authorization(id, secret, tenant);
43 +
        if let Some(endpoint) = url.param("endpoint") {
44 +
            b = b
45 +
                .with_endpoint(endpoint.to_string())
46 +
                .with_allow_http(endpoint.starts_with("http://"));
47 +
        }
48 +
        let store = b
49 +
            .build()
50 +
            .map_err(|e| Error::Backend(format!("{}: {e}", url.plain())))?;
51 +
        return Ok((store, prefix));
52 +
    }
32 53
    let mut b = MicrosoftAzureBuilder::from_env()
33 54
        .with_account(&url.host)
34 55
        .with_container_name(&container)
…
32 unchanged lines
modified crates/oxy-strata/src/gcp.rs +14 −1
…
26 unchanged lines
27 27
        .unwrap_or(false)
28 28
}
29 29
30 -
pub fn open(url: &StoreUrl) -> Result<GoogleCloudStorage> {
30 +
/// With `key` (a service account key's JSON) given, only it is used.
31 +
pub fn open(url: &StoreUrl, key: Option<&str>) -> Result<GoogleCloudStorage> {
32 +
    if let Some(key) = key {
33 +
        let mut b = GoogleCloudStorageBuilder::new()
34 +
            .with_bucket_name(&url.host)
35 +
            .with_http_connector(crate::http::UreqConnector)
36 +
            .with_service_account_key(key);
37 +
        if let Some(endpoint) = url.param("endpoint") {
38 +
            b = b.with_base_url(endpoint);
39 +
        }
40 +
        return b
41 +
            .build()
42 +
            .map_err(|e| Error::Backend(format!("{}: {e}", url.plain())));
43 +
    }
31 44
    let mut b = GoogleCloudStorageBuilder::from_env()
32 45
        .with_bucket_name(&url.host)
33 46
        .with_http_connector(crate::http::UreqConnector);
…
24 unchanged lines
modified crates/oxy-strata/src/lib.rs +704 −28
…
45 unchanged lines
46 46
    if !matches!(scheme, "s3" | "gs" | "az") {
47 47
        return None;
48 48
    }
49 -
    Some(CloudStore::open(url).map(|s| Box::new(s) as Box<dyn Store>))
49 +
    let lookup = *CREDENTIALS.lock().unwrap_or_else(|p| p.into_inner());
50 +
    let given = lookup.and_then(|f| f(url));
51 +
    Some(shared(url, given).map(|s| Box::new(Shared(s)) as Box<dyn Store>))
52 +
}
53 +
54 +
/// One client per store and credentials, kept for the life of the process
55 +
/// (oxygenhub#234): a server opens a repository for every request, and a
56 +
/// new client each time meant a new runtime, new connections and signing
57 +
/// in to the provider again before the first byte.
58 +
type OpenStores = std::collections::HashMap<(String, Option<Credentials>), Arc<CloudStore>>;
59 +
60 +
fn shared(url: &str, given: Option<Credentials>) -> Result<Arc<CloudStore>> {
61 +
    static OPEN: Mutex<Option<OpenStores>> = Mutex::new(None);
62 +
    let key = (url.to_string(), given);
63 +
    let mut open = OPEN.lock().unwrap_or_else(|p| p.into_inner());
64 +
    let map = open.get_or_insert_with(Default::default);
65 +
    if let Some(s) = map.get(&key) {
66 +
        return Ok(s.clone());
67 +
    }
68 +
    let store = Arc::new(CloudStore::open_with(url, key.1.as_ref())?);
69 +
    map.insert(key, store.clone());
70 +
    Ok(store)
71 +
}
72 +
73 +
/// A shared [`CloudStore`] as a [`Store`].
74 +
struct Shared(Arc<CloudStore>);
75 +
76 +
impl Store for Shared {
77 +
    fn describe(&self) -> String {
78 +
        self.0.describe()
79 +
    }
80 +
    fn has(&self, key: &str) -> Result<bool> {
81 +
        self.0.has(key)
82 +
    }
83 +
    fn get(&self, key: &str) -> Result<Vec<u8>> {
84 +
        self.0.get(key)
85 +
    }
86 +
    fn get_range(&self, key: &str, range: Range<u64>) -> Result<Vec<u8>> {
87 +
        self.0.get_range(key, range)
88 +
    }
89 +
    fn put_new(&self, key: &str, bytes: &[u8]) -> Result<()> {
90 +
        self.0.put_new(key, bytes)
91 +
    }
92 +
    fn list(&self, prefix: &str) -> Result<Vec<Entry>> {
93 +
        self.0.list(prefix)
94 +
    }
95 +
    fn stat(&self, key: &str) -> Result<Option<Entry>> {
96 +
        self.0.stat(key)
97 +
    }
98 +
    fn delete(&self, key: &str) -> Result<()> {
99 +
        self.0.delete(key)
100 +
    }
101 +
    fn read_ref(&self, key: &str) -> Result<Option<(Vec<u8>, Version)>> {
102 +
        self.0.read_ref(key)
103 +
    }
104 +
    fn write_ref(&self, key: &str, bytes: &[u8], expected: Option<&Version>) -> Result<Version> {
105 +
        self.0.write_ref(key, bytes, expected)
106 +
    }
107 +
}
108 +
109 +
static OPENS: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
110 +
static CALLS: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
111 +
static MICROS: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
112 +
static BY_OP: [std::sync::atomic::AtomicU64; Op::ALL.len()] =
113 +
    [const { std::sync::atomic::AtomicU64::new(0) }; Op::ALL.len()];
114 +
115 +
/// The kinds of request a cloud store makes, counted apart so a server can
116 +
/// tell which of them its time goes to.
117 +
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
118 +
pub enum Op {
119 +
    /// `has` that found the object.
120 +
    Has,
121 +
    /// `has` that did not: an absent object is never cached.
122 +
    HasMissing,
123 +
    Get,
124 +
    Range,
125 +
    Put,
126 +
    List,
127 +
    /// A listing of a repository's packs, sent.
128 +
    ListPacks,
129 +
    /// A listing answered from [`Listings`], not sent.
130 +
    ListKept,
131 +
    Delete,
132 +
    ReadRef,
133 +
    /// A read of a repository's journal of signatures and index entries
134 +
    /// (`pack-entries`, spec §13.7): one per open of a folded store, in
135 +
    /// place of a listing per lookup.
136 +
    ReadJournal,
137 +
    WriteRef,
138 +
}
139 +
140 +
impl Op {
141 +
    pub const ALL: [Op; 12] = [
142 +
        Op::Has,
143 +
        Op::HasMissing,
144 +
        Op::Get,
145 +
        Op::Range,
146 +
        Op::Put,
147 +
        Op::List,
148 +
        Op::ListPacks,
149 +
        Op::ListKept,
150 +
        Op::Delete,
151 +
        Op::ReadRef,
152 +
        Op::ReadJournal,
153 +
        Op::WriteRef,
154 +
    ];
155 +
156 +
    pub fn name(self) -> &'static str {
157 +
        match self {
158 +
            Op::Has => "has",
159 +
            Op::HasMissing => "has_missing",
160 +
            Op::Get => "get",
161 +
            Op::Range => "range",
162 +
            Op::Put => "put",
163 +
            Op::List => "list",
164 +
            Op::ListPacks => "list_packs",
165 +
            Op::ListKept => "list_kept",
166 +
            Op::Delete => "delete",
167 +
            Op::ReadRef => "read_ref",
168 +
            Op::ReadJournal => "read_journal",
169 +
            Op::WriteRef => "write_ref",
170 +
        }
171 +
    }
172 +
}
173 +
174 +
fn counted(op: Op) {
175 +
    BY_OP[op as usize].fetch_add(1, std::sync::atomic::Ordering::Relaxed);
176 +
}
177 +
178 +
/// What the cloud stores have done since the process started: clients
179 +
/// opened, requests made and the time spent in them, for a server to show
180 +
/// (a hub's /health).
181 +
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
182 +
pub struct Stats {
183 +
    pub opens: u64,
184 +
    pub calls: u64,
185 +
    pub millis: u64,
186 +
    /// Requests by kind, in [`Op::ALL`]'s order.
187 +
    pub by_op: [u64; Op::ALL.len()],
188 +
}
189 +
190 +
impl Stats {
191 +
    /// Each kind of request with how many were made.
192 +
    pub fn ops(&self) -> impl Iterator<Item = (&'static str, u64)> + '_ {
193 +
        Op::ALL
194 +
            .iter()
195 +
            .map(|op| (op.name(), self.by_op[*op as usize]))
196 +
    }
197 +
}
198 +
199 +
pub fn stats() -> Stats {
200 +
    use std::sync::atomic::Ordering::Relaxed;
201 +
    Stats {
202 +
        opens: OPENS.load(Relaxed),
203 +
        calls: CALLS.load(Relaxed),
204 +
        millis: MICROS.load(Relaxed) / 1000,
205 +
        by_op: std::array::from_fn(|i| BY_OP[i].load(Relaxed)),
206 +
    }
207 +
}
208 +
209 +
/// Credentials a program keeps itself and hands over, instead of the
210 +
/// provider's usual places: a hub keeps each repository's sealed. When a
211 +
/// store is opened with them, nothing from the environment or the
212 +
/// provider's files is used.
213 +
#[derive(Clone, PartialEq, Eq, Hash)]
214 +
pub enum Credentials {
215 +
    /// An S3 access key.
216 +
    Aws {
217 +
        access_key_id: String,
218 +
        secret_access_key: String,
219 +
        session_token: Option<String>,
220 +
    },
221 +
    /// A Google Cloud service account key, as the JSON Google issues.
222 +
    Gcp { service_account_key: String },
223 +
    /// An Azure service principal.
224 +
    Azure {
225 +
        tenant_id: String,
226 +
        client_id: String,
227 +
        client_secret: String,
228 +
    },
229 +
}
230 +
231 +
impl std::fmt::Debug for Credentials {
232 +
    /// Names the kind and the public part only: never the secret.
233 +
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
234 +
        match self {
235 +
            Credentials::Aws { access_key_id, .. } => {
236 +
                write!(f, "Aws({access_key_id}, secret hidden)")
237 +
            }
238 +
            Credentials::Gcp { .. } => write!(f, "Gcp(service account key hidden)"),
239 +
            Credentials::Azure { client_id, .. } => write!(f, "Azure({client_id}, secret hidden)"),
240 +
        }
241 +
    }
242 +
}
243 +
244 +
impl Credentials {
245 +
    /// The scheme of the stores these are for.
246 +
    pub fn scheme(&self) -> &'static str {
247 +
        match self {
248 +
            Credentials::Aws { .. } => "s3",
249 +
            Credentials::Gcp { .. } => "gs",
250 +
            Credentials::Azure { .. } => "az",
251 +
        }
252 +
    }
253 +
254 +
    /// As JSON, for keeping (sealed) and reading back with [`Credentials::from_json`].
255 +
    pub fn to_json(&self) -> String {
256 +
        let v = match self {
257 +
            Credentials::Aws {
258 +
                access_key_id,
259 +
                secret_access_key,
260 +
                session_token,
261 +
            } => serde_json::json!({
262 +
                "kind": "aws", "access_key_id": access_key_id, "secret_access_key": secret_access_key,
263 +
                "session_token": session_token,
264 +
            }),
265 +
            Credentials::Gcp {
266 +
                service_account_key,
267 +
            } => {
268 +
                serde_json::json!({ "kind": "gcp", "service_account_key": service_account_key })
269 +
            }
270 +
            Credentials::Azure {
271 +
                tenant_id,
272 +
                client_id,
273 +
                client_secret,
274 +
            } => serde_json::json!({
275 +
                "kind": "azure", "tenant_id": tenant_id, "client_id": client_id, "client_secret": client_secret,
276 +
            }),
277 +
        };
278 +
        v.to_string()
279 +
    }
280 +
281 +
    pub fn from_json(text: &str) -> Result<Credentials> {
282 +
        let v: serde_json::Value = serde_json::from_str(text)
283 +
            .map_err(|e| Error::Backend(format!("credentials are not JSON: {e}")))?;
284 +
        let field = |k: &str| -> Result<String> {
285 +
            v.get(k)
286 +
                .and_then(|x| x.as_str())
287 +
                .filter(|x| !x.trim().is_empty())
288 +
                .map(|x| x.trim().to_string())
289 +
                .ok_or_else(|| Error::Backend(format!("credentials need {k}")))
290 +
        };
291 +
        match v.get("kind").and_then(|k| k.as_str()) {
292 +
            Some("aws") => Ok(Credentials::Aws {
293 +
                access_key_id: field("access_key_id")?,
294 +
                secret_access_key: field("secret_access_key")?,
295 +
                session_token: field("session_token").ok(),
296 +
            }),
297 +
            Some("gcp") => {
298 +
                let key = field("service_account_key")?;
299 +
                let parsed: serde_json::Value = serde_json::from_str(&key).map_err(|_| {
300 +
                    Error::Backend("the service account key is not the JSON Google issues".into())
301 +
                })?;
302 +
                if parsed.get("type").and_then(|t| t.as_str()) != Some("service_account") {
303 +
                    return Err(Error::Backend(
304 +
                        "the key is not a service account key (its type is not service_account)"
305 +
                            .into(),
306 +
                    ));
307 +
                }
308 +
                Ok(Credentials::Gcp {
309 +
                    service_account_key: key,
310 +
                })
311 +
            }
312 +
            Some("azure") => Ok(Credentials::Azure {
313 +
                tenant_id: field("tenant_id")?,
314 +
                client_id: field("client_id")?,
315 +
                client_secret: field("client_secret")?,
316 +
            }),
317 +
            _ => Err(Error::Backend(
318 +
                "credentials need a kind: aws, gcp or azure".into(),
319 +
            )),
320 +
        }
321 +
    }
50 322
}
51 323
324 +
/// Finds the credentials a program keeps for a store URL.
325 +
pub type CredentialsLookup = fn(&str) -> Option<Credentials>;
326 +
327 +
static CREDENTIALS: Mutex<Option<CredentialsLookup>> = Mutex::new(None);
328 +
329 +
/// Have stores opened by URL (as a repository's `storage.url` is) use the
330 +
/// credentials `lookup` finds for it, where it finds any.
331 +
pub fn set_credentials_lookup(lookup: CredentialsLookup) {
332 +
    *CREDENTIALS.lock().unwrap_or_else(|p| p.into_inner()) = Some(lookup);
333 +
}
334 +
52 335
/// A parsed store URL: `SCHEME://HOST/PATH?PARAMS`.
53 336
#[derive(Clone, Debug, PartialEq, Eq)]
54 337
pub struct StoreUrl {
…
73 unchanged lines
128 411
    prefix: String,
129 412
    name: String,
130 413
    hint: &'static str,
131 -
    runtime: tokio::runtime::Runtime,
132 -
    /// One call at a time: the runtime runs on the calling thread.
133 -
    turn: Mutex<()>,
414 +
    /// Always there until the store is dropped: two workers, so the
415 +
    /// requests of a shared store run side by side.
416 +
    runtime: Option<tokio::runtime::Runtime>,
417 +
    /// Keys `has` found missing lately (see [`Misses`]).
418 +
    misses: Misses,
419 +
    /// Pack listings, kept a moment (see [`Listings`]).
420 +
    listings: Listings,
421 +
}
422 +
423 +
/// How long a key found missing is taken as missing without asking again.
424 +
const MISS_FOR: std::time::Duration = std::time::Duration::from_secs(30);
425 +
426 +
/// Keys found missing in the last [`MISS_FOR`], so a server answering the
427 +
/// same question again and again does not send a request each time
428 +
/// (oxygenhub#228: a client's negotiate asked about ~190 objects the hub
429 +
/// lacked every 8 s, 45 ms a request). This process forgets a key the
430 +
/// moment it writes it, so its own writes are never hidden; another writer
431 +
/// to the same keys is seen within `MISS_FOR`.
432 +
#[derive(Default)]
433 +
struct Misses(Mutex<std::collections::HashMap<String, std::time::Instant>>);
434 +
435 +
impl Misses {
436 +
    /// At most this many kept; past it the oldest half is let go.
437 +
    const MAX: usize = 100_000;
438 +
439 +
    fn known(&self, key: &str) -> bool {
440 +
        let map = self.0.lock().unwrap_or_else(|p| p.into_inner());
441 +
        map.get(key).is_some_and(|t| t.elapsed() < MISS_FOR)
442 +
    }
443 +
444 +
    fn missed(&self, key: &str) {
445 +
        let mut map = self.0.lock().unwrap_or_else(|p| p.into_inner());
446 +
        if map.len() >= Self::MAX {
447 +
            map.retain(|_, t| t.elapsed() < MISS_FOR / 2);
448 +
            if map.len() >= Self::MAX {
449 +
                map.clear();
450 +
            }
451 +
        }
452 +
        map.insert(key.to_string(), std::time::Instant::now());
453 +
    }
454 +
455 +
    fn written(&self, key: &str) {
456 +
        self.0.lock().unwrap_or_else(|p| p.into_inner()).remove(key);
457 +
    }
458 +
}
459 +
460 +
/// Listings kept for [`MISS_FOR`] (oxygen#116): the `pack-` keys, which
461 +
/// every open of a repository's store lists, so a forge serving a
462 +
/// repository from a bucket lists them once a while rather than once a
463 +
/// request. Like [`Misses`], this process forgets a listing the moment it
464 +
/// writes or deletes a key under it; another writer's pack is seen within
465 +
/// `MISS_FOR`, as a missing key is.
466 +
#[derive(Default)]
467 +
struct Listings(Mutex<std::collections::HashMap<String, (std::time::Instant, Vec<Entry>)>>);
468 +
469 +
impl Listings {
470 +
    /// The prefixes kept: those every open lists.
471 +
    fn kept(prefix: &str) -> bool {
472 +
        prefix == "pack-"
473 +
    }
474 +
475 +
    fn get(&self, prefix: &str) -> Option<Vec<Entry>> {
476 +
        if !Self::kept(prefix) {
477 +
            return None;
478 +
        }
479 +
        let map = self.0.lock().unwrap_or_else(|p| p.into_inner());
480 +
        map.get(prefix)
481 +
            .filter(|(t, _)| t.elapsed() < MISS_FOR)
482 +
            .map(|(_, e)| e.clone())
483 +
    }
484 +
485 +
    fn put(&self, prefix: &str, entries: &[Entry]) {
486 +
        if Self::kept(prefix) {
487 +
            self.0.lock().unwrap_or_else(|p| p.into_inner()).insert(
488 +
                prefix.to_string(),
489 +
                (std::time::Instant::now(), entries.to_vec()),
490 +
            );
491 +
        }
492 +
    }
493 +
494 +
    fn written(&self, key: &str) {
495 +
        self.0
496 +
            .lock()
497 +
            .unwrap_or_else(|p| p.into_inner())
498 +
            .retain(|prefix, _| !key.starts_with(prefix.as_str()));
499 +
    }
500 +
}
501 +
502 +
/// Run a request that writes or deletes `key`, forgetting what the caches
503 +
/// hold for it both before it is sent and after it completes, whatever it
504 +
/// returns: a listing or a miss taken by another thread while the request
505 +
/// was in flight may predate it, and would otherwise be kept for
506 +
/// `MISS_FOR` (oxygenhub#249).
507 +
fn writing<T>(misses: &Misses, listings: &Listings, key: &str, request: impl FnOnce() -> T) -> T {
508 +
    misses.written(key);
509 +
    listings.written(key);
510 +
    let out = request();
511 +
    misses.written(key);
512 +
    listings.written(key);
513 +
    out
514 +
}
515 +
516 +
impl Drop for CloudStore {
517 +
    /// Without waiting, so a store can be dropped inside a runtime too.
518 +
    fn drop(&mut self) {
519 +
        if let Some(rt) = self.runtime.take() {
520 +
            rt.shutdown_background();
521 +
        }
522 +
    }
134 523
}
135 524
136 525
impl CloudStore {
137 526
    /// Open the store a URL names. Nothing is sent yet: credentials are
138 527
    /// resolved here, but the bucket is first spoken to on the first call.
139 528
    pub fn open(url: &str) -> Result<CloudStore> {
529 +
        Self::open_with(url, None)
530 +
    }
531 +
532 +
    /// [`CloudStore::open`] with credentials given, which must be for
533 +
    /// the URL's kind of store; then only they are used.
534 +
    pub fn open_with(url: &str, given: Option<&Credentials>) -> Result<CloudStore> {
140 535
        let parsed = StoreUrl::parse(url)?;
141 -
        let runtime = tokio::runtime::Builder::new_current_thread()
536 +
        if let Some(c) = given.filter(|c| c.scheme() != parsed.scheme) {
537 +
            return Err(Error::Backend(format!(
538 +
                "{}: the credentials given are for {}:// stores",
539 +
                parsed.plain(),
540 +
                c.scheme()
541 +
            )));
542 +
        }
543 +
        let aws_keys = match given {
544 +
            Some(Credentials::Aws {
545 +
                access_key_id,
546 +
                secret_access_key,
547 +
                session_token,
548 +
            }) => Some((
549 +
                access_key_id.as_str(),
550 +
                secret_access_key.as_str(),
551 +
                session_token.as_deref(),
552 +
            )),
553 +
            _ => None,
554 +
        };
555 +
        let gcp_key = match given {
556 +
            Some(Credentials::Gcp {
557 +
                service_account_key,
558 +
            }) => Some(service_account_key.as_str()),
559 +
            _ => None,
560 +
        };
561 +
        let principal = match given {
562 +
            Some(Credentials::Azure {
563 +
                tenant_id,
564 +
                client_id,
565 +
                client_secret,
566 +
            }) => Some((
567 +
                tenant_id.as_str(),
568 +
                client_id.as_str(),
569 +
                client_secret.as_str(),
570 +
            )),
571 +
            _ => None,
572 +
        };
573 +
        OPENS.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
574 +
        let runtime = tokio::runtime::Builder::new_multi_thread()
575 +
            .worker_threads(2)
576 +
            .thread_name("oxy-strata")
142 577
            .enable_all()
143 578
            .build()
144 579
            .map_err(|e| Error::Backend(format!("starting the HTTP runtime: {e}")))?;
145 580
        let (inner, prefix, hint): (Arc<dyn ObjectStore>, String, &'static str) =
146 581
            match parsed.scheme.as_str() {
147 582
                "s3" => (
148 -
                    Arc::new(aws::open(&parsed)?),
583 +
                    Arc::new(aws::open(&parsed, aws_keys)?),
149 584
                    parsed.path.clone(),
150 585
                    aws::HINT,
151 586
                ),
152 587
                "gs" => (
153 -
                    Arc::new(gcp::open(&parsed)?),
588 +
                    Arc::new(gcp::open(&parsed, gcp_key)?),
154 589
                    parsed.path.clone(),
155 590
                    gcp::HINT,
156 591
                ),
157 592
                "az" => {
158 -
                    let (store, prefix) = azure::open(&parsed)?;
593 +
                    let (store, prefix) = azure::open(&parsed, principal)?;
159 594
                    (Arc::new(store), prefix, azure::HINT)
160 595
                }
161 596
                other => {
…
7 unchanged lines
169 604
            prefix,
170 605
            name: parsed.plain(),
171 606
            hint,
172 -
            runtime,
173 -
            turn: Mutex::new(()),
607 +
            runtime: Some(runtime),
608 +
            misses: Misses::default(),
609 +
            listings: Listings::default(),
174 610
        })
175 611
    }
176 612
177 -
    fn block<F: Future>(&self, f: F) -> F::Output {
178 -
        let _turn = self.turn.lock().unwrap_or_else(|p| p.into_inner());
179 -
        self.runtime.block_on(f)
613 +
    /// Run a request to completion. Called from a thread that is itself
614 +
    /// driving async tasks (a server's handler), where blocking on another
615 +
    /// runtime is not allowed, the request runs on a thread of its own.
616 +
    fn block<F>(&self, f: F) -> F::Output
617 +
    where
618 +
        F: Future + Send,
619 +
        F::Output: Send,
620 +
    {
621 +
        let runtime = self
622 +
            .runtime
623 +
            .as_ref()
624 +
            .expect("the runtime lives as long as the store");
625 +
        let started = std::time::Instant::now();
626 +
        let out = if tokio::runtime::Handle::try_current().is_ok() {
627 +
            std::thread::scope(|s| {
628 +
                s.spawn(|| runtime.block_on(f))
629 +
                    .join()
630 +
                    .unwrap_or_else(|p| std::panic::resume_unwind(p))
631 +
            })
632 +
        } else {
633 +
            runtime.block_on(f)
634 +
        };
635 +
        CALLS.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
636 +
        MICROS.fetch_add(
637 +
            started.elapsed().as_micros() as u64,
638 +
            std::sync::atomic::Ordering::Relaxed,
639 +
        );
640 +
        out
180 641
    }
181 642
182 643
    fn path(&self, key: &str) -> Result<ObjectPath> {
…
73 unchanged lines
256 717
257 718
    fn has(&self, key: &str) -> Result<bool> {
258 719
        let path = self.path(key)?;
720 +
        if self.misses.known(key) {
721 +
            return Ok(false);
722 +
        }
259 723
        match self.block(self.inner.head(&path)) {
260 -
            Ok(_) => Ok(true),
261 -
            Err(object_store::Error::NotFound { .. }) => Ok(false),
724 +
            Ok(_) => {
725 +
                counted(Op::Has);
726 +
                Ok(true)
727 +
            }
728 +
            Err(object_store::Error::NotFound { .. }) => {
729 +
                counted(Op::HasMissing);
730 +
                self.misses.missed(key);
731 +
                Ok(false)
732 +
            }
262 733
            Err(e) => Err(self.map(e, key)),
263 734
        }
264 735
    }
265 736
266 737
    fn get(&self, key: &str) -> Result<Vec<u8>> {
267 738
        let path = self.path(key)?;
739 +
        counted(Op::Get);
268 740
        self.block(async {
269 741
            let r = self.inner.get(&path).await?;
270 742
            r.bytes().await
…
14 unchanged lines
285 757
            return Ok(Vec::new());
286 758
        }
287 759
        let want = (range.end - range.start) as usize;
760 +
        counted(Op::Range);
288 761
        let bytes = self
289 762
            .block(self.inner.get_range(&path, range.clone()))
290 763
            .map_err(|e| self.map(e, key))?;
…
14 unchanged lines
305 778
            mode: PutMode::Create,
306 779
            ..Default::default()
307 780
        };
308 -
        self.block(
309 -
            self.inner
310 -
                .put_opts(&path, PutPayload::from(bytes.to_vec()), opts),
311 -
        )
781 +
        counted(Op::Put);
782 +
        writing(&self.misses, &self.listings, key, || {
783 +
            self.block(
784 +
                self.inner
785 +
                    .put_opts(&path, PutPayload::from(bytes.to_vec()), opts),
786 +
            )
787 +
        })
312 788
        .map(|_| ())
313 789
        .map_err(|e| self.map(e, key))
314 790
    }
315 791
316 792
    fn list(&self, prefix: &str) -> Result<Vec<Entry>> {
793 +
        if let Some(kept) = self.listings.get(prefix) {
794 +
            counted(Op::ListKept);
795 +
            return Ok(kept);
796 +
        }
317 797
        // Providers list by whole path segments; the asked prefix may end
318 798
        // mid-segment, so list its directory and filter here.
319 799
        let dir = prefix.rsplit_once('/').map(|(d, _)| d).unwrap_or("");
…
6 unchanged lines
326 806
        let full_dir = full_dir
327 807
            .map(|d| ObjectPath::parse(&d).map_err(|e| Error::Backend(format!("{d}: {e}"))))
328 808
            .transpose()?;
809 +
        counted(if Listings::kept(prefix) {
810 +
            Op::ListPacks
811 +
        } else {
812 +
            Op::List
813 +
        });
329 814
        let metas: Vec<object_store::ObjectMeta> = self
330 815
            .block(self.inner.list(full_dir.as_ref()).try_collect())
331 816
            .map_err(|e| self.map(e, prefix))?;
…
15 unchanged lines
347 832
            })
348 833
            .collect();
349 834
        out.sort_by(|a, b| a.key.cmp(&b.key));
835 +
        self.listings.put(prefix, &out);
350 836
        Ok(out)
351 837
    }
352 838
839 +
    /// A HEAD of the key, not a listing of its directory (oxygen#116).
840 +
    fn stat(&self, key: &str) -> Result<Option<Entry>> {
841 +
        let path = self.path(key)?;
842 +
        counted(Op::Has);
843 +
        match self.block(self.inner.head(&path)) {
844 +
            Ok(m) => Ok(Some(Entry {
845 +
                key: key.to_string(),
846 +
                size: m.size,
847 +
                modified_ms: u64::try_from(m.last_modified.timestamp_millis()).ok(),
848 +
            })),
849 +
            Err(object_store::Error::NotFound { .. }) => Ok(None),
850 +
            Err(e) => Err(self.map(e, key)),
851 +
        }
852 +
    }
853 +
353 854
    fn delete(&self, key: &str) -> Result<()> {
354 855
        let path = self.path(key)?;
355 -
        match self.block(self.inner.delete(&path)) {
856 +
        counted(Op::Delete);
857 +
        match writing(&self.misses, &self.listings, key, || {
858 +
            self.block(self.inner.delete(&path))
859 +
        }) {
356 860
            Ok(()) | Err(object_store::Error::NotFound { .. }) => Ok(()),
357 861
            Err(e) => Err(self.map(e, key)),
358 862
        }
…
1 unchanged lines
360 864
361 865
    fn read_ref(&self, key: &str) -> Result<Option<(Vec<u8>, Version)>> {
362 866
        let path = self.path(key)?;
867 +
        counted(if key == oxy_core::store::JOURNAL_KEY {
868 +
            Op::ReadJournal
869 +
        } else {
870 +
            Op::ReadRef
871 +
        });
363 872
        let got = self.block(async {
364 873
            let r = self.inner.get_opts(&path, GetOptions::default()).await?;
365 874
            let meta = r.meta.clone();
…
20 unchanged lines
386 895
            mode,
387 896
            ..Default::default()
388 897
        };
389 -
        let put = self
390 -
            .block(
898 +
        counted(Op::WriteRef);
899 +
        let put = writing(&self.misses, &self.listings, key, || {
900 +
            self.block(
391 901
                self.inner
392 902
                    .put_opts(&path, PutPayload::from(bytes.to_vec()), opts),
393 903
            )
394 -
            .map_err(|e| match self.map(e, key) {
395 -
                // A record that exists when none was expected is a lost
396 -
                // race too, not a misuse of put_new.
397 -
                Error::Exists(k) => Error::Conflict(k),
398 -
                other => other,
399 -
            })?;
904 +
        })
905 +
        .map_err(|e| match self.map(e, key) {
906 +
            // A record that exists when none was expected is a lost
907 +
            // race too, not a misuse of put_new.
908 +
            Error::Exists(k) => Error::Conflict(k),
909 +
            other => other,
910 +
        })?;
400 911
        Ok(Self::version_of(put.e_tag, put.version))
401 912
    }
402 913
}
403 914
404 915
#[cfg(test)]
405 916
mod tests {
917 +
918 +
    /// A miss is remembered until the key is written, or for MISS_FOR.
919 +
    #[test]
920 +
    fn a_miss_is_remembered_until_written() {
921 +
        let m = super::Misses::default();
922 +
        assert!(!m.known("loose/ab/cd"));
923 +
        m.missed("loose/ab/cd");
924 +
        assert!(m.known("loose/ab/cd") && !m.known("loose/ab/ce"));
925 +
        m.written("loose/ab/cd");
926 +
        assert!(!m.known("loose/ab/cd"), "a write forgets the miss");
927 +
        m.missed("old");
928 +
        m.0.lock()
929 +
            .unwrap()
930 +
            .insert("old".into(), std::time::Instant::now() - super::MISS_FOR);
931 +
        assert!(!m.known("old"), "a miss expires");
932 +
    }
933 +
934 +
    /// Every kind of request is reported, once and by its own name.
935 +
    /// The pack listing is kept until a pack key is written or deleted
936 +
    /// here, or for MISS_FOR; other prefixes are never kept (oxygen#116).
937 +
    #[test]
938 +
    fn the_pack_listing_is_kept_until_a_pack_is_written() {
939 +
        let l = super::Listings::default();
940 +
        let e = |k: &str| oxy_core::strata::Entry {
941 +
            key: k.into(),
942 +
            size: 1,
943 +
            modified_ms: None,
944 +
        };
945 +
        l.put("pack-", &[e("pack-1.idx")]);
946 +
        assert_eq!(l.get("pack-").map(|v| v.len()), Some(1));
947 +
        l.put("loose/", &[e("loose/ab")]);
948 +
        assert!(l.get("loose/").is_none(), "only the pack listing is kept");
949 +
        l.written("loose/cd");
950 +
        assert!(l.get("pack-").is_some(), "a loose object leaves it");
951 +
        l.written("pack-2.pack");
952 +
        assert!(l.get("pack-").is_none(), "a new pack forgets it");
953 +
        l.put("pack-", &[e("pack-1.idx")]);
954 +
        l.0.lock().unwrap().get_mut("pack-").unwrap().0 -= super::MISS_FOR;
955 +
        assert!(l.get("pack-").is_none(), "and it ages out");
956 +
    }
957 +
958 +
    /// A listing or miss taken by another thread while a write is in
959 +
    /// flight is forgotten when the write completes, not kept stale for
960 +
    /// MISS_FOR (oxygenhub#249), whether the write succeeded or not.
961 +
    #[test]
962 +
    fn a_listing_taken_during_a_write_is_forgotten_after_it() {
963 +
        let (m, l) = (super::Misses::default(), super::Listings::default());
964 +
        let e = |k: &str| oxy_core::strata::Entry {
965 +
            key: k.into(),
966 +
            size: 1,
967 +
            modified_ms: None,
968 +
        };
969 +
        l.put("pack-", &[e("pack-1.idx")]);
970 +
        let got = super::writing(&m, &l, "pack-2.idx", || {
971 +
            assert!(l.get("pack-").is_none(), "forgotten before it is sent");
972 +
            // Another thread lists, and misses, mid-request.
973 +
            l.put("pack-", &[e("pack-1.idx")]);
974 +
            m.missed("pack-2.idx");
975 +
            7
976 +
        });
977 +
        assert_eq!(got, 7);
978 +
        assert!(l.get("pack-").is_none(), "the mid-request listing is gone");
979 +
        assert!(!m.known("pack-2.idx"), "and the miss");
980 +
        // A delete that fails is treated the same.
981 +
        l.put("pack-", &[e("pack-1.idx")]);
982 +
        let r: std::result::Result<(), &str> = super::writing(&m, &l, "pack-1.idx", || {
983 +
            l.put("pack-", &[e("pack-1.idx")]);
984 +
            Err("network")
985 +
        });
986 +
        assert!(r.is_err() && l.get("pack-").is_none());
987 +
        // Other prefixes' writes leave the pack listing.
988 +
        l.put("pack-", &[e("pack-1.idx")]);
989 +
        super::writing(&m, &l, "loose/ab/cd", || ());
990 +
        assert!(l.get("pack-").is_some());
991 +
    }
992 +
993 +
    #[test]
994 +
    fn stats_name_every_kind_of_request() {
995 +
        let names: Vec<&str> = super::stats().ops().map(|(n, _)| n).collect();
996 +
        assert_eq!(names.len(), super::Op::ALL.len());
997 +
        let mut unique = names.clone();
998 +
        unique.sort();
999 +
        unique.dedup();
1000 +
        assert_eq!(unique.len(), names.len());
1001 +
        assert!(names.contains(&"has_missing") && names.contains(&"range"));
1002 +
    }
1003 +
1004 +
    /// One client per store and credentials (oxygenhub#234): opening the
1005 +
    /// same again shares it, and only a new one counts as opened.
1006 +
    #[test]
1007 +
    fn a_store_opened_again_shares_its_client() {
1008 +
        let creds = |k: &str| Credentials::Aws {
1009 +
            access_key_id: k.into(),
1010 +
            secret_access_key: "s".into(),
1011 +
            session_token: None,
1012 +
        };
1013 +
        let url = "s3://oxy-shared-test/repos/a?region=eu-west-2";
1014 +
        let before = super::stats().opens;
1015 +
        let a = super::shared(url, Some(creds("one"))).unwrap();
1016 +
        let b = super::shared(url, Some(creds("one"))).unwrap();
1017 +
        assert!(std::sync::Arc::ptr_eq(&a, &b));
1018 +
        let other_keys = super::shared(url, Some(creds("two"))).unwrap();
1019 +
        assert!(
1020 +
            !std::sync::Arc::ptr_eq(&a, &other_keys),
1021 +
            "other credentials, another client"
1022 +
        );
1023 +
        assert!(
1024 +
            super::stats().opens - before >= 2,
1025 +
            "two clients opened (other tests may open more)"
1026 +
        );
1027 +
    }
1028 +
1029 +
    #[tokio::test]
1030 +
    async fn a_store_can_be_used_from_inside_a_runtime() {
1031 +
        // As a server's handler would: blocking on the store's own runtime
1032 +
        // from here must neither panic nor deadlock, and nor must dropping it.
1033 +
        let store = CloudStore::open(
1034 +
            "s3://oxy-no-such-bucket/p?region=eu-west-2&endpoint=http://127.0.0.1:9",
1035 +
        )
1036 +
        .unwrap();
1037 +
        assert_eq!(store.block(async { 7 }), 7);
1038 +
        drop(store);
1039 +
    }
1040 +
1041 +
    #[test]
1042 +
    fn credentials_round_trip_and_never_show_their_secret() {
1043 +
        let aws = Credentials::Aws {
1044 +
            access_key_id: "AKIDEXAMPLE".into(),
1045 +
            secret_access_key: "wJalrXUtnFEMI".into(),
1046 +
            session_token: None,
1047 +
        };
1048 +
        assert_eq!(Credentials::from_json(&aws.to_json()).unwrap(), aws);
1049 +
        assert_eq!(format!("{aws:?}"), "Aws(AKIDEXAMPLE, secret hidden)");
1050 +
        let az = Credentials::Azure {
1051 +
            tenant_id: "t".into(),
1052 +
            client_id: "c".into(),
1053 +
            client_secret: "s3cr3t".into(),
1054 +
        };
1055 +
        assert_eq!(Credentials::from_json(&az.to_json()).unwrap(), az);
1056 +
        assert!(!format!("{az:?}").contains("s3cr3t"));
1057 +
        let key = r#"{"type":"service_account","client_email":"x@y","private_key":"k"}"#;
1058 +
        let gcp = Credentials::Gcp {
1059 +
            service_account_key: key.into(),
1060 +
        };
1061 +
        assert_eq!(Credentials::from_json(&gcp.to_json()).unwrap(), gcp);
1062 +
        assert!(!format!("{gcp:?}").contains("private_key"));
1063 +
        for (bad, why) in [
1064 +
            (r#"{"kind":"aws","access_key_id":"a"}"#, "secret_access_key"),
1065 +
            (
1066 +
                r#"{"kind":"gcp","service_account_key":"{\"type\":\"authorized_user\"}"}"#,
1067 +
                "not a service account key",
1068 +
            ),
1069 +
            (r#"{"kind":"ftp"}"#, "aws, gcp or azure"),
1070 +
            ("nope", "not JSON"),
1071 +
        ] {
1072 +
            let e = Credentials::from_json(bad).unwrap_err().to_string();
1073 +
            assert!(e.contains(why), "{bad}: {e}");
1074 +
        }
1075 +
        let e = CloudStore::open_with("gs://bucket/p", Some(&aws))
1076 +
            .err()
1077 +
            .unwrap()
1078 +
            .to_string();
1079 +
        assert!(e.contains("are for s3:// stores"), "{e}");
1080 +
    }
1081 +
406 1082
    use super::*;
407 1083
408 1084
    #[test]
…
30 unchanged lines
modified crates/oxy-strata/tests/cloud.rs +129 −0
…
141 unchanged lines
142 142
    }
143 143
}
144 144
145 +
/// oxygen#68: a small repository kept on each store (`storage.url`), with
146 +
/// the local cache in front: commits, a reopen with the cache dropped, and
147 +
/// the time each step took. Then every key the test wrote is removed.
148 +
#[test]
149 +
fn a_repository_runs_on_every_configured_store() {
150 +
    for (url, base_store) in each_store() {
151 +
        remove_stale(&base_store);
152 +
        let dir = tempfile::tempdir().unwrap();
153 +
        let root = dir.path();
154 +
        std::fs::create_dir_all(root.join("home")).unwrap();
155 +
        std::env::set_var("OXY_HOME", root.join("home"));
156 +
        let (base, query) = match url.split_once('?') {
157 +
            Some((b, q)) => (b.to_string(), format!("?{q}")),
158 +
            None => (url.clone(), String::new()),
159 +
        };
160 +
        let store_url = format!("{base}/repo-{}{query}", std::process::id());
161 +
        let plain = format!("{base}/repo-{}", std::process::id());
162 +
        let store = CloudStore::open(&store_url).unwrap();
163 +
        let t = std::time::Instant::now();
164 +
        let repo = Repo::init_new(&root.join("w"), false, Some(&store_url))
165 +
            .unwrap_or_else(|e| panic!("{plain}: init: {e}"));
166 +
        let init_ms = t.elapsed().as_millis();
167 +
        let t = std::time::Instant::now();
168 +
        let mut last = None;
169 +
        for i in 0..5 {
170 +
            write(
171 +
                root,
172 +
                "w/f.txt",
173 +
                format!("line {i}\n").repeat(i + 1).as_bytes(),
174 +
            );
175 +
            last = Some(repo.commit(&format!("commit {i}"), &[]).unwrap());
176 +
        }
177 +
        let commit_ms = t.elapsed().as_millis() / 5;
178 +
        drop(repo);
179 +
        std::fs::remove_dir_all(root.join("w/.oxy/cache")).unwrap();
180 +
        let t = std::time::Instant::now();
181 +
        let repo = Repo::open(&root.join("w")).unwrap();
182 +
        let heads = repo.bookmark_heads().unwrap();
183 +
        let state = repo.load_state(&heads).unwrap().materialize();
184 +
        let cold_ms = t.elapsed().as_millis();
185 +
        assert_eq!(heads, vec![last.unwrap()], "{plain}");
186 +
        assert_eq!(state.files.len(), 1);
187 +
        assert!(
188 +
            store.read_ref("ops/head").unwrap().is_some(),
189 +
            "{plain}: the head is in the store"
190 +
        );
191 +
        let keys = store.list("").unwrap();
192 +
        let total: u64 = keys.iter().map(|e| e.size).sum();
193 +
        eprintln!(
194 +
            "{plain}: init {init_ms} ms, commit {commit_ms} ms each, cold open and read {cold_ms} ms; {} keys, {total} bytes",
195 +
            keys.len()
196 +
        );
197 +
        remove_all(&store, "");
198 +
        assert!(store.list("").unwrap().is_empty(), "{plain}: cleanup");
199 +
    }
200 +
}
201 +
202 +
/// Each store opened with credentials handed over (`CloudStore::open_with`,
203 +
/// as a hub does with the ones it keeps sealed), the profile and key file
204 +
/// parameters left off the URL, passes the conformance check.
205 +
#[test]
206 +
fn every_configured_store_conforms_with_credentials_handed_over() {
207 +
    use oxy_strata::Credentials;
208 +
    let env = |k: &str| std::env::var(k).ok().filter(|v| !v.is_empty());
209 +
    let mut tried = 0;
210 +
    if let (Some(url), Some(profile)) = (env("OXY_TEST_S3"), env("OXY_TEST_S3_PROFILE")) {
211 +
        let home = std::path::PathBuf::from(env("HOME").unwrap());
212 +
        let text = std::fs::read_to_string(home.join(".aws/credentials")).unwrap_or_default();
213 +
        let mut section = String::new();
214 +
        let mut keys = std::collections::BTreeMap::new();
215 +
        for line in text.lines().map(str::trim) {
216 +
            if let Some(name) = line.strip_prefix('[').and_then(|l| l.strip_suffix(']')) {
217 +
                section = name.to_string();
218 +
            } else if let Some((k, v)) = line.split_once('=') {
219 +
                if section == profile {
220 +
                    keys.insert(k.trim().to_string(), v.trim().to_string());
221 +
                }
222 +
            }
223 +
        }
224 +
        if let (Some(id), Some(secret)) = (
225 +
            keys.get("aws_access_key_id"),
226 +
            keys.get("aws_secret_access_key"),
227 +
        ) {
228 +
            let c = Credentials::Aws {
229 +
                access_key_id: id.clone(),
230 +
                secret_access_key: secret.clone(),
231 +
                session_token: keys.get("aws_session_token").cloned(),
232 +
            };
233 +
            check_handed_over(&url, &c);
234 +
            tried += 1;
235 +
        }
236 +
    }
237 +
    if let (Some(url), Some(file)) = (env("OXY_TEST_GCS"), env("OXY_TEST_GCS_CREDENTIALS")) {
238 +
        let c = Credentials::Gcp {
239 +
            service_account_key: std::fs::read_to_string(file).unwrap(),
240 +
        };
241 +
        check_handed_over(&url, &c);
242 +
        tried += 1;
243 +
    }
244 +
    if let (Some(url), Some(file)) = (env("OXY_TEST_AZURE"), env("OXY_TEST_AZURE_CREDENTIALS")) {
245 +
        let mut vars = std::collections::BTreeMap::new();
246 +
        for line in std::fs::read_to_string(file).unwrap().lines() {
247 +
            if let Some((k, v)) = line.split_once('=') {
248 +
                vars.insert(k.trim().to_string(), v.trim().to_string());
249 +
            }
250 +
        }
251 +
        let c = Credentials::Azure {
252 +
            tenant_id: vars["AZURE_TENANT_ID"].clone(),
253 +
            client_id: vars["AZURE_CLIENT_ID"].clone(),
254 +
            client_secret: vars["AZURE_CLIENT_SECRET"].clone(),
255 +
        };
256 +
        check_handed_over(&url, &c);
257 +
        tried += 1;
258 +
    }
259 +
    eprintln!("{tried} stores checked with credentials handed over");
260 +
}
261 +
262 +
fn check_handed_over(url: &str, c: &oxy_strata::Credentials) {
263 +
    let store =
264 +
        oxy_strata::CloudStore::open_with(url, Some(c)).unwrap_or_else(|e| panic!("{url}: {e}"));
265 +
    let prefix = format!("probe-{}", std::process::id());
266 +
    let passed = conformance(&store, &prefix).unwrap_or_else(|e| {
267 +
        remove_all(&store, &prefix);
268 +
        panic!("{url} with {c:?}: {e}")
269 +
    });
270 +
    eprintln!("{url} with {c:?}: {} checks passed", passed.len());
271 +
    assert_eq!(passed.len(), 8);
272 +
}
273 +
145 274
fn write(root: &Path, rel: &str, content: &[u8]) {
146 275
    let p = root.join(rel);
147 276
    std::fs::create_dir_all(p.parent().unwrap()).unwrap();
…
2 unchanged lines
added docs/cylinder.md +456 −0
1 +
# Cylinder: artifacts, the Oxygen way
2 +
3 +
Status: Draft 0.3 (proposal revision 3), 2026-10-12. Draft 0.2 added the
4 +
object model, deployment leases, inverted vulnerability indexing,
5 +
multi-party consensus, agent scopes and revset queries; 0.3 corrects the
6 +
MicroVM mount path, the reach of the file index, revocation, encryption,
7 +
leases and upstream quarantine, and keeps §7 open. C0 is built; the rest
8 +
is not. "Cylinder" is a working name (the artifact-repository idea parked
9 +
on 2026-10-02); nothing here is decided until Andy says so.
10 +
11 +
**Cylinder** keeps and moves everything a build produces: runner images,
12 +
container images, packages, binaries, SBOMs and reports. It would be a
13 +
service of its own, embedded in OxygenHub by default and shown through
14 +
the hub's UI, the same pattern as Jetstream. It is meant to be better
15 +
than what exists, not compatible with it; compatibility is an edge we add
16 +
for adoption, never the core.
17 +
18 +
## 1. What exists today
19 +
20 +
| | Strengths | Weaknesses |
21 +
|---|---|---|
22 +
| **Docker Hub** | The default namespace; huge public catalogue; official images; scanning (Scout). | Pull limits since April 2025: 10 an hour anonymous, 100 an hour on a free account. SaaS only. Tags are mutable. Signatures and provenance are separate artifacts; referrers are still in beta. Nothing links an image to the source it came from. |
23 +
| **GitHub** (GHCR, Packages) | Next to the code and Actions; free container storage for now; the best Sigstore integration; artifact attestations. | Storage billing for private images deferred, not settled. One silo per package format. Auth by personal access tokens. Tags are mutable: the tj-actions attack (March 2025, CVE-2025-30066) moved every tag of a popular action to a malicious commit at once. No self-hosting outside Enterprise Server. |
24 +
| **GitLab** | A registry per project; many package formats; a dependency proxy; self-hostable, so it can be sovereign. | Cleanup policies delete tags only; layers stay until an administrator runs garbage collection. The registry is a separate service to run and scale. Signing means bringing cosign yourself. |
25 +
| **Bitbucket** (Packages) | GA since December 2025; permissions inherited from the repository; garbage collection runs by itself; Maven and npm since April 2026. | Young; Cloud only; few formats; no provenance story. |
26 +
| **Artifactory, Nexus** | Every format; proxies and caches; enterprise features. | Costly; heavy JVM operations; slow UIs. A second system beside the forge, with its own identities and permissions. Provenance and scanning are add-on products. |
27 +
| **Harbor, Quay** | Self-hosted OCI with signing (cosign, Notation), replication and scanning. | Many components to run. A central bottleneck: peer-to-peer means adding Dragonfly. |
28 +
| **ECR, Artifact Registry, ACR** | Cloud IAM; regional; scanning; lazy pulling on AWS (SOCI). | Lock-in; egress costs; not sovereign. |
29 +
30 +
And the bolt-ons each of them needs:
31 +
32 +
- **Distribution:** Dragonfly, Kraken or Spegel, run beside the registry.
33 +
- **Lazy pulling:** eStargz, SOCI, Nydus or zstd:chunked, each a format
34 +
  plus a snapshotter. It matters: pulling took 76% of container start time
35 +
  while only 6.4% of the pulled data was read (Slacker, FAST '16).
36 +
- **Signing and provenance:** cosign and Rekor, SLSA generators,
37 +
  in-toto, Notation, attached afterwards through the OCI referrers API.
38 +
- **Retention:** cleanup policies plus a separate garbage collection.
39 +
40 +
## 2. The pattern behind the weaknesses
41 +
42 +
1. **The unit is the tarball layer.** One changed file re-pushes and
43 +
   re-pulls a whole layer; layers share nothing below their own size; and a
44 +
   layer has two digests (compressed and not), so tools disagree on what
45 +
   it is called.
46 +
2. **Names are mutable pointers.** A tag moves silently and leaves no
47 +
   history. Pinning by digest is safe but unreadable, so most people
48 +
   don't.
49 +
3. **Trust is bolted on afterwards.** The registry stores bytes; a
50 +
   separate tool signs them, a third says how they were built, and each
51 +
   brings its own keys and identities. Nothing stops an unsigned image
52 +
   from being pushed, and most pulls never verify anything.
53 +
4. **Distribution is someone else's problem.** Every registry is a
54 +
   central server; peer-to-peer and lazy loading are extra products.
55 +
5. **The forge and the registry are strangers.** Permissions, identity,
56 +
   retention and "which commit is this?" are kept twice and drift apart.
57 +
58 +
## 3. What Oxygen already has
59 +
60 +
Every piece Cylinder needs exists in some form:
61 +
62 +
- **Content-defined chunks with BLAKE3** (FastCDC, 32 to 256 KiB), shared
63 +
  by repositories, release assets, and runner images: deduplication below
64 +
  file level, across everything.
65 +
- **Hybrid post-quantum signatures** (Ed25519 with ML-DSA-65), identities
66 +
  with delegation chains, and rotation recorded in the repository.
67 +
- **Signed Flow receipts:** a runner signs what it built, from which
68 +
  state, with which pipeline and environment, producing which outputs.
69 +
  That is provenance by construction, not a statement added later.
70 +
- **flow-casfs:** a content-addressed tree mounted lazily over FUSE or
71 +
  virtio-fs. Lazy pulling is native, not a format bolted on.
72 +
- **The Jetstream swarm:** peer-to-peer chunks, rarest first, TLS pinned
73 +
  by the hub, tickets that only the hub signs.
74 +
- **Strata:** storage on disk, S3, GCS or Azure, with no server in between,
75 +
  sovereign keys and client-side hash verification.
76 +
- **OxygenHub:** one identity and permission model, checks, proposals and
77 +
  reports.
78 +
- **Scoped delegation:** key scopes (`changes`, `refs`, `seal`) allowing
79 +
  principals and AI agents to act within strict cryptographic boundaries.
80 +
81 +
## 4. The design
82 +
83 +
### 4.1 One kind of thing: a tree of chunks
84 +
85 +
An **artifact** is an immutable, content-addressed object in the Oxygen
86 +
store naming a file tree:
87 +
88 +
```
89 +
Artifact {
90 +
  version     [0]: 1
91 +
  kind        [1]: "container" | "runner-image" | "package" | "archive" | "report"
92 +
  arch        [2]: text | null              // "x86_64", "riscv64", "aarch64"
93 +
  os          [3]: text | null              // "linux", "darwin", "none"
94 +
  tree        [4]: ObjectRef                // DirTree blob: paths, modes, blobs, sizes
95 +
  entrypoint  [5]: [text] | null            // default exec argv
96 +
  env         [6]: map<text, text> | null   // environment variables
97 +
  labels      [7]: map<text, text> | null   // metadata / OCI labels
98 +
  total_size  [8]: u64                      // uncompressed bytes
99 +
  receipt     [9]: ObjectRef | null         // Receipt object that produced it
100 +
}
101 +
```
102 +
103 +
Files in the `DirTree` are ordinary Oxygen `blob` objects composed of
104 +
FastCDC BLAKE3 `chunk`s. So:
105 +
106 +
- An image rebuilt with one changed file moves only that file's changed
107 +
  chunks: no layers.
108 +
- A release tarball, an npm package, and the container image containing
109 +
  the same binary share its chunks verbatim. If a 100 MB model or library
110 +
  is committed in the repo and then packaged, zero new chunk bytes are
111 +
  written to Strata.
112 +
- A single file can be fetched independently: a container or job starts
113 +
  after fetching only the files it reads.
114 +
- Every artifact has exactly one name for its content: the BLAKE3 of its
115 +
  canonical CBOR encoding.
116 +
- **MicroVMs:** Flow's MicroVM tier runs on Firecracker, which has no
117 +
  virtio-fs, so a microVM gets the artifact as a block device: an ext4
118 +
  image served on the fly from the chunk tree (a userspace block device,
119 +
  ublk, or an NBD server), fetching blocks as the guest reads them and
120 +
  duplicating no bytes. That removes copying whole images onto each
121 +
  runner's disk without changing hypervisor. The syscall tier and
122 +
  containers mount the tree directly over `flow-casfs`; mounting it with
123 +
  virtio-fs would need Cloud Hypervisor or QEMU instead of Firecracker.
124 +
125 +
### 4.2 Names are signed histories, not pointers
126 +
127 +
A **channel** (`app:stable`, `runner-image:rust`, `pkg:cargo/my-crate`) is an
128 +
append-only log of signed moves, recorded as an attestation of kind
129 +
`channel`:
130 +
131 +
```
132 +
Attest (kind = "channel") {
133 +
  channel   [0]: text             // e.g. "app:stable"
134 +
  old_art   [1]: ObjectRef | null // previous artifact hash (CAS check)
135 +
  new_art   [2]: ObjectRef        // target artifact hash
136 +
  serial    [3]: u64              // monotonically increasing
137 +
  receipt   [4]: ObjectRef | null // Receipt that built new_art
138 +
  reason    [5]: text             // commit message or promotion note
139 +
}
140 +
```
141 +
142 +
A move needs policy: signed by an authorized key under the repository's
143 +
trust policy (`spec.md §14.5`), from an approved bookmark, after
144 +
required checks or human approvals pass. Clients follow a channel and
145 +
verify each step:
146 +
147 +
- A tag cannot move silently. The tj-actions attack needed one stolen
148 +
  token; here it needs a move signed by an authorized key, and the move
149 +
  is logged in the append-only history.
150 +
- Rolling back is a move to an earlier artifact, recorded and attested
151 +
  like any other move.
152 +
- Pinning stays readable and tamper-evident: `app:stable@42` names the
153 +
  42nd entry of an audited log.
154 +
- On Strata stores, updates use compare-and-swap on the channel record
155 +
  (`old_art` and `serial`), matching how bookmark `refs` operate.
156 +
157 +
### 4.3 Provenance by construction and multi-party consensus
158 +
159 +
An artifact is published by a Flow job, and its record is the unbroken
160 +
chain:
161 +
162 +
> source state → pipeline → runner (identity, tier, image) → signed
163 +
> receipt → artifact
164 +
165 +
Each link is a hash or signature verifiable offline. Beyond what
166 +
attestations give today:
167 +
168 +
- **Assurance levels:** a MicroVM job on a registered runner is worth
169 +
  more than a syscall-tier job, which is worth more than a signed local
170 +
  build. Trust policies can enforce minimum assurance floors for
171 +
  sensitive channels.
172 +
- **Multi-party consensus (`reproduced`):** when multiple independent
173 +
  runners on disjoint machines or architectures build the same artifact
174 +
  hash from the same state, each signs a receipt. A policy rule can
175 +
  require threshold consensus:
176 +
  `require: "reproduced(2)"` demands two independent runner receipts
177 +
  before an artifact can be promoted to a production channel.
178 +
- **Upstream lineage:** an artifact built from other artifacts (a base
179 +
  image, a package) names their hashes in its receipt inputs, so the chain
180 +
  reaches back transitively to their sources.
181 +
- **An SBOM with every artifact:** the job that publishes an artifact also
182 +
  records what it contains by package and version (an SBOM, itself an
183 +
  artifact named in the receipt's outputs), so "which artifacts hold a
184 +
  vulnerable version of this package" is a lookup, not a scan (§4.6).
185 +
186 +
### 4.4 Distribution is built in
187 +
188 +
- **The store is the seed; the swarm delivers.** Jetstream's swarm,
189 +
  generalized: any runner, cluster node or developer machine in a site
190 +
  shares the chunks it has, over TLS pinned by the hub, admitted by
191 +
  hub-signed tickets.
192 +
- **Lazy by default:** artifacts are mounted via `flow-casfs` and fetched
193 +
  on demand.
194 +
- **Learned prefetch:** each run records which chunks were accessed
195 +
  during its initial execution window; subsequent pulls request those
196 +
  chunks first.
197 +
- **Sites and air gaps:** a site syncs chunks and channel logs from
198 +
  another hub as a signed bundle, so a disconnected network gets
199 +
  verified artifacts with no live connection.
200 +
201 +
### 4.5 Sovereign storage, and our own transparency
202 +
203 +
- Bytes live on any Strata store: local disk, S3, GCS or Azure, in the
204 +
  customer's own account and region, encrypted at rest by the store with
205 +
  customer-managed keys (for example S3 with a KMS key). Chunks stay named
206 +
  by the hash of their plain bytes: encrypting them on the client with
207 +
  separate keys would stop them deduplicating and stop peers checking them
208 +
  by hash.
209 +
- Channel moves and signatures are recorded in the hub's append-only
210 +
  Merkle log. Auditing requires no dependence on public ledgers like
211 +
  Rekor, though Cylinder can bridge to them when desired.
212 +
213 +
### 4.6 Policy at point of use, and the inverted vulnerability index
214 +
215 +
Verification happens where the artifact is used, not only where it is
216 +
stored:
217 +
218 +
- A runner checks the job's image against policy before execution.
219 +
- A Kubernetes admission hook checks pod images against channel policies
220 +
  and assurance levels.
221 +
- `oxy` and `flow` verify what they fetch.
222 +
223 +
**The inverted file index:** Because artifacts are trees of
224 +
content-addressed files rather than opaque tarballs, Cylinder keeps an
225 +
index from each file's hash to the artifacts that contain it:
226 +
227 +
$$\text{FileHash} \longrightarrow [\text{ArtifactHash}]$$
228 +
229 +
- **Exact files, at once.** Given a known-bad file (a compromised binary,
230 +
  a leaked key, the exact `libcrypto.so.3` an advisory names), one lookup
231 +
  lists every container image, runner image and package across the fleet
232 +
  holding it, with no layers to unpack.
233 +
- **Not versions.** A patched or rebuilt library is a different file with
234 +
  a different hash, so this index does not answer "which images have a
235 +
  vulnerable version of OpenSSL". That comes from package identity: the
236 +
  SBOM each artifact carries (§4.3), indexed by package and version. The
237 +
  two together answer both questions; the file index alone answers the
238 +
  first.
239 +
- **Revocation without an outage.** When an artifact, base image or file
240 +
  is revoked in policy, admission hooks refuse new pods that use it at
241 +
  once. Pods already running are not touched by admission; a controller
242 +
  marks them, then replaces or evicts them within a grace period the
243 +
  policy sets, so one policy change cannot take production down.
244 +
245 +
### 4.7 Retention by reachability and deployment leases
246 +
247 +
What a channel, a release, a pinned pipeline or an active deployment
248 +
reaches is kept; unreferenced chunks are collected by `oxy gc`.
249 +
250 +
**Deployment leases:** Container registries often suffer from garbage
251 +
collection deleting images still running on long-lived cluster pods.
252 +
To prevent this, nodes or Kubernetes operators register periodic
253 +
**deployment leases**:
254 +
255 +
```
256 +
Attest (kind = "lease") {
257 +
  artifact [0]: ObjectRef  // artifact hash currently running
258 +
  scope    [1]: text       // e.g. "k8s:cluster-lon/ns/workload"
259 +
  expires  [2]: u64        // timestamp
260 +
}
261 +
```
262 +
263 +
Active leases act as garbage collection roots. If a workload terminates
264 +
or a cluster is decommissioned, the lease expires naturally, enabling
265 +
safe automated retention without operational risk. A lease is signed by
266 +
a key scoped to leases and to its `scope`, and has a maximum length (a
267 +
day, renewed while the workload runs), so a compromised node cannot hold
268 +
storage forever or pin artifacts outside its own scope.
269 +
270 +
### 4.8 Compatibility at the edge
271 +
272 +
- **An OCI registry façade:** `docker pull` and Kubernetes work
273 +
  unchanged. Manifests and layers are assembled on the fly from chunks.
274 +
  `docker push` is accepted and chunked via FastCDC, applying
275 +
  deduplication to legacy images.
276 +
- **A containerd snapshotter** speaking the native protocol, mounting
277 +
  file trees via `flow-casfs` for instant sub-second container starts
278 +
  and P2P delivery on Kubernetes.
279 +
- **Package façades (Cargo, npm, PyPI):** Served from the same chunk
280 +
  store. Façade endpoints synthesize package metadata (Cargo sparse
281 +
  indexes, npm packuments, PyPI simple indexes) dynamically from artifact
282 +
  manifests.
283 +
- **Upstream dependency quarantine:** When Flow builds fetch third-party
284 +
  packages from crates.io or npm, Cylinder proxies them into the
285 +
  organization's Strata store, locking the hashes into internal mirror
286 +
  channels. Upstream outages or deletions cannot break builds. Packages
287 +
  arrive compressed (`.crate` and `.tgz` are gzip), which chunks poorly,
288 +
  so the original archive is kept for clients that check its hash and its
289 +
  unpacked files are chunked as an artifact tree, where they deduplicate
290 +
  with everything else.
291 +
292 +
### 4.9 AI-native agent integration
293 +
294 +
Extending Oxygen's scoped delegation model (`ai_native_agent_workflows.md §2`),
295 +
agent keys can hold artifact-specific scopes:
296 +
297 +
```
298 +
delegates [8]: [{ key: KeyId, scope: ["changes" | "refs" | "seal" | "artifacts" | "channels"], expires: u64? }]
299 +
```
300 +
301 +
- An AI agent (e.g. Aider, Claude Code) working autonomously can be
302 +
  granted `--scope artifacts` and restricted channel rights:
303 +
  - Permitted: building artifacts and moving `agents/NAME/**` channels.
304 +
  - Forbidden: moving production channels (`app:stable`, `release/*`).
305 +
- Agents can test containerized workloads end-to-end in isolated
306 +
  channels, while production promotion remains guarded by human landing
307 +
  rules or multi-party consensus.
308 +
309 +
### 4.10 Revset query integration
310 +
311 +
Oxygen's revset language (`revset_query_language.md`) bridges source
312 +
changes and deployed artifacts:
313 +
314 +
| Expression | Selects |
315 +
|---|---|
316 +
| `artifact(CHANNEL)` | Changes contained within the artifact currently at `CHANNEL` |
317 +
| `deployed(ENV)` | Changes currently active in environment `ENV` (via active leases) |
318 +
| `unshipped()` | Equivalent to `main ~ artifact(app:stable)` (changes on `main` that have not reached production) |
319 +
| `reproduced()` | Changes whose built artifacts have 2+ independent runner receipts |
320 +
| `vulnerable(CVE)` | Changes that touch or introduced files present in an artifact flagged with `CVE` |
321 +
322 +
Running `oxy log -r 'main ~ artifact(app:stable)'` immediately shows all
323 +
changes pending deployment.
324 +
325 +
### 4.11 The UI (inside the hub)
326 +
327 +
- **Per repository, an Artifacts tab:** published artifacts, channel
328 +
  histories, and the provenance graph from commit to run, artifact, and
329 +
  deployment lease.
330 +
- **Site-wide:** chunk distribution across hubs and runners, active
331 +
  swarm transfers, deduplication savings, and the global CVE inverted index.
332 +
- **Runner images** are listed as standard artifacts with their build
333 +
  receipts and hardware architecture badges.
334 +
335 +
```mermaid
336 +
flowchart LR
337 +
  subgraph Build
338 +
    S[Source state] --> P[Pipeline]
339 +
    P --> R1[Runner 1: microvm]
340 +
    P --> R2[Runner 2: disjoint host]
341 +
    R1 -->|receipt 1| A[Artifact: tree of chunks]
342 +
    R2 -->|receipt 2| A
343 +
  end
344 +
  A -->|consensus move| C[Channel log: app:stable@42]
345 +
  A --> ST[(Strata store)]
346 +
  ST -->|seed| SW{{Swarm}}
347 +
  SW --> N1[Runner]
348 +
  SW --> N2[K8s node: containerd snapshotter]
349 +
  SW --> N3[Developer]
350 +
  N2 -->|deployment lease| DL[Active Leases: GC Roots]
351 +
  C -->|policy check| N2
352 +
  OCI[OCI façade] --- ST
353 +
  PKG[Package façades: cargo/npm/pypi] --- ST
354 +
  H[OxygenHub UI & CVE Index] --- C
355 +
  H --- A
356 +
```
357 +
358 +
### 4.12 Where it stands among the services
359 +
360 +
- **OxygenHub** owns repositories, proposals, checks, user identity,
361 +
  and the UI.
362 +
- **Jetstream** owns the fleet: queue, placement, and runner swarms. It
363 +
  fetches runner images from Cylinder by artifact hash.
364 +
- **Flow** executes pipelines and produces signed execution receipts.
365 +
- **Cylinder** owns artifacts: manifests, channel logs, provenance
366 +
  chains, distribution, package façades, and deployment leases.
367 +
- **Strata** provides the underlying content-addressed chunk storage.
368 +
369 +
Each service runs embedded in OxygenHub by default, or standalone for
370 +
distributed scale.
371 +
372 +
## 5. Better how, in one table
373 +
374 +
| | Today's best | Cylinder |
375 +
|---|---|---|
376 +
| Rebuild with one changed file | Re-push and re-pull the layer | Only that file's changed chunks |
377 +
| Dedupe | Whole identical layers | FastCDC chunks across repos, images and packages |
378 +
| Names | Mutable tags; digests nobody reads | Signed, append-only channels (`app:stable@42`) |
379 +
| Provenance | Attestations added later, if at all | The build's signed receipt is the record |
380 +
| Signing | A separate tool and keys | Native Ed25519 + ML-DSA-65 composite signatures |
381 +
| Consensus | Single-builder attestation | Multi-party consensus receipts (`reproduced(N)`) |
382 +
| Lazy start | Bolted-on snapshotter (SOCI, Nydus) | Native: file trees mounted via `flow-casfs` |
383 +
| Peer to peer | Separate mesh daemon (Dragonfly) | Built-in Jetstream swarm, TLS-pinned |
384 +
| Vulnerability search | Unpack and scan every layer | Exact files by one index lookup; versions by SBOM package index |
385 +
| Retention | Tag rules + manual GC | Reachability + active deployment leases |
386 +
| AI workflows | Uncontrolled service tokens | Scoped delegation (`artifacts`, `channels`) |
387 +
| Querying | Disconnected CI/CD dashboards | Unified revsets (`main ~ artifact(app:stable)`) |
388 +
| Sovereignty | Complex multi-tier server stack | Single binary; customer KMS and Strata bucket |
389 +
390 +
## 6. How we could build it
391 +
392 +
- **C0. Images in the hub (built 2026-10-12, oxygenhub #146).** Admin ›
393 +
  Runners › Images: every runner image, its size and architecture, where
394 +
  it is (the hub, each runner, swarm fetches in progress), the
395 +
  repositories whose jobs used it, and an image held nowhere marked.
396 +
- **C1. The service boundary and artifact model.** Implement the
397 +
  `Artifact` CBOR object type and FastCDC chunk engine in `cylinder`;
398 +
  unify runner images with artifacts.
399 +
  *Started 2026-10-28 (oxygen#108):* `crates/cylinder` has the
400 +
  `Artifact` and `DirTree` encodings of §4.1 (canonical CBOR; a name is
401 +
  the BLAKE3 of the encoding; `env` and `labels` as sorted pairs, since
402 +
  the profile's maps take integer keys), packing a directory into the
403 +
  repository's own blobs and chunks, verifying and unpacking, and the
404 +
  roots garbage collection keeps; `oxy artifact pack | list | show |
405 +
  unpack | verify`. The artifact is an object type of its own
406 +
  (`artifact`, in oxy-core's `artifact` module, checked by `oxy fsck`),
407 +
  named by its object hash like every object; its tree is a blob, and
408 +
  `.oxy/artifacts/NAME` records the artifacts a repository keeps. Next:
409 +
  runner images as artifacts, then moving them between repositories (C2).
410 +
- **C2. Channels, provenance and consensus.** Flow publishes artifacts;
411 +
  signed channel move attestations; multi-party `reproduced(N)` consensus;
412 +
  revset predicates.
413 +
- **C3. The OCI façade and containerd snapshotter.** OCI pull/push
414 +
  translation; `containerd` snapshotter over `flow-casfs`.
415 +
- **C4. Leases and the indexes.** Kubernetes deployment lease operator
416 +
  for safe GC; the inverted file index and the SBOM package index; the
417 +
  controller that replaces revoked running workloads within their grace
418 +
  period.
419 +
- **C5. Reach and package façades.** Package façades for Cargo, npm,
420 +
  and PyPI; upstream dependency quarantine mirroring.
421 +
422 +
## 7. Open questions
423 +
424 +
These are Andy's decisions. Each carries the position proposed in review,
425 +
not an answer.
426 +
427 +
- **The name.** Proposed: keep **Cylinder**. It fits the elemental
428 +
  progression: *Oxygen* (VCS) is carried in *Cylinders* (artifacts) by
429 +
  *Jetstream* (the fleet), driven by *Flow* (pipelines).
430 +
- **Where it lives.** Proposed: first in `oxygenflow` as
431 +
  `crates/oxygen-cylinder`, beside `jet` and `flow`, to share `flow-casfs`
432 +
  and the chunking code; its own repository from C3.
433 +
- **Local builds.** Proposed: allowed, at the lowest assurance level
434 +
  (`local`); production channels require more (`tier-microvm` and
435 +
  `reproduced(2)`).
436 +
- **A public catalogue and federation.** Proposed: public hubs federate
437 +
  by publishing signed Strata bundles of channels and artifacts, checked
438 +
  offline with no central intermediary.
439 +
- **Its place in the queue.** Proposed: C1 alongside Jetstream's service
440 +
  boundary, so Jetstream's image store is built on Cylinder from the
441 +
  start rather than on a temporary mechanism. Against: the core work of
442 +
  2026-10-03 (Strata's remaining stores, agents, hub polish) waits
443 +
  longer.
444 +
445 +
## Sources
446 +
447 +
- Docker Hub pull limits: [docs.docker.com, pull usage and limits](https://docs.docker.com/docker-hub/usage/pulls/); [heise, "only 10 pulls per hour"](https://heise.de/-10329047)
448 +
- GHCR billing: [GitHub community discussion #183054](https://github.com/orgs/community/discussions/183054)
449 +
- GitLab cleanup policies and garbage collection: [GitLab docs, reduce container registry storage](https://docs.gitlab.com/user/packages/container_registry/reduce_container_registry_storage/)
450 +
- Bitbucket Packages: [Atlassian blog, Bitbucket Packages](https://www.atlassian.com/blog/bitbucket/bitbucket-packages-container-registry); [GA announcement](https://www.atlassian.com/blog/bitbucket/open-beta-introducing-packages-native-container-registry-in-bitbucket-cloud)
451 +
- Pull time versus data read: [SOCI paper (Amazon), citing Slacker](https://cdn.amazon.science/84/96/51ea2e0d40fca8cfaee9328278b9/soci-paper.pdf); [Grab engineering, lazy loading](https://engineering.grab.com/docker-lazy-loading)
452 +
- zstd:chunked: [Red Hat, faster container image pulls](https://www.redhat.com/sysadmin/faster-container-image-pulls)
453 +
- Peer-to-peer meshes: [Spegel vs Kraken vs Dragonfly](https://www.pistack.xyz/posts/2026-04-28-spegel-vs-kraken-vs-dragonfly-self-hosted-p2p-container-image-distribution-guide-2026/)
454 +
- Sigstore in registries: [Sigstore in image registries: adoption and reality](https://jacar.es/en/sigstore-in-image-registries-adoption-and-reality/); [cosign and the referrers API](https://oneuptime.com/blog/post/2026-08-11-cosign-signature-storage-oci-referrers-separate-repositories/markdown)
455 +
- tj-actions tag attack: [CVE-2025-30066](https://inventivehq.com/tools/security/cve-lookup/cve-2025-30066); [Legit Security, immutable tags](https://legitsecurity.com/blog/what-are-immutable-tags-and-can-they-protect-you-from-supply-chain-attacks)
456 +
- Artifactory and Nexus: [Codefresh, Artifactory alternatives](https://codefresh.io/learn/jfrog-artifactory/top-8-artifactory-alternatives-in-2025); [PeerSpot, Artifactory improvements](https://origin.peerspot.com/questions/what-needs-improvement-with-jfrog-artifactory)
modified docs/guide.md +122 −11
…
117 unchanged lines
118 118
oxy bookmark switch main
119 119
oxy merge feature                  # main now holds both; working copy follows if clean
120 120
oxy bookmark list                  # ● current, heads, ↑n/↓n against origin
121 +
oxy bookmark show feature          # each head, newest first (list -v: every bookmark's)
121 122
oxy push -b feature                # publish a bookmark other than the current one
122 -
oxy bookmark set feature 658990aa  # move it; oxy bookmark delete feature to drop it
123 +
oxy bookmark set feature 658990aa  # move it (onto work built on it: kept heads stay); delete drops it
123 124
```
124 125
125 126
`oxy checkout feature` materialises a bookmark without switching to it, and
126 127
a bookmark name works wherever a change is expected when it has one head.
128 +
In a revset it means the bookmark's heads (`oxy log -r feature`);
129 +
`bookmark(feature)` means only the bookmark, never a tag or change prefix.
127 130
Each workspace remembers its own bookmark, so two directories can sit on
128 131
two bookmarks of one repository.
129 132
…
17 unchanged lines
147 150
bookmark to resolve conflicts, `oxy bookmark set NAME CHANGE --upto`
148 151
keeps everything the bookmark held.
149 152
153 +
`--only` is exact, and that can surprise: other changes to the same files
154 +
are left out, so a file can come out partly edited (a pipeline file that
155 +
no longer parses, say). To put one change on top of a bookmark's work,
156 +
give that bookmark's heads with it, as `oxy bookmark show main` prints
157 +
them: `oxy bookmark set fix <main's heads> 745307b4`.
158 +
150 159
`oxy bookmark switch` says when the bookmark you switch to lacks changes
151 160
that the one you left has.
152 161
…
4 unchanged lines
157 166
`oxy bookmark switch develop` makes a local `develop` from `origin/develop`
158 167
when there is none, and `oxy pull` then keeps every local bookmark up to
159 168
date with the remote's of the same name, not only the current one. Pulling
160 -
is a union, so this never loses work.
169 +
is a union, so this never loses work. A local bookmark that is only behind
170 +
its remote (`-3` in `bookmark list`) is caught up when you switch to it.
161 171
162 172
```
163 173
oxy clone https://oxygenhub.io/owner/game --recursive   # subrepos too
…
151 unchanged lines
315 325
name, appear as `<name>.oxy-conflict-<n>` candidates: keep the one you
316 326
want under the right name and delete the rest.
317 327
328 +
A binary file or a subrepo pin changed two ways has no markers to edit:
329 +
the checkout writes one version, and you say which content wins by
330 +
naming it. Put the bytes you want at the path (or move the subrepo to the
331 +
pin you want), then:
332 +
318 333
```
334 +
oxy resolve assets/logo.png deps/lib   # these paths' content settles them
335 +
```
336 +
337 +
Two changes that each add lines at the same place (two new imports, two
338 +
changelog entries) conflict too, since oxy will not guess their order.
339 +
`oxy status` says when a conflict is one of these; keep both sides, the
340 +
earlier change's first, with one command:
341 +
342 +
```
343 +
oxy resolve --both CHANGELOG.md   # --reverse for the later change's first
344 +
```
345 +
346 +
For files where the order never matters, such as a changelog, say so once
347 +
in `.oxyattributes` and checkouts keep both sides without asking:
348 +
349 +
```
350 +
CHANGELOG.md merge=union
351 +
```
352 +
353 +
```
319 354
oxy resolve      # which conflicts your edits resolve, which still have markers
320 355
oxy commit -m "Resolve retry vs refactor"
321 356
```
…
44 unchanged lines
366 401
367 402
A token is found in this order: the remote's own `remote.<name>.token`, then
368 403
`$OXY_TOKEN` (for CI), then the store, then a remote on the same hub in an
369 -
enclosing checkout (so a subrepo uses its parent's). oxy sends a token over
404 +
enclosing checkout (so a subrepo uses its parent's). When the token came
405 +
from the store or an enclosing checkout and the hub refuses it (a stale login,
406 +
or another account's), oxy tries the other tokens enclosing checkouts hold for
407 +
that hub before giving up, and says so once; a remote's own token and
408 +
`$OXY_TOKEN` are never swapped for another. oxy sends a token over
370 409
`https` only, or over plain `http` to this machine; it is never sent to
371 410
another host after a redirect, and a credentials file that others can read
372 411
is ignored.
…
2 unchanged lines
375 414
- `oxy pull` fetches, then makes your bookmark, and every other local
376 415
  bookmark the remote also has, the union of both sides.
377 416
  There is no merge commit; unrelated changes simply coexist, and a real
378 -
  conflict is recorded as in section 6. The working copy follows when it was
379 -
  clean.
417 +
  conflict is recorded as in section 6. The working copy follows, taking
418 +
  uncommitted work along when the pulled changes leave its files alone;
419 +
  otherwise it stays and `oxy checkout` updates it. A working copy that had
420 +
  fallen behind its bookmark catches up on the next pull too, and `oxy
421 +
  status` says when it is behind.
380 422
- `oxy push` sends your changes and moves the remote bookmark. If the remote
381 423
  moved since you last fetched, the push is refused: pull and push again.
382 424
  `--force` overrides the compare-and-swap.
383 425
- `oxy remote add|remove|list` manages remotes; the clone source is `origin`.
426 +
  Commands that take a remote use `origin`, or the only remote there is; with
427 +
  several and none named `origin`, name one or set `oxy config core.remote
428 +
  NAME`. `bookmark list` compares each bookmark with a remote that has it,
429 +
  and names the remote when there is more than one.
384 430
385 431
A repository can also be cloned from a local path, which is handy for tests
386 432
and for a bare "hub" on a shared disk.
…
64 unchanged lines
451 497
lists them, `oxy verify` names the agent, and its own `oxy push` is
452 498
refused under any policy, so you land its work.
453 499
500 +
**Keyless agents** (on an OxygenHub) hold no key at all. You give the
501 +
agent a grant instead, signed by your key, and the hub certifies a fresh
502 +
key for it each time it signs:
503 +
504 +
```
505 +
oh agent grant claude --for 30d       # your shell: prints the agent's token, once
506 +
oxy auth grant < token                # the agent's shell (its own OXY_HOME)
507 +
oxy config --user author.key    <YOUR KeyId>
508 +
oxy config --user author.signer keyless
509 +
oxy config --user author.for    <YOUR ACCOUNT>   # follow your newest identity
454 510
```
511 +
512 +
Each certificate lasts ten minutes and carries your grant, so the
513 +
agent's commits count as yours, and a push seals them with the fresh
514 +
key. A repository with a trust policy also names the agent:
515 +
`oxy policy grant-subject agent:claude@YOU --issuer hub.identity --scope
516 +
changes` (add `refs` to let it push), with the hub's identity from
517 +
`/.well-known/oxyhub-key`. `oh agent list` shows your grants and `oh
518 +
agent revoke ID` stops one at once; there is no key to rotate. When you
519 +
rotate your own key (every delegation does), an agent with `author.for`
520 +
follows: the hub names your newest identity with each certificate, and
521 +
the agent authors as it from then on, rather than as the key you gave it.
522 +
A repository whose own config pins `author.key` keeps that pin.
523 +
524 +
```
455 525
oxy identity revoke                   # retire your own key
456 526
oxy identity revoke 3f2a… --reason "laptop stolen"   # any key, if you hold a root
457 527
```
…
31 unchanged lines
489 559
oxy receipt export 3f2a9c build.receipt  # hand it to someone
490 560
oxy receipt verify build.receipt         # check one without keeping it
491 561
oxy receipt import build.receipt         # verify and keep it
562 +
oxy receipt export 3f2a9c prov.json --format slsa   # SLSA provenance, signed with your key
563 +
oxy identity export --pem me.pem         # the key cosign checks it with
492 564
```
493 565
566 +
The SLSA form is for tools outside Oxygen: in-toto and SLSA verifiers, and
567 +
`cosign verify-blob-attestation --key me.pem --type slsaprovenance1
568 +
--insecure-ignore-tlog=true --check-claims=false --signature prov.json`. Its subjects are named by
569 +
Oxygen hashes, which cosign cannot compute from a file, hence
570 +
`--check-claims=false`.
571 +
494 572
## 11. Big repositories: partial, sparse, shallow
495 573
496 574
```
…
25 unchanged lines
522 600
Committing the parent records whatever the nested repository is at, and
523 601
checking out an older parent state moves the nested repository back, unless
524 602
it has uncommitted changes, which are never overwritten. An absent nested
525 -
repository keeps its pin until you run `update --init`. Git submodules
526 -
survive `oxy git import` as Git pins and go back out as gitlinks on export.
603 +
repository keeps its pin until you run `update --init`. A `.oxymodules`
604 +
section written by hand has no pin until `oxy submodule add` clones it;
605 +
`status` and `update` name such entries, and `update` exits non-zero. Git
606 +
submodules survive `oxy git import` as Git pins and go back out as gitlinks
607 +
on export.
527 608
528 609
## 12. Git interoperability
529 610
…
11 unchanged lines
541 622
has been fetched, so it always fast-forwards. Exported commits carry
542 623
`Change-Id` and `Oxy-Hash` trailers.
543 624
625 +
A long import moves its bookmark to the commits done so far every 2000
626 +
commits, so if it stops, `oxy git fetch GIT_DIR --branch BRANCH` in the new
627 +
repository carries on from there. One stopped before its first checkpoint
628 +
(or by an older oxy) is recovered first with `oxy git recover`, which
629 +
indexes the objects it had written and points the bookmark at them.
630 +
544 631
## 13. Automate: hooks
545 632
546 633
Executables in `.oxy/hooks/<event>` (never cloned) receive one JSON object
…
34 unchanged lines
581 668
| Key | Meaning |
582 669
|---|---|
583 670
| `author.name`, `author.email`, `author.key` | who you are; `key` is set by `identity create` |
584 -
| `author.signer` | the key that signs instead of `author.key`: an agent's own key, which your identity lists as a delegate (also `$OXY_SIGNER`) |
671 +
| `author.for` | in a keyless agent's home, the account it acts for: it authors as that person's newest identity, as the hub names it (a repository's own `author.key` pin still wins) |
672 +
| `author.signer` | the key that signs instead of `author.key`: an agent's own key, which your identity lists as a delegate; or `keyless`, for fresh keys the hub certifies under the agent's grant (also `$OXY_SIGNER`) |
673 +
| `keyless.hub` | the hub that certifies keyless keys (default: the default remote's hub) |
585 674
| `core.sign` | `change` to sign every commit; otherwise seals at push |
586 675
| `core.op-retention` | days for which old operations keep their states alive (90) |
587 676
| `core.gc-grace` | days an unreachable object survives (14) |
…
17 unchanged lines
605 694
oxy gc              # prune unreachable objects past the grace period, repack into one pack
606 695
oxy gc --now        # ignore the grace period (only when nothing else is writing)
607 696
oxy fsck            # check every object, the op log, references, signatures, checkpoints
608 -
oxy doctor          # check identity, config and remotes, with what to do about each problem
697 +
oxy doctor          # check identity, config, trust policy and remotes, with what to do about each problem
609 698
oxy checkpoint      # serialise the bookmark's state so large histories load instantly
610 699
```
611 700
…
6 unchanged lines
618 707
`diff`, `hunks`, `blame`, `resolve`, `op log`, `bookmark list`, `remote list`,
619 708
`fetch`, `pull`, `push`, `submodule status`, `bisect status`, `verify`,
620 709
`policy show`, `root`, `version`). `oxy cat PATH -r CHANGE` prints a file as
621 -
it was, `oxy discard PATH..` restores files to the base, and `oxy hunks`
622 -
numbers the working copy's hunks for `commit --hunks` and `discard --hunks`.
710 +
it was in that change's dependency closure (the change and what it depends
711 +
on; `-r BOOKMARK` or a tag reads the whole state), `oxy discard PATH..`
712 +
restores files to the base, and `oxy hunks` numbers the working copy's hunks
713 +
for `commit --hunks` and `discard --hunks`.
623 714
624 715
## 18. If something looks wrong
625 716
…
18 unchanged lines
644 735
```
645 736
646 737
The oh-my-bash plugin in `contrib/oh-my-bash` loads the bash completions itself.
738 +
739 +
## Transparency logs
740 +
741 +
A hub that keeps a transparency log (OxygenHub does, at `/log`) logs every signed object it accepts. `oxy fetch` and `oxy pull` check it as they go. They remember the log's key and its last checkpoint, and fail if the hub ever shows a log that does not extend what you saw before. That is the sign of a hub showing different histories to different people. To audit a hub's whole log and see what it holds for this repository:
742 +
743 +
```
744 +
oxy tlog verify                         # this checkout's hub
745 +
oxy tlog verify https://oxygenhub.io --all
746 +
oxy tlog verify --key <KeyId>           # what a key signed
747 +
```
748 +
749 +
Anyone can **witness** a hub's log: a witness cosigns each checkpoint that extends the last one it cosigned, with its own key and clock, so a hub cannot show two histories without the witness noticing. The hub lists its witnesses in `hub.toml` (`[tlog] witnesses`), and its served checkpoint carries their cosignatures.
750 +
751 +
```
752 +
curl -fsSL https://oxygenhub.io/.well-known/oxyhub-key -o oxygenhub.identity
753 +
oxy witness serve --name witness.example.org --log oxygenhub.io/log=oxygenhub.identity \
754 +
    --bind 0.0.0.0:8891 --tls-cert cert.pem --tls-key key.pem
755 +
```
756 +
757 +
It signs with your oxy home's key and keeps what it cosigned in `--state` (default `witness-state`).
added docs/lazy-clone.md +195 −0
1 +
# Instant clone: a lazy working copy (oxygen#132)
2 +
3 +
Status: design, 2026-10-25. Nothing here is built yet beyond what it names
4 +
as existing. An experimental `oxy clone --lazy` (Linux) follows if this
5 +
measures well (§9).
6 +
7 +
## 1. The aim
8 +
9 +
`oxy clone URL` of any repository, however big, gives a working copy in
10 +
about a second: `ls`, `cd`, editors and builds see every file at once, a
11 +
file's bytes arrive the first time something reads it, and `status`,
12 +
`diff`, `commit` and `push` work at once and cost what the edits cost, not
13 +
what the repository weighs. Git's partial clone and VFS for Git do part of
14 +
this (VFS needs Windows and a special server); Oxygen can do all of it
15 +
with what it already has, because every object is content addressed and
16 +
a state's tree is known without its file contents.
17 +
18 +
## 2. What exists
19 +
20 +
- **Shallow clones** (spec §17): `oxy clone --depth 0` takes a checkpoint
21 +
  of the bookmark's heads and no history. The `checkpoint` message brings
22 +
  the checkpoint, its payload blob and **every file's graph blob** (spec
23 +
  §11, payload version 2; `Repo::checkpoint_bundle_for`,
24 +
  crates/oxy-core/src/repo/checkpoint.rs). A file graph holds the file's
25 +
  current text and its tombstones, so this is the whole state's text and
26 +
  more: for VS Code most of the clone's bytes.
27 +
- **Lazy blobs** (spec §13.5, §17): `--max-blob N` leaves opaque blobs
28 +
  larger than N to be fetched with `objects` on first read; `status` and
29 +
  `commit` never need them.
30 +
- **Sparse filters** (`--sparse PATH`): only those paths are materialized.
31 +
- **flow-casfs** (OxygenFlow, `crates/flow-casfs`): a read-only FUSE mount
32 +
  of a state from a local repository. The tree comes from the state
33 +
  index; a text file is rendered from its graph on first read or stat and
34 +
  cached on disk by graph key; an opaque file is its blob.
35 +
- **Content addressing everywhere**: an object fetched later is checked
36 +
  against the hash that named it, so a lazily fetched byte is as
37 +
  trustworthy as one cloned eagerly (spec §13).
38 +
39 +
## 3. The design
40 +
41 +
`oxy clone --lazy URL DIR`:
42 +
43 +
1. **Refs and the checkpoint skeleton.** Fetch `refs` (with the bookmark's
44 +
   attestation, checked against the policy as any clone does), then the
45 +
   checkpoint of its heads **without its graph blobs**: a `checkpoint`
46 +
   request with a new `graphs: false` field (additive; an older server
47 +
   ignores it and sends them, so the clone is merely not lazy). The
48 +
   payload's `files` entries name each graph blob by ObjectRef (graphs
49 +
   under 2 KiB are inline and come anyway), and its `tree` gives every
50 +
   path, mode and node. Clone writes the repository as a shallow one at
51 +
   depth 0 whose missing graph blobs are **lazy objects**, like lazy blobs
52 +
   today.
53 +
2. **The mount.** DIR is a FUSE mount (lower layer) of that state, built on
54 +
   flow-casfs: the directory tree is answered from the payload's tree at
55 +
   once; `stat` of a text file needs its length, so the payload gains
56 +
   (payload version 3, writer-optional) each file's rendered length, and
57 +
   until a server writes it the length comes with the graph on first
58 +
   `stat`. A read of a file whose graph is not stored fetches it with
59 +
   `objects` (hash checked), renders it, keeps both, and answers. Opaque
60 +
   files fetch their blob the same way. A small daemon (`oxy mount`,
61 +
   started by the clone, restarted by any oxy command that finds it gone)
62 +
   serves the mount; the repository itself stays in `DIR/.oxy`, which is
63 +
   a real directory beside the mount (bind or an overlay entry), so oxy
64 +
   never reads through FUSE.
65 +
3. **The writable layer.** Writes go to an upper directory
66 +
   (`.oxy/overlay/`): a written file is copied up whole on first write
67 +
   (its bytes fetched first if they were never read), a new file is
68 +
   created there, a removal is a whiteout, a rename is a whiteout plus a
69 +
   copy up. This is the ordinary overlay model, done in the daemon so it
70 +
   needs no privileges (kernel overlayfs over FUSE needs a user namespace
71 +
   and is not available everywhere).
72 +
4. **Status, diff and commit read the overlay only.** The upper layer is
73 +
   exactly the set of paths that may differ from the base state, so
74 +
   `status` lists it (comparing each copied-up file's bytes with the
75 +
   base rendering, which is stored, since copying up read it) and never
76 +
   walks the tree. `diff` and `commit` do the same, and the changes they
77 +
   build need only the touched files' graphs, all stored by then.
78 +
   `commit` clears the overlay entries it recorded: the mount's base
79 +
   state moves to the new heads.
80 +
5. **Fetch, pull, checkout.** A fetch brings changes as today (their
81 +
   blobs lazy). Moving the working copy to another state swaps the
82 +
   mount's base: the new state's tree is known from the changes and the
83 +
   stored checkpoint, the overlay is kept when its paths are unchanged
84 +
   between the two states (the `--carry` rule, §10.1), and graphs needed
85 +
   to render the new state are lazy again.
86 +
6. **Push.** As today: the pushed changes and the blobs they bring. A
87 +
   change built on a lazy base never needs the base's unread files.
88 +
89 +
## 4. Offline and failures
90 +
91 +
A file read once is kept, so it reads offline. Reading a file never read
92 +
while the hub cannot be reached fails with `EIO`, and the daemon logs the
93 +
path and the reason; `oxy status` says how many files are not local yet.
94 +
`oxy lazy fetch [PATH..]` brings them in ahead (a directory, the whole
95 +
state, or the files a build reads, as a background job), and `oxy lazy
96 +
materialize` turns the working copy into an ordinary one (unmounted, every
97 +
file written), after which nothing is lazy. A hash that does not match is
98 +
an error for that read, never a wrong byte.
99 +
100 +
## 5. Without FUSE
101 +
102 +
Where FUSE is not available (macOS without macFUSE, Windows, a container
103 +
without /dev/fuse), `--lazy` falls back to what exists: a depth-0 clone
104 +
with lazy blobs, and `--sparse` paths when given, saying so. Windows
105 +
ProjFS and macOS File Provider are later backends behind the same daemon
106 +
interface (lookup, read, write-through to the upper layer).
107 +
108 +
## 6. The protocol
109 +
110 +
- `checkpoint` gains `graphs: false`: leave out graph blobs (and their
111 +
  chunks) the payload names by reference.
112 +
- `objects` already fetches any object by hash, with a blob's chunks; it
113 +
  needs batching limits only (as `push.batch-bytes` bounds pushes).
114 +
- Optional later: `render` by graph key (the rendered bytes rather than
115 +
  the graph with its tombstones), smaller for long-lived files; a client
116 +
  checks it by rendering the graph when it has it, so it is an
117 +
  optimization a client may refuse.
118 +
- A host should keep `objects` cheap: it is now on the read path of
119 +
  editors and builds. OxygenHub's state cache and bucket stores (Strata)
120 +
  already serve objects by hash.
121 +
122 +
## 7. Trust
123 +
124 +
Nothing is weaker than an eager clone. The clone checks the bookmark's
125 +
attestation and the checkpoint against its heads (a checkpoint is valid
126 +
only if it reproduces from its heads, spec §11, which a lazy client cannot
127 +
check without the graphs: it trusts the attested heads and checks each
128 +
graph it fetches against the checkpoint's reference). `oxy verify` on a
129 +
lazy clone checks what is stored and says how much is not.
130 +
131 +
## 8. Not in scope
132 +
133 +
History (`log`, `blame` beyond the checkpoint) on a lazy clone fetches
134 +
changes on demand later (`oxy deepen` exists); the first version answers
135 +
`log` from the checkpoint's change list as a shallow clone does.
136 +
137 +
## 9. Measurements
138 +
139 +
On Earth (4 cores, SSD), 2026-10-25, the VS Code history
140 +
(`~/bench/vscode-full9.oxy`, 166k changes; `~/bench/vscode.git` for Git)
141 +
served over loopback by `oxy serve` and `git daemon`, oxy at main
142 +
(0.5.1 + #137), Git 2.53; one run each, a fresh directory, the server
143 +
warm (its checkpoint of the heads already made). Seconds:
144 +
145 +
| Step | git clone | git clone --filter=blob:none | oxy clone --depth 0 | oxy clone --depth 0 --max-blob 0 |
146 +
|---|---|---|---|---|
147 +
| clone | 91.4 | 30.5 | 457.9 | 849.0 |
148 +
| `ls -R \| wc -l` | 0.11 | 0.11 | 0.54 | 0.11 |
149 +
| read `src/vs/code/electron-main/main.ts` | 0.01 | 0.01 | 0.01 | 0.01 |
150 +
| append a line, `status` | 0.18 | 0.72 | 8.07 | 8.50 |
151 +
| `diff` | 0.10 | 0.03 | 0.23 | 0.23 |
152 +
| `commit` | 0.12 | 0.05 | 0.98 | 0.92 |
153 +
| disk after | 1.7 GB | 693 MB | 977 MB | 979 MB |
154 +
155 +
Before #137 the depth-0 clone failed after 11 minutes (the state's binary
156 +
files' blobs came with nothing) and, before that fix's other half, wrote
157 +
the checkpoint object by object: stopped after 38 minutes with 57,498
158 +
loose objects. The full `oxy clone` (12 GB of history) was not run
159 +
tonight.
160 +
161 +
What this says:
162 +
163 +
- A depth-0 clone is five times slower than Git's full clone and fifteen
164 +
  times slower than its blobless one. Timed on both sides (`OXY_TIMINGS=1`,
165 +
  oxygen#137): the server spent 375 s gathering the checkpoint's 57,000
166 +
  graph blobs and chunks one read at a time (239 s once read in parallel),
167 +
  19 s encoding one 2 GB answer, and the client 120 s on checkout, 111 s
168 +
  of it rendering 19,545 files on one core. Depth 0 also took the 33,835
169 +
  head changes, which it no longer does (spec §17), though that was not
170 +
  where the time went. A lazy clone avoids all three: no graph blobs move
171 +
  at clone time, and nothing is rendered until read.
172 +
- `--max-blob 0` without a mount is slower, not faster: the checkout then
173 +
  fetches every binary file's blob one request at a time. Lazy contents
174 +
  only pay with a mount that reads them when asked.
175 +
- The first `status` after a clone is 8 s (it builds the manifest); Git's
176 +
  is 0.2 s. A lazy clone's `status` from the overlay (§3.4) would not
177 +
  depend on the tree at all.
178 +
179 +
So the prototype's bar: clone in seconds (refs, the checkpoint skeleton
180 +
without graphs, the mount), `ls -R` at once from the payload's tree, a
181 +
first read in tens of milliseconds, `status` in milliseconds; and,
182 +
whatever happens to `--lazy`, the depth-0 clone itself should come down
183 +
to Git's order (streamed checkpoint in parts, rendering on every core).
184 +
185 +
## 10. The prototype (L2)
186 +
187 +
1. `checkpoint` with `graphs: false` in oxy-core and oxy-server, and a
188 +
   repository flag for lazy graph objects (fetch on load, like lazy blobs).
189 +
2. `oxy mount` (Linux, `fuser`, as flow-casfs) with the upper layer, in a
190 +
   new crate `oxy-mount` beside the CLI, behind a cargo feature so builds
191 +
   without FUSE are unaffected.
192 +
3. `status`/`diff`/`commit` from the overlay when the working copy is
193 +
   mounted.
194 +
4. `oxy clone --lazy`, `oxy lazy fetch`, `oxy lazy materialize`.
195 +
5. Measured as in §9; numbers on oxygen#132.
added docs/mcp.md +78 −0
1 +
# oxy mcp: Oxygen for AI agents
2 +
3 +
`oxy mcp` serves one repository to an AI agent over the
4 +
[Model Context Protocol](https://modelcontextprotocol.io) (MCP), on stdio
5 +
(oxygen#159). The agent calls typed tools (`status`, `log`, `commit`, …)
6 +
and gets JSON back, instead of running `oxy` in a shell and reading its
7 +
output.
8 +
9 +
## Who changes are signed as
10 +
11 +
The server signs as whatever identity its environment configures, exactly
12 +
as `oxy` would: it runs each tool as this same `oxy` binary, in the
13 +
repository, with the server's environment. Start it from an agent home and
14 +
every change the agent makes is sealed **via the agent**, with no
15 +
`OXY_HOME` to remember on each command:
16 +
17 +
```sh
18 +
OXY_HOME=~/.oxy-agents/claude oxy mcp ~/development/myproject
19 +
```
20 +
21 +
The server says who it signs as in its `initialize` reply (the first line
22 +
of `oxy identity show`), and the `identity` tool shows the rest. Without an
23 +
identity, reading works and committing is refused with oxy's own message.
24 +
25 +
## Tools
26 +
27 +
Each tool returns the command's `--json` output as structured content (an
28 +
array comes wrapped as `{"result": […]}`) and as text. A failed command is a
29 +
tool result with `isError: true` and oxy's error, so the agent sees why.
30 +
31 +
| Tool | Runs | Arguments |
32 +
|---|---|---|
33 +
| `status` | `oxy status --json` | |
34 +
| `diff` | `oxy diff [CHANGE] --json` | `change` |
35 +
| `log` | `oxy log --json` | `revset`, `path`, `limit` (default 20, at most 1000) |
36 +
| `show` | `oxy show CHANGE --json` | `change` (default `@`), `stat` |
37 +
| `cat` | `oxy cat PATH [-r CHANGE]` | `path`, `change` |
38 +
| `blame` | `oxy blame PATH --json` | `path` |
39 +
| `bookmarks` | `oxy bookmark list --json` | `all` |
40 +
| `tags` | `oxy tag list --json` | |
41 +
| `conflicts` | `oxy resolve --json` (report only) | |
42 +
| `op_log` | `oxy op log --json` | `limit` |
43 +
| `identity` | `oxy identity show` | |
44 +
| `commit` | `oxy commit -m MESSAGE PATHS…` | `message`, and `paths` or `all: true` |
45 +
| `amend` | `oxy amend [-m] PATHS…` | `paths` or `all: true`, `message` |
46 +
| `describe` | `oxy describe [CHANGE] -m MESSAGE` | `message`, `change` |
47 +
| `fetch`, `pull` | `oxy fetch\|pull [REMOTE] --json` | `remote` |
48 +
| `push` | `oxy push [REMOTE] --json` | `remote`, `bookmark`, `force` |
49 +
| `tlog_verify` | `oxy tlog verify --json` | |
50 +
| `undo` | `oxy undo` | only with `--allow-destructive` |
51 +
52 +
## Safety
53 +
54 +
- **Commits are deliberate.** `commit` and `amend` take the paths to
55 +
  include, or `all: true`; never neither.
56 +
- **Nothing destructive by default.** `undo` is not offered, and `push`
57 +
  refuses `force`, unless the server is started with `--allow-destructive`.
58 +
  Tag withdrawal, bookmark deletion and history rewriting are not tools at
59 +
  all. Each tool carries MCP's `readOnlyHint` and `destructiveHint`.
60 +
- **Values are never flags.** Arguments are passed as `--flag=value` and
61 +
  after `--`, so a path or message starting with `-` stays a value.
62 +
- **Your policy still applies.** Pushes go through the remote's checks and
63 +
  the repository's trust policy like any other; an agent's delegation
64 +
  scope limits what its seals may do.
65 +
- **Output is bounded.** A tool's text stops at 256 KiB, saying to narrow
66 +
  the request.
67 +
68 +
## Adding it to an agent
69 +
70 +
Claude Code, for one project (run in the project's root):
71 +
72 +
```sh
73 +
claude mcp add oxy -- env OXY_HOME=$HOME/.oxy-agents/claude oxy mcp
74 +
```
75 +
76 +
Any MCP client that starts stdio servers takes the same command: `oxy mcp
77 +
[DIR] [--allow-destructive]`, with `OXY_HOME` set for an agent's identity.
78 +
`DIR` defaults to the current directory and must be a repository's root.
modified docs/protocol.md +31 −6
…
51 unchanged lines
52 52
  when the server was started with tokens; a missing or wrong token is `401`.
53 53
- Errors are plain-text bodies with a 4xx or 5xx status; the client shows the
54 54
  text. A push that is *refused* (policy, compare-and-swap) is a `200` with
55 -
  `ok = false` in the push response, not an HTTP error.
55 +
  `ok = false` in the push response, not an HTTP error; one a host
56 +
  refuses by its content (`vet_push`, spec §16.4) is an HTTP error with
57 +
  the host's reason as its text, and nothing of it is stored.
56 58
- TLS is ordinary HTTPS. The reference client trusts the system web roots
57 59
  plus an optional PEM file (`remote.<name>.ca` or `OXY_CA`); servers with
58 60
  self-signed certificates are expected to hand that file out.
…
8 unchanged lines
67 69
  with `400`. Clients send the header on every request when they have an
68 70
  id; servers ignore it on bound repositories.
69 71
72 +
Client capabilities: the reference client sends `User-Agent:
73 +
oxy/<version>` and `Oxy-Features: <feature>[,<feature>…]` on every request,
74 +
a comma-separated list (blanks around entries and case are ignored, unknown
75 +
entries too) of what it understands beyond the oxy 0.5.0 feature set.
76 +
Servers may tailor their answers to the features a request declares, and a
77 +
request without the header is a client of that era. Features so far:
78 +
79 +
- `certified-tags`: the client authorizes a tag signed by a short-lived,
80 +
  certified key by a subject grant, its certificate checked at the time
81 +
  the host accepted the tag (spec §14.7; oxygen#129). A client without it
82 +
  authorizes a tag only by a key the policy's rule lists, a delegate of one
83 +
  or a rotation predecessor, and fails the whole clone or fetch on any
84 +
  other. So under a policy, a server leaves out of a `refs` answer to such
85 +
  a client every tag whose attestation's signer none of those authorizes
86 +
  (`Repo::refs_for_client`, applied by `oxy serve` and by default by every
87 +
  `oxy_server` host): that client sees the repository as if those tags did
88 +
  not exist. If it holds one of them already and pushes, it sends the tag
89 +
  again (it is not in the answer); the server already has it in the same
90 +
  state and takes it as it is.
91 +
70 92
Embedding: `oxy_server::Server` answers any endpoint for a repository name
71 93
a host's `Backend` resolves and authorises (`open`, `authorize`, and an
72 -
`after_push` hook), so a hub with `owner/name` paths and its own accounts
94 +
`after_push` hook, `adjust_refs`, which tailors a refs answer to the
95 +
request's headers and by default applies the rule above, and `vet_push`, with which a host refuses a push by
96 +
what it adds before anything is stored: an HTTP error, `422` say, whose
97 +
text, several lines perhaps, the client shows as it is; spec §16.4; and `push_notes`, with which a host adds a note to a reference the push moved, which the client shows in place of "updated", #148), so a hub with `owner/name` paths and its own accounts
73 98
routes `/{owner}/{name}/oxy/v1/{endpoint}` to `Server::handle`; `oxy serve`
74 99
is `DirBackend` on that same server.
75 100
…
11 unchanged lines
87 112
| Signed attestation | `[attest bytes, [Signature]]` |
88 113
| Signed policy | `[policy bytes, [Signature]]` (also the policy proposal file format) |
89 114
| Capabilities | `{0: [protocol versions], 1: repo id bytes(16), 2: [feature names]}`; version `1`; features currently `["seal"]` |
90 -
| Refs response | `{0: [ref], 1: repo id, 2: signed policy or null, 3: [tag], 4: [signed policy]}` |
91 -
| ref | `{0: name, 1: state ref, 2: [heads], 3: signed attestation or null, 4: last serial or null}`; the attestation is present only when it describes the current state; `last serial` is the highest attestation serial ever recorded for the name |
92 -
| tag | `{0: name, 1: state ref, 2: [heads], 3: signed attestation or null}` |
115 +
| Refs response | `{0: [ref], 1: repo id, 2: signed policy or null, 3: [tag], 4: [signed policy], 8: [tag]}`; key 8 (optional, only when there are any; oxygenhub#301) lists the tags withdrawn there, each a tag whose attestation is its `untag` attestation and whose key 4 is that attestation's acceptance when a short-lived key signed it; older readers ignore it |
116 +
| ref | `{0: name, 1: state ref, 2: [heads], 3: signed attestation or null, 4: last serial or null, 5: signed acceptance}`; the attestation is present only when it describes the current state; `last serial` is the highest attestation serial ever recorded for the name; key 5 (optional, only when there is one) is the host's `accepted` attestation of that attestation (spec §14.7); key 6 (optional) is `[entry number, entry bytes, [proof hash]]`, the attestation's inclusion proof in the host's transparency log against the response's key 6, the latest checkpoint as a signed note (spec §14.8) |
117 +
| tag | `{0: name, 1: state ref, 2: [heads], 3: signed attestation or null, 4: signed acceptance, 5: log place, 6: [cosigned checkpoint]}`; keys 4 to 6 (optional, each only when there is one; oxygen#129) are for the tag's attestation what a ref's keys 5 to 7 are for its own: the issuer's `accepted` attestation when a short-lived key signed the tag (spec §14.7), which a fetch checks the certificate at, and its place in the host's transparency log with the cosigned checkpoints that first held it (spec §14.8); a push carries key 4 when the client has one; older readers ignore them |
93 118
| Negotiate request | `{0: [have], 1: [want]}`; response `{0: [hashes]}` in dependency order (no change before its dependencies) |
94 119
| Fetch request | `{0: [hashes], 1: max blob size or null}`; response a Bundle whose blobs above the size (and their chunks) are omitted |
95 120
| Objects request | `{0: [refs]}`; response a Bundle carrying only objects; a blob is accompanied by its chunks |
96 121
| Checkpoint request | `{0: [heads]}`; response a Bundle of objects: the checkpoint of exactly those heads, its payload blob, and the blob's chunks; the server computes one when it has none for those heads |
97 122
| Shallow request | `{0: [heads], 1: depth}`; response `{0: [hashes]}`: the changes within `depth` dependency steps of the heads, dependencies first |
98 123
| Bundle | `{0: [[change bytes, [Signature]]], 1: [signed attestation], 2: [[type name, object bytes]], 3: [signed policy]}` |
99 -
| Push request | `{0: Bundle, 1: [update], 2: signed policy or null, 3: [tag]}` |
124 +
| Push request | `{0: Bundle, 1: [update], 2: signed policy or null, 3: [tag]}`; a tag whose attestation is an `untag` attestation withdraws that tag (oxygenhub#301), reported as `tags/<name>`; an older server refuses it as an attestation of the wrong kind |
100 125
| update | `{0: name, 1: old state ref, 2: new state ref, 3: force (bool), 4: signed attestation or null}` |
101 126
| Push response | `{0: [{0: name, 1: ok (bool), 2: reason (text)}]}`; a result named `policy` reports the policy update, `tags/<name>` each tag |
102 127
…
101 unchanged lines
modified docs/revset_query_language.md +1 −0
…
48 unchanged lines
49 49
| Symbol | Means |
50 50
|---|---|
51 51
| `main`, `v0.3.1` | the heads of that bookmark or tag |
52 +
| `bookmark(main)`, `bookmark(origin/main)` | the heads of that bookmark only; an error if there is none (a bare name falls through to tags and change prefixes) |
52 53
| `origin/main` | the heads of a remote-tracking bookmark |
53 54
| `8c5ea12a` | the change with that ChangeId prefix |
54 55
| `#fc68459c` | the change with that hash prefix |
…
108 unchanged lines
modified docs/spec.md +181 −32
…
355 unchanged lines
356 356
357 357
### 9.2 Tree conflicts
358 358
359 -
- **Name conflict.** Two live nodes share a placement.
359 +
- **Name conflict.** Two live nodes share a placement. The same path added with identical content on several lines of history is not one, as Git unifies an addition both sides of a merge made: when every node at the placement is a file, symlink, or subrepo of one kind and one mode, none is in another tree conflict, and their contents are the same and unconflicted (one rendering for text files; one live blob for opaque files, symlink targets, and subrepo pins), the state shows a single entry, the node whose placement has the least `PlacementRef`, so every peer shows the same one. Nothing is resolved and no change is recorded. The first change that touches the node shown (an edit, `Move`, `RemoveNode`, or `SetMode`) also deletes the others' placements, or they would differ from it and the conflict would return; a working-copy snapshot adds those deletions with the touch and reports no change of its own for them. Directories are not unified. A Git import builds against the tree without this unification, so an import of a merge records Git's tree for such a path itself (§23).
360 360
- **Multi-placement.** One node has two live placements (concurrent renames).
361 361
- **Orphan.** A live node's parent directory is not live (concurrent delete of a directory and modification within it).
362 362
- **Cycle.** A directory is placed inside its own descendant, which arises when two commuting changes each move a directory under the other. Both placements are live and the tree is conflicted. For materialization the placement whose introducing change has the greater `(hlc, ChangeHash)` is treated as absent; resolution deletes one of the two placements.
…
3 unchanged lines
366 366
367 367
Two live `Replace` edits with the same `old` and different `new` are a **replace conflict**. The state exposes both candidate blobs.
368 368
369 -
Which blobs are live: a replace record is superseded by a later replacement of its blob, a record whose `old` is its `new` and whose change depends on its change (§8.1 makes every replacement depend on the change that made its `old` current). Replacements that do not know each other both stay live, and a file reverted to an earlier blob keeps it. A replace conflict is **resolved** by a `kind = resolution` change with one `Replace` from each live blob to the chosen one, depending on every change that made a candidate live, so each is superseded and one blob remains. A working-copy snapshot never settles a replace conflict on its own (a checkout writes one candidate at the path); a tool that knows the user's choice, such as a forge's resolver, builds the resolution. A replacement recorded without that dependency (before Draft 0.8i) supersedes every record with its `new` that does not come after it, the rule until then.
369 +
Which blobs are live: a replace record is superseded by a later replacement of its blob, a record whose `old` is its `new` and whose change depends on its change (§8.1 makes every replacement depend on the change that made its `old` current). Replacements that do not know each other both stay live, and a file reverted to an earlier blob keeps it. A replace conflict is **resolved** by a `kind = resolution` change with one `Replace` from each live blob to the chosen one, depending on every change that made a candidate live, so each is superseded and one blob remains. A working-copy snapshot never settles a replace conflict on its own (a checkout writes one candidate at the path); a tool that knows the user's choice, such as a forge's resolver or `oxy resolve PATH` (which takes the working copy's content at the path), builds the resolution. A replacement recorded without that dependency (before Draft 0.8i) supersedes every record with its `new` that does not come after it, the rule until then.
370 370
371 371
### 9.4 Properties
372 372
…
13 unchanged lines
386 386
>>>>>>> oxy conflict 1 of 2 end
387 387
```
388 388
389 -
The short identifiers are ChangeId prefixes. Delete/insert conflicts render the deleted context as a `-------` section. Tree conflicts are materialized by writing each candidate with a suffix `.oxy-conflict-<n>` and are listed by `oxy status`. Replace conflicts materialize as `<name>.oxy-conflict-<n>` files.
389 +
The short identifiers are ChangeId prefixes. A conflict's sides are written earliest first, by the clock (§6.4) of the oldest change on each side, then by ChangeHash. Delete/insert conflicts render the deleted context as a `-------` section. Tree conflicts are materialized by writing each candidate with a suffix `.oxy-conflict-<n>` and are listed by `oxy status`. Replace conflicts materialize as `<name>.oxy-conflict-<n>` files.
390 390
391 391
`oxy status` MUST list every conflict, MUST exit with status 2 when any conflict exists in the working-copy state, MUST exit with status 3 when only workspace conflicts (§9.7) exist, and MUST support `--json` output enumerating conflicts by node, kind, and contributing ChangeIds. The JSON object has `bookmark`, `working_copy` (ChangeId), `heads`, `clean`, `conflicted`, `paths` (`[{path, status}]`) and `conflicts`: for a text or replace conflict `{kind, path, node, changes: [ChangeId], resolved: true | false | "markers"}`, for a tree conflict `{kind: name | multi-placement | orphan | cycle | mode, nodes, path?}`.
392 392
393 +
An **adjacent insertion** is an order conflict whose sides only add lines between two lines that were next to each other: no line between them was deleted by a change that not every side builds on (two changes that each replaced the same line conflict the same way, and are not one). Keeping every side settles it without losing or repeating a line, though either order can be wrong for code, so it is never done silently: `oxy resolve --both [PATH]..` replaces the markers of each adjacent insertion in the working copy with its sides, earliest first (`--reverse`: latest first), and leaves the markers of any other conflict, or of one whose lines between the markers were edited. `oxy status` says which conflicts qualify. A file whose `.oxyattributes` say `merge=union` (§19; changelogs, lists where order does not matter) needs no command: a checkout settles its adjacent insertions the same way at once, and the next commit records the resolution. The state is still conflicted until then; nothing about what a conflict is changes. A conflict whose sides are **identical**, byte for byte, settles the same way in any file, with one side kept: two bookmarks that resolved the same conflict to the same text, then merged, agree (oxygen#146; Git merges identical changes cleanly too). A host landing a proposal settles both kinds, and only them, as a checkout does.
394 +
393 395
### 9.6 Resolution
394 396
395 397
A **resolution** is a change with `kind = resolution` whose deletions and insertions leave the affected graphs totally ordered and the tree conflict-free. `oxy resolve` computes it from the edited working copy by parsing the markers; a resolution MAY also be authored by any tool that can produce a change. Because a resolution depends on all contributing changes (§8.1), any peer that fetches it necessarily has the conflict it resolves.
…
18 unchanged lines
414 416
```
415 417
bookmarks/<name>                 local bookmarks
416 418
remotes/<remote>/bookmarks/<name>  last-seen remote bookmarks
417 -
tags/<name>                      tags (immutable once created)
419 +
tags/<name>                      tags (immutable once created; may be withdrawn)
418 420
policy                           the trust policy head (§14.5)
419 421
```
420 422
…
5 unchanged lines
426 428
State { version [0]: 1, heads [1]: [ChangeHash] }   // sorted, unique; may be empty
427 429
```
428 430
429 -
A repository holds any number of bookmarks. Each workspace has a **current bookmark** (`oxy bookmark switch NAME`, recorded per workspace) that its working copy sits on and that `commit`, `log`, `push`, and `pull` act on by default. Switching refuses uncommitted work unless told what to do with it: `--discard` drops it, and `--carry` takes it along when every path it touches is the same in NAME's state as in the working copy's base (so the same edits apply there), refusing and naming the paths otherwise. `oxy bookmark create feat && oxy bookmark switch feat --carry` moves work started on one bookmark onto a new one. `oxy bookmark create NAME [CHANGE..]` names a new state (default: the current working copy's base heads); given CHANGE.., it refuses when the current state has changes recorded before the latest of them that they do not contain, since a state holds its heads' dependency closure and not everything recorded before them, unless `--upto` (the current state's changes up to CHANGE.., as Git ancestry would have it) or `--only` (exactly CHANGE.. and their dependencies) says which is meant, and `bookmark switch` reports how many changes the new state lacks of the old; `set NAME CHANGE..` moves one, `delete NAME` removes it (its changes stay until unreachable, §21), `list` shows all with their heads and the remote-tracking difference, and `oxy merge NAME` unions the named bookmark's heads into the current one, which is the whole of merging in this model: no merge change, conflicts reified (§9); a working copy without changes follows the merge, whichever state it sat on. `oxy checkout NAME` materializes a bookmark without switching to it; with uncommitted work it refuses, naming each changed path and how it changed (a *removed* path is in the working copy's base but not on disk), unless `--discard` or `--carry` (as for switching) says what to do with the work. Bookmark names, then tag names, are resolved before change prefixes wherever a change is expected (a bookmark or tag names a change when its state has one head). For builds, `oxy describe` without `-m` names the working copy as `git describe` does: its base state's tag (`v1.2`), else the nearest tag whose state the base contains with the number of changes it adds and the newest head's ChangeId (`v1.2-3-8c5ea12a`), else the bookmark and that ChangeId (`develop-8c5ea12a`), with `-dirty` when the working copy has uncommitted changes; `--json` gives the parts.
431 +
A repository holds any number of bookmarks. Each workspace has a **current bookmark** (`oxy bookmark switch NAME`, recorded per workspace) that its working copy sits on and that `commit`, `log`, `push`, and `pull` act on by default. Switching refuses uncommitted work unless told what to do with it: `--discard` drops it, and `--carry` takes it along when every path it touches is the same in NAME's state as in the working copy's base (so the same edits apply there), refusing and naming the paths otherwise. `oxy bookmark create feat && oxy bookmark switch feat --carry` moves work started on one bookmark onto a new one. `oxy bookmark create NAME [CHANGE..]` names a new state (default: the current working copy's base heads); given CHANGE.., it refuses when the current state has changes recorded before the latest of them that they do not contain, since a state holds its heads' dependency closure and not everything recorded before them, unless `--upto` (the current state's changes up to CHANGE.., as Git ancestry would have it) or `--only` (exactly CHANGE.. and their dependencies) says which is meant, and `bookmark switch` reports how many changes the new state lacks of the old; `set NAME CHANGE..` moves one, `delete NAME` removes it (its changes stay until unreachable, §21), `list` shows all with their heads and the remote-tracking difference, and `oxy merge NAME` unions the named bookmark's heads into the current one (NAME is a bookmark here or a remote's tracking bookmark, `origin/develop`, as last fetched: oxygen#117), which is the whole of merging in this model: no merge change, conflicts reified (§9); a working copy without changes follows the merge, whichever state it sat on. `oxy checkout NAME` materializes a bookmark without switching to it; with uncommitted work it refuses, naming each changed path and how it changed (a *removed* path is in the working copy's base but not on disk), unless `--discard` or `--carry` (as for switching) says what to do with the work. Bookmark names, then tag names, are resolved before change prefixes wherever a change is expected (a bookmark or tag names a change when its state has one head). For builds, `oxy describe` without `-m` names the working copy as `git describe` does: its base state's tag (`v1.2`), else the nearest tag whose state the base contains with the number of changes it adds and the newest head's ChangeId (`v1.2-3-8c5ea12a`), else the bookmark and that ChangeId (`develop-8c5ea12a`), with `-dirty` when the working copy has uncommitted changes; `--json` gives the parts.
430 432
431 433
Every reference, attestation, view, and checkpoint therefore names a state by one ObjectRef however many heads it has. Multiple heads are normal in this model: every independent change is a head until something depends on it, so a busy bookmark may have dozens. That is not a conflict in the §9 sense; `oxy log` shows the heads as parallel, and a state object keeps the cost of naming them constant for everything that refers to the bookmark. The empty state, `State { heads: [] }`, is what a new bookmark points at.
432 434
433 435
### 10.2 Reference attestations (G2, G3)
434 436
435 -
Every published bookmark update is accompanied by an attestation (§14.4) of kind `ref` that binds the repository ID, the reference name, the previous state, the new state, and a per-reference monotonically increasing `serial`. Servers MUST store the latest attestation per reference and serve it with the reference. Clients MUST verify the attestation against policy before updating a `remotes/` reference and MUST reject an attestation whose `serial` is lower than one they have already recorded for that reference (rollback protection). Clients SHOULD warn when the newest attestation's `hlc.wall_ms` is older than `remote.max-age` (default 30 days) and the remote claims no update.
437 +
Every published bookmark update is accompanied by an attestation (§14.4) of kind `ref` that binds the repository ID, the reference name, the previous state, the new state, and a per-reference monotonically increasing `serial`. Servers MUST store the latest attestation per reference and serve it with the reference. Clients MUST verify the attestation against policy before updating a `remotes/` reference and MUST reject an attestation whose `serial` is lower than one they have already recorded for that reference (rollback protection). Clients SHOULD warn when the newest attestation's `hlc.wall_ms` is older than `remote.max-age` (default 30 days) and the remote claims no update. A fetched bookmark whose attestation cannot be verified because its signer's identity is unknown to the client or its signature does not verify is skipped, not fatal, when it is not the current bookmark: its `remotes/` reference stays as it was, the client warns naming it and why, and the other bookmarks are fetched (oxygenhub#266). For the current bookmark (the one a `pull` takes, or a `clone` checks out) the fetch, pull or clone still fails, and an attestation that is missing under a policy, unauthorized, rolled back or not describing the update fails the fetch as before.
438 +
439 +
Tags are attestations of kind `tag` and are never updated; a server MUST reject a second attestation for an existing tag name, and any receiver MUST refuse a tag whose name it already holds with a different state. `oxy tag create NAME [CHANGE..]` tags the named state (default the bookmark's heads); tags travel with `refs` and `push` (§16.3), and a policy rule for `tags/<name>` governs who may create them. A tag signed by a short-lived key (§14.7), as a keyless agent's is, is authorized by a subject grant like a bookmark update: on a push at the receiver's clock, on a fetch at a time its signer did not choose (§14.7, *Acceptance*). The push carries the tag signer's identity and certificate as it does the bookmark signer's, since the tag may have been made by an earlier process's key (oxygen#129).
436 440
437 -
Tags are attestations of kind `tag` and are never updated; a server MUST reject a second attestation for an existing tag name, and any receiver MUST refuse a tag whose name it already holds with a different state. `oxy tag create NAME [CHANGE..]` tags the named state (default the bookmark's heads); tags travel with `refs` and `push` (§16.3), and a policy rule for `tags/<name>` governs who may create them.
441 +
**Withdrawing a tag** (oxygenhub#301). A tag is never moved, but it can be withdrawn: `oxy tag delete NAME [-r REASON]` records an attestation of kind `untag`, `{0: name, 1: state, 2: reason}`, naming the tag's state and signed by the repository's key, and the tag leaves the view; its `tag` attestation stays beside the withdrawal. The next `oxy push` sends the withdrawal to a remote that lists the tag at that state (`oxy push REMOTE --delete tags/NAME` withdraws it here and there at once), in the push's tags as a tag whose attestation is the withdrawal. A receiver adopts a withdrawal only of a tag it holds at the state named; under a policy the signer MUST be a key the rule for `tags/<name>` authorizes to create the tag (a listed key, its delegate, a rotation predecessor, or a short-lived key under a subject grant, at the receiver's clock on a push and at its issuer's acceptance on a fetch), and an unsigned withdrawal is refused. A server lists the tags withdrawn there in its `refs` answer (`withdrawn`), and a fetch withdraws a tag this repository holds at the same state when the withdrawal verifies, noting it, and otherwise keeps the tag, saying why. A withdrawn name is never used again: `tag create`, a push and a fetch refuse or skip a tag of that name, so a tag name never names two states. A host MAY refuse to withdraw a tag it depends on (the reference server's `Backend::vet_tag_withdrawal`; a forge refuses while a release names the tag).
438 442
439 443
## 11. Checkpoints
440 444
…
32 unchanged lines
473 477
  hlc     [2]: HLC
474 478
  command [3]: text                   // e.g. "oxy rebase -s k7m2p9 -d main"
475 479
  view    [4]: { bookmarks: map<name, state ObjectRef>, remotes: map, tags: map, workspaces: map<WorkspaceId, ChangeHash> }
480 +
  workspace [5]: WorkspaceId          // optional: the workspace whose command recorded it
481 +
  undoes  [6]: ObjectRef              // optional: the operation an in-place undo reverted
476 482
}
477 483
```
478 484
479 485
`view` is the complete state of all references and workspaces after the operation. The operation log is a DAG. The head (`.oxy/ops/head`) moves by compare-and-swap under a short lock (`.oxy/ops/lock`), held only while the pointer is written: a process that finds the head is no longer the parent it built on records a merge operation with both as parents, whose view is the three-way merge of the two views against that parent. A reference changed on one side takes that side's value; changed differently on both, a bookmark keeps both sides' heads (it then has several), a change beats a deletion, a tag keeps its first value, remote references and the policy take the later process's value, and a workspace changed by both is an error: the later operation is not recorded, since the files on disk match only one of the two working-copy changes. `oxy undo` refuses when the head moved while it ran.
480 486
481 -
`oxy undo` moves the current operation to its parent's view. `oxy redo` and `oxy op log` expose the DAG. Operations are retained for `core.op-retention` (default 90 days) and then eligible for garbage collection.
487 +
`oxy undo` undoes **this workspace's** last command (oxygen#99): the newest operation recorded by this workspace (for operations older than the `workspace` field, one that moved only other workspaces' working copies is theirs), passing over working-copy snapshots, pushes (the remote keeps what it was sent) and commands an earlier undo reverted, and taking a command's own `(checkout)` with it. When no other workspace recorded an operation since, the head moves back to the command's parent. Otherwise the command is reverted in place as a new operation naming it in `undoes`: each reference it changed returns to its value before it; a bookmark another workspace moved since is put back head by head (the heads the command added go, those it replaced return, the other workspace's stay); and a reference that cannot be put back so (a later change builds on one the command added, or another workspace changed the same tag, remote reference, policy or working copy) refuses the undo, naming it. Undo reports the command it undid, what that changed, the other workspaces' operations it kept, and bookmarks whose undone changes a remote already holds. `oxy redo` and `oxy op log` expose the DAG. Operations are retained for `core.op-retention` (default 90 days) and then eligible for garbage collection.
488 +
489 +
A process reads the operation head once, when it first needs it, and answers every later read of references, bookmarks and the view from that operation (oxygenhub#249): one command, or one request a host serves, never mixes two heads, and never waits on another process's operation to read. Its own operations move it to the head each one recorded. It reads the head again only where it must see what others did meanwhile: once it holds the working-copy lock (whoever held the lock may have moved the head), when a host receives a push (the updates are checked against the head as it is now), in `oxy fsck`, and wherever the implementation asks for it explicitly (`Repo::refresh`). An operation recorded from an older head merges with the newer one as any concurrent operation does.
482 490
483 491
Because all mutation goes through the operation log and object writes are idempotent, no long-held repository lock is needed: the operation head's lock covers one pointer write. Each workspace has a **working-copy lock** (`.oxy/workspaces/<id>/lock`), held by every command that reads or writes the working copy and replaces its working-copy change (snapshot, checkout, bookmark switch, merge, commit, amend and rewrites, discard, pull), from reading the operation head until its operation is recorded. A snapshot therefore never sees a half-written checkout, and never records one as edits. The lock is re-entrant within a process thread; a waiting command gives up after a minute and says why; a lock whose process is gone is taken over, and where that cannot be told, one older than a minute.
484 492
…
6 unchanged lines
491 499
The working copy is a real change with `kind = working-copy`. Before running any command that reads repository state, `oxy` snapshots the working copy: it diffs the file system against the working-copy change's materialized state and rewrites the working-copy change to include the difference. The change therefore always describes the working directory. To prevent operation log bloat, a command that snapshots the working copy MUST NOT record a new Operation (§12.1) if the filesystem diff was empty (i.e., the working-copy change's hash did not change).
492 500
493 501
- The working-copy change is unsigned and never pushed while `kind = working-copy`.
502 +
- A **new, untracked file larger than `snapshot.max-new-file-size`** (default 100 MiB; 0 for no limit) is left out of the snapshot and named by `oxy status` (and `status --json` as `skipped_large`), with how to ignore it or raise the limit (oxygen#98): a build's output or a downloaded image is not stored unseen by the next command. A tracked file, or one already in the working copy, is never left out for its size.
494 503
- `oxy commit` (`-m MSG`, or `-F FILE` with `-` for standard input) sets the message, sets `kind = normal`, and starts a fresh working-copy change on top of it. The change is signed immediately if `core.sign = "change"`; with the default `core.sign = "seal"` it is left unsigned locally and covered by a seal at push time (§14.4).
495 504
- `oxy new`, `oxy edit <change>`, and `oxy abandon` move which change the workspace edits.
496 505
- There is no staging area and no stash. A partial commit is a split of the working copy: `oxy commit PATH..` records only the named paths (and every edit of theirs) and leaves the rest as uncommitted work; `oxy commit --interactive` offers each hunk (one `Insert` or `Delete` edit on one file, or one tree operation) for inclusion, like `git add -p`; `oxy split` does the same to an existing change. A hunk that another selected hunk refers to is selected with it, so the committed change is always well-formed (§8.3).
506 +
- A **resolution** (a working copy that resolves its base's conflicts, §9.6) may be committed by paths too, so other work need not be put aside (oxygen#118): the named paths must include every file it resolves, whole, or the commit is refused naming them; what is left stays uncommitted on top of the resolved state. `oxy amend PATH..` folds only those paths into the head (committed as a change on it, then squashed into it, §8.5), leaving the rest uncommitted; `-m` replaces the head's message as usual.
497 507
498 508
### 12.4 Snapshot performance
499 509
…
14 unchanged lines
514 524
  objects/
515 525
    pack-<id>.pack    append-only pack file
516 526
    pack-<id>.idx     sorted (ObjectRef → offset, length, type) index
527 +
    pack-<id>.dict    the pack's zstd dictionary, when a full repack trained one
528 +
    pack-retired      packs, loose objects and entry lists a repack or fold replaced, kept for readers (§13.4)
529 +
    pack-entries      journal of signatures and index entries written since the last fold (§13.7)
530 +
    pack-<id>.entries       an entry list: every signature and index entry, as a fold found them (§13.7)
531 +
    pack-<id>.entries-part  an entry list of journal lines a writer folded (§13.7)
532 +
    pack-legacy-folded      written once a collection has folded the older one-list keys (§13.7)
517 533
    loose/            objects written since the last pack, as header || body
518 -
    sigs/             signature lists keyed by the ObjectRef they cover (§5.4)
534 +
    nocompress/       chunks packs keep raw (§13.3)
535 +
    sig/              one signature per signer, keyed by the ObjectRef it covers (§5.4, §13.7)
536 +
    seal/ receipt/ approval/ cert/ accepted/
537 +
                      index entries, one key each, the value in the key (§13.7)
519 538
  ops/                operation objects and op heads
520 539
  refs/               current view cache (derived from ops; rebuildable)
521 540
  workspaces/<id>/    snapshot cache, fsmonitor cookie
541 +
  index/              derived caches, rebuilt when missing or damaged: each
542 +
                      workspace's state indexes, and the nodes each change touches
522 543
  policy/             cached policy and identities
523 544
```
524 545
…
21 unchanged lines
546 567
547 568
### 13.4 Packing
548 569
549 -
An automatic trigger when the loose object count exceeds 4096 writes loose objects into a new pack and merges packs when there are more than 16; `oxy gc` repacks every surviving object into one pack, compressed harder (zstd level 9, against level 3 for everyday writes) and on every core, as `git gc` repacks. Writers SHOULD batch: a snapshot, a fetched bundle, a Git import, or a checkpoint is one write batch, and a batch that grows beyond a small threshold (256 objects or 8 MiB) is written as a single pack with one `fsync` rather than as loose objects, each with its own; smaller batches stay loose so ordinary commits do not multiply packs. Objects of an open batch MUST be readable by the writer before the batch completes. Pack indexes MUST support lookup by ObjectRef prefix and MUST be memory-mappable. A repository with two million objects MUST be openable with under 64 MiB of resident index memory on the Full profile.
570 +
An automatic trigger when the loose object count exceeds 4096 writes loose objects into a new pack and merges packs when there are more than 16; `oxy gc` repacks every surviving object into one pack, compressed harder (zstd level 9, against level 3 for everyday writes) and on every core, as `git gc` repacks. A store that is already one such pack (it has a dictionary) with nothing loose is left as it is: repacking it would write the same pack. Writers SHOULD batch: a snapshot, a fetched bundle, a Git import, or a checkpoint is one write batch, and a batch that grows beyond a small threshold (256 objects or 8 MiB) is written as a single pack with one `fsync` rather than as loose objects, each with its own; smaller batches stay loose so ordinary commits do not multiply packs. Objects of an open batch MUST be readable by the writer before the batch completes. Pack indexes MUST support lookup by ObjectRef prefix and MUST be memory-mappable. A repository with two million objects MUST be openable with under 64 MiB of resident index memory on the Full profile.
571 +
572 +
Readers never wait for writers (oxygenhub#249). A store loads its list of packs when it opens and keeps it; a read that misses is not believed at once. When the pack an object was found in is gone, or its dictionary is, the list is loaded again and the read tried once more; when the object is in no pack and not loose, the store lists its packs, and if the listing differs from the one it loaded from, loads the new packs and tries once more. Only then is the object missing (`has` answers false). A pack whose index is gone between the listing and the read, as when a collection deletes it meanwhile, is passed over and the store listed once more, so opening a repository never fails for it. A writer asking whether it already holds many objects lists the packs at most once a second while nothing new turns up.
573 +
574 +
A repack does not delete what it replaces. The packs it folded in and the loose objects it packed are **retired**: recorded, with the time, in `pack-retired`, a reference record of lines `<wall ms> <key>` updated by compare-and-swap, so concurrent repacks and collections never lose each other's lines. A store opening, or loading its packs again, does not load a retired pack, and retired loose objects are not counted or packed again as loose; a reader that loaded its packs before the repack reads on from the old files. Each `oxy gc`, and each automatic pack check after a commit, first deletes what was retired at least an hour ago: each such pack with its index and dictionary, and each such loose object a live pack holds. A retired loose object no live pack holds (its pack copy was pruned since) is kept as an ordinary loose object. Readers that loaded their packs before the files went find the objects in the new pack by the reload above. The hour is independent of `core.gc-grace` (§21): `oxy gc --now` waives the grace period for unreachable objects, not the hour readers have.
550 575
551 576
The pack and index formats are fixed so that tools can read them without the implementation:
552 577
…
13 unchanged lines
566 591
567 592
Every update that must survive a crash replaces a file atomically: write a uniquely named temporary file in the same directory, `fsync` it, rename it over the target, and `fsync` the directory. Loose objects and packs are written the same way, a pack before its index. The operation head (`.oxy/ops/head`) is the commit point of every command: before it moves, every object the new operation names MUST be durable, including objects still held by an open write batch (§13.4), which is flushed first. A crash therefore leaves the repository at the operation before or after the interrupted command, never between; stray temporary files may remain and are ignored. The reference implementation's tests stop `commit` and `push` at each durable step and require both repositories to open, pass `oxy fsck` (§21.1) and accept the same command again.
568 593
594 +
### 13.7 Signatures and index entries
595 +
596 +
Signatures (§5.4) and the indexes kept beside the objects are stored one key per entry, with the value in the key where there is no other. `sig/<hh>/<rest>/<signer>` holds one signer's signature over the object `<hh><rest>`; a new signature by the same signer replaces it. `seal/`, `receipt/`, `approval/`, `cert/` and `accepted/` hold empty `<hh>/<rest>/<value>` keys: the seal attestations covering a change, the receipts and approvals on a state, the certificate kept for a short-lived key, and a reference's accepted attestation. Each entry is its own write-once key, so writers sharing a store never lose each other's entries. Stores written before this layout kept one list per object (`sigs/`, `seals/`, `receipts/`, `approvals/`, `certs/`, `accepted-lists/`); readers still read these until a collection folds them (below).
597 +
598 +
To find an object's entries in this layout, a reader lists its prefix. On a directory that is one directory read. On a bucket it is one LIST request per lookup, and most find nothing, because most objects are unsigned (oxygen#116). So a store that is not this disk's directory also keeps its entries as lists that a reader reads once per open. Neither changes the per-object keys, which every writer still writes first, so older readers keep working.
599 +
600 +
- **The journal**, `pack-entries`: a reference record, updated by compare-and-swap as `pack-retired` is, of lines `<folded at, wall ms; 0 until folded> <entry key>`. After writing an entry's own key, a writer adds the key to the journal. A write batch (§13.4) adds all of its entries in one update when it is flushed, before the operation head moves. A writer that loses the race reads the record again and retries (up to 32 times), so concurrent writers never lose each other's lines.
601 +
- **Entry lists**: zstd-compressed UTF-8 text, written once: the line `oxy-entries 1`, then one entry key per line, sorted. `pack-<id>.entries` is **complete**: it holds every entry the store held when it was written. `pack-<id>.entries-part` is **partial**: it holds the journal lines a writer folded out. The lists are named under `pack-` so that the pack listing every open makes already finds them. They are not tied to a pack: a repack by an implementation that does not know them cannot hide them, and a collection with nothing to repack still folds.
602 +
- **Folding.** `oxy gc` (and a forge's collection) folds **completely**. It reads the journal, lists every entry prefix, writes a complete list of both, retires every list it read (in `pack-retired`, like a pack: a collection deletes it an hour later), and marks the journal lines it read as folded. When nothing changed since the last complete list, no list is written. A collection that deletes an object deletes its entry keys and their journal lines too, and the next complete list leaves them out. A writer whose update leaves more than 1000 unfolded lines folds them into a partial list and marks them. The next update after a folded line's hour (§13.4) drops it. Until then, a reader whose pack listing predates the fold still finds every entry, in the old lists or in the journal. Lines added while a fold runs are not marked, so they stay.
603 +
- **Reading.** A store is **folded** when its live lists (listed and not retired) include a complete one. On its first lookup, a handle on a folded store reads every live list and the journal, folded lines included, and answers every lookup from that set. It adds what it writes itself, and it never lists the per-object prefixes or reads the older lists. A signature's bytes are still read from its own key, which a cache may keep, since that key is written once. A handle reads them again when it loads its packs again (§13.4) or is refreshed, and on the first lookup more than 30 seconds after it last read them. At that point it lists its packs again, reads only the lists that are new, and reads the journal. 30 seconds is as long as a cloud client keeps the pack listing, so another writer's entry is seen within that interval, as another writer's pack is, even by a handle a server keeps open across requests. On a store that is not folded (no collection has folded it since writers kept a journal, or another implementation's collection retired its lists), a lookup lists the per-object prefix as before.
604 +
- **Older lists.** Before listing the entry prefixes, a complete fold copies every older one-list key's contents into per-object entries, keeping an entry already there. It then writes the marker `pack-legacy-folded` once. A reader whose pack listing shows the marker does not read the older lists. Nothing writes them any more.
605 +
606 +
**Older writers.** Writers that predate the journal (oxy 0.4.6 and before) write only the per-object keys. Readers of a folded store miss what such a writer adds until the next complete fold picks it up. A store becomes folded at its first complete fold. So the operator of a repository kept in a bucket first upgrades every writer to it (clients with `storage.url`, runners, the forge), then runs `oxy gc` there, or lets the forge's collection run. After that, a lookup costs no request apart from reading a signature's own key. A forge that collects nightly picks up an older writer's entries within a day. A repository's own directory (`.oxy/objects`) keeps no journal or lists: reading a directory costs no request, so its lookups list the per-object prefixes as before.
607 +
569 608
## 14. Identity, signatures, and trust policy
570 609
571 610
### 14.1 Algorithms
…
32 unchanged lines
604 643
605 644
The two public keys live in one object, so a `KeyId` names the pair. `delegates` lists keys that may act for this identity, each within a **scope**: `changes` lets the key sign changes whose `author` is this identity, `seal` lets it seal them, and `refs` lets it attest references and tags as this identity would (§14.5). A proxy acting for a Thin client (§22) or a host that commits browser edits for a user gets `changes,refs`; an AI agent gets `changes` alone, so its signature authenticates the changes it makes but it can never move a reference, whatever its owner may do. A delegation with `expires` authenticates nothing whose `hlc.wall_ms` is later than that, so an agent can be let in for a session without changing the owner's own expiry. Delegation is one level deep and is not transitive. A bare `KeyId` in `delegates`, as identities carried before scopes existed, means every scope with no expiry. Rotation is a new identity with `previous` set, plus an attestation of kind `rotate` signed by the previous key. Revocation is an attestation of kind `revoke` signed by the revoked key, by the identity that delegated to it, or by a policy root key (§14.5); the delegator's revocation marks the changes the key already signed as valid but revoked, and the delegator then re-issues their identity without it.
606 645
607 -
**Lifecycle rules.** Because the `KeyId` is the hash of the whole object, any change to an identity, including its `delegates` or `expires`, is a rotation: a successor object with `previous` set. A successor MAY keep the same key pair (`oxy identity renew`, `oxy identity delegate add|remove`) or carry a fresh one (`oxy identity rotate`); either way earlier signatures remain verifiable under the earlier object, which stays stored. A rotation **supersedes** the previous KeyId: a policy that named it MUST be updated to name the successor (§14.5), and a rule key covers every predecessor in its `previous` chain that is not revoked, so changes signed before a rotation remain authorized. Revocation is for compromised or abandoned keys: a revoked key authorizes nothing, whatever its chain. `expires` bounds the changes a key can authenticate: a signature or seal over a change whose `hlc.wall_ms` is later than the identity's `expires` is invalid. A client records the revocations it makes or verifies under `.oxy/revoked/` and reports signatures by such keys as valid but revoked.
646 +
**Lifecycle rules.** Because the `KeyId` is the hash of the whole object, any change to an identity, including its `delegates` or `expires`, is a rotation: a successor object with `previous` set. A successor MAY keep the same key pair (`oxy identity renew`, `oxy identity delegate add|remove`) or carry a fresh one (`oxy identity rotate`); either way earlier signatures remain verifiable under the earlier object, which stays stored. A rotation **supersedes** the previous KeyId: a policy that named it MUST be updated to name the successor (§14.5), and a rule key covers every predecessor in its `previous` chain that is not revoked, so changes signed before a rotation remain authorized. A tag's attestation (§10) signed by such a predecessor is authorized only when its `hlc.wall_ms` is earlier than the `created` of the identity that succeeded the signer, so tags made before a rotation stay valid and a superseded key makes no new ones. Revocation is for compromised or abandoned keys: a revoked key authorizes nothing, whatever its chain. `expires` bounds the changes a key can authenticate: a signature or seal over a change whose `hlc.wall_ms` is later than the identity's `expires` is invalid. A client records the revocations it makes or verifies under `.oxy/revoked/` and reports signatures by such keys as valid but revoked.
608 647
609 648
### 14.3 Composite signature (G2, G4)
610 649
…
5 unchanged lines
616 655
  ml   [3]: bytes(3309)
617 656
}
618 657
message = "oxy1 sig " || domain || 0x00 || KeyId || ObjectRef-being-signed
619 -
domain  = "change" | "attest" | "policy"
658 +
domain  = "change" | "attest" | "policy" | "certificate" | "grant" | "checkpoint"
620 659
```
621 660
622 661
Both signatures are computed over the same `message`, which includes the `KeyId` of the composite pair. Verification MUST require both signatures to verify with the keys of the identity named by `key`. A signature with either half missing, empty, or failing is invalid. This is the same construction as the IETF LAMPS composite signature drafts and prevents stripping the post-quantum half.
…
7 unchanged lines
630 669
```
631 670
Attest {
632 671
  version [0]: 1
633 -
  kind    [1]: "ref" | "tag" | "seal" | "rotate" | "revoke" | "lock" | "receipt"
672 +
  kind    [1]: "ref" | "tag" | "seal" | "rotate" | "revoke" | "lock" | "receipt" | "approval" | "accepted"
634 673
  repo    [2]: bytes(16)               // repository ID, chosen at init
635 674
  hlc     [3]: HLC
636 675
  body    [4]: map                     // per kind
…
5 unchanged lines
642 681
revoke: { key: KeyId, reason }
643 682
lock:   { node: NodeId, path, acquire: bool, expires }
644 683
receipt: { receipt: ObjectRef, state: ObjectRef, run: text }   // §14.6
684 +
approval: { run: text, job: text, job_hash: ObjectRef, state: ObjectRef,
685 +
            inputs: [ObjectRef], environment: text, approve: 0 | 1 }  // sorted, unique inputs
686 +
accepted: { attest: ObjectRef }        // §14.7
645 687
```
646 688
689 +
An **approval** is a person's decision about one job of one OxygenFlow run (an approval gate, OxygenFlow spec §5.3.2): the job as hashed with its sandbox, on that state, taking exactly those input blobs (for a deploy, the reviewed plan), for that environment. It approves nothing else: another run, state, plan or environment needs another approval. It is handed to the hub as `[attest bytes, [Signature], identity bytes]`, signed in domain `attest` by the approver's own key (`oxy seal --flow URL --job NAME`, `--reject` to reject), so it verifies with nothing else; whether the signer may approve is the hub's rule (a key linked to an account with the gate's role; delegated keys only where the repository allows). The hub keeps it in the repository, so it can be verified without the hub.
690 +
647 691
A **seal** is one signature over a batch of change hashes. A composite signature is 3373 bytes, which is larger than a typical change, so signing every change individually roughly doubles the size of history in high-frequency workflows. A seal replaces that with one signature per push.
648 692
649 693
**Cadence.** With the default `core.sign = "seal"`, changes are left unsigned locally and `oxy push` creates one seal covering every change in the push that carries no individual signature, immediately before the reference attestation. Seals are created only at push: a seal made earlier would be invalidated by any amend, since rewriting produces a new hash. With `core.sign = "change"`, every change is signed at commit and no seal is needed. Policy (§14.5) decides which the receiving repository accepts.
650 694
651 -
**Verification.** A sealed change `C` is authenticated iff a seal `S` exists such that `hash(C) ∈ S.body.hashes`, `S` carries a valid signature from key `K`, and `K` equals `C.author` or is listed in `C.author`'s identity as a delegate with the `seal` scope, not expired at `C.hlc.wall_ms` (§14.2). The same author-or-delegate rule applies to individual change signatures, with the `changes` scope. This rule is what stops a collaborator from sealing a change whose `author` field names someone else (§3.1). A change whose `author` is the zero KeyId names nobody (history imported from Git, §21, or made before an identity existed), so a seal from any key authenticates it: the sealer **adopts** it, and policy authorizes the sealer for the reference as usual. A push seals the pusher's own unauthenticated changes and adopts anonymous ones in the same seal. A seal's signer must additionally be authorized by policy for the reference being pushed.
695 +
**Verification.** A sealed change `C` is authenticated iff a seal `S` exists such that `hash(C) ∈ S.body.hashes`, `S` carries a valid signature from key `K`, and `K` equals `C.author` or is listed in `C.author`'s identity as a delegate with the `seal` scope, not expired at `C.hlc.wall_ms` (§14.2). The same author-or-delegate rule applies to individual change signatures, with the `changes` scope. This rule is what stops a collaborator from sealing a change whose `author` field names someone else (§3.1). A change whose `author` is the zero KeyId names nobody (history imported from Git, §21, or made before an identity existed), so a seal from any key authenticates it: the sealer **adopts** it, and policy authorizes the sealer for the reference as usual. A push seals the pusher's own unauthenticated changes and adopts anonymous ones in the same seal; a pusher that signs for its configured author (`author.signer`: a delegate, or a short-lived key under the author's grant, §14.7) seals the author's as well, and the receiver checks it may. A seal's signer must additionally be authorized by policy for the reference being pushed.
652 696
653 697
**Handling.** A seal is stored and transported with the changes it covers: `fetch` (§16.1) streams the seals covering every change it returns, `push` (§16.4) MUST include a seal or signature for every change, and a seal covering any reachable change is itself reachable (§21). Seals from different keys MAY overlap; a change needs at least one valid authenticator, and policy thresholds count distinct authenticating keys.
654 698
…
8 unchanged lines
663 707
  serial    [2]: u64
664 708
  roots     [3]: { keys: [KeyId], threshold: u32 }
665 709
  rules     [4]: [ { pattern: glob over reference names, keys: [KeyId], threshold: u32,
666 -
                     require: "signed" | "sealed" | "any" /* default */, allow_conflicts: bool } ]
710 +
                     require: "signed" | "sealed" | "any" /* default */, allow_conflicts: bool,
711 +
                     subjects [5]: [ { pattern: glob over subjects, issuer: KeyId, scope: [text] } ] /* only when any */ } ]
667 712
  identities[5]: [Identity]           // embedded so verification needs no fetch
668 713
  revoked   [6]: [KeyId]
669 714
  expires   [7]: u64
…
2 unchanged lines
672 717
673 718
- The initial policy is signed by the root keys; every update MUST be signed by `roots.threshold` root keys of the policy it replaces and have a higher `serial`. An update that a single holder cannot complete is a **proposal**: the signed policy (`[policy bytes, [Signature]]`, §16.3) is passed to other root holders, each of whom adds a signature with `oxy policy sign`; whoever completes the threshold applies it. Rotating a root key (§14.2) substitutes the successor in the roots and rules; such a policy is signed by both the previous and the successor key so a fresh clone can bootstrap from it.
674 719
- A change's signer is authorized by a rule if the rule lists it, or lists a successor of it, or lists the change's `author` (or a successor) when the signer is that author's delegate.
675 -
- A reference attestation's signer is authorized by a rule if the rule lists it, or lists an identity whose `delegates` name it with the `refs` scope, not expired at the attestation's `hlc.wall_ms` (§14.2). Delegation is the identity's own signed statement and the policy names that identity, so a delegate with `refs` moves references exactly as far as its delegator may; revoking either key, or the delegation expiring, ends it. Hosts use this to land and to commit browser edits on a user's behalf with their own key. A delegate without `refs` (an agent) is never authorized for a reference or tag, even when its delegator is.
720 +
- A reference attestation's signer is authorized by a rule if the rule lists it, or lists an identity whose `delegates` name it with the `refs` scope, not expired at the attestation's `hlc.wall_ms` (§14.2), or, on a fetch, at its integrated time when the host's log gives one (§14.8). Delegation is the identity's own signed statement and the policy names that identity, so a delegate with `refs` moves references exactly as far as its delegator may; revoking either key, or the delegation expiring, ends it. Hosts use this to land and to commit browser edits on a user's behalf with their own key. A delegate without `refs` (an agent) is never authorized for a reference or tag, even when its delegator is.
721 +
- A rule may also grant **subjects an issuer certifies** (§14.7): `oxy policy grant-subject SUBJECT --issuer IDENTITY --scope SCOPES` adds `{pattern, issuer, scope}` and embeds the issuer's identity. A signer is then authorized when it is a short-lived key whose certificate verifies with that issuer, names a subject the pattern matches, gives the scope needed (`changes` for changes, `refs` for reference attestations and tags) and covers a time the signer did not choose; it counts as one signer toward the threshold. A rule with subjects only has no keys, and its threshold counts keys and subjects together. The time is the receiver's own clock when it receives a push (changes and the reference attestation), and, on a fetch, the attestation's integrated time when the host's log gives one (§14.8, *Integrated time*); without one, the time the certificate's issuer says it accepted the update (§14.7, *Acceptance*), or the attestation's own time when there is no such acceptance, which is weaker: a key whose certificate expired could backdate an attestation within its scope. A tag is never checked at its own time: a fetch without an integrated time or an acceptance for it leaves the tag out and says why, and fetches the rest (oxygen#129). A policy without subjects encodes as before (field 5 is left out), so its hash is unchanged.
676 722
- A change or attestation is **authorized** for reference `r` if its signer is listed in the first rule whose pattern matches `r`, and the rule's threshold of distinct authorized signers is met across the attestation and each change in the closure of `new_state` not previously in the closure of `old_state`. `require` selects which authenticators count for changes: individual signatures only, seals only, or either. Every change MUST additionally satisfy the author-or-delegate rule of §14.4 regardless of `require`.
677 723
- Clients MUST verify policy on fetch, MUST refuse to update a `remotes/` reference whose attestation is unauthorized, and MUST warn loudly on an expired policy. Servers MUST enforce the same rules on push.
678 724
- A repository without a `policy` reference is **unverified**; every command that talks to a remote MUST print a one-line warning until one is created. `oxy policy init` creates one with the author's key as the sole root; `oxy policy grant` and `oxy policy revoke` produce successors.
…
21 unchanged lines
700 746
701 747
A receipt names no state: the same job on the same scope in the same environment gives the same receipt, which is what lets a runner reuse it for a later state whose scope is unchanged. An attestation of kind `receipt` (§14.4) binds a receipt to the `state` it ran on and names the `run` (usually its URL). The runner signs that attestation with its own identity (§14.2), an ordinary composite key, in domain `attest`.
702 748
703 -
**Signed receipt.** A runner hands a receipt over as `[receipt bytes, attest bytes, [Signature], identity bytes]`, so it verifies with nothing else: the attestation's `receipt` is the receipt's ObjectRef, and a signature by the included identity verifies over the attestation. Whether a runner is trusted is the reader's decision (a host lists its runners' keys); a revoked runner key is reported.
749 +
**Signed receipt.** A runner hands a receipt over as `[receipt bytes, attest bytes, [Signature], identity bytes]`, so it verifies with nothing else: the attestation's `receipt` is the receipt's ObjectRef, and a signature by the included identity verifies over the attestation. Whether a runner is trusted is the reader's decision (a host lists its runners' keys); a revoked runner key is reported. A runner that signs with a short-lived key (§14.7) adds a fifth element, `[certificate bytes, issuer identity bytes]`: the receipt then verifies only if the certificate does (signed by the identity given, which its `issuer` names), certifies the key that signed, gives the `seal` scope and covers the attestation's time; a host checks the time it accepted the receipt as well, since the attestation's is the signer's claim. A repository keeps the certificate (type `certificate`, with its issuer's signature, indexed by the key it certifies) and the issuer's identity beside the receipt, and gc keeps them with it.
704 750
705 751
**Storage.** A repository keeps receipts beside the state they name: the receipt, the attestation with its signatures, and the runner's identity, indexed by state ObjectRef. Receipts live as long as the changes of their state, and keep their outputs and inputs alive (§21). `oxy receipt list|show|export|verify|import` reads and exchanges them; a receipt for another repository is refused.
706 752
753 +
**SLSA provenance.** For tools outside Oxygen, `oxy receipt export --format slsa` writes what a valid receipt says as an in-toto Statement v1 with an SLSA provenance v1 predicate (`buildType` `https://oxygenhub.io/oxygenflow/job/v1`), in a DSSE envelope (payload type `application/vnd.in-toto+json`). Its subjects are the job's outputs, with the digest `oxygen-blob` (their ObjectRefs); a job with none has the state as its subject (`oxygen-state`). Resolved dependencies are the state, the scope (`oxygen-scope`) and the inputs; the builder is `urn:oxygen:runner:<runner KeyId>`; `startedOn` is the attestation's wall time less `duration_ms`, `finishedOn` that time. The signed receipt itself is a byproduct (`oxygen-receipt`, media type `application/vnd.oxygen.receipt+cbor`, base64 `content`), so an Oxygen verifier can still check the runner's signature. The envelope is signed by the **exporter**, with the Ed25519 half of its signing key over DSSE's pre-authentication encoding (which cannot be mistaken for an Oxygen signature message, §14.3); `keyid` is the KeyId in hex. DSSE verifiers know no ML-DSA, so this signature is classical: it vouches that the exporter checked the receipt, and the receipt inside keeps the runner's composite signature. `oxy identity export --pem` writes the signing key's Ed25519 public key as a PEM SubjectPublicKeyInfo, as cosign's `--key` takes it. A host does the same with its own key for the artifacts it keeps (hub spec).
754 +
755 +
### 14.7 Certificates for short-lived keys
756 +
757 +
An issuer (a hub's key) may vouch for a fresh key for a few minutes, so a Flow job or an agent signs with a key that dies with it instead of one kept on disk (oxygenhub#241, oxygenhub docs/keyless.md):
758 +
759 +
```
760 +
Certificate {
761 +
  version     [0]: 1
762 +
  key         [1]: bytes          // the short-lived key's identity object (§14.2)
763 +
  issuer      [2]: KeyId          // the key that signs this certificate
764 +
  subject     [3]: text           // who proved itself: "repo:OWNER/NAME:bookmark:B", "agent:NAME@ACCOUNT"
765 +
  claims      [4]: [[text, text]] // what else the proof said, sorted by name
766 +
  scope       [5]: [text]         // changes, refs, seal
767 +
  not_before  [6]: u64            // wall milliseconds
768 +
  expires     [7]: u64
769 +
  grant       [8]: bytes          // optional: a signed grant (below), only when present
770 +
}
771 +
```
772 +
773 +
A signed certificate is `[certificate bytes, Signature]`: the issuer's composite signature, in domain `certificate` (§14.3), over the certificate's ObjectRef (type `certificate`). It is valid when the signature verifies with the identity its `issuer` names, `expires` is after `not_before` and at most an hour later, and its scope is not empty. A signature by the short-lived key names the key's KeyId as any signature does; it counts only within the certificate's scope and window, and the time it is checked at must be one the signer did not choose (a host's record of when it accepted the signature, or the transparency log's), never the signed object's own clock. Which issuers a repository trusts, and for which subjects, its trust policy says (§14.5).
774 +
775 +
**Grants.** An author lets the keys an issuer certifies for a subject act for it, as a delegate does but without a key of their own (an agent's fresh keys, oxygenhub docs/keyless.md):
776 +
777 +
```
778 +
Grant {
779 +
  version     [0]: 1
780 +
  grantor     [1]: KeyId          // the author, whose key signs the grant
781 +
  subject     [2]: text           // "agent:NAME@ACCOUNT"
782 +
  issuer      [3]: KeyId          // whose certificates for the subject count
783 +
  scope       [4]: [text]         // what certificates under it may give: changes, seal, refs
784 +
  not_before  [5]: u64            // wall milliseconds
785 +
  expires     [6]: u64
786 +
}
787 +
```
788 +
789 +
A signed grant is `[grant bytes, Signature]`: the grantor's composite signature, in domain `grant`, over the grant's ObjectRef (type `grant`). It needs no maximum lifetime: it gives nothing until an issuer certifies a key under it, and the grantor revokes it by asking the issuer to stop. The issuer puts it in each certificate it issues under it (field 8), so it travels with the certificate and a verifier needs nothing else. A signature by a key that is neither a change's author nor its delegate (§14.4) still satisfies the author-or-delegate rule when the key's certificate verifies with its issuer and carries a grant that verifies with the change's author (or with a key the author's identity was rotated or renewed from, by its `previous` chain, when the grant's `not_before` is before that step's `created`: an owner's rotation keeps the grants they made, but a key rotated away grants nothing new; oxygen#135), names the certificate's subject and issuer, gives the scope needed (`changes` for a change signature, `seal` for a seal) in both the grant and the certificate, and covers the change's `hlc.wall_ms`, as a delegation does. Whether the certificate covered the time the key signed is the policy's check (§14.5), at a time the signer did not choose.
790 +
791 +
**Acceptance.** A host that issues certificates records when it accepted a reference or tag attestation signed by a short-lived key: an `accepted` attestation (§14.4) naming that attestation's ObjectRef, its `hlc` the host's clock at the push, signed in domain `attest` by the issuer's key. The host keeps it indexed by the attestation it accepts (gc keeps it beside the bookmark's latest attestation or the tag's) and serves it with the bookmark (§16, `refs` entry key 5) or the tag (tag key 4). A fetch checks the certificate at the acceptance's time only when the acceptance names exactly the attestation being checked and verifies with the certificate's issuer; otherwise, including when there is none, at the attestation's own time for a bookmark, and not at all for a tag, which is left out (§14.5). Tags are kept for good and fetched long after a ten-minute certificate expired, so the issuer's acceptance travels with the tag: a client that fetched one keeps it and serves it with the tag, a push carries it, and a receiver checks a tag that comes with its issuer's acceptance at that time when its own clock is past the certificate's window (a tag passed on from the host that accepted it to another), keeping the acceptance rather than recording its own. An acceptance by any other key changes nothing: the issuer is already trusted to vouch for the key, so it adds no trust it did not have. A fetch that has the attestation's integrated time (§14.8) checks at that instead: the acceptance is the issuer's clock, the integrated time its witnesses'.
792 +
793 +
### 14.8 The transparency log
794 +
795 +
A host keeps an append-only log of what it accepts, so it cannot show one history to some readers and another to others (oxygenhub#238, oxygenhub docs/transparency.md). Each accepted signed object (a reference or tag attestation, a seal, a rotation or revocation, a receipt, an approval, a policy, an identity, a certificate, an acceptance) becomes one entry, in the order the host accepted it:
796 +
797 +
```
798 +
Entry {
799 +
  version   [0]: 1
800 +
  kind      [1]: text           // "ref", "seal", "tag", "rotate", "revoke", "receipt", "approval",
801 +
                                //  "policy", "identity", "certificate", "accepted"
802 +
  repo      [2]: bytes(16)      // the repository's id; zero for identities
803 +
  object    [3]: ObjectRef      // the attestation, policy, identity or certificate
804 +
  signers   [4]: [KeyId]        // the keys whose signatures the host checked
805 +
  received  [5]: u64            // the host's clock, milliseconds (its claim)
806 +
}
807 +
```
808 +
809 +
**The tree** is RFC 6962's Merkle tree over the entries, with BLAKE3: a leaf's hash is `BLAKE3("oxygen-tlog-leaf\0" || cbor(Entry))`, an interior node's `BLAKE3("oxygen-tlog-node\0" || left || right)`, and the empty tree's root `BLAKE3("")`. A tree of n > 1 leaves splits at k, the largest power of two below n. Inclusion and consistency proofs are RFC 6962's (§2.1.1, §2.1.2), their hashes listed from the leaf upwards.
810 +
811 +
**Stored order.** Every hash of the tree is stored once, as soon as it is known: adding leaf n stores its hash and then the hash of each subtree it completes (one per trailing 1 bit of n). The hash of tree level L at position n is therefore at stored index `i(L, n)`, where n is first replaced L times by 2n+1, then i = n + ⌊n/2⌋ + ⌊n/4⌋ + … + L. The store is append-only.
812 +
813 +
**Tiles** (C2SP tlog-tiles, height 8). Level-L tile N holds the hashes of tree level 8L for leaves from N·256 on, 32 bytes each: 256 of them when full, fewer in a partial tile. Its path is `tile/<L>/<N>`, with `.p/<W>` appended for a partial tile of W hashes; N is written in groups of three digits, every group but the last prefixed `x` (1234067 is `x001/x234/067`). The entries of level-0 tile N are at `tile/entries/<N>[.p/<W>]`, each as a big-endian u16 length and its bytes. Full tiles never change.
814 +
815 +
**Checkpoints** (C2SP tlog-checkpoint) are C2SP signed notes. The text is the origin (the host's name and `/log`), the tree size in decimal and the root in base64, each line ending in a newline. It is signed twice under the origin's name:
816 +
- with the host key's Ed25519 half, as a standard note signature (type 0x01: key hash = SHA-256(name ‖ "\n" ‖ 0x01 ‖ public key)[:4], signature over the text), which note tools read;
817 +
- with the composite key, as type 0x4f (key hash over the identity object's bytes): the CBOR of a composite Signature in domain `checkpoint` over `BLAKE3("oxygen-tlog-note\0" || text)`.
818 +
819 +
A reader accepts a checkpoint only when both verify, so the log stays post-quantum. Which witnesses must cosign it is oxygenhub docs/transparency.md's.
820 +
821 +
**Witnesses** (C2SP tlog-witness) cosign a log's checkpoints with their own keys and clocks, so a log cannot show two histories without someone seeing both. A log sends a witness `POST add-checkpoint`: `old <size>` (of the last checkpoint that witness cosigned), the consistency proof's hashes in base64 one per line, a blank line, then the checkpoint note. The witness:
822 +
- answers 409 with its own size when `old` is not it;
823 +
- answers 403 when the note does not verify with the log's key;
824 +
- answers 422 when the checkpoint does not extend the one it cosigned;
825 +
- otherwise remembers the new size and root, and returns its cosignature lines, which the log adds to the checkpoint it serves.
826 +
827 +
A cosignature is two lines under the witness's name:
828 +
- C2SP cosignature/v1 (type 0x04): the witness's time in seconds as 8 big-endian bytes, then an Ed25519 signature over `cosignature/v1\ntime <seconds>\n` followed by the checkpoint's text;
829 +
- a composite line (type 0x4e): the same time, then the CBOR of a composite Signature in domain `checkpoint` over `BLAKE3("oxygen-tlog-cosig\0" || that message)`.
830 +
831 +
A reader counts a cosignature only when both lines verify and agree on the time.
832 +
833 +
**On fetch**, when a refs response carries a checkpoint (§16), a client:
834 +
- pins the host's log key on first sight (the hub's published identity), per user and hub, with the checkpoint;
835 +
- requires every later checkpoint to verify with that key, keep its origin, and extend the last one seen: a smaller tree, the same size with another root, or a consistency proof that fails (computed from the host's tiles) means the host showed two histories, and the fetch fails;
836 +
- checks each bookmark's attestation against its entry (kind `ref`, the attestation's ObjectRef and repository id) and inclusion proof, reporting one that is missing or does not verify. While logs are new this is a warning, not a refusal.
837 +
838 +
**Integrated time** (oxygenhub#240). A cosignature is a witness's statement, by its own clock, that the checkpoint existed then, so every entry the checkpoint holds existed by that time, whatever its signer or the host claims. An entry's **integrated time** is the earliest time a witness the reader trusts cosigned a checkpoint that holds it: an upper bound on when its object was signed, in seconds, by a clock neither the signer nor the host chose.
839 +
- **Witness keys.** A host lists the witnesses it asks to cosign, each as its name in cosignatures and its identity (refs response key 7, §16.3; and `GET /log/witnesses`, one `<name> <base64 identity>` line each). A client pins them per user and hub beside the log key (`<oxy home>/tlog/<hub>.witnesses`): the first list silently, as the log key on first sight; a witness listed later, or listed under a pinned name with another key, is pinned with a note, as a new log key is. A witness the host stops listing stays pinned. A cosignature counts only when both its lines verify with a pinned witness's key; lines by anyone else are ignored. Trust-on-first-use gives no more than the host's own word on first contact, which is what the acceptance gives today (§14.7); after it, a host that names a witness of its own to move a time earlier is noted by every client that pinned the real ones.
840 +
- **What a host serves.** With a bookmark's entry (ref key 6), the cosigned checkpoints that first held it (ref key 7): each a checkpoint note with the log's signatures and its witnesses' cosignatures, and the entry's inclusion proof against it, one for each witness when they first cosigned different checkpoints. A client takes one when its origin is the log's, it verifies with the pinned log key, the entry is the attestation's (as above) and the proof verifies; the integrated time is the earliest trusted cosignature among those it takes. A host can only make the time later by serving a later checkpoint, which weakens the bound and helps no one but a signer it would refuse; it cannot make it earlier without a witness's key.
841 +
- **Use.** On a fetch, a check that depends on when a reference or tag attestation was signed (whether a delegation with `refs` was in force, §14.5, or whether a certificate covered it, §14.7) is made at the integrated time when there is one. Without one (the host keeps no log, the entry is in no cosigned checkpoint yet, or no trusted witness cosigned it), it is made as before, at the issuer's acceptance or the attestation's own time (for a tag, at the acceptance only, §14.7), and the fetch says so when the host keeps a log. While logs are new (oxygenhub docs/transparency.md, decision 3), an attestation that fails at its integrated time but passes as before is accepted with a note rather than refused; one that fails both is refused, as before. A key the rule lists itself is authorized whenever it signed, and revocations are absolute (§14.5): neither depends on the time. A push is checked at the receiver's own clock, as before.
842 +
- **A large gap.** For an attestation whose check depends on the time, a claimed time (`hlc.wall_ms`) more than 10 minutes before its integrated time, or more than 10 minutes after it, is noted, not refused: the attestation was backdated, held back before it was logged (one a host accepted before its log began, logged when first served), or signed on a wrong clock. A reference attestation is made at push and logged and cosigned within seconds, so ten minutes leaves room for drift and for a witness that was briefly away.
843 +
844 +
**`oxy tlog verify [HUB]`** audits a whole log: it accepts the checkpoint as a fetch does, downloads every entry bundle, and requires the entries to make the root the checkpoint signs. It names the trusted witnesses that cosigned the checkpoint and when (pinning those `GET /log/witnesses` lists, as a fetch does), a bound on when every entry was made. It then lists this repository's entries (`--key KEYID` for a key's, `--all` for every one, `--json` for machines).
845 +
707 846
## 15. Transport
708 847
709 848
Every transport carries the same wire protocol (§16). The transport provides the channel; integrity and authenticity of history come from the object model regardless of channel.
…
51 unchanged lines
761 900
| Message | Encoding |
762 901
|---|---|
763 902
| Signed attestation | `[attest bytes, [Signature]]` |
764 -
| `Bundle` | `{0: [[change bytes, [Signature]]], 1: [signed attestation], 2: [[type name, object bytes]], 3: [signed policy]}`; changes in dependency order; objects are chunks, blobs, identities, and states only, and include the text blobs of every change in the bundle and their chunks, so a bundle can be read on its own; key 3 (optional) holds the policies between the receiver's serial and the one sent with a push, ascending |
903 +
| `Bundle` | `{0: [[change bytes, [Signature]]], 1: [signed attestation], 2: [[type name, object bytes]], 3: [signed policy]}`; changes in dependency order; objects are chunks, blobs, identities, and states only, and include the text blobs of every change in the bundle and their chunks, so a bundle can be read on its own, and the `new` blob of every `Replace` (with its chunks) unless left out by size; never a replacement's `old` blob, which is the `new` of a change it depends on (§8.1), held by the receiver or carried by the same bundle (oxygen#136); a push that moves a bookmark also carries the identity that signed the move and, for a delegate, the identity it acts for, so a receiver that has seen no change by that key can check it (oxygenhub#129), and a push that sends tags carries each tag signer's identity and, for a short-lived key, its certificate likewise (oxygen#129); key 3 (optional) holds the policies between the receiver's serial and the one sent with a push, ascending; key 4 (optional, only when there are any) holds signed certificates (§14.7) of short-lived keys that signed what the bundle carries or the push's attestation, their keys' and issuers' identities among the objects; a receiver keeps each that verifies with its issuer and refuses a bundle with one that does not |
765 904
| `capabilities` response | `{0: [versions], 1: repo id bytes(16), 2: [feature names]}` |
766 905
| Signed policy | `[policy bytes, [Signature]]` |
767 -
| `refs` response | `{0: [{0: name, 1: state ref, 2: [heads], 3: signed attestation or null, 4: last serial or null}], 1: repo id, 2: signed policy or null, 3: [tag], 4: [signed policy]}`; key 4 (optional) lists earlier policies ascending by serial; key 5 (optional, text or null) names the default bookmark, which a clone checks out; the attestation is present only when it describes the current state; a tag is `{0: name, 1: state ref, 2: [heads], 3: signed attestation or null}` |
906 +
| `refs` response | `{0: [{0: name, 1: state ref, 2: [heads], 3: signed attestation or null, 4: last serial or null, 5: signed acceptance}], 1: repo id, 2: signed policy or null, 3: [tag], 4: [signed policy]}`; key 4 (optional) lists earlier policies ascending by serial; key 5 (optional, text or null) names the default bookmark, which a clone checks out; the attestation is present only when it describes the current state; an entry's key 5 (optional, only when there is one) is the host's acceptance of that attestation (§14.7); an entry's key 6 (optional) is the attestation's place in the host's transparency log, `[entry number, entry bytes, [proof hash]]`, an inclusion proof against the checkpoint in the response's key 6 (§14.8); an entry's key 7 (optional, only beside key 6 and when there are any) lists the cosigned checkpoints that first held that log entry, `[[checkpoint note text, [proof hash]]]`, each with the entry's inclusion proof against it, from which its integrated time is read (§14.8); the response's key 6 (optional, text) is the host's latest checkpoint, a signed note; the response's key 7 (optional, only when there are any) lists the witnesses the host asks to cosign, `[[name, identity bytes]]`, which a client pins (§14.8); a tag is `{0: name, 1: state ref, 2: [heads], 3: signed attestation or null}`, and optionally, each only when there is one, `4: signed acceptance`, `5: log place`, `6: [cosigned checkpoint]`, meaning for the tag's attestation what a bookmark entry's keys 5 to 7 mean for its attestation (oxygen#129); a reader that does not know them ignores them |
768 907
| `negotiate` request | `{0: [have], 1: [want]}`; response `{0: [hashes]}` in dependency order |
769 -
| `fetch` request | `{0: [hashes], 1: max blob size or null}`; response a `Bundle` without blobs above the size |
908 +
| `fetch` request | `{0: [hashes], 1: max blob size or null}`; response a `Bundle` without blobs above the size; also when it names no change, the bundle carries the identities of the keys that signed the attestations served with `refs` (each bookmark's latest and each tag's) and, for a short-lived key, its certificate and its issuer's identity (§14.7), each once, so a client can check an attestation whose signer signed none of the changes it receives, such as a delegate's or a keyless agent's bookmark move (oxygenhub#266) |
770 909
| `objects` request | `{0: [refs]}`; response a `Bundle` carrying only objects, a blob accompanied by its chunks |
771 -
| `checkpoint` request | `{0: [heads]}`; response a `Bundle` of objects: the checkpoint of exactly those heads, its payload blob, and the blob's chunks |
910 +
| `checkpoint` request | `{0: [heads]}`; response a `Bundle` of objects: the checkpoint of exactly those heads, its payload blob, and the blob's chunks, with the payload's graph blobs and their chunks (§11) |
911 +
| `checkpoint-index` request | as `checkpoint`, without the graph blobs: the client asks for those it lacks with `objects`, in parts (oxygen#137), so neither side holds a big state's whole answer; a server without it answers 404 and the client falls back to `checkpoint` |
772 912
| `shallow` request | `{0: [heads], 1: depth}`; response `{0: [hashes]}`, the changes within `depth` steps of the heads, dependencies first |
773 913
| `push` request | `{0: Bundle, 1: [{0: name, 1: old state, 2: new state, 3: force, 4: signed attestation or null}], 2: signed policy or null, 3: [tag]}`; response `{0: [{0: name, 1: ok, 2: reason}]}`, where a result named `policy` reports the policy update and `tags/<name>` each tag |
774 914
…
3 unchanged lines
778 918
779 919
On push, the server MUST, in order: verify that every change has a valid signature or seal from a key embedded in policy and satisfies the author-or-delegate rule (§14.4), verify the attestation for each reference, check that `old_state` matches its current state (compare-and-swap), check policy rules for the reference including `allow_conflicts`, check that every dependency of every change is either included or already stored, and only then update references atomically. A failure rejects the whole push. A refusal's reason names the reference and, for a failure of the server's own storage, the step that failed; the reference server also logs every refusal and every failed request with its repository and operation, and answers a storage failure with 500 rather than 400.
780 920
921 +
**A push in parts.** A push may carry no reference update and no tag: its changes and objects alone, which the server validates as above and stores, reporting the outcome as one result named `objects` (`ok`, and as its reason how many changes it stored or why it refused them). A client sends a long history this way, in dependency order, in parts of about `push.batch-bytes` (default 64 MiB, under the 100 MB a proxy in front of a hub may take) of changes and their texts and blobs, and the last part carries the reference updates; when one part's changes bring more than that in blobs, the blobs and chunks go first in parts of their own, objects alone (oxygen#133), so that neither side holds the whole history at once and no request passes the server's body limit. A part already stored is harmless if the push stops, and a server MAY collect its changes under §21 if no reference ever reaches them.
922 +
923 +
**Refused by content.** Before any of the checks above, and before a push binds an unbound repository, a host MAY refuse a push by what it adds: the lines its changes insert, the blobs it carries (a forge's secret scanning is one such check). The refusal rejects the whole push, or the whole part of a push in parts, with nothing stored, bound or moved, and is an HTTP error (the reference server's hosts choose `422`, never `401`, `403` or `404`, which a client takes for a credential to change) whose plain-text body is the host's reason, which MAY run to several lines; a client MUST show it as it is. In the reference server this is `Backend::vet_push`, which accepts everything unless a host implements it.
924 +
781 925
**Deleting a reference.** An update whose `new_state` is the zero ObjectRef deletes the bookmark (`oxy push REMOTE --delete NAME`). The bookmark MUST exist at `old_state`. A server MUST refuse to delete its default bookmark: in a bare repository that is the host's to name (the reference server's is the repository's configured bookmark, a forge's its default-bookmark setting), and in a non-bare repository it is the checked-out bookmark. The update is attested like any other: under a policy the attestation is required and its key MUST be authorized for the reference. The deletion's attestation stays the reference's latest, so an older attestation cannot be replayed to bring the bookmark back. The first attestation of a reference the remote does not list takes the client's clock in milliseconds as its serial, which is past the serial of any deleted namesake; later ones increment it. Changes of a deleted bookmark stay until unreachable (§21).
782 926
783 927
## 17. Partial, sparse, and shallow repositories
…
9 unchanged lines
793 937
794 938
- **Sparse** repositories hold the full change graph but only materialize the listed paths: the working copy contains nothing else, files outside the filter are neither reported nor removed, and changes touching other paths are stored and applied to the graph. A sparse clone sets `max-blob` to zero unless a size is given, so blobs outside the filter are fetched only if the filter later widens (§13.5). `oxy sparse PATH..` changes the filter of a clean working copy and re-checks it out.
795 939
- **Lazy blobs** are fetched via `objects` on first read. `oxy status` and `oxy commit` MUST NOT need them.
796 -
- **Shallow** repositories hold a checkpoint of their bookmark's heads (obtained with the `checkpoint` message) and the changes within `depth` dependency steps of those heads (the `shallow` message lists them). The *shallow boundary* is the set of dependencies of stored changes that are not stored (`.oxy/shallow`). Dependencies into the boundary are accepted as well-formed, negotiation stops at the boundary (the remote is assumed to hold it), and the log shows every change from the checkpoint's metadata. `oxy deepen N` fetches `N` more levels below the boundary; when nothing remains the repository is no longer shallow.
940 +
- **Shallow** repositories hold a checkpoint of their bookmark's heads (obtained with the `checkpoint` message) and the changes within `depth` dependency steps of those heads (the `shallow` message lists them); at depth 0 no change at all, the checkpoint's heads being the boundary (oxygen#137), and a shallow repository takes and builds on changes whose dependencies lie below what it cloned, which its remote is assumed to hold. The *shallow boundary* is the set of dependencies of stored changes that are not stored (`.oxy/shallow`). Dependencies into the boundary are accepted as well-formed, negotiation stops at the boundary (the remote is assumed to hold it), and the log shows every change from the checkpoint's metadata. `oxy deepen N` fetches `N` more levels below the boundary; when nothing remains the repository is no longer shallow.
797 941
- Thin clients (§22) always use `depth = 0`: a checkpoint plus zero history.
798 942
799 943
Servers MUST honour `filter` on `negotiate`, `fetch`, and `checkpoint`.
…
12 unchanged lines
812 956
| File | Purpose |
813 957
|---|---|
814 958
| `.oxyignore` | Ignore rules. Gitignore syntax and semantics, per directory, plus `.oxy/info/ignore` and the global ignore file. A directory's `.gitignore` and `.ignore` files are read too, with lower precedence than its `.oxyignore` (a `!` rule there re-includes what they exclude), so a project coming from Git leaves out what Git did. Rules apply to untracked paths only: a tracked path stays tracked, and its edits are seen, until it is deleted or removed from the state. |
815 -
| `.oxyattributes` | Per-path attributes: `binary` (force opaque), `text` (text unless NUL bytes), `eol=lf|crlf` (store LF; check out as given), `nocompress` (chunks packed raw), `lockable` (read-only until `oxy lock`), `filter=<name>`. Gitattributes syntax: a pattern without `/` matches basenames below the file's directory, one with `/` is anchored to it; later rules and deeper files override. The attributes in force for a change are those in the files being committed. |
959 +
| `.oxyattributes` | Per-path attributes: `binary` (force opaque), `text` (text unless NUL bytes), `eol=lf|crlf` (store LF; check out as given), `nocompress` (chunks packed raw), `lockable` (read-only until `oxy lock`), `merge=union` (adjacent insertions settled by keeping both sides, §9.5), `filter=<name>`. Gitattributes syntax: a pattern without `/` matches basenames below the file's directory, one with `/` is anchored to it; later rules and deeper files override. The attributes in force for a change are those in the files being committed. |
816 960
| `.oxy/config` | Repository-local settings, TOML. |
817 -
| `~/.config/oxy/config.toml` | User settings, the same keys as `.oxy/config`: `author.name`, `author.email`, `author.key`, `author.signer` (the key that signs and seals in place of `author.key`, which must list it as a delegate: an agent's own key; also `$OXY_SIGNER`), `core.sign`, and `remote.<name>.token`; also `auth.helper`. Overridden by `$OXY_HOME/config.toml` when `$OXY_HOME` is set. |
961 +
| `~/.config/oxy/config.toml` | User settings, the same keys as `.oxy/config`: `author.name`, `author.email`, `author.key`, `author.signer` (the key that signs and seals in place of `author.key`, which must list it as a delegate: an agent's own key; or `keyless`, for fresh keys the hub certifies under an agent's grant, §14.7, from the grant token `oxy auth grant` keeps; also `$OXY_SIGNER`), `keyless.hub` (the hub that certifies them; default the default remote's), `author.for` (in a keyless home, the account it acts for: it authors as that account's newest identity, which the hub sends with each certificate with its chain back to the grant's key and the home keeps in `keys/author.for`; a repository's own `author.key` wins), `core.sign`, and `remote.<name>.token`; also `auth.helper`. Overridden by `$OXY_HOME/config.toml` when `$OXY_HOME` is set. |
818 962
| `~/.config/oxy/credentials.toml` | Tokens from `oxy auth login`, one `[scheme://host[:port]]` section per hub with `token`, `account` and `ca`. Written owner-only; ignored when others can read it. With `auth.helper` set, the token is kept by that helper (Git's credential-helper protocol, user name `oxy-token`) and the section holds the rest. |
819 963
820 964
Settings resolve in order: the repository file overrides the user file, which overrides environment variables (`OXY_AUTHOR_NAME`, `OXY_AUTHOR_EMAIL`, `OXY_TOKEN`) and built-in defaults. A remote's token and CA without `remote.<name>.token` or `.ca` come from `OXY_TOKEN` and `OXY_CA`, then from `credentials.toml` for the remote's hub, then from a remote on the same hub in an enclosing repository's `.oxy/config`. A client MUST NOT send a token over plain `http` except to a loopback address, and MUST NOT forward it across a redirect. `oxy config` writes the repository file; `oxy config --user` writes the user file. Secret keys live beside the user file in `keys/`, one `<KeyId>.secret` and one `<KeyId>.identity` per key, so any repository the user works in can store the identity object when it first signs.
…
10 unchanged lines
831 975
832 976
## 21. Garbage collection
833 977
834 -
An object is **reachable** if it is referenced, directly or transitively, from any bookmark, tag, policy, the current operation heads, any operation newer than `core.op-retention`, any workspace's working-copy change, or any checkpoint of a reachable state. `oxy gc` deletes unreachable objects that are older than `core.gc-grace` (default 14 days, to protect concurrent writers) and repacks. Tombstone atoms are never collected independently of their change. Divergent changes are reachable if any of their variants is. A change's text blobs and their chunks are reachable with it (§7.4). A seal is reachable if any change it covers is reachable, and reference attestations are reachable while they are the latest for their reference or are within `core.op-retention`.
978 +
An object is **reachable** if it is referenced, directly or transitively, from any bookmark, tag, policy, the current operation heads, any operation newer than `core.op-retention`, any workspace's current working-copy change, the working-copy changes of operations newer than `core.snapshot-retention` (default 7 days; `oxy gc --expire-snapshots` keeps none but the current ones, oxygen#98), or any checkpoint of a reachable state. A file that was only ever in a working copy, never committed, so goes once the snapshots that held it expire. `oxy gc` first deletes the packs and loose objects earlier repacks retired over an hour ago (§13.4), then deletes unreachable objects that are older than `core.gc-grace` (default 14 days, to protect concurrent writers) and repacks; what this repack replaces is retired in turn, for a later collection. On a store that is not this disk's directory it then folds the signatures and index entries completely (§13.7). Tombstone atoms are never collected independently of their change. Divergent changes are reachable if any of their variants is. A change's text blobs and their chunks are reachable with it (§7.4). A seal is reachable if any change it covers is reachable, and reference attestations are reachable while they are the latest for their reference or are within `core.op-retention`.
835 979
836 980
### 21.1 Integrity check
837 981
…
1 unchanged lines
839 983
840 984
### 21.2 Doctor
841 985
842 -
`oxy doctor` checks what commonly stands between a user and a working repository and says, for each problem, what to do next: the operation log opens and no lock was left by a dead process; the signing identity exists, its secret key file is present, and it has not expired (a warning within 30 days); the author name and email are not the defaults; and for each remote, that plain `http://` is only used on loopback, a token is set, the CA file exists, and (unless `--offline`) the remote answers and is this repository or an unbound one. Failures of remote requests anywhere in `oxy` carry the same next step (an untrusted certificate, a rejected or insufficient token, no repository at the URL, nothing listening, an unknown host, a timeout). The exit status is 1 when a check fails; `--json` gives the checks.
986 +
`oxy doctor` checks what commonly stands between a user and a working repository and says, for each problem, what to do next: the operation log opens and no lock was left by a dead process; the signing identity exists, its secret key file is present, and it has not expired (a warning within 30 days); the author name and email are not the defaults; the trust policy (§14.5) loads and has not expired (a warning within 30 days), and its absence is a warning, since nothing fetched is then verified, with the steps by which the repository's owner sets one (oxygen#127); and for each remote, that plain `http://` is only used on loopback, a token is set, the CA file exists, and (unless `--offline`) the remote answers and is this repository or an unbound one. Failures of remote requests anywhere in `oxy` carry the same next step (an untrusted certificate, a rejected or insufficient token, no repository at the URL, nothing listening, an unknown host, a timeout). The exit status is 1 when a check fails; `--json` gives the checks.
843 987
844 988
## 22. Client profiles and retro hardware
845 989
…
17 unchanged lines
863 1007
864 1008
`oxy git import <repo>` and `oxy git export <repo>` provide a lossless-enough bridge; `oxy git fetch/push` operate against a Git remote directly.
865 1009
866 -
- **Import.** Each Git commit becomes one change whose edits are the diff between the union of its parents' states and its tree, so a merge commit becomes a resolution exactly when the parents conflict in the change model, and an ordinary change otherwise. The commit's author and hash are recorded as `Git-Author` and `Git-Commit` trailers; the author key is the zero KeyId, since nobody can sign on the original author's behalf. Submodule entries are skipped. Renames are detected on import exactly as for a snapshot (§7.1, similarity ≥ 50%), then tracked exactly. The Git commit's `Change-Id` trailer, if present and 128 bits, becomes the ChangeId; otherwise one is derived as BLAKE3-128 of the Git commit hash, so re-import is idempotent. New files' and directories' NodeIds are derived too, as BLAKE3-128 of the commit hash, the node kind and the path, so importing the same history twice, anywhere, gives the same changes. Note that a change's own closure is what it depends on, not the Git snapshot: the imported bookmark's state equals the Git tree, but checking out an individual imported change yields only the files it and its dependencies touch.
1010 +
- **Import.** Each Git commit becomes one change whose edits are the diff between the union of its parents' states and its tree, so a merge commit becomes a resolution exactly when the parents conflict in the change model, and an ordinary change otherwise. The commit's author and hash are recorded as `Git-Author` and `Git-Commit` trailers; the author key is the zero KeyId, since nobody can sign on the original author's behalf. The exception is the importing identity's own history: with `git.claim` set (`oxy git import --claim`, `oxy git fetch --claim`), a commit whose Git author email is that identity's email, compared without case, gets its key as author, and is sealed by it on push like any of its changes; every other commit keeps the zero key. Submodule entries are skipped. Renames are detected on import exactly as for a snapshot (§7.1, similarity ≥ 50%), then tracked exactly. The Git commit's `Change-Id` trailer, if present and 128 bits, becomes the ChangeId; otherwise one is derived as BLAKE3-128 of the Git commit hash, so re-import is idempotent. New files' and directories' NodeIds are derived too, as BLAKE3-128 of the commit hash, the node kind and the path, so importing the same history twice, anywhere, gives the same changes. Note that a change's own closure is what it depends on, not the Git snapshot: the imported bookmark's state equals the Git tree, but checking out an individual imported change yields only the files it and its dependencies touch.
867 1011
- **Directories.** Git gives a directory no identity: its path is all it is. An import therefore never makes a second node for a directory path: when a commit needs a directory where the parent's state has none, and the parent's history has a directory last placed at that (parent, name) and now placed nowhere, the change brings that node back with a `Move` from its last placement (the newest by operation key when there are several). Otherwise lines of history that placed files in the directory before it was deleted and after it came back would place them in two different directories, a conflict Git never had. Likewise a node the commit keeps is in exactly the one place its tree names: any other live placement it has in the parents' union (one left under a directory another line removed is out of sight, and no diff of the tree would reach it) is retired by the commit's change, and so is every live placement of a node the union cannot show at all (an orphan, whose only placements are under removed directories): Git's tree has no path for it.
868 1012
- **Export.** Each state along a bookmark becomes a Git commit whose tree is the materialized state. The ChangeId is written as a `Change-Id:` trailer and the ChangeHash as `Oxy-Hash:`. Conflicted states are exported with §9.5 markers and an `Oxy-Conflicted: true` trailer. Signatures are not exported; Git commits MAY be additionally signed with the Ed25519 key via SSH signature format.
869 1013
- **Fetch and push against a Git remote.** `oxy git fetch URL` keeps a bare mirror under `.oxy/git/` for the remote, fetches the branch into it, and imports the commits the bookmark does not have yet, seeding the import with the commits already known: an imported commit maps to its change, and a commit this repository exported maps to the bookmark state it was made from (so a Git commit on top of exported work is based on everything that work contained, including changes Git never saw). The bookmark moves to the union of its heads and the new changes, like a pull; the working copy follows when it was clean. A fetched state whose previous state was itself a Git tree is remembered as equal to the fetched tip. `oxy git push URL` refuses until the remote's tip is known, exports the bookmark's later states as commits chained onto that tip, and pushes the branch, so the push is always a fast-forward.
870 1014
- **Submodules.** A Git gitlink entry (mode `160000`) imports as a `subrepo` node with a Git pin (§7.8) and the matching `.gitmodules` entry becomes a `.oxymodules` entry with `branch` set to the submodule's configured branch when there is one. Export writes a Git pin back as a gitlink, keeps a tracked `.gitmodules` as is, and otherwise derives `.gitmodules` from `.oxymodules`; an Oxygen pin has no Git form and is omitted from the exported tree, which the export report counts.
871 1015
- **Merges of opaque files.** When the union of a merge commit's parents holds several live blobs for a file or submodule, the merge's change replaces each of them with the commit's blob and is a resolution: Git's merge is the authority, unlike a working copy, where a candidate on disk is not yet a decision.
1016 +
- **A side-by-side `.git`.** `oxy git colocate` (config `git.colocate`) keeps a `.git` beside `.oxy` so editors' and other tools' Git features work on an Oxygen checkout: made with `git init` when missing, `/.oxy/` added to its `info/exclude`, the current bookmark exported into it (as `oxy git export` does) after every oxy command that leaves the bookmark somewhere new, HEAD on that branch and the index at its tree, so `git status` shows exactly the working copy's uncommitted work. It is one way: commits made with Git are not read back. oxy never takes a `.git` directory as content, and checkout refuses to write a path with a `.git` component at any depth or case (as Git does), so a change cannot plant hooks there. `oxy git colocate --off` stops the updates and leaves `.git` in place.
1017 +
- **Export by change graph.** For a server serving Git clients (a hub's Git view), `git_export_graph` exports every bookmark and tag from the changes alone, so the same changes give the same commits wherever it runs: each change is a commit whose tree is the state of the change and its dependency closure and whose parents are its dependencies' commits (a change that depends on nothing is a root), with the change's author, time, message and `Change-Id:`/`Oxy-Hash:` trailers; a bookmark with one head is a branch at its commit, with several a merge of their commits (the bookmark's state, at the newest head's time, `Oxy-Heads:`); tags likewise. Marks make it incremental; branches of deleted bookmarks are removed.
872 1018
- **Round trip.** Import followed by export MUST reproduce every tree hash. Export followed by import MUST reproduce every ChangeId. `oxy git verify GIT_DIR` checks an import commit by commit: for each commit, the state formed by the changes imported for it and all its ancestors must materialize to exactly that commit's tree (content, executable bits, symlink targets, submodule pins; `.oxymodules` is allowed beside submodules). `--last N` rebuilds only the last N commits, following the others, to look again at where an import went wrong.
873 1019
874 1020
## 24. Command-line interface
…
4 unchanged lines
879 1025
oxy init [PATH] [--bare]            oxy clone URL [--sparse PATH].. [--depth N] [--max-blob SIZE]
880 1026
oxy status [--json]                 oxy diff                     oxy prompt [--format F]
881 1027
oxy commit -m MSG [--depends-on CHANGE]..
882 -
oxy log [-r REVSET] [-n N] [-v] [-L S,E:PATH]   oxy show CHANGE | -r REVSET
883 -
oxy describe [CHANGE] [-m MSG]      oxy amend [-m MSG] [--into CHANGE]
884 -
oxy split CHANGE [PATH]..           oxy abandon CHANGE | -r REVSET [--yes]   oxy resolve
1028 +
oxy log [-r REVSET] [-n N] [-v] [-L S,E:PATH]   oxy show CHANGE | -r REVSET [--stat]
1029 +
oxy describe [CHANGE] [-m MSG]      oxy amend [-m MSG] [--into CHANGE | PATH..]
1030 +
oxy split CHANGE [PATH]..           oxy abandon CHANGE | -r REVSET [--yes]   oxy resolve [--both [--reverse]] [PATH]..
885 1031
oxy checkout [CHANGE].. [--discard|--carry] oxy undo                     oxy op log
886 1032
oxy tag create NAME [CHANGE].. [-m MSG]   oxy tag list
887 1033
oxy bookmark create|set|delete|list|switch  oxy merge BOOKMARK
…
7 unchanged lines
895 1041
oxy seal [CHANGE].. | --proposal URL  seal your unauthenticated and anonymous changes (§14.4), or a hub proposal's changes as your approval
896 1042
oxy proposal show|land URL          hub proposals from a client (hub spec §7)
897 1043
oxy git import|export|fetch|push|verify   (fetch and push go through a bare mirror under .oxy/git/)
898 -
oxy checkpoint                      oxy gc [--now]               oxy config [--user] KEY VALUE
1044 +
oxy checkpoint                      oxy gc [--now] [--expire-snapshots]               oxy config [--user] KEY VALUE
899 1045
oxy root                            oxy cat PATH [-r CHANGE]     oxy version
900 1046
oxy fsck [--json]                   check integrity (§21.1)
901 1047
oxy doctor [--offline] [--json]     check identity, config and remotes (§21.2)
902 1048
oxy completions SHELL               shell completion script     oxy man [--dir DIR]   man pages
903 1049
oxy hunks                           oxy discard PATH.. | --hunks N,M
1050 +
oxy mcp [DIR] [--allow-destructive] serve the repository to an AI agent (MCP, stdio)
904 1051
```
905 1052
906 1053
**Machine interface.** `status`, `log` (with `--path`), `show`, `diff`, `hunks`, `blame`, `resolve`, `op log`, `bookmark list`, `remote list`, `fetch`, `pull`, `push`, `submodule status`, `bisect status`, `verify`, `policy show`, `root` and `version` take `--json` and then print exactly one JSON document on stdout, with ids as hex strings and times in milliseconds since the epoch; errors go to stderr with a non-zero status. `commit --hunks N,M` and `discard --hunks N,M` take hunk numbers as `hunks` prints them. Editors and other tools MUST use these rather than reading `.oxy/`.
907 1054
1055 +
**Agents.** `oxy mcp` serves the repository to an AI agent over the Model Context Protocol on stdio (docs/mcp.md): each tool runs one of the commands above with `--json`, in the repository and with the server's environment, so it signs as the identity `OXY_HOME` configures (an agent home seals every change via the agent). `commit` and `amend` take explicit paths or `all: true`; `undo` and a forced push are offered only with `--allow-destructive`.
1056 +
908 1057
Change arguments accept a ChangeId prefix, a ChangeHash prefix (with `#`), a bookmark or tag name, and `@` for the working-copy change. Ambiguous prefixes are an error.
909 1058
910 1059
Reserved for a later version, with these names: `oxy new`, `oxy edit`, `oxy redo`, revsets (`log -r`), and `oxy remote trust` (with the Noise transport, §15).
…
49 unchanged lines
960 1109
| HTTPS transport and endpoints (§16) | Implemented, HTTP only; Noise and ZCT (§15) reserved; negotiation is exact (no Merkle buckets); `refs` takes no `patterns` |
961 1110
| Partial, sparse, shallow (§17) | Implemented |
962 1111
| Locks (§18) | Local advisory locks only; server locks and the `lock` endpoint reserved |
963 -
| Configuration, attributes, hooks (§19, §20) | Implemented (`.oxyattributes`: binary, text, eol, nocompress, lockable) |
1112 +
| Configuration, attributes, hooks (§19, §20) | Implemented (`.oxyattributes`: binary, text, eol, nocompress, lockable, merge=union) |
964 1113
| Thin profile and proxy (§22) | Reserved |
965 1114
| Git interoperability (§23) | Implemented: import, export, fetch, push, rename detection |
966 1115
| Conformance suite (§26) | Partial: encoding, signature, permutation, conflict, rewrite, protocol, chunking and Git tests exist as unit and integration tests in the crates; a standalone vector suite is not published yet |
…
84 unchanged lines
modified docs/stores.md +36 −0
…
142 unchanged lines
143 143
  holds no copy of its own and the data lives where its owner chose.
144 144
  B1 (the store over the trait) is done; B2 is the caching store and
145 145
  the references by compare-and-swap.
146 +
- **The operation head on a store** (B2, 2026-10-05): a repository whose
147 +
  objects are on any store but this disk's directory keeps its
148 +
  operation head in that store's `ops/head` reference record. Recording
149 +
  an operation reads the record and its version, and moves it only if
150 +
  the version is unchanged. When another process moved it first, the
151 +
  two operations are merged by the same rules as on disk, and the write
152 +
  is tried again (eight times at most). A repository on this disk keeps
153 +
  `.oxy/ops/head` under its lock file, as before, so older oxy builds
154 +
  share it safely.
155 +
- **The caching store** (B2, 2026-10-05): `strata::CachingStore` puts a
156 +
  directory store in front of any other. Reads go through it: a missing
157 +
  object is fetched from the remote and kept. Writes go to the remote
158 +
  first and then the cache, so the remote always holds everything and
159 +
  the cache can be deleted at any time. The cache has a size limit; past
160 +
  it the objects used least recently are removed, down to nine tenths of
161 +
  the limit. An object larger than an eighth of the limit is not kept:
162 +
  ranges of it are read from the remote each time. References are never
163 +
  cached, so compare-and-swap is always the remote's.
164 +
- **A repository on a store** (B2, 2026-10-05): `oxy init --store URL`
165 +
  (or `storage.url` in `.oxy/config`, set at init) keeps the objects and
166 +
  the operation head on that store, behind a cache in `.oxy/cache` of
167 +
  `storage.cache` bytes (2 GiB by default). `.oxy/objects` then holds
168 +
  only scratch: packs being written and derived caches. The cache can be
169 +
  deleted at any time. A second repository cannot start on a store that
170 +
  already holds one. Measured from the workstation against the real
171 +
  services with five small commits (`a_repository_runs_on_every_configured_store`):
172 +
173 +
  | Store | init | each commit | open with no cache, read the files |
174 +
  |---|---|---|---|
175 +
  | S3 | 441 ms | 703 ms | 481 ms |
176 +
  | GCS | 769 ms | 857 ms | 570 ms |
177 +
  | Azure | 664 ms | 511 ms | 639 ms |
178 +
179 +
  A commit writes its objects one request each and moves the head, so
180 +
  its cost is round trips, not bytes; batching a commit's objects into
181 +
  one pack is the next step for speed.
146 182
147 183
## 4. Layout in the bucket (as built, 2026-10-03)
148 184
…
150 unchanged lines
modified docs/strata.md +49 −0
…
119 unchanged lines
120 120
later (`--now` removes them at once), so a clone that was listing them at
121 121
the time still completes.
122 122
123 +
## A repository kept on a store
124 +
125 +
A store can also hold a repository itself rather than a copy of it.
126 +
Start one with `--store`:
127 +
128 +
```sh
129 +
oxy init myproject --store s3://my-bucket/repos/myproject?profile=work
130 +
```
131 +
132 +
- **Where things live:** the objects and the operation log are in the
133 +
  store. `.oxy/cache` keeps the objects this machine has used, up to
134 +
  `storage.cache` (2 GiB unless you set it, for example
135 +
  `oxy config storage.cache 10G`).
136 +
- **The cache:** you can delete `.oxy/cache` at any time. Objects come
137 +
  back from the store as they are needed.
138 +
- **One repository per prefix:** `oxy init --store` refuses a prefix
139 +
  that already holds a repository's operation log.
140 +
- **Speed:** each command that records something makes a few requests to
141 +
  the store, so a commit takes about half a second to a second.
142 +
- **Choose at init:** set `storage.url` only when the repository is
143 +
  created. Changing it later does not move the objects.
144 +
- **Signatures and seals:** each one is stored under a key of its own,
145 +
  as in any repository. A bucket can only find those keys with a LIST
146 +
  request, so the store also keeps a journal of them (`pack-entries`)
147 +
  and lists of them (`pack-<id>.entries`). These are read when the
148 +
  repository is opened and again every 30 seconds (spec §13.7). After
149 +
  every client that writes to the bucket runs this version of oxy, run
150 +
  `oxy gc` once. From then on, looking up a signature or a seal sends no
151 +
  LIST request, and another client's new signature is seen within 30
152 +
  seconds. A client still on oxy 0.4.6 or older writes signatures that
153 +
  the others miss until the next `oxy gc`, so upgrade every client
154 +
  first.
155 +
156 +
A repository kept on a store holds:
157 +
158 +
```
159 +
PREFIX/
160 +
  pack-<id>.pack, .idx, .dict   packs, their indexes and dictionaries
161 +
  pack-retired                  what repacks and folds replaced, deleted an hour later
162 +
  pack-entries                  journal: signatures and index entries since the last fold
163 +
  pack-<id>.entries             every signature and index entry, as the last gc found them
164 +
  pack-<id>.entries-part        journal lines folded out once there were more than 1000
165 +
  pack-legacy-folded            the older one-list keys have been folded
166 +
  loose/  nocompress/           objects not yet packed; chunks kept raw
167 +
  sig/  seal/  receipt/  approval/  cert/  accepted/
168 +
                                one key per signature or index entry
169 +
  ops/head                      the operation head
170 +
```
171 +
123 172
## What a store holds
124 173
125 174
```
…
37 unchanged lines
added packaging/haikuports/README.md +40 −0
1 +
# oxy and oh on HaikuDepot
2 +
3 +
[haikuports](https://github.com/haikuports/haikuports) recipes for oxy
4 +
(`dev-vcs/oxy`) and OxygenHub's `oh` (`dev-vcs/oh`) are written by
5 +
`make-recipe.py` from a release's source tarball (oxygen#156, #157).
6 +
7 +
Each release's dist job attaches `NAME-VERSION-src.tar.gz`: the source as
8 +
released, reproducible (sorted, fixed times and owners, gzip without a
9 +
timestamp). oh's holds the oxygenhub tree with its subrepos inside, so it
10 +
builds alone. The recipe fetches it from the release's stable address,
11 +
`https://oxygenhub.io/andymccall/REPO/releases/download/vVERSION/NAME-VERSION-src.tar.gz`,
12 +
and every crates.io crate in its `Cargo.lock` as a source of its own with its
13 +
checksum, building offline as haikuports' other Rust recipes (ripgrep) do.
14 +
15 +
For a release (oxy shown; oh is the same with `oh` and `oxygenhub`):
16 +
17 +
```sh
18 +
curl -LO https://oxygenhub.io/andymccall/oxygen/releases/download/vX.Y.Z/oxy-X.Y.Z-src.tar.gz
19 +
python3 packaging/haikuports/make-recipe.py oxy oxy-X.Y.Z-src.tar.gz X.Y.Z > oxy-X.Y.Z.recipe
20 +
```
21 +
22 +
Test it on Haiku R1/beta6 with haikuports checked out and `haikuporter`
23 +
installed: copy the recipe to `haikuports/dev-vcs/oxy/` and run
24 +
`haikuporter --no-source-packages -j1 -y oxy`. Submit it as a pull request
25 +
to haikuports (`oxy: new recipe`, then `oxy: bump to X.Y.Z`).
26 +
27 +
Haiku needs the system's zstd (`ZSTD_SYS_USE_PKG_CONFIG=1`: the bundled
28 +
zstd's hidden symbols break the link), which the recipe sets. oh's tests
29 +
pull in the hub, whose pipeline parser needs a newer Rust than HaikuPorts'
30 +
1.94, so its recipe tests `hub-api` only.
31 +
32 +
## Icons
33 +
34 +
HaikuDepot takes a package's icon from HaikuDepotServer, not from the
35 +
recipe. `icons/` holds them, written by `make-icons.py icons/` (the PNGs
36 +
rendered from the SVGs): Oxygen (`oxy`) is the mark on a pale tile,
37 +
OxygenHub (`oh`) the mark on the hub's dark tile, as the VS Code
38 +
extensions' icons are. Once the packages are listed on
39 +
depot.haiku-os.org, upload `NAME.hvif` with `NAME-16.png`, `-32` and `-64`
40 +
as each package's icon.
added packaging/haikuports/dev-vcs/oh/oh-0.9.16.recipe +1472 −0
1 +
SUMMARY="The command line of OxygenHub, the forge for Oxygen repositories"
2 +
DESCRIPTION="oh talks to an OxygenHub (such as https://oxygenhub.io): issues, proposals \
3 +
and their reviews, releases, Flow runs, their logs and reports, and checks, \
4 +
from the terminal. It works alongside oxy, the Oxygen version control \
5 +
client."
6 +
HOMEPAGE="https://oxygenhub.io/andymccall/oxygenhub"
7 +
COPYRIGHT="2026 Andy McCall"
8 +
LICENSE="MIT
9 +
	Apache v2"
10 +
REVISION="1"
11 +
SOURCE_URI="$HOMEPAGE/releases/download/v$portVersion/oh-$portVersion-src.tar.gz"
12 +
CHECKSUM_SHA256="55770a8668647a44e57ed085315dc11bd97028d2a5c1a1640cd85af87b461200"
13 +
SOURCE_DIR="oxygenhub-$portVersion"
14 +
SOURCE_URI_2="https://static.crates.io/crates/adler2/adler2-2.0.1.crate"
15 +
CHECKSUM_SHA256_2="320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
16 +
SOURCE_FILENAME_2="adler2-2.0.1.tar.gz"
17 +
SOURCE_URI_3="https://static.crates.io/crates/aho-corasick/aho-corasick-1.1.5.crate"
18 +
CHECKSUM_SHA256_3="c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba"
19 +
SOURCE_FILENAME_3="aho-corasick-1.1.5.tar.gz"
20 +
SOURCE_URI_4="https://static.crates.io/crates/allocator-api2/allocator-api2-0.2.21.crate"
21 +
CHECKSUM_SHA256_4="683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
22 +
SOURCE_FILENAME_4="allocator-api2-0.2.21.tar.gz"
23 +
SOURCE_URI_5="https://static.crates.io/crates/ammonia/ammonia-4.2.0.crate"
24 +
CHECKSUM_SHA256_5="3f7ed3ac4252be4d3e36fb589ed5659356842587b21a31596a9dc89d3b2f5e2a"
25 +
SOURCE_FILENAME_5="ammonia-4.2.0.tar.gz"
26 +
SOURCE_URI_6="https://static.crates.io/crates/android_system_properties/android_system_properties-0.1.6.crate"
27 +
CHECKSUM_SHA256_6="ae221649c9976a6f6c56ae1facf410f3ddb33cc661c4b7b61020a912d4237fbc"
28 +
SOURCE_FILENAME_6="android_system_properties-0.1.6.tar.gz"
29 +
SOURCE_URI_7="https://static.crates.io/crates/anstream/anstream-1.0.0.crate"
30 +
CHECKSUM_SHA256_7="824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d"
31 +
SOURCE_FILENAME_7="anstream-1.0.0.tar.gz"
32 +
SOURCE_URI_8="https://static.crates.io/crates/anstyle/anstyle-1.0.14.crate"
33 +
CHECKSUM_SHA256_8="940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000"
34 +
SOURCE_FILENAME_8="anstyle-1.0.14.tar.gz"
35 +
SOURCE_URI_9="https://static.crates.io/crates/anstyle-parse/anstyle-parse-1.0.0.crate"
36 +
CHECKSUM_SHA256_9="52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e"
37 +
SOURCE_FILENAME_9="anstyle-parse-1.0.0.tar.gz"
38 +
SOURCE_URI_10="https://static.crates.io/crates/anstyle-query/anstyle-query-1.1.5.crate"
39 +
CHECKSUM_SHA256_10="40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
40 +
SOURCE_FILENAME_10="anstyle-query-1.1.5.tar.gz"
41 +
SOURCE_URI_11="https://static.crates.io/crates/anstyle-wincon/anstyle-wincon-3.0.11.crate"
42 +
CHECKSUM_SHA256_11="291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
43 +
SOURCE_FILENAME_11="anstyle-wincon-3.0.11.tar.gz"
44 +
SOURCE_URI_12="https://static.crates.io/crates/anyhow/anyhow-1.0.104.crate"
45 +
CHECKSUM_SHA256_12="330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470"
46 +
SOURCE_FILENAME_12="anyhow-1.0.104.tar.gz"
47 +
SOURCE_URI_13="https://static.crates.io/crates/arbitrary/arbitrary-1.4.2.crate"
48 +
CHECKSUM_SHA256_13="c3d036a3c4ab069c7b410a2ce876bd74808d2d0888a82667669f8e783a898bf1"
49 +
SOURCE_FILENAME_13="arbitrary-1.4.2.tar.gz"
50 +
SOURCE_URI_14="https://static.crates.io/crates/arc-swap/arc-swap-1.9.2.crate"
51 +
CHECKSUM_SHA256_14="c049c0be4daef0b145cb3555416b3b8ef5b7888a38aea1a3a155801fe7b0810b"
52 +
SOURCE_FILENAME_14="arc-swap-1.9.2.tar.gz"
53 +
SOURCE_URI_15="https://static.crates.io/crates/argon2/argon2-0.5.3.crate"
54 +
CHECKSUM_SHA256_15="3c3610892ee6e0cbce8ae2700349fcf8f98adb0dbfbee85aec3c9179d29cc072"
55 +
SOURCE_FILENAME_15="argon2-0.5.3.tar.gz"
56 +
SOURCE_URI_16="https://static.crates.io/crates/arrayvec/arrayvec-0.7.8.crate"
57 +
CHECKSUM_SHA256_16="d3fb67a6e08acf24fdeccbac2cb6ac4305825bd1f117462e0e6f2f193345ad56"
58 +
SOURCE_FILENAME_16="arrayvec-0.7.8.tar.gz"
59 +
SOURCE_URI_17="https://static.crates.io/crates/askama/askama-0.14.0.crate"
60 +
CHECKSUM_SHA256_17="f75363874b771be265f4ffe307ca705ef6f3baa19011c149da8674a87f1b75c4"
61 +
SOURCE_FILENAME_17="askama-0.14.0.tar.gz"
62 +
SOURCE_URI_18="https://static.crates.io/crates/askama_derive/askama_derive-0.14.0.crate"
63 +
CHECKSUM_SHA256_18="129397200fe83088e8a68407a8e2b1f826cf0086b21ccdb866a722c8bcd3a94f"
64 +
SOURCE_FILENAME_18="askama_derive-0.14.0.tar.gz"
65 +
SOURCE_URI_19="https://static.crates.io/crates/askama_parser/askama_parser-0.14.0.crate"
66 +
CHECKSUM_SHA256_19="d6ab5630b3d5eaf232620167977f95eb51f3432fc76852328774afbd242d4358"
67 +
SOURCE_FILENAME_19="askama_parser-0.14.0.tar.gz"
68 +
SOURCE_URI_20="https://static.crates.io/crates/asn1-rs/asn1-rs-0.7.2.crate"
69 +
CHECKSUM_SHA256_20="b7f43a50ac4fdca5df8e885c21b835997f0a1cdee65494a6847694a98652d9d8"
70 +
SOURCE_FILENAME_20="asn1-rs-0.7.2.tar.gz"
71 +
SOURCE_URI_21="https://static.crates.io/crates/asn1-rs-derive/asn1-rs-derive-0.6.0.crate"
72 +
CHECKSUM_SHA256_21="3109e49b1e4909e9db6515a30c633684d68cdeaa252f215214cb4fa1a5bfee2c"
73 +
SOURCE_FILENAME_21="asn1-rs-derive-0.6.0.tar.gz"
74 +
SOURCE_URI_22="https://static.crates.io/crates/asn1-rs-impl/asn1-rs-impl-0.2.0.crate"
75 +
CHECKSUM_SHA256_22="7b18050c2cd6fe86c3a76584ef5e0baf286d038cda203eb6223df2cc413565f7"
76 +
SOURCE_FILENAME_22="asn1-rs-impl-0.2.0.tar.gz"
77 +
SOURCE_URI_23="https://static.crates.io/crates/async-compression/async-compression-0.4.48.crate"
78 +
CHECKSUM_SHA256_23="fb61aea1a7def73ee7c350a184f0e70b32c182344e2e75bf70c9b621b83417fd"
79 +
SOURCE_FILENAME_23="async-compression-0.4.48.tar.gz"
80 +
SOURCE_URI_24="https://static.crates.io/crates/async-trait/async-trait-0.1.92.crate"
81 +
CHECKSUM_SHA256_24="82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667"
82 +
SOURCE_FILENAME_24="async-trait-0.1.92.tar.gz"
83 +
SOURCE_URI_25="https://static.crates.io/crates/atoi/atoi-2.0.0.crate"
84 +
CHECKSUM_SHA256_25="f28d99ec8bfea296261ca1af174f24225171fea9664ba9003cbebee704810528"
85 +
SOURCE_FILENAME_25="atoi-2.0.0.tar.gz"
86 +
SOURCE_URI_26="https://static.crates.io/crates/atomic-waker/atomic-waker-1.1.2.crate"
87 +
CHECKSUM_SHA256_26="1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0"
88 +
SOURCE_FILENAME_26="atomic-waker-1.1.2.tar.gz"
89 +
SOURCE_URI_27="https://static.crates.io/crates/autocfg/autocfg-1.5.1.crate"
90 +
CHECKSUM_SHA256_27="f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
91 +
SOURCE_FILENAME_27="autocfg-1.5.1.tar.gz"
92 +
SOURCE_URI_28="https://static.crates.io/crates/axum/axum-0.8.9.crate"
93 +
CHECKSUM_SHA256_28="31b698c5f9a010f6573133b09e0de5408834d0c82f8d7475a89fc1867a71cd90"
94 +
SOURCE_FILENAME_28="axum-0.8.9.tar.gz"
95 +
SOURCE_URI_29="https://static.crates.io/crates/axum-core/axum-core-0.5.6.crate"
96 +
CHECKSUM_SHA256_29="08c78f31d7b1291f7ee735c1c6780ccde7785daae9a9206026862dab7d8792d1"
97 +
SOURCE_FILENAME_29="axum-core-0.5.6.tar.gz"
98 +
SOURCE_URI_30="https://static.crates.io/crates/axum-extra/axum-extra-0.10.3.crate"
99 +
CHECKSUM_SHA256_30="9963ff19f40c6102c76756ef0a46004c0d58957d87259fc9208ff8441c12ab96"
100 +
SOURCE_FILENAME_30="axum-extra-0.10.3.tar.gz"
101 +
SOURCE_URI_31="https://static.crates.io/crates/axum-macros/axum-macros-0.5.1.crate"
102 +
CHECKSUM_SHA256_31="7aa268c23bfbbd2c4363b9cd302a4f504fb2a9dfe7e3451d66f35dd392e20aca"
103 +
SOURCE_FILENAME_31="axum-macros-0.5.1.tar.gz"
104 +
SOURCE_URI_32="https://static.crates.io/crates/axum-server/axum-server-0.8.0.crate"
105 +
CHECKSUM_SHA256_32="b1df331683d982a0b9492b38127151e6453639cd34926eb9c07d4cd8c6d22bfc"
106 +
SOURCE_FILENAME_32="axum-server-0.8.0.tar.gz"
107 +
SOURCE_URI_33="https://static.crates.io/crates/base32/base32-0.5.1.crate"
108 +
CHECKSUM_SHA256_33="022dfe9eb35f19ebbcb51e0b40a5ab759f46ad60cadf7297e0bd085afb50e076"
109 +
SOURCE_FILENAME_33="base32-0.5.1.tar.gz"
110 +
SOURCE_URI_34="https://static.crates.io/crates/base64/base64-0.22.1.crate"
111 +
CHECKSUM_SHA256_34="72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
112 +
SOURCE_FILENAME_34="base64-0.22.1.tar.gz"
113 +
SOURCE_URI_35="https://static.crates.io/crates/base64/base64-0.23.1.crate"
114 +
CHECKSUM_SHA256_35="ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
115 +
SOURCE_FILENAME_35="base64-0.23.1.tar.gz"
116 +
SOURCE_URI_36="https://static.crates.io/crates/base64ct/base64ct-1.8.3.crate"
117 +
CHECKSUM_SHA256_36="2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
118 +
SOURCE_FILENAME_36="base64ct-1.8.3.tar.gz"
119 +
SOURCE_URI_37="https://static.crates.io/crates/basic-toml/basic-toml-0.1.10.crate"
120 +
CHECKSUM_SHA256_37="ba62675e8242a4c4e806d12f11d136e626e6c8361d6b829310732241652a178a"
121 +
SOURCE_FILENAME_37="basic-toml-0.1.10.tar.gz"
122 +
SOURCE_URI_38="https://static.crates.io/crates/bincode/bincode-1.3.3.crate"
123 +
CHECKSUM_SHA256_38="b1f45e9417d87227c7a56d22e471c6206462cba514c7590c09aff4cf6d1ddcad"
124 +
SOURCE_FILENAME_38="bincode-1.3.3.tar.gz"
125 +
SOURCE_URI_39="https://static.crates.io/crates/bit-set/bit-set-0.8.0.crate"
126 +
CHECKSUM_SHA256_39="08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3"
127 +
SOURCE_FILENAME_39="bit-set-0.8.0.tar.gz"
128 +
SOURCE_URI_40="https://static.crates.io/crates/bit-vec/bit-vec-0.8.0.crate"
129 +
CHECKSUM_SHA256_40="5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7"
130 +
SOURCE_FILENAME_40="bit-vec-0.8.0.tar.gz"
131 +
SOURCE_URI_41="https://static.crates.io/crates/bit-vec/bit-vec-0.9.1.crate"
132 +
CHECKSUM_SHA256_41="b71798fca2c1fe1086445a7258a4bc81e6e49dcd24c8d0dd9a1e57395b603f51"
133 +
SOURCE_FILENAME_41="bit-vec-0.9.1.tar.gz"
134 +
SOURCE_URI_42="https://static.crates.io/crates/bitflags/bitflags-2.13.2.crate"
135 +
CHECKSUM_SHA256_42="3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06"
136 +
SOURCE_FILENAME_42="bitflags-2.13.2.tar.gz"
137 +
SOURCE_URI_43="https://static.crates.io/crates/blake2/blake2-0.10.6.crate"
138 +
CHECKSUM_SHA256_43="46502ad458c9a52b69d4d4d32775c788b7a1b85e8bc9d482d92250fc0e3f8efe"
139 +
SOURCE_FILENAME_43="blake2-0.10.6.tar.gz"
140 +
SOURCE_URI_44="https://static.crates.io/crates/blake3/blake3-1.8.7.crate"
141 +
CHECKSUM_SHA256_44="6d9e454fc11f76977dc803893aff6304ed33d6a26efae8696573bea74baa27ae"
142 +
SOURCE_FILENAME_44="blake3-1.8.7.tar.gz"
143 +
SOURCE_URI_45="https://static.crates.io/crates/block-buffer/block-buffer-0.10.4.crate"
144 +
CHECKSUM_SHA256_45="3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
145 +
SOURCE_FILENAME_45="block-buffer-0.10.4.tar.gz"
146 +
SOURCE_URI_46="https://static.crates.io/crates/block-buffer/block-buffer-0.12.1.crate"
147 +
CHECKSUM_SHA256_46="d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa"
148 +
SOURCE_FILENAME_46="block-buffer-0.12.1.tar.gz"
149 +
SOURCE_URI_47="https://static.crates.io/crates/bstr/bstr-1.13.1.crate"
150 +
CHECKSUM_SHA256_47="6bb31b46c14244e20ee9984b11bf5c992b91fb6939fea616e3512c8baecdbe5f"
151 +
SOURCE_FILENAME_47="bstr-1.13.1.tar.gz"
152 +
SOURCE_URI_48="https://static.crates.io/crates/bumpalo/bumpalo-3.20.3.crate"
153 +
CHECKSUM_SHA256_48="72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
154 +
SOURCE_FILENAME_48="bumpalo-3.20.3.tar.gz"
155 +
SOURCE_URI_49="https://static.crates.io/crates/bytemuck/bytemuck-1.25.2.crate"
156 +
CHECKSUM_SHA256_49="95832e849adfb21180ccb6826a99da14e5d266ae5c2e668e1602cf234f153797"
157 +
SOURCE_FILENAME_49="bytemuck-1.25.2.tar.gz"
158 +
SOURCE_URI_50="https://static.crates.io/crates/byteorder/byteorder-1.5.0.crate"
159 +
CHECKSUM_SHA256_50="1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b"
160 +
SOURCE_FILENAME_50="byteorder-1.5.0.tar.gz"
161 +
SOURCE_URI_51="https://static.crates.io/crates/byteorder-lite/byteorder-lite-0.1.0.crate"
162 +
CHECKSUM_SHA256_51="8f1fe948ff07f4bd06c30984e69f5b4899c516a3ef74f34df92a2df2ab535495"
163 +
SOURCE_FILENAME_51="byteorder-lite-0.1.0.tar.gz"
164 +
SOURCE_URI_52="https://static.crates.io/crates/bytes/bytes-1.12.1.crate"
165 +
CHECKSUM_SHA256_52="fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04"
166 +
SOURCE_FILENAME_52="bytes-1.12.1.tar.gz"
167 +
SOURCE_URI_53="https://static.crates.io/crates/cc/cc-1.5.1.crate"
168 +
CHECKSUM_SHA256_53="f360145194ee8e21db5ee7f3fcd4fe52210864c75c985dae33218202c8bbe040"
169 +
SOURCE_FILENAME_53="cc-1.5.1.tar.gz"
170 +
SOURCE_URI_54="https://static.crates.io/crates/cfg-if/cfg-if-1.0.5.crate"
171 +
CHECKSUM_SHA256_54="4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600"
172 +
SOURCE_FILENAME_54="cfg-if-1.0.5.tar.gz"
173 +
SOURCE_URI_55="https://static.crates.io/crates/cfg_aliases/cfg_aliases-0.2.2.crate"
174 +
CHECKSUM_SHA256_55="f079e83a288787bcd14a6aea84cee5c87a67c5a3e660c30f557a3d24761b3527"
175 +
SOURCE_FILENAME_55="cfg_aliases-0.2.2.tar.gz"
176 +
SOURCE_URI_56="https://static.crates.io/crates/chacha20/chacha20-0.10.2.crate"
177 +
CHECKSUM_SHA256_56="65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06"
178 +
SOURCE_FILENAME_56="chacha20-0.10.2.tar.gz"
179 +
SOURCE_URI_57="https://static.crates.io/crates/chrono/chrono-0.4.45.crate"
180 +
CHECKSUM_SHA256_57="1aa79e62e7697b8e29b513a68abacf485adcd1fe8284a4316c5ae868e6633327"
181 +
SOURCE_FILENAME_57="chrono-0.4.45.tar.gz"
182 +
SOURCE_URI_58="https://static.crates.io/crates/clap/clap-4.6.7.crate"
183 +
CHECKSUM_SHA256_58="aa8876b300ab35ba921adea3dfd70157a46249b33f95c9084ae5709785478946"
184 +
SOURCE_FILENAME_58="clap-4.6.7.tar.gz"
185 +
SOURCE_URI_59="https://static.crates.io/crates/clap_builder/clap_builder-4.6.7.crate"
186 +
CHECKSUM_SHA256_59="ec0797fb7aeb1406c84efac526901f7ec3ead2124f946b494e72879d4b54704d"
187 +
SOURCE_FILENAME_59="clap_builder-4.6.7.tar.gz"
188 +
SOURCE_URI_60="https://static.crates.io/crates/clap_derive/clap_derive-4.6.7.crate"
189 +
CHECKSUM_SHA256_60="f9c751b79415d4e559e3d1fcf128e09e720eb673a06d26cf6f392d37d75b66e0"
190 +
SOURCE_FILENAME_60="clap_derive-4.6.7.tar.gz"
191 +
SOURCE_URI_61="https://static.crates.io/crates/clap_lex/clap_lex-1.1.1.crate"
192 +
CHECKSUM_SHA256_61="1c133bc6a41be0d194c306b5506d15e6feeea7b1d6604bd3f8310dfb2ca96486"
193 +
SOURCE_FILENAME_61="clap_lex-1.1.1.tar.gz"
194 +
SOURCE_URI_62="https://static.crates.io/crates/cmov/cmov-0.5.4.crate"
195 +
CHECKSUM_SHA256_62="0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a"
196 +
SOURCE_FILENAME_62="cmov-0.5.4.tar.gz"
197 +
SOURCE_URI_63="https://static.crates.io/crates/colorchoice/colorchoice-1.0.5.crate"
198 +
CHECKSUM_SHA256_63="1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
199 +
SOURCE_FILENAME_63="colorchoice-1.0.5.tar.gz"
200 +
SOURCE_URI_64="https://static.crates.io/crates/compression-codecs/compression-codecs-0.4.43.crate"
201 +
CHECKSUM_SHA256_64="bef16c47ba2797aa6a909cc37d39911f3a6743811fe7408ac0b0cc0276b656e9"
202 +
SOURCE_FILENAME_64="compression-codecs-0.4.43.tar.gz"
203 +
SOURCE_URI_65="https://static.crates.io/crates/compression-core/compression-core-0.4.33.crate"
204 +
CHECKSUM_SHA256_65="6e8ccc4ea9f6acc32d102c0f6d471d11d913ad15f20c04de743374861fa1d414"
205 +
SOURCE_FILENAME_65="compression-core-0.4.33.tar.gz"
206 +
SOURCE_URI_66="https://static.crates.io/crates/const-oid/const-oid-0.10.2.crate"
207 +
CHECKSUM_SHA256_66="a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c"
208 +
SOURCE_FILENAME_66="const-oid-0.10.2.tar.gz"
209 +
SOURCE_URI_67="https://static.crates.io/crates/const-oid/const-oid-0.9.6.crate"
210 +
CHECKSUM_SHA256_67="c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8"
211 +
SOURCE_FILENAME_67="const-oid-0.9.6.tar.gz"
212 +
SOURCE_URI_68="https://static.crates.io/crates/constant_time_eq/constant_time_eq-0.4.2.crate"
213 +
CHECKSUM_SHA256_68="3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b"
214 +
SOURCE_FILENAME_68="constant_time_eq-0.4.2.tar.gz"
215 +
SOURCE_URI_69="https://static.crates.io/crates/cookie/cookie-0.18.2.crate"
216 +
CHECKSUM_SHA256_69="1a373e3602691c3cdea496d2f0ee5935151e6168fe87739483c463db1b2f2f87"
217 +
SOURCE_FILENAME_69="cookie-0.18.2.tar.gz"
218 +
SOURCE_URI_70="https://static.crates.io/crates/core_detect/core_detect-1.0.0.crate"
219 +
CHECKSUM_SHA256_70="7f8f80099a98041a3d1622845c271458a2d73e688351bf3cb999266764b81d48"
220 +
SOURCE_FILENAME_70="core_detect-1.0.0.tar.gz"
221 +
SOURCE_URI_71="https://static.crates.io/crates/cpufeatures/cpufeatures-0.2.17.crate"
222 +
CHECKSUM_SHA256_71="59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
223 +
SOURCE_FILENAME_71="cpufeatures-0.2.17.tar.gz"
224 +
SOURCE_URI_72="https://static.crates.io/crates/cpufeatures/cpufeatures-0.3.1.crate"
225 +
CHECKSUM_SHA256_72="5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566"
226 +
SOURCE_FILENAME_72="cpufeatures-0.3.1.tar.gz"
227 +
SOURCE_URI_73="https://static.crates.io/crates/crc/crc-3.4.0.crate"
228 +
CHECKSUM_SHA256_73="5eb8a2a1cd12ab0d987a5d5e825195d372001a4094a0376319d5a0ad71c1ba0d"
229 +
SOURCE_FILENAME_73="crc-3.4.0.tar.gz"
230 +
SOURCE_URI_74="https://static.crates.io/crates/crc-catalog/crc-catalog-2.5.0.crate"
231 +
CHECKSUM_SHA256_74="217698eaf96b4a3f0bc4f3662aaa55bdf913cd54d7204591faa790070c6d0853"
232 +
SOURCE_FILENAME_74="crc-catalog-2.5.0.tar.gz"
233 +
SOURCE_URI_75="https://static.crates.io/crates/crc32fast/crc32fast-1.5.2.crate"
234 +
CHECKSUM_SHA256_75="01a7799fd6b852db0e61728dde9a204c423b44d689dbd432522543614b490e78"
235 +
SOURCE_FILENAME_75="crc32fast-1.5.2.tar.gz"
236 +
SOURCE_URI_76="https://static.crates.io/crates/crossbeam-deque/crossbeam-deque-0.8.8.crate"
237 +
CHECKSUM_SHA256_76="622f3fc73690be383c7214310406f28a90e6edeadc3cea882f9d71e495b9711a"
238 +
SOURCE_FILENAME_76="crossbeam-deque-0.8.8.tar.gz"
239 +
SOURCE_URI_77="https://static.crates.io/crates/crossbeam-epoch/crossbeam-epoch-0.9.21.crate"
240 +
CHECKSUM_SHA256_77="dc74980687109a3b14c72fd458107bf0baa1da1a1a805e178d15501ba9b86d9d"
241 +
SOURCE_FILENAME_77="crossbeam-epoch-0.9.21.tar.gz"
242 +
SOURCE_URI_78="https://static.crates.io/crates/crossbeam-queue/crossbeam-queue-0.3.14.crate"
243 +
CHECKSUM_SHA256_78="03e8bd762f7479489c70ed6c768ddca99d7296857de437a68dcb2a94365b3fae"
244 +
SOURCE_FILENAME_78="crossbeam-queue-0.3.14.tar.gz"
245 +
SOURCE_URI_79="https://static.crates.io/crates/crossbeam-utils/crossbeam-utils-0.8.23.crate"
246 +
CHECKSUM_SHA256_79="a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6"
247 +
SOURCE_FILENAME_79="crossbeam-utils-0.8.23.tar.gz"
248 +
SOURCE_URI_80="https://static.crates.io/crates/crypto-common/crypto-common-0.1.7.crate"
249 +
CHECKSUM_SHA256_80="78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
250 +
SOURCE_FILENAME_80="crypto-common-0.1.7.tar.gz"
251 +
SOURCE_URI_81="https://static.crates.io/crates/crypto-common/crypto-common-0.2.2.crate"
252 +
CHECKSUM_SHA256_81="ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453"
253 +
SOURCE_FILENAME_81="crypto-common-0.2.2.tar.gz"
254 +
SOURCE_URI_82="https://static.crates.io/crates/cssparser/cssparser-0.38.0.crate"
255 +
CHECKSUM_SHA256_82="11119743ad110e8c1bdccd930d7f5c30c99e5fc76a7b63ec9807e84eef0c5f59"
256 +
SOURCE_FILENAME_82="cssparser-0.38.0.tar.gz"
257 +
SOURCE_URI_83="https://static.crates.io/crates/ctutils/ctutils-0.4.2.crate"
258 +
CHECKSUM_SHA256_83="7d5515a3834141de9eafb9717ad39eea8247b5674e6066c404e8c4b365d2a29e"
259 +
SOURCE_FILENAME_83="ctutils-0.4.2.tar.gz"
260 +
SOURCE_URI_84="https://static.crates.io/crates/curve25519-dalek/curve25519-dalek-5.0.0.crate"
261 +
CHECKSUM_SHA256_84="b5eed333089e2e1c1ac8c6c0398e5e2497b4c9926ca6d0365ed1e099afa5bc23"
262 +
SOURCE_FILENAME_84="curve25519-dalek-5.0.0.tar.gz"
263 +
SOURCE_URI_85="https://static.crates.io/crates/curve25519-dalek-derive/curve25519-dalek-derive-0.1.1.crate"
264 +
CHECKSUM_SHA256_85="f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3"
265 +
SOURCE_FILENAME_85="curve25519-dalek-derive-0.1.1.tar.gz"
266 +
SOURCE_URI_86="https://static.crates.io/crates/data-encoding/data-encoding-2.11.1.crate"
267 +
CHECKSUM_SHA256_86="4583a4551df46e2792f82ceeac45e850d2e2d5debba0b91f102385cda5b11f06"
268 +
SOURCE_FILENAME_86="data-encoding-2.11.1.tar.gz"
269 +
SOURCE_URI_87="https://static.crates.io/crates/der/der-0.7.10.crate"
270 +
CHECKSUM_SHA256_87="e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb"
271 +
SOURCE_FILENAME_87="der-0.7.10.tar.gz"
272 +
SOURCE_URI_88="https://static.crates.io/crates/der/der-0.8.2.crate"
273 +
CHECKSUM_SHA256_88="a878c850e9e421b20262e9b41f9c860e4785fa07541c266b62ff9d1ef998a80a"
274 +
SOURCE_FILENAME_88="der-0.8.2.tar.gz"
275 +
SOURCE_URI_89="https://static.crates.io/crates/der-parser/der-parser-10.0.0.crate"
276 +
CHECKSUM_SHA256_89="07da5016415d5a3c4dd39b11ed26f915f52fc4e0dc197d87908bc916e51bc1a6"
277 +
SOURCE_FILENAME_89="der-parser-10.0.0.tar.gz"
278 +
SOURCE_URI_90="https://static.crates.io/crates/deranged/deranged-0.5.8.crate"
279 +
CHECKSUM_SHA256_90="7cd812cc2bc1d69d4764bd80df88b4317eaef9e773c75226407d9bc0876b211c"
280 +
SOURCE_FILENAME_90="deranged-0.5.8.tar.gz"
281 +
SOURCE_URI_91="https://static.crates.io/crates/derive_arbitrary/derive_arbitrary-1.4.2.crate"
282 +
CHECKSUM_SHA256_91="1e567bd82dcff979e4b03460c307b3cdc9e96fde3d73bed1496d2bc75d9dd62a"
283 +
SOURCE_FILENAME_91="derive_arbitrary-1.4.2.tar.gz"
284 +
SOURCE_URI_92="https://static.crates.io/crates/digest/digest-0.10.7.crate"
285 +
CHECKSUM_SHA256_92="9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
286 +
SOURCE_FILENAME_92="digest-0.10.7.tar.gz"
287 +
SOURCE_URI_93="https://static.crates.io/crates/digest/digest-0.11.3.crate"
288 +
CHECKSUM_SHA256_93="f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
289 +
SOURCE_FILENAME_93="digest-0.11.3.tar.gz"
290 +
SOURCE_URI_94="https://static.crates.io/crates/displaydoc/displaydoc-0.2.7.crate"
291 +
CHECKSUM_SHA256_94="c6232dd377dcc64799954cbd3a9bb882e9cdc1308ccd87b1c098f1fb2eaf82a8"
292 +
SOURCE_FILENAME_94="displaydoc-0.2.7.tar.gz"
293 +
SOURCE_URI_95="https://static.crates.io/crates/dotenvy/dotenvy-0.15.7.crate"
294 +
CHECKSUM_SHA256_95="1aaf95b3e5c8f23aa320147307562d361db0ae0d51242340f558153b4eb2439b"
295 +
SOURCE_FILENAME_95="dotenvy-0.15.7.tar.gz"
296 +
SOURCE_URI_96="https://static.crates.io/crates/dtoa/dtoa-1.0.11.crate"
297 +
CHECKSUM_SHA256_96="4c3cf4824e2d5f025c7b531afcb2325364084a16806f6d47fbc1f5fbd9960590"
298 +
SOURCE_FILENAME_96="dtoa-1.0.11.tar.gz"
299 +
SOURCE_URI_97="https://static.crates.io/crates/dtoa-short/dtoa-short-0.3.5.crate"
300 +
CHECKSUM_SHA256_97="cd1511a7b6a56299bd043a9c167a6d2bfb37bf84a6dfceaba651168adfb43c87"
301 +
SOURCE_FILENAME_97="dtoa-short-0.3.5.tar.gz"
302 +
SOURCE_URI_98="https://static.crates.io/crates/dyn-clone/dyn-clone-1.0.20.crate"
303 +
CHECKSUM_SHA256_98="d0881ea181b1df73ff77ffaaf9c7544ecc11e82fba9b5f27b262a3c73a332555"
304 +
SOURCE_FILENAME_98="dyn-clone-1.0.20.tar.gz"
305 +
SOURCE_URI_99="https://static.crates.io/crates/ed25519/ed25519-3.0.0.crate"
306 +
CHECKSUM_SHA256_99="29fcf32e6c73d1079f83ab4d782de2d81620346a5f38c6237a86a22f8368980a"
307 +
SOURCE_FILENAME_99="ed25519-3.0.0.tar.gz"
308 +
SOURCE_URI_100="https://static.crates.io/crates/ed25519-dalek/ed25519-dalek-3.0.0.crate"
309 +
CHECKSUM_SHA256_100="6ebaa1a2bf1290ab3bfe5a7b771d050ebffab2711c19a81691c683a5144a25de"
310 +
SOURCE_FILENAME_100="ed25519-dalek-3.0.0.tar.gz"
311 +
SOURCE_URI_101="https://static.crates.io/crates/either/either-1.18.0.crate"
312 +
CHECKSUM_SHA256_101="252afb9ae5eaa683babdc6a068b3f5726eb19e05070c731f9b2a23a7c3e8ed34"
313 +
SOURCE_FILENAME_101="either-1.18.0.tar.gz"
314 +
SOURCE_URI_102="https://static.crates.io/crates/email-encoding/email-encoding-0.4.2.crate"
315 +
CHECKSUM_SHA256_102="420b9da095f052ea597503e39073b5b3c522f7db933fbac202d91d24492693fd"
316 +
SOURCE_FILENAME_102="email-encoding-0.4.2.tar.gz"
317 +
SOURCE_URI_103="https://static.crates.io/crates/email_address/email_address-0.2.9.crate"
318 +
CHECKSUM_SHA256_103="e079f19b08ca6239f47f8ba8509c11cf3ea30095831f7fed61441475edd8c449"
319 +
SOURCE_FILENAME_103="email_address-0.2.9.tar.gz"
320 +
SOURCE_URI_104="https://static.crates.io/crates/encoding_rs/encoding_rs-0.8.41.crate"
321 +
CHECKSUM_SHA256_104="7b5ef0006ac9ab233c38522f5ae99cae3625151de8f706cacee1cba4b8e2832a"
322 +
SOURCE_FILENAME_104="encoding_rs-0.8.41.tar.gz"
323 +
SOURCE_URI_105="https://static.crates.io/crates/equivalent/equivalent-1.0.2.crate"
324 +
CHECKSUM_SHA256_105="877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
325 +
SOURCE_FILENAME_105="equivalent-1.0.2.tar.gz"
326 +
SOURCE_URI_106="https://static.crates.io/crates/errno/errno-0.3.14.crate"
327 +
CHECKSUM_SHA256_106="39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
328 +
SOURCE_FILENAME_106="errno-0.3.14.tar.gz"
329 +
SOURCE_URI_107="https://static.crates.io/crates/etcetera/etcetera-0.8.0.crate"
330 +
CHECKSUM_SHA256_107="136d1b5283a1ab77bd9257427ffd09d8667ced0570b6f938942bc7568ed5b943"
331 +
SOURCE_FILENAME_107="etcetera-0.8.0.tar.gz"
332 +
SOURCE_URI_108="https://static.crates.io/crates/event-listener/event-listener-5.4.2.crate"
333 +
CHECKSUM_SHA256_108="5a23add41df1562121a9393cb065eab5146a1242410f23a644851e90cfd669d2"
334 +
SOURCE_FILENAME_108="event-listener-5.4.2.tar.gz"
335 +
SOURCE_URI_109="https://static.crates.io/crates/fancy-regex/fancy-regex-0.16.2.crate"
336 +
CHECKSUM_SHA256_109="998b056554fbe42e03ae0e152895cd1a7e1002aec800fdc6635d20270260c46f"
337 +
SOURCE_FILENAME_109="fancy-regex-0.16.2.tar.gz"
338 +
SOURCE_URI_110="https://static.crates.io/crates/fastrand/fastrand-2.5.0.crate"
339 +
CHECKSUM_SHA256_110="da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223"
340 +
SOURCE_FILENAME_110="fastrand-2.5.0.tar.gz"
341 +
SOURCE_URI_111="https://static.crates.io/crates/fdeflate/fdeflate-0.3.7.crate"
342 +
CHECKSUM_SHA256_111="1e6853b52649d4ac5c0bd02320cddc5ba956bdb407c4b75a2c6b75bf51500f8c"
343 +
SOURCE_FILENAME_111="fdeflate-0.3.7.tar.gz"
344 +
SOURCE_URI_112="https://static.crates.io/crates/fiat-crypto/fiat-crypto-0.3.0.crate"
345 +
CHECKSUM_SHA256_112="64cd1e32ddd350061ae6edb1b082d7c54915b5c672c389143b9a63403a109f24"
346 +
SOURCE_FILENAME_112="fiat-crypto-0.3.0.tar.gz"
347 +
SOURCE_URI_113="https://static.crates.io/crates/filetime/filetime-0.2.29.crate"
348 +
CHECKSUM_SHA256_113="5c287a33c7f0a620c38e641e7f60827713987b3c0f26e8ddc9462cc69cf75759"
349 +
SOURCE_FILENAME_113="filetime-0.2.29.tar.gz"
350 +
SOURCE_URI_114="https://static.crates.io/crates/find-msvc-tools/find-msvc-tools-0.1.14.crate"
351 +
CHECKSUM_SHA256_114="aedcfb3409746eddb02b9e19ebda1c3394f759a152e48ee875a0844d1b955484"
352 +
SOURCE_FILENAME_114="find-msvc-tools-0.1.14.tar.gz"
353 +
SOURCE_URI_115="https://static.crates.io/crates/flate2/flate2-1.1.10.crate"
354 +
CHECKSUM_SHA256_115="6e634e2e0ebac1ee034020da1ca582e17ffe4e0f5e985823721e168928136dcb"
355 +
SOURCE_FILENAME_115="flate2-1.1.10.tar.gz"
356 +
SOURCE_URI_116="https://static.crates.io/crates/flume/flume-0.11.1.crate"
357 +
CHECKSUM_SHA256_116="da0e4dd2a88388a1f4ccc7c9ce104604dab68d9f408dc34cd45823d5a9069095"
358 +
SOURCE_FILENAME_116="flume-0.11.1.tar.gz"
359 +
SOURCE_URI_117="https://static.crates.io/crates/fnv/fnv-1.0.7.crate"
360 +
CHECKSUM_SHA256_117="3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
361 +
SOURCE_FILENAME_117="fnv-1.0.7.tar.gz"
362 +
SOURCE_URI_118="https://static.crates.io/crates/foldhash/foldhash-0.1.5.crate"
363 +
CHECKSUM_SHA256_118="d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
364 +
SOURCE_FILENAME_118="foldhash-0.1.5.tar.gz"
365 +
SOURCE_URI_119="https://static.crates.io/crates/form_urlencoded/form_urlencoded-1.2.2.crate"
366 +
CHECKSUM_SHA256_119="cb4cb245038516f5f85277875cdaa4f7d2c9a0fa0468de06ed190163b1581fcf"
367 +
SOURCE_FILENAME_119="form_urlencoded-1.2.2.tar.gz"
368 +
SOURCE_URI_120="https://static.crates.io/crates/fs-err/fs-err-3.3.1.crate"
369 +
CHECKSUM_SHA256_120="b91aa448ca50d7e79433bdf3ee8d99215430d2ec02ade5aefab2a073a1822e8a"
370 +
SOURCE_FILENAME_120="fs-err-3.3.1.tar.gz"
371 +
SOURCE_URI_121="https://static.crates.io/crates/futures/futures-0.3.34.crate"
372 +
CHECKSUM_SHA256_121="9a31d2a3fbaaeb2af2368bbdd904aa8e812d3c04a1ee10d3171f52d556e5d0a3"
373 +
SOURCE_FILENAME_121="futures-0.3.34.tar.gz"
374 +
SOURCE_URI_122="https://static.crates.io/crates/futures-channel/futures-channel-0.3.34.crate"
375 +
CHECKSUM_SHA256_122="b1f9e3d69d39e4862ffed03ed071a76f9a13ba1d9109d355b0f0aa6b15e393c4"
376 +
SOURCE_FILENAME_122="futures-channel-0.3.34.tar.gz"
377 +
SOURCE_URI_123="https://static.crates.io/crates/futures-core/futures-core-0.3.34.crate"
378 +
CHECKSUM_SHA256_123="92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e"
379 +
SOURCE_FILENAME_123="futures-core-0.3.34.tar.gz"
380 +
SOURCE_URI_124="https://static.crates.io/crates/futures-executor/futures-executor-0.3.34.crate"
381 +
CHECKSUM_SHA256_124="031b47cf1a3c6cc8bc2fc76cd437f521619387907d469316e7c0bc278f1f5432"
382 +
SOURCE_FILENAME_124="futures-executor-0.3.34.tar.gz"
383 +
SOURCE_URI_125="https://static.crates.io/crates/futures-intrusive/futures-intrusive-0.5.0.crate"
384 +
CHECKSUM_SHA256_125="1d930c203dd0b6ff06e0201a4a2fe9149b43c684fd4420555b26d21b1a02956f"
385 +
SOURCE_FILENAME_125="futures-intrusive-0.5.0.tar.gz"
386 +
SOURCE_URI_126="https://static.crates.io/crates/futures-io/futures-io-0.3.34.crate"
387 +
CHECKSUM_SHA256_126="53c0fa8157de1303bfffdaa1cc2a673bfffb60102f76b0ef4441659124373fed"
388 +
SOURCE_FILENAME_126="futures-io-0.3.34.tar.gz"
389 +
SOURCE_URI_127="https://static.crates.io/crates/futures-macro/futures-macro-0.3.34.crate"
390 +
CHECKSUM_SHA256_127="9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44"
391 +
SOURCE_FILENAME_127="futures-macro-0.3.34.tar.gz"
392 +
SOURCE_URI_128="https://static.crates.io/crates/futures-sink/futures-sink-0.3.34.crate"
393 +
CHECKSUM_SHA256_128="1944426bf7d03f1d14f708785e4b33efd750b36d48a157b836b3efc15ede8e1d"
394 +
SOURCE_FILENAME_128="futures-sink-0.3.34.tar.gz"
395 +
SOURCE_URI_129="https://static.crates.io/crates/futures-task/futures-task-0.3.34.crate"
396 +
CHECKSUM_SHA256_129="cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd"
397 +
SOURCE_FILENAME_129="futures-task-0.3.34.tar.gz"
398 +
SOURCE_URI_130="https://static.crates.io/crates/futures-util/futures-util-0.3.34.crate"
399 +
CHECKSUM_SHA256_130="0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc"
400 +
SOURCE_FILENAME_130="futures-util-0.3.34.tar.gz"
401 +
SOURCE_URI_131="https://static.crates.io/crates/generic-array/generic-array-0.14.7.crate"
402 +
CHECKSUM_SHA256_131="85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
403 +
SOURCE_FILENAME_131="generic-array-0.14.7.tar.gz"
404 +
SOURCE_URI_132="https://static.crates.io/crates/getrandom/getrandom-0.2.17.crate"
405 +
CHECKSUM_SHA256_132="ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0"
406 +
SOURCE_FILENAME_132="getrandom-0.2.17.tar.gz"
407 +
SOURCE_URI_133="https://static.crates.io/crates/getrandom/getrandom-0.4.3.crate"
408 +
CHECKSUM_SHA256_133="300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099"
409 +
SOURCE_FILENAME_133="getrandom-0.4.3.tar.gz"
410 +
SOURCE_URI_134="https://static.crates.io/crates/globset/globset-0.4.20.crate"
411 +
CHECKSUM_SHA256_134="07c34a9410465b45bd9787443bc7370f37735bad04b0f0cd57ff1a3186c98988"
412 +
SOURCE_FILENAME_134="globset-0.4.20.tar.gz"
413 +
SOURCE_URI_135="https://static.crates.io/crates/h2/h2-0.4.19.crate"
414 +
CHECKSUM_SHA256_135="ef8e5e5a340588f4452631496976cf8636d4a7ecf600239fdc27615d2530bc16"
415 +
SOURCE_FILENAME_135="h2-0.4.19.tar.gz"
416 +
SOURCE_URI_136="https://static.crates.io/crates/hashbrown/hashbrown-0.15.5.crate"
417 +
CHECKSUM_SHA256_136="9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
418 +
SOURCE_FILENAME_136="hashbrown-0.15.5.tar.gz"
419 +
SOURCE_URI_137="https://static.crates.io/crates/hashbrown/hashbrown-0.17.1.crate"
420 +
CHECKSUM_SHA256_137="ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
421 +
SOURCE_FILENAME_137="hashbrown-0.17.1.tar.gz"
422 +
SOURCE_URI_138="https://static.crates.io/crates/hashlink/hashlink-0.10.0.crate"
423 +
CHECKSUM_SHA256_138="7382cf6263419f2d8df38c55d7da83da5c18aef87fc7a7fc1fb1e344edfe14c1"
424 +
SOURCE_FILENAME_138="hashlink-0.10.0.tar.gz"
425 +
SOURCE_URI_139="https://static.crates.io/crates/heck/heck-0.5.0.crate"
426 +
CHECKSUM_SHA256_139="2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
427 +
SOURCE_FILENAME_139="heck-0.5.0.tar.gz"
428 +
SOURCE_URI_140="https://static.crates.io/crates/hex/hex-0.4.3.crate"
429 +
CHECKSUM_SHA256_140="7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
430 +
SOURCE_FILENAME_140="hex-0.4.3.tar.gz"
431 +
SOURCE_URI_141="https://static.crates.io/crates/hkdf/hkdf-0.12.4.crate"
432 +
CHECKSUM_SHA256_141="7b5f8eb2ad728638ea2c7d47a21db23b7b58a72ed6a38256b8a1849f15fbbdf7"
433 +
SOURCE_FILENAME_141="hkdf-0.12.4.tar.gz"
434 +
SOURCE_URI_142="https://static.crates.io/crates/hmac/hmac-0.12.1.crate"
435 +
CHECKSUM_SHA256_142="6c49c37c09c17a53d937dfbb742eb3a961d65a994e6bcdcf37e7399d0cc8ab5e"
436 +
SOURCE_FILENAME_142="hmac-0.12.1.tar.gz"
437 +
SOURCE_URI_143="https://static.crates.io/crates/home/home-0.5.12.crate"
438 +
CHECKSUM_SHA256_143="cc627f471c528ff0c4a49e1d5e60450c8f6461dd6d10ba9dcd3a61d3dff7728d"
439 +
SOURCE_FILENAME_143="home-0.5.12.tar.gz"
440 +
SOURCE_URI_144="https://static.crates.io/crates/html5ever/html5ever-0.40.1.crate"
441 +
CHECKSUM_SHA256_144="456a1a377e608e555d22ddab27ac0114bc7a7b4199078108e34c2aeae6c9b130"
442 +
SOURCE_FILENAME_144="html5ever-0.40.1.tar.gz"
443 +
SOURCE_URI_145="https://static.crates.io/crates/http/http-1.5.0.crate"
444 +
CHECKSUM_SHA256_145="918d3568bebf352712bc2ef3d46a8bcf1a75b373be6539de198e9105cbbf9ce0"
445 +
SOURCE_FILENAME_145="http-1.5.0.tar.gz"
446 +
SOURCE_URI_146="https://static.crates.io/crates/http-body/http-body-1.1.0.crate"
447 +
CHECKSUM_SHA256_146="ca2a8f2913ee65f60facd6a5905613afaa448497a0230cc41ce022d93290bc2c"
448 +
SOURCE_FILENAME_146="http-body-1.1.0.tar.gz"
449 +
SOURCE_URI_147="https://static.crates.io/crates/http-body-util/http-body-util-0.1.5.crate"
450 +
CHECKSUM_SHA256_147="23169fe34a5fbcdd3f3862e78fb9b6fccd5f02a6dc6f732547005d45631ce71c"
451 +
SOURCE_FILENAME_147="http-body-util-0.1.5.tar.gz"
452 +
SOURCE_URI_148="https://static.crates.io/crates/httparse/httparse-1.10.1.crate"
453 +
CHECKSUM_SHA256_148="6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87"
454 +
SOURCE_FILENAME_148="httparse-1.10.1.tar.gz"
455 +
SOURCE_URI_149="https://static.crates.io/crates/httpdate/httpdate-1.0.3.crate"
456 +
CHECKSUM_SHA256_149="df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
457 +
SOURCE_FILENAME_149="httpdate-1.0.3.tar.gz"
458 +
SOURCE_URI_150="https://static.crates.io/crates/humantime/humantime-2.4.0.crate"
459 +
CHECKSUM_SHA256_150="15cdd26707701c53297e2fa6afb323d55fbc1d0810c3aec078ae3ef0424c3c15"
460 +
SOURCE_FILENAME_150="humantime-2.4.0.tar.gz"
461 +
SOURCE_URI_151="https://static.crates.io/crates/hybrid-array/hybrid-array-0.4.15.crate"
462 +
CHECKSUM_SHA256_151="27f864f10dfb56725ce5ce5472bc52252c8f93a4ab86327122cebf62c5f59a17"
463 +
SOURCE_FILENAME_151="hybrid-array-0.4.15.tar.gz"
464 +
SOURCE_URI_152="https://static.crates.io/crates/hyper/hyper-1.11.1.crate"
465 +
CHECKSUM_SHA256_152="27b501faa50e7a26c3d3560ca625132f4078a17771f4810baf70475ae48cbe43"
466 +
SOURCE_FILENAME_152="hyper-1.11.1.tar.gz"
467 +
SOURCE_URI_153="https://static.crates.io/crates/hyper-rustls/hyper-rustls-0.27.10.crate"
468 +
CHECKSUM_SHA256_153="dfa8e654703247911e29c23fbeaa261834bd9bb74efba2f9acddc37bfb127f53"
469 +
SOURCE_FILENAME_153="hyper-rustls-0.27.10.tar.gz"
470 +
SOURCE_URI_154="https://static.crates.io/crates/hyper-util/hyper-util-0.1.21.crate"
471 +
CHECKSUM_SHA256_154="ddc03d96684f9226b8a787cdb71488417b53ab5ea8fdb1dac946cb9431cc8bff"
472 +
SOURCE_FILENAME_154="hyper-util-0.1.21.tar.gz"
473 +
SOURCE_URI_155="https://static.crates.io/crates/iana-time-zone-haiku/iana-time-zone-haiku-0.1.2.crate"
474 +
CHECKSUM_SHA256_155="f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f"
475 +
SOURCE_FILENAME_155="iana-time-zone-haiku-0.1.2.tar.gz"
476 +
SOURCE_URI_156="https://static.crates.io/crates/icu_collections/icu_collections-2.3.0.crate"
477 +
CHECKSUM_SHA256_156="fa68d21081c4a05d5a901a1c62add574c77048b6a1c67be3b50ce0b60d4ca513"
478 +
SOURCE_FILENAME_156="icu_collections-2.3.0.tar.gz"
479 +
SOURCE_URI_157="https://static.crates.io/crates/icu_locale_core/icu_locale_core-2.3.0.crate"
480 +
CHECKSUM_SHA256_157="d56e28588da92eee5c3201a6eff33fabdd49b62269c8938d4ff050ce4d900deb"
481 +
SOURCE_FILENAME_157="icu_locale_core-2.3.0.tar.gz"
482 +
SOURCE_URI_158="https://static.crates.io/crates/icu_normalizer/icu_normalizer-2.3.0.crate"
483 +
CHECKSUM_SHA256_158="12f9cf5f235641ed274641dd81c3f28d870e276763d0797aeeab72317b1c646f"
484 +
SOURCE_FILENAME_158="icu_normalizer-2.3.0.tar.gz"
485 +
SOURCE_URI_159="https://static.crates.io/crates/icu_normalizer_data/icu_normalizer_data-2.3.0.crate"
486 +
CHECKSUM_SHA256_159="1563da1ed3e0b3bf3d74c9b85917ac9c56464d2f57242270c09c9e752f8021a0"
487 +
SOURCE_FILENAME_159="icu_normalizer_data-2.3.0.tar.gz"
488 +
SOURCE_URI_160="https://static.crates.io/crates/icu_properties/icu_properties-2.3.0.crate"
489 +
CHECKSUM_SHA256_160="7e7ca276ad3145661a65914e6daf131ca5120cd3dcee8f8f3214b8875184a148"
490 +
SOURCE_FILENAME_160="icu_properties-2.3.0.tar.gz"
491 +
SOURCE_URI_161="https://static.crates.io/crates/icu_properties_data/icu_properties_data-2.3.0.crate"
492 +
CHECKSUM_SHA256_161="e590f038c1464a96894fd6d10127e90a8be4509f56ff7ecef851b15cee0b7caa"
493 +
SOURCE_FILENAME_161="icu_properties_data-2.3.0.tar.gz"
494 +
SOURCE_URI_162="https://static.crates.io/crates/icu_provider/icu_provider-2.3.1.crate"
495 +
CHECKSUM_SHA256_162="d27bbb9d3abbefac45d55f647c9de1d44aafcd1186eb91879afef17c396c3e73"
496 +
SOURCE_FILENAME_162="icu_provider-2.3.1.tar.gz"
497 +
SOURCE_URI_163="https://static.crates.io/crates/idna/idna-1.1.0.crate"
498 +
CHECKSUM_SHA256_163="3b0875f23caa03898994f6ddc501886a45c7d3d62d04d2d90788d47be1b1e4de"
499 +
SOURCE_FILENAME_163="idna-1.1.0.tar.gz"
500 +
SOURCE_URI_164="https://static.crates.io/crates/idna_adapter/idna_adapter-1.2.2.crate"
501 +
CHECKSUM_SHA256_164="cb68373c0d6620ef8105e855e7745e18b0d00d3bdb07fb532e434244cdb9a714"
502 +
SOURCE_FILENAME_164="idna_adapter-1.2.2.tar.gz"
503 +
SOURCE_URI_165="https://static.crates.io/crates/ignore/ignore-0.4.33.crate"
504 +
CHECKSUM_SHA256_165="00b69833ed729dc5aa7d19541d96d6cf8e9137194207a04916d658e43168402f"
505 +
SOURCE_FILENAME_165="ignore-0.4.33.tar.gz"
506 +
SOURCE_URI_166="https://static.crates.io/crates/image/image-0.25.10.crate"
507 +
CHECKSUM_SHA256_166="85ab80394333c02fe689eaf900ab500fbd0c2213da414687ebf995a65d5a6104"
508 +
SOURCE_FILENAME_166="image-0.25.10.tar.gz"
509 +
SOURCE_URI_167="https://static.crates.io/crates/image-webp/image-webp-0.2.4.crate"
510 +
CHECKSUM_SHA256_167="525e9ff3e1a4be2fbea1fdf0e98686a6d98b4d8f937e1bf7402245af1909e8c3"
511 +
SOURCE_FILENAME_167="image-webp-0.2.4.tar.gz"
512 +
SOURCE_URI_168="https://static.crates.io/crates/indexmap/indexmap-2.14.2.crate"
513 +
CHECKSUM_SHA256_168="cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855"
514 +
SOURCE_FILENAME_168="indexmap-2.14.2.tar.gz"
515 +
SOURCE_URI_169="https://static.crates.io/crates/ipnet/ipnet-2.12.2.crate"
516 +
CHECKSUM_SHA256_169="791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
517 +
SOURCE_FILENAME_169="ipnet-2.12.2.tar.gz"
518 +
SOURCE_URI_170="https://static.crates.io/crates/is_terminal_polyfill/is_terminal_polyfill-1.70.2.crate"
519 +
CHECKSUM_SHA256_170="a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695"
520 +
SOURCE_FILENAME_170="is_terminal_polyfill-1.70.2.tar.gz"
521 +
SOURCE_URI_171="https://static.crates.io/crates/itertools/itertools-0.15.0.crate"
522 +
CHECKSUM_SHA256_171="8b4baf93f58d4425749ca49a51c50ebab072c5df6994d08fed93541c331481dc"
523 +
SOURCE_FILENAME_171="itertools-0.15.0.tar.gz"
524 +
SOURCE_URI_172="https://static.crates.io/crates/itoa/itoa-1.0.18.crate"
525 +
CHECKSUM_SHA256_172="8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
526 +
SOURCE_FILENAME_172="itoa-1.0.18.tar.gz"
527 +
SOURCE_URI_173="https://static.crates.io/crates/jobserver/jobserver-0.1.35.crate"
528 +
CHECKSUM_SHA256_173="1c00acbd29eabad4a2392fa0e921c874934dbbf4194312ad20f04a0ed67a3cb3"
529 +
SOURCE_FILENAME_173="jobserver-0.1.35.tar.gz"
530 +
SOURCE_URI_174="https://static.crates.io/crates/js-sys/js-sys-0.3.106.crate"
531 +
CHECKSUM_SHA256_174="7883d941dae510fb2d978fc3fe018c71c9e2892fd38854de3e8b92c2e5ad9cc5"
532 +
SOURCE_FILENAME_174="js-sys-0.3.106.tar.gz"
533 +
SOURCE_URI_175="https://static.crates.io/crates/kdl/kdl-6.7.1.crate"
534 +
CHECKSUM_SHA256_175="082ddf81b2acd76fe04412655d17befedfff1837db772f8e74c38050d25ed670"
535 +
SOURCE_FILENAME_175="kdl-6.7.1.tar.gz"
536 +
SOURCE_URI_176="https://static.crates.io/crates/keccak/keccak-0.2.2.crate"
537 +
CHECKSUM_SHA256_176="d8f198d1db720e4940b5a493201d199d9f24f568f8f746bd13706243a2f71598"
538 +
SOURCE_FILENAME_176="keccak-0.2.2.tar.gz"
539 +
SOURCE_URI_177="https://static.crates.io/crates/lazy_static/lazy_static-1.5.0.crate"
540 +
CHECKSUM_SHA256_177="bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
541 +
SOURCE_FILENAME_177="lazy_static-1.5.0.tar.gz"
542 +
SOURCE_URI_178="https://static.crates.io/crates/lettre/lettre-0.11.23.crate"
543 +
CHECKSUM_SHA256_178="f2c646bd5cc763b1087b15493e29a64be6147ba8f19342004fa52048ee596eae"
544 +
SOURCE_FILENAME_178="lettre-0.11.23.tar.gz"
545 +
SOURCE_URI_179="https://static.crates.io/crates/libc/libc-0.2.189.crate"
546 +
CHECKSUM_SHA256_179="3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
547 +
SOURCE_FILENAME_179="libc-0.2.189.tar.gz"
548 +
SOURCE_URI_180="https://static.crates.io/crates/libm/libm-0.2.16.crate"
549 +
CHECKSUM_SHA256_180="b6d2cec3eae94f9f509c767b45932f1ada8350c4bdb85af2fcab4a3c14807981"
550 +
SOURCE_FILENAME_180="libm-0.2.16.tar.gz"
551 +
SOURCE_URI_181="https://static.crates.io/crates/libredox/libredox-0.1.25.crate"
552 +
CHECKSUM_SHA256_181="61ff90caf6077a803a240f62fdbe88645a890bbca49ef8174c3cb0404362171d"
553 +
SOURCE_FILENAME_181="libredox-0.1.25.tar.gz"
554 +
SOURCE_URI_182="https://static.crates.io/crates/libsqlite3-sys/libsqlite3-sys-0.30.1.crate"
555 +
CHECKSUM_SHA256_182="2e99fb7a497b1e3339bc746195567ed8d3e24945ecd636e3619d20b9de9e9149"
556 +
SOURCE_FILENAME_182="libsqlite3-sys-0.30.1.tar.gz"
557 +
SOURCE_URI_183="https://static.crates.io/crates/linked-hash-map/linked-hash-map-0.5.6.crate"
558 +
CHECKSUM_SHA256_183="0717cef1bc8b636c6e1c1bbdefc09e6322da8a9321966e8928ef80d20f7f770f"
559 +
SOURCE_FILENAME_183="linked-hash-map-0.5.6.tar.gz"
560 +
SOURCE_URI_184="https://static.crates.io/crates/linux-raw-sys/linux-raw-sys-0.12.1.crate"
561 +
CHECKSUM_SHA256_184="32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
562 +
SOURCE_FILENAME_184="linux-raw-sys-0.12.1.tar.gz"
563 +
SOURCE_URI_185="https://static.crates.io/crates/litemap/litemap-0.8.3.crate"
564 +
CHECKSUM_SHA256_185="47d9d19d1d6efa0109d2f65ff4c85cddd50bd572e5a00127ab10987290bcefae"
565 +
SOURCE_FILENAME_185="litemap-0.8.3.tar.gz"
566 +
SOURCE_URI_186="https://static.crates.io/crates/lock_api/lock_api-0.4.14.crate"
567 +
CHECKSUM_SHA256_186="224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965"
568 +
SOURCE_FILENAME_186="lock_api-0.4.14.tar.gz"
569 +
SOURCE_URI_187="https://static.crates.io/crates/log/log-0.4.34.crate"
570 +
CHECKSUM_SHA256_187="f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
571 +
SOURCE_FILENAME_187="log-0.4.34.tar.gz"
572 +
SOURCE_URI_188="https://static.crates.io/crates/lru-slab/lru-slab-0.1.3.crate"
573 +
CHECKSUM_SHA256_188="4050469837a6ff301cd14c1f8f24f88549e6d548f24f64e2148eb0f72cebc51f"
574 +
SOURCE_FILENAME_188="lru-slab-0.1.3.tar.gz"
575 +
SOURCE_URI_189="https://static.crates.io/crates/lzma-rs/lzma-rs-0.3.0.crate"
576 +
CHECKSUM_SHA256_189="297e814c836ae64db86b36cf2a557ba54368d03f6afcd7d947c266692f71115e"
577 +
SOURCE_FILENAME_189="lzma-rs-0.3.0.tar.gz"
578 +
SOURCE_URI_190="https://static.crates.io/crates/maplit/maplit-1.0.2.crate"
579 +
CHECKSUM_SHA256_190="3e2e65a1a2e43cfcb47a895c4c8b10d1f4a61097f9f254f183aee60cad9c651d"
580 +
SOURCE_FILENAME_190="maplit-1.0.2.tar.gz"
581 +
SOURCE_URI_191="https://static.crates.io/crates/markup5ever/markup5ever-0.40.0.crate"
582 +
CHECKSUM_SHA256_191="0ab3dc68ac4a0f5719e560136778c1ee716e296030d75dbd4484e37e39e3a842"
583 +
SOURCE_FILENAME_191="markup5ever-0.40.0.tar.gz"
584 +
SOURCE_URI_192="https://static.crates.io/crates/matchers/matchers-0.2.0.crate"
585 +
CHECKSUM_SHA256_192="d1525a2a28c7f4fa0fc98bb91ae755d1e2d1505079e05539e35bc876b5d65ae9"
586 +
SOURCE_FILENAME_192="matchers-0.2.0.tar.gz"
587 +
SOURCE_URI_193="https://static.crates.io/crates/matchit/matchit-0.8.4.crate"
588 +
CHECKSUM_SHA256_193="47e1ffaa40ddd1f3ed91f717a33c8c0ee23fff369e3aa8772b9605cc1d22f4c3"
589 +
SOURCE_FILENAME_193="matchit-0.8.4.tar.gz"
590 +
SOURCE_URI_194="https://static.crates.io/crates/md-5/md-5-0.10.6.crate"
591 +
CHECKSUM_SHA256_194="d89e7ee0cfbedfc4da3340218492196241d89eefb6dab27de5df917a6d2e78cf"
592 +
SOURCE_FILENAME_194="md-5-0.10.6.tar.gz"
593 +
SOURCE_URI_195="https://static.crates.io/crates/md-5/md-5-0.11.0.crate"
594 +
CHECKSUM_SHA256_195="69b6441f590336821bb897fb28fc622898ccceb1d6cea3fde5ea86b090c4de98"
595 +
SOURCE_FILENAME_195="md-5-0.11.0.tar.gz"
596 +
SOURCE_URI_196="https://static.crates.io/crates/memchr/memchr-2.8.3.crate"
597 +
CHECKSUM_SHA256_196="cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
598 +
SOURCE_FILENAME_196="memchr-2.8.3.tar.gz"
599 +
SOURCE_URI_197="https://static.crates.io/crates/miette/miette-7.6.0.crate"
600 +
CHECKSUM_SHA256_197="5f98efec8807c63c752b5bd61f862c165c115b0a35685bdcfd9238c7aeb592b7"
601 +
SOURCE_FILENAME_197="miette-7.6.0.tar.gz"
602 +
SOURCE_URI_198="https://static.crates.io/crates/miette-derive/miette-derive-7.6.0.crate"
603 +
CHECKSUM_SHA256_198="db5b29714e950dbb20d5e6f74f9dcec4edbcc1067bb7f8ed198c097b8c1a818b"
604 +
SOURCE_FILENAME_198="miette-derive-7.6.0.tar.gz"
605 +
SOURCE_URI_199="https://static.crates.io/crates/mime/mime-0.3.17.crate"
606 +
CHECKSUM_SHA256_199="6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a"
607 +
SOURCE_FILENAME_199="mime-0.3.17.tar.gz"
608 +
SOURCE_URI_200="https://static.crates.io/crates/minimal-lexical/minimal-lexical-0.2.1.crate"
609 +
CHECKSUM_SHA256_200="68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
610 +
SOURCE_FILENAME_200="minimal-lexical-0.2.1.tar.gz"
611 +
SOURCE_URI_201="https://static.crates.io/crates/miniz_oxide/miniz_oxide-0.8.9.crate"
612 +
CHECKSUM_SHA256_201="1fa76a2c86f704bdb222d66965fb3d63269ce38518b83cb0575fca855ebb6316"
613 +
SOURCE_FILENAME_201="miniz_oxide-0.8.9.tar.gz"
614 +
SOURCE_URI_202="https://static.crates.io/crates/miniz_oxide/miniz_oxide-0.9.1.crate"
615 +
CHECKSUM_SHA256_202="b63fbc4a50860e98e7b2aa7804ded1db5cbc3aff9193adaff57a6931bf7c4b4c"
616 +
SOURCE_FILENAME_202="miniz_oxide-0.9.1.tar.gz"
617 +
SOURCE_URI_203="https://static.crates.io/crates/mio/mio-1.2.3.crate"
618 +
CHECKSUM_SHA256_203="4b18443e9c262bfe8fa82f51666e2642c53393f7e5c27b3e1aeab922cff5b9d8"
619 +
SOURCE_FILENAME_203="mio-1.2.3.tar.gz"
620 +
SOURCE_URI_204="https://static.crates.io/crates/ml-dsa/ml-dsa-0.1.1.crate"
621 +
CHECKSUM_SHA256_204="add6b9d92e496f16f4526d68ff29da1483aba4b119baeab8bed3b9e3544a6f3d"
622 +
SOURCE_FILENAME_204="ml-dsa-0.1.1.tar.gz"
623 +
SOURCE_URI_205="https://static.crates.io/crates/module-lattice/module-lattice-0.2.3.crate"
624 +
CHECKSUM_SHA256_205="0c61b87c9683ab7cb1c6871d261ad5479b6b10ceb52c4352aaca3b5d35a8febe"
625 +
SOURCE_FILENAME_205="module-lattice-0.2.3.tar.gz"
626 +
SOURCE_URI_206="https://static.crates.io/crates/moxcms/moxcms-0.8.1.crate"
627 +
CHECKSUM_SHA256_206="bb85c154ba489f01b25c0d36ae69a87e4a1c73a72631fc6c0eb6dde34a73e44b"
628 +
SOURCE_FILENAME_206="moxcms-0.8.1.tar.gz"
629 +
SOURCE_URI_207="https://static.crates.io/crates/multer/multer-3.1.0.crate"
630 +
CHECKSUM_SHA256_207="83e87776546dc87511aa5ee218730c92b666d7264ab6ed41f9d215af9cd5224b"
631 +
SOURCE_FILENAME_207="multer-3.1.0.tar.gz"
632 +
SOURCE_URI_208="https://static.crates.io/crates/multiversion/multiversion-0.9.0.crate"
633 +
CHECKSUM_SHA256_208="b4ca4bea16ffc3f443cf7d866912118196bfef4c6a1556ca00f9f9b00bb43f7c"
634 +
SOURCE_FILENAME_208="multiversion-0.9.0.tar.gz"
635 +
SOURCE_URI_209="https://static.crates.io/crates/multiversion-macros/multiversion-macros-0.9.0.crate"
636 +
CHECKSUM_SHA256_209="0d416831a7317ef4b08bee00b69cbbb9c8763da7959a7026244d6266869f9c83"
637 +
SOURCE_FILENAME_209="multiversion-macros-0.9.0.tar.gz"
638 +
SOURCE_URI_210="https://static.crates.io/crates/multiversion_no_op/multiversion_no_op-1.0.0.crate"
639 +
CHECKSUM_SHA256_210="743fb55ba31b18fb1ecef6bdc9aa2743314978ac084044301a7eee33fb99a20d"
640 +
SOURCE_FILENAME_210="multiversion_no_op-1.0.0.tar.gz"
641 +
SOURCE_URI_211="https://static.crates.io/crates/new_debug_unreachable/new_debug_unreachable-1.0.6.crate"
642 +
CHECKSUM_SHA256_211="650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086"
643 +
SOURCE_FILENAME_211="new_debug_unreachable-1.0.6.tar.gz"
644 +
SOURCE_URI_212="https://static.crates.io/crates/nom/nom-7.1.3.crate"
645 +
CHECKSUM_SHA256_212="d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
646 +
SOURCE_FILENAME_212="nom-7.1.3.tar.gz"
647 +
SOURCE_URI_213="https://static.crates.io/crates/nom/nom-8.0.0.crate"
648 +
CHECKSUM_SHA256_213="df9761775871bdef83bee530e60050f7e54b1105350d6884eb0fb4f46c2f9405"
649 +
SOURCE_FILENAME_213="nom-8.0.0.tar.gz"
650 +
SOURCE_URI_214="https://static.crates.io/crates/nu-ansi-term/nu-ansi-term-0.50.3.crate"
651 +
CHECKSUM_SHA256_214="7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
652 +
SOURCE_FILENAME_214="nu-ansi-term-0.50.3.tar.gz"
653 +
SOURCE_URI_215="https://static.crates.io/crates/num-bigint/num-bigint-0.4.8.crate"
654 +
CHECKSUM_SHA256_215="c89e69e7e0f03bea5ef08013795c25018e101932225a656383bd384495ecc367"
655 +
SOURCE_FILENAME_215="num-bigint-0.4.8.tar.gz"
656 +
SOURCE_URI_216="https://static.crates.io/crates/num-bigint-dig/num-bigint-dig-0.8.6.crate"
657 +
CHECKSUM_SHA256_216="e661dda6640fad38e827a6d4a310ff4763082116fe217f279885c97f511bb0b7"
658 +
SOURCE_FILENAME_216="num-bigint-dig-0.8.6.tar.gz"
659 +
SOURCE_URI_217="https://static.crates.io/crates/num-conv/num-conv-0.2.2.crate"
660 +
CHECKSUM_SHA256_217="521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441"
661 +
SOURCE_FILENAME_217="num-conv-0.2.2.tar.gz"
662 +
SOURCE_URI_218="https://static.crates.io/crates/num-integer/num-integer-0.1.47.crate"
663 +
CHECKSUM_SHA256_218="7ce2d95d4b3734dc35aa2f45e1aa22cd416814592a4f9d9205e11affd5b8e10b"
664 +
SOURCE_FILENAME_218="num-integer-0.1.47.tar.gz"
665 +
SOURCE_URI_219="https://static.crates.io/crates/num-iter/num-iter-0.1.46.crate"
666 +
CHECKSUM_SHA256_219="c92800bd69a1eac91786bcfe9da64a897eb72911b8dc3095decbd07429e8048b"
667 +
SOURCE_FILENAME_219="num-iter-0.1.46.tar.gz"
668 +
SOURCE_URI_220="https://static.crates.io/crates/num-traits/num-traits-0.2.19.crate"
669 +
CHECKSUM_SHA256_220="071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
670 +
SOURCE_FILENAME_220="num-traits-0.2.19.tar.gz"
671 +
SOURCE_URI_221="https://static.crates.io/crates/object_store/object_store-0.14.2.crate"
672 +
CHECKSUM_SHA256_221="f1796bc93603f78c5760a69f2d58badc9618d22adade0a95385bb2adbae4eb94"
673 +
SOURCE_FILENAME_221="object_store-0.14.2.tar.gz"
674 +
SOURCE_URI_222="https://static.crates.io/crates/oid-registry/oid-registry-0.8.1.crate"
675 +
CHECKSUM_SHA256_222="12f40cff3dde1b6087cc5d5f5d4d65712f34016a03ed60e9c08dcc392736b5b7"
676 +
SOURCE_FILENAME_222="oid-registry-0.8.1.tar.gz"
677 +
SOURCE_URI_223="https://static.crates.io/crates/once_cell/once_cell-1.21.4.crate"
678 +
CHECKSUM_SHA256_223="9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
679 +
SOURCE_FILENAME_223="once_cell-1.21.4.tar.gz"
680 +
SOURCE_URI_224="https://static.crates.io/crates/once_cell_polyfill/once_cell_polyfill-1.70.2.crate"
681 +
CHECKSUM_SHA256_224="384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
682 +
SOURCE_FILENAME_224="once_cell_polyfill-1.70.2.tar.gz"
683 +
SOURCE_URI_225="https://static.crates.io/crates/parking/parking-2.2.1.crate"
684 +
CHECKSUM_SHA256_225="f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba"
685 +
SOURCE_FILENAME_225="parking-2.2.1.tar.gz"
686 +
SOURCE_URI_226="https://static.crates.io/crates/parking_lot/parking_lot-0.12.5.crate"
687 +
CHECKSUM_SHA256_226="93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a"
688 +
SOURCE_FILENAME_226="parking_lot-0.12.5.tar.gz"
689 +
SOURCE_URI_227="https://static.crates.io/crates/parking_lot_core/parking_lot_core-0.9.12.crate"
690 +
CHECKSUM_SHA256_227="2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1"
691 +
SOURCE_FILENAME_227="parking_lot_core-0.9.12.tar.gz"
692 +
SOURCE_URI_228="https://static.crates.io/crates/password-hash/password-hash-0.5.0.crate"
693 +
CHECKSUM_SHA256_228="346f04948ba92c43e8469c1ee6736c7563d71012b17d40745260fe106aac2166"
694 +
SOURCE_FILENAME_228="password-hash-0.5.0.tar.gz"
695 +
SOURCE_URI_229="https://static.crates.io/crates/pem/pem-4.0.0.crate"
696 +
CHECKSUM_SHA256_229="d354a98a3d1251555de99e8fdd8afda05573c31b82f59063a7b0a29b5527f120"
697 +
SOURCE_FILENAME_229="pem-4.0.0.tar.gz"
698 +
SOURCE_URI_230="https://static.crates.io/crates/pem-rfc7468/pem-rfc7468-0.7.0.crate"
699 +
CHECKSUM_SHA256_230="88b39c9bfcfc231068454382784bb460aae594343fb030d46e9f50a645418412"
700 +
SOURCE_FILENAME_230="pem-rfc7468-0.7.0.tar.gz"
701 +
SOURCE_URI_231="https://static.crates.io/crates/percent-encoding/percent-encoding-2.3.2.crate"
702 +
CHECKSUM_SHA256_231="9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
703 +
SOURCE_FILENAME_231="percent-encoding-2.3.2.tar.gz"
704 +
SOURCE_URI_232="https://static.crates.io/crates/phf/phf-0.14.0.crate"
705 +
CHECKSUM_SHA256_232="010378780309880b08997fae13be7834dba947d36393bd372f2b1556deb2a2f6"
706 +
SOURCE_FILENAME_232="phf-0.14.0.tar.gz"
707 +
SOURCE_URI_233="https://static.crates.io/crates/phf_codegen/phf_codegen-0.14.0.crate"
708 +
CHECKSUM_SHA256_233="41b585a510fb76fdebead6897982ef2a03a21d8e6cbcca904999742a4afc6ffe"
709 +
SOURCE_FILENAME_233="phf_codegen-0.14.0.tar.gz"
710 +
SOURCE_URI_234="https://static.crates.io/crates/phf_generator/phf_generator-0.14.0.crate"
711 +
CHECKSUM_SHA256_234="aeb62e0959d5a1bebc965f4d15d9e2b7cea002b6b0f5ba8cde6cc26738467100"
712 +
SOURCE_FILENAME_234="phf_generator-0.14.0.tar.gz"
713 +
SOURCE_URI_235="https://static.crates.io/crates/phf_shared/phf_shared-0.14.0.crate"
714 +
CHECKSUM_SHA256_235="c6fd9027e2d9319be6349febd1db4e8d02aa544921200c9b777720ac34a3aa89"
715 +
SOURCE_FILENAME_235="phf_shared-0.14.0.tar.gz"
716 +
SOURCE_URI_236="https://static.crates.io/crates/pin-project-lite/pin-project-lite-0.2.17.crate"
717 +
CHECKSUM_SHA256_236="a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
718 +
SOURCE_FILENAME_236="pin-project-lite-0.2.17.tar.gz"
719 +
SOURCE_URI_237="https://static.crates.io/crates/pkcs1/pkcs1-0.7.5.crate"
720 +
CHECKSUM_SHA256_237="c8ffb9f10fa047879315e6625af03c164b16962a5368d724ed16323b68ace47f"
721 +
SOURCE_FILENAME_237="pkcs1-0.7.5.tar.gz"
722 +
SOURCE_URI_238="https://static.crates.io/crates/pkcs8/pkcs8-0.10.2.crate"
723 +
CHECKSUM_SHA256_238="f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7"
724 +
SOURCE_FILENAME_238="pkcs8-0.10.2.tar.gz"
725 +
SOURCE_URI_239="https://static.crates.io/crates/pkcs8/pkcs8-0.11.0.crate"
726 +
CHECKSUM_SHA256_239="451913da69c775a56034ea8d9003d27ee8948e12443eae7c038ba100a4f21cb7"
727 +
SOURCE_FILENAME_239="pkcs8-0.11.0.tar.gz"
728 +
SOURCE_URI_240="https://static.crates.io/crates/pkg-config/pkg-config-0.3.34.crate"
729 +
CHECKSUM_SHA256_240="f6b464fbc74e149a392436b17d523f769e057cb6877f6a5c4618bc6f11800548"
730 +
SOURCE_FILENAME_240="pkg-config-0.3.34.tar.gz"
731 +
SOURCE_URI_241="https://static.crates.io/crates/plain/plain-0.2.3.crate"
732 +
CHECKSUM_SHA256_241="b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
733 +
SOURCE_FILENAME_241="plain-0.2.3.tar.gz"
734 +
SOURCE_URI_242="https://static.crates.io/crates/plist/plist-1.10.1.crate"
735 +
CHECKSUM_SHA256_242="2896bade328c13f7042a297ea5ac5b0951f6cf989dea5f32c2fd98da398195cb"
736 +
SOURCE_FILENAME_242="plist-1.10.1.tar.gz"
737 +
SOURCE_URI_243="https://static.crates.io/crates/png/png-0.18.1.crate"
738 +
CHECKSUM_SHA256_243="60769b8b31b2a9f263dae2776c37b1b28ae246943cf719eb6946a1db05128a61"
739 +
SOURCE_FILENAME_243="png-0.18.1.tar.gz"
740 +
SOURCE_URI_244="https://static.crates.io/crates/potential_utf/potential_utf-0.1.6.crate"
741 +
CHECKSUM_SHA256_244="d83eb9bc6d8e5cf568e7a1101d60ee05e81ed50ea106026f3d18deeb046d7661"
742 +
SOURCE_FILENAME_244="potential_utf-0.1.6.tar.gz"
743 +
SOURCE_URI_245="https://static.crates.io/crates/powerfmt/powerfmt-0.2.0.crate"
744 +
CHECKSUM_SHA256_245="439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391"
745 +
SOURCE_FILENAME_245="powerfmt-0.2.0.tar.gz"
746 +
SOURCE_URI_246="https://static.crates.io/crates/ppv-lite86/ppv-lite86-0.2.21.crate"
747 +
CHECKSUM_SHA256_246="85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9"
748 +
SOURCE_FILENAME_246="ppv-lite86-0.2.21.tar.gz"
749 +
SOURCE_URI_247="https://static.crates.io/crates/precomputed-hash/precomputed-hash-0.1.1.crate"
750 +
CHECKSUM_SHA256_247="925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c"
751 +
SOURCE_FILENAME_247="precomputed-hash-0.1.1.tar.gz"
752 +
SOURCE_URI_248="https://static.crates.io/crates/proc-macro2/proc-macro2-1.0.107.crate"
753 +
CHECKSUM_SHA256_248="985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
754 +
SOURCE_FILENAME_248="proc-macro2-1.0.107.tar.gz"
755 +
SOURCE_URI_249="https://static.crates.io/crates/pulldown-cmark/pulldown-cmark-0.13.4.crate"
756 +
CHECKSUM_SHA256_249="e9f068eba8e7071c5f9511831b44f32c740d5adf574e990f946ddb53db2f314e"
757 +
SOURCE_FILENAME_249="pulldown-cmark-0.13.4.tar.gz"
758 +
SOURCE_URI_250="https://static.crates.io/crates/pulldown-cmark-escape/pulldown-cmark-escape-0.11.0.crate"
759 +
CHECKSUM_SHA256_250="007d8adb5ddab6f8e3f491ac63566a7d5002cc7ed73901f72057943fa71ae1ae"
760 +
SOURCE_FILENAME_250="pulldown-cmark-escape-0.11.0.tar.gz"
761 +
SOURCE_URI_251="https://static.crates.io/crates/pxfm/pxfm-0.1.30.crate"
762 +
CHECKSUM_SHA256_251="d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
763 +
SOURCE_FILENAME_251="pxfm-0.1.30.tar.gz"
764 +
SOURCE_URI_252="https://static.crates.io/crates/qrcode/qrcode-0.14.1.crate"
765 +
CHECKSUM_SHA256_252="d68782463e408eb1e668cf6152704bd856c78c5b6417adaee3203d8f4c1fc9ec"
766 +
SOURCE_FILENAME_252="qrcode-0.14.1.tar.gz"
767 +
SOURCE_URI_253="https://static.crates.io/crates/quick-error/quick-error-2.0.1.crate"
768 +
CHECKSUM_SHA256_253="a993555f31e5a609f617c12db6250dedcac1b0a85076912c436e6fc9b2c8e6a3"
769 +
SOURCE_FILENAME_253="quick-error-2.0.1.tar.gz"
770 +
SOURCE_URI_254="https://static.crates.io/crates/quick-xml/quick-xml-0.41.0.crate"
771 +
CHECKSUM_SHA256_254="e660451e55124f798a69a5af3f49ccfbefbd41910eefd25caf2393e1f3473ec1"
772 +
SOURCE_FILENAME_254="quick-xml-0.41.0.tar.gz"
773 +
SOURCE_URI_255="https://static.crates.io/crates/quick-xml/quick-xml-0.42.0.crate"
774 +
CHECKSUM_SHA256_255="41b1177fdf999d2321d3fb46ff47159d9c1fb9ad66a4879f8c50a0b504615e9b"
775 +
SOURCE_FILENAME_255="quick-xml-0.42.0.tar.gz"
776 +
SOURCE_URI_256="https://static.crates.io/crates/quinn/quinn-0.11.12.crate"
777 +
CHECKSUM_SHA256_256="4051e23e9185c255a7e33ef59cdbca87a22d359052eecd22fc6b901fb37d9d11"
778 +
SOURCE_FILENAME_256="quinn-0.11.12.tar.gz"
779 +
SOURCE_URI_257="https://static.crates.io/crates/quinn-proto/quinn-proto-0.11.18.crate"
780 +
CHECKSUM_SHA256_257="a9746dbde176634f4f2f1faf2404e30a31b2bc1e9cafb5329c95d8177a18c9fc"
781 +
SOURCE_FILENAME_257="quinn-proto-0.11.18.tar.gz"
782 +
SOURCE_URI_258="https://static.crates.io/crates/quinn-udp/quinn-udp-0.5.15.crate"
783 +
CHECKSUM_SHA256_258="35a133f956daabe89a61a685c2649f13d82d5aa4bd5d12d1277e1072a21c0694"
784 +
SOURCE_FILENAME_258="quinn-udp-0.5.15.tar.gz"
785 +
SOURCE_URI_259="https://static.crates.io/crates/quote/quote-1.0.47.crate"
786 +
CHECKSUM_SHA256_259="1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
787 +
SOURCE_FILENAME_259="quote-1.0.47.tar.gz"
788 +
SOURCE_URI_260="https://static.crates.io/crates/quoted_printable/quoted_printable-0.5.2.crate"
789 +
CHECKSUM_SHA256_260="478e0585659a122aa407eb7e3c0e1fa51b1d8a870038bd29f0cf4a8551eea972"
790 +
SOURCE_FILENAME_260="quoted_printable-0.5.2.tar.gz"
791 +
SOURCE_URI_261="https://static.crates.io/crates/r-efi/r-efi-6.0.0.crate"
792 +
CHECKSUM_SHA256_261="f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
793 +
SOURCE_FILENAME_261="r-efi-6.0.0.tar.gz"
794 +
SOURCE_URI_262="https://static.crates.io/crates/rand/rand-0.10.3.crate"
795 +
CHECKSUM_SHA256_262="65c9fb96cbc91e3478eaae79a69fcd3f1ae4ad052e471fe6732fff548984b4af"
796 +
SOURCE_FILENAME_262="rand-0.10.3.tar.gz"
797 +
SOURCE_URI_263="https://static.crates.io/crates/rand/rand-0.8.8.crate"
798 +
CHECKSUM_SHA256_263="e058c7de0b26af77780c769414d6257830bb240f3c38477dbc2c16e5f54d6d4c"
799 +
SOURCE_FILENAME_263="rand-0.8.8.tar.gz"
800 +
SOURCE_URI_264="https://static.crates.io/crates/rand_chacha/rand_chacha-0.3.1.crate"
801 +
CHECKSUM_SHA256_264="e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88"
802 +
SOURCE_FILENAME_264="rand_chacha-0.3.1.tar.gz"
803 +
SOURCE_URI_265="https://static.crates.io/crates/rand_core/rand_core-0.10.1.crate"
804 +
CHECKSUM_SHA256_265="63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69"
805 +
SOURCE_FILENAME_265="rand_core-0.10.1.tar.gz"
806 +
SOURCE_URI_266="https://static.crates.io/crates/rand_core/rand_core-0.6.4.crate"
807 +
CHECKSUM_SHA256_266="ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c"
808 +
SOURCE_FILENAME_266="rand_core-0.6.4.tar.gz"
809 +
SOURCE_URI_267="https://static.crates.io/crates/rand_pcg/rand_pcg-0.10.2.crate"
810 +
CHECKSUM_SHA256_267="caa0f4137e1c0a72f4c651489402276c8e8e1cf081f3b0ba156d2cbeef09e86a"
811 +
SOURCE_FILENAME_267="rand_pcg-0.10.2.tar.gz"
812 +
SOURCE_URI_268="https://static.crates.io/crates/rayon/rayon-1.12.0.crate"
813 +
CHECKSUM_SHA256_268="fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d"
814 +
SOURCE_FILENAME_268="rayon-1.12.0.tar.gz"
815 +
SOURCE_URI_269="https://static.crates.io/crates/rayon-core/rayon-core-1.13.0.crate"
816 +
CHECKSUM_SHA256_269="22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91"
817 +
SOURCE_FILENAME_269="rayon-core-1.13.0.tar.gz"
818 +
SOURCE_URI_270="https://static.crates.io/crates/rcgen/rcgen-0.14.10.crate"
819 +
CHECKSUM_SHA256_270="8774e05a7d0de114588e6a28fe7e71694b82614ed569d86d8b389dfbc98b8ad8"
820 +
SOURCE_FILENAME_270="rcgen-0.14.10.tar.gz"
821 +
SOURCE_URI_271="https://static.crates.io/crates/redox_syscall/redox_syscall-0.5.18.crate"
822 +
CHECKSUM_SHA256_271="ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
823 +
SOURCE_FILENAME_271="redox_syscall-0.5.18.tar.gz"
824 +
SOURCE_URI_272="https://static.crates.io/crates/redox_syscall/redox_syscall-0.9.4.crate"
825 +
CHECKSUM_SHA256_272="737970939a87c6fa31e7acad13307bccbb017a073b695b6089a2c484f929e20e"
826 +
SOURCE_FILENAME_272="redox_syscall-0.9.4.tar.gz"
827 +
SOURCE_URI_273="https://static.crates.io/crates/ref-cast/ref-cast-1.0.27.crate"
828 +
CHECKSUM_SHA256_273="7e440fb4e4b4147295338efb76001ab9e4efc0e5839df2c47fc5ac2381d365c3"
829 +
SOURCE_FILENAME_273="ref-cast-1.0.27.tar.gz"
830 +
SOURCE_URI_274="https://static.crates.io/crates/ref-cast-impl/ref-cast-impl-1.0.27.crate"
831 +
CHECKSUM_SHA256_274="92ecd8964f8453721699a1ed72037b0db49ce2f5a5138486ee89bed6f67cdf3a"
832 +
SOURCE_FILENAME_274="ref-cast-impl-1.0.27.tar.gz"
833 +
SOURCE_URI_275="https://static.crates.io/crates/regex/regex-1.13.1.crate"
834 +
CHECKSUM_SHA256_275="f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d"
835 +
SOURCE_FILENAME_275="regex-1.13.1.tar.gz"
836 +
SOURCE_URI_276="https://static.crates.io/crates/regex-automata/regex-automata-0.4.18.crate"
837 +
CHECKSUM_SHA256_276="ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
838 +
SOURCE_FILENAME_276="regex-automata-0.4.18.tar.gz"
839 +
SOURCE_URI_277="https://static.crates.io/crates/regex-syntax/regex-syntax-0.8.11.crate"
840 +
CHECKSUM_SHA256_277="d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
841 +
SOURCE_FILENAME_277="regex-syntax-0.8.11.tar.gz"
842 +
SOURCE_URI_278="https://static.crates.io/crates/reqwest/reqwest-0.12.28.crate"
843 +
CHECKSUM_SHA256_278="eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147"
844 +
SOURCE_FILENAME_278="reqwest-0.12.28.tar.gz"
845 +
SOURCE_URI_279="https://static.crates.io/crates/ring/ring-0.17.14.crate"
846 +
CHECKSUM_SHA256_279="a4689e6c2294d81e88dc6261c768b63bc4fcdb852be6d1352498b114f61383b7"
847 +
SOURCE_FILENAME_279="ring-0.17.14.tar.gz"
848 +
SOURCE_URI_280="https://static.crates.io/crates/rsa/rsa-0.9.10.crate"
849 +
CHECKSUM_SHA256_280="b8573f03f5883dcaebdfcf4725caa1ecb9c15b2ef50c43a07b816e06799bb12d"
850 +
SOURCE_FILENAME_280="rsa-0.9.10.tar.gz"
851 +
SOURCE_URI_281="https://static.crates.io/crates/rustc-hash/rustc-hash-2.1.3.crate"
852 +
CHECKSUM_SHA256_281="6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d"
853 +
SOURCE_FILENAME_281="rustc-hash-2.1.3.tar.gz"
854 +
SOURCE_URI_282="https://static.crates.io/crates/rustc_version/rustc_version-0.4.1.crate"
855 +
CHECKSUM_SHA256_282="cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92"
856 +
SOURCE_FILENAME_282="rustc_version-0.4.1.tar.gz"
857 +
SOURCE_URI_283="https://static.crates.io/crates/rusticata-macros/rusticata-macros-4.1.0.crate"
858 +
CHECKSUM_SHA256_283="faf0c4a6ece9950b9abdb62b1cfcf2a68b3b67a10ba445b3bb85be2a293d0632"
859 +
SOURCE_FILENAME_283="rusticata-macros-4.1.0.tar.gz"
860 +
SOURCE_URI_284="https://static.crates.io/crates/rustix/rustix-1.1.5.crate"
861 +
CHECKSUM_SHA256_284="891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
862 +
SOURCE_FILENAME_284="rustix-1.1.5.tar.gz"
863 +
SOURCE_URI_285="https://static.crates.io/crates/rustls/rustls-0.23.45.crate"
864 +
CHECKSUM_SHA256_285="0d41d731c7d2f962d1ccc364cec258de3c0e93b38c2fb3ba97ac74513048d634"
865 +
SOURCE_FILENAME_285="rustls-0.23.45.tar.gz"
866 +
SOURCE_URI_286="https://static.crates.io/crates/rustls-pemfile/rustls-pemfile-2.2.0.crate"
867 +
CHECKSUM_SHA256_286="dce314e5fee3f39953d46bb63bb8a46d40c2f8fb7cc5a3b6cab2bde9721d6e50"
868 +
SOURCE_FILENAME_286="rustls-pemfile-2.2.0.tar.gz"
869 +
SOURCE_URI_287="https://static.crates.io/crates/rustls-pki-types/rustls-pki-types-1.15.1.crate"
870 +
CHECKSUM_SHA256_287="2f4925028c7eb5d1fcdaf196971378ed9d2c1c4efc7dc5d011256f76c99c0a96"
871 +
SOURCE_FILENAME_287="rustls-pki-types-1.15.1.tar.gz"
872 +
SOURCE_URI_288="https://static.crates.io/crates/rustls-webpki/rustls-webpki-0.103.15.crate"
873 +
CHECKSUM_SHA256_288="f3c3cf1d8b1e7d4927e2d154c3fcb02979afb9939629c62cd9048d4f07b60ac2"
874 +
SOURCE_FILENAME_288="rustls-webpki-0.103.15.tar.gz"
875 +
SOURCE_URI_289="https://static.crates.io/crates/rustversion/rustversion-1.0.23.crate"
876 +
CHECKSUM_SHA256_289="cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f"
877 +
SOURCE_FILENAME_289="rustversion-1.0.23.tar.gz"
878 +
SOURCE_URI_290="https://static.crates.io/crates/ryu/ryu-1.0.23.crate"
879 +
CHECKSUM_SHA256_290="9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f"
880 +
SOURCE_FILENAME_290="ryu-1.0.23.tar.gz"
881 +
SOURCE_URI_291="https://static.crates.io/crates/same-file/same-file-1.0.6.crate"
882 +
CHECKSUM_SHA256_291="93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502"
883 +
SOURCE_FILENAME_291="same-file-1.0.6.tar.gz"
884 +
SOURCE_URI_292="https://static.crates.io/crates/schemars/schemars-1.2.2.crate"
885 +
CHECKSUM_SHA256_292="687274d293b6cdc6e73e0fee520bf2049650090d7164f87672d212a3c530cf4a"
886 +
SOURCE_FILENAME_292="schemars-1.2.2.tar.gz"
887 +
SOURCE_URI_293="https://static.crates.io/crates/schemars_derive/schemars_derive-1.2.2.crate"
888 +
CHECKSUM_SHA256_293="d98c67716b46af2f0b8cf752abc930f6f9aecfbf671ecfb531db8a31dbe4e2ba"
889 +
SOURCE_FILENAME_293="schemars_derive-1.2.2.tar.gz"
890 +
SOURCE_URI_294="https://static.crates.io/crates/scopeguard/scopeguard-1.2.0.crate"
891 +
CHECKSUM_SHA256_294="94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
892 +
SOURCE_FILENAME_294="scopeguard-1.2.0.tar.gz"
893 +
SOURCE_URI_295="https://static.crates.io/crates/semver/semver-1.0.28.crate"
894 +
CHECKSUM_SHA256_295="8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd"
895 +
SOURCE_FILENAME_295="semver-1.0.28.tar.gz"
896 +
SOURCE_URI_296="https://static.crates.io/crates/serde/serde-1.0.229.crate"
897 +
CHECKSUM_SHA256_296="4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
898 +
SOURCE_FILENAME_296="serde-1.0.229.tar.gz"
899 +
SOURCE_URI_297="https://static.crates.io/crates/serde_core/serde_core-1.0.229.crate"
900 +
CHECKSUM_SHA256_297="67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
901 +
SOURCE_FILENAME_297="serde_core-1.0.229.tar.gz"
902 +
SOURCE_URI_298="https://static.crates.io/crates/serde_derive/serde_derive-1.0.229.crate"
903 +
CHECKSUM_SHA256_298="e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
904 +
SOURCE_FILENAME_298="serde_derive-1.0.229.tar.gz"
905 +
SOURCE_URI_299="https://static.crates.io/crates/serde_derive_internals/serde_derive_internals-0.30.0.crate"
906 +
CHECKSUM_SHA256_299="f852137cce035d6a4df67ccce505ff6b3e9fd3a10e3e52b24dc71e650bb1a9bd"
907 +
SOURCE_FILENAME_299="serde_derive_internals-0.30.0.tar.gz"
908 +
SOURCE_URI_300="https://static.crates.io/crates/serde_html_form/serde_html_form-0.2.8.crate"
909 +
CHECKSUM_SHA256_300="b2f2d7ff8a2140333718bb329f5c40fc5f0865b84c426183ce14c97d2ab8154f"
910 +
SOURCE_FILENAME_300="serde_html_form-0.2.8.tar.gz"
911 +
SOURCE_URI_301="https://static.crates.io/crates/serde_json/serde_json-1.0.151.crate"
912 +
CHECKSUM_SHA256_301="c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
913 +
SOURCE_FILENAME_301="serde_json-1.0.151.tar.gz"
914 +
SOURCE_URI_302="https://static.crates.io/crates/serde_path_to_error/serde_path_to_error-0.1.20.crate"
915 +
CHECKSUM_SHA256_302="10a9ff822e371bb5403e391ecd83e182e0e77ba7f6fe0160b795797109d1b457"
916 +
SOURCE_FILENAME_302="serde_path_to_error-0.1.20.tar.gz"
917 +
SOURCE_URI_303="https://static.crates.io/crates/serde_spanned/serde_spanned-0.6.9.crate"
918 +
CHECKSUM_SHA256_303="bf41e0cfaf7226dca15e8197172c295a782857fcb97fad1808a166870dee75a3"
919 +
SOURCE_FILENAME_303="serde_spanned-0.6.9.tar.gz"
920 +
SOURCE_URI_304="https://static.crates.io/crates/serde_urlencoded/serde_urlencoded-0.7.1.crate"
921 +
CHECKSUM_SHA256_304="d3491c14715ca2294c4d6a88f15e84739788c1d030eed8c110436aafdaa2f3fd"
922 +
SOURCE_FILENAME_304="serde_urlencoded-0.7.1.tar.gz"
923 +
SOURCE_URI_305="https://static.crates.io/crates/sha1/sha1-0.10.7.crate"
924 +
CHECKSUM_SHA256_305="a978451301f4db1d02937a4ab3ccce137717b81826e79b7d49ffe3244a13c3b8"
925 +
SOURCE_FILENAME_305="sha1-0.10.7.tar.gz"
926 +
SOURCE_URI_306="https://static.crates.io/crates/sha2/sha2-0.10.9.crate"
927 +
CHECKSUM_SHA256_306="a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283"
928 +
SOURCE_FILENAME_306="sha2-0.10.9.tar.gz"
929 +
SOURCE_URI_307="https://static.crates.io/crates/sha2/sha2-0.11.0.crate"
930 +
CHECKSUM_SHA256_307="446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
931 +
SOURCE_FILENAME_307="sha2-0.11.0.tar.gz"
932 +
SOURCE_URI_308="https://static.crates.io/crates/shake/shake-0.1.0.crate"
933 +
CHECKSUM_SHA256_308="09057cb2149ad4cbd2da1e26b351f9a4c354219421229c69c3063e6f61947c4a"
934 +
SOURCE_FILENAME_308="shake-0.1.0.tar.gz"
935 +
SOURCE_URI_309="https://static.crates.io/crates/sharded-slab/sharded-slab-0.1.7.crate"
936 +
CHECKSUM_SHA256_309="f40ca3c46823713e0d4209592e8d6e826aa57e928f09752619fc696c499637f6"
937 +
SOURCE_FILENAME_309="sharded-slab-0.1.7.tar.gz"
938 +
SOURCE_URI_310="https://static.crates.io/crates/shlex/shlex-2.0.1.crate"
939 +
CHECKSUM_SHA256_310="f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
940 +
SOURCE_FILENAME_310="shlex-2.0.1.tar.gz"
941 +
SOURCE_URI_311="https://static.crates.io/crates/signal-hook-registry/signal-hook-registry-1.4.8.crate"
942 +
CHECKSUM_SHA256_311="c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b"
943 +
SOURCE_FILENAME_311="signal-hook-registry-1.4.8.tar.gz"
944 +
SOURCE_URI_312="https://static.crates.io/crates/signature/signature-2.2.0.crate"
945 +
CHECKSUM_SHA256_312="77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de"
946 +
SOURCE_FILENAME_312="signature-2.2.0.tar.gz"
947 +
SOURCE_URI_313="https://static.crates.io/crates/signature/signature-3.0.0.crate"
948 +
CHECKSUM_SHA256_313="28d567dcbaf0049cb8ac2608a76cd95ff9e4412e1899d389ee400918ca7537f5"
949 +
SOURCE_FILENAME_313="signature-3.0.0.tar.gz"
950 +
SOURCE_URI_314="https://static.crates.io/crates/simd-adler32/simd-adler32-0.3.10.crate"
951 +
CHECKSUM_SHA256_314="3a219298ac11a56ea9a6d2120044824d6f01aeb034955e7af7bc16858527deea"
952 +
SOURCE_FILENAME_314="simd-adler32-0.3.10.tar.gz"
953 +
SOURCE_URI_315="https://static.crates.io/crates/simdutf8/simdutf8-0.1.5.crate"
954 +
CHECKSUM_SHA256_315="e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e"
955 +
SOURCE_FILENAME_315="simdutf8-0.1.5.tar.gz"
956 +
SOURCE_URI_316="https://static.crates.io/crates/similar/similar-2.7.0.crate"
957 +
CHECKSUM_SHA256_316="bbbb5d9659141646ae647b42fe094daf6c6192d1620870b449d9557f748b2daa"
958 +
SOURCE_FILENAME_316="similar-2.7.0.tar.gz"
959 +
SOURCE_URI_317="https://static.crates.io/crates/siphasher/siphasher-1.0.4.crate"
960 +
CHECKSUM_SHA256_317="33f4fe9184a62d842c9ef383018f3306d8ba224fd9d836f56d7288308847c256"
961 +
SOURCE_FILENAME_317="siphasher-1.0.4.tar.gz"
962 +
SOURCE_URI_318="https://static.crates.io/crates/slab/slab-0.4.12.crate"
963 +
CHECKSUM_SHA256_318="0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
964 +
SOURCE_FILENAME_318="slab-0.4.12.tar.gz"
965 +
SOURCE_URI_319="https://static.crates.io/crates/smallvec/smallvec-1.16.2.crate"
966 +
CHECKSUM_SHA256_319="f9395f0f0eee849a9b707b2f06bb92a6a422090e2123bb2ef8e87a0e61892a8e"
967 +
SOURCE_FILENAME_319="smallvec-1.16.2.tar.gz"
968 +
SOURCE_URI_320="https://static.crates.io/crates/socket2/socket2-0.6.5.crate"
969 +
CHECKSUM_SHA256_320="c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4"
970 +
SOURCE_FILENAME_320="socket2-0.6.5.tar.gz"
971 +
SOURCE_URI_321="https://static.crates.io/crates/spin/spin-0.10.1.crate"
972 +
CHECKSUM_SHA256_321="023a211cb3138dbc438680b32560ad89f699977624c9f8dbb95a47d5b4c07dd3"
973 +
SOURCE_FILENAME_321="spin-0.10.1.tar.gz"
974 +
SOURCE_URI_322="https://static.crates.io/crates/spin/spin-0.9.9.crate"
975 +
CHECKSUM_SHA256_322="3763264f6b73151db08c50ff20d7d8a0b8796e021cdea7ceedad07b80155fa0e"
976 +
SOURCE_FILENAME_322="spin-0.9.9.tar.gz"
977 +
SOURCE_URI_323="https://static.crates.io/crates/spki/spki-0.7.3.crate"
978 +
CHECKSUM_SHA256_323="d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d"
979 +
SOURCE_FILENAME_323="spki-0.7.3.tar.gz"
980 +
SOURCE_URI_324="https://static.crates.io/crates/spki/spki-0.8.0.crate"
981 +
CHECKSUM_SHA256_324="1d9efca8738c78ee9484207732f728b1ef517bbb1833d6fc0879ca898a522f6f"
982 +
SOURCE_FILENAME_324="spki-0.8.0.tar.gz"
983 +
SOURCE_URI_325="https://static.crates.io/crates/sponge-cursor/sponge-cursor-0.1.0.crate"
984 +
CHECKSUM_SHA256_325="3a0219bd7d979d58245a4f41f695e1ac9f8befdffadd7f61f1bae9e39abc6620"
985 +
SOURCE_FILENAME_325="sponge-cursor-0.1.0.tar.gz"
986 +
SOURCE_URI_326="https://static.crates.io/crates/sqlx/sqlx-0.8.6.crate"
987 +
CHECKSUM_SHA256_326="1fefb893899429669dcdd979aff487bd78f4064e5e7907e4269081e0ef7d97dc"
988 +
SOURCE_FILENAME_326="sqlx-0.8.6.tar.gz"
989 +
SOURCE_URI_327="https://static.crates.io/crates/sqlx-core/sqlx-core-0.8.6.crate"
990 +
CHECKSUM_SHA256_327="ee6798b1838b6a0f69c007c133b8df5866302197e404e8b6ee8ed3e3a5e68dc6"
991 +
SOURCE_FILENAME_327="sqlx-core-0.8.6.tar.gz"
992 +
SOURCE_URI_328="https://static.crates.io/crates/sqlx-macros/sqlx-macros-0.8.6.crate"
993 +
CHECKSUM_SHA256_328="a2d452988ccaacfbf5e0bdbc348fb91d7c8af5bee192173ac3636b5fb6e6715d"
994 +
SOURCE_FILENAME_328="sqlx-macros-0.8.6.tar.gz"
995 +
SOURCE_URI_329="https://static.crates.io/crates/sqlx-macros-core/sqlx-macros-core-0.8.6.crate"
996 +
CHECKSUM_SHA256_329="19a9c1841124ac5a61741f96e1d9e2ec77424bf323962dd894bdb93f37d5219b"
997 +
SOURCE_FILENAME_329="sqlx-macros-core-0.8.6.tar.gz"
998 +
SOURCE_URI_330="https://static.crates.io/crates/sqlx-mysql/sqlx-mysql-0.8.6.crate"
999 +
CHECKSUM_SHA256_330="aa003f0038df784eb8fecbbac13affe3da23b45194bd57dba231c8f48199c526"
1000 +
SOURCE_FILENAME_330="sqlx-mysql-0.8.6.tar.gz"
1001 +
SOURCE_URI_331="https://static.crates.io/crates/sqlx-postgres/sqlx-postgres-0.8.6.crate"
1002 +
CHECKSUM_SHA256_331="db58fcd5a53cf07c184b154801ff91347e4c30d17a3562a635ff028ad5deda46"
1003 +
SOURCE_FILENAME_331="sqlx-postgres-0.8.6.tar.gz"
1004 +
SOURCE_URI_332="https://static.crates.io/crates/sqlx-sqlite/sqlx-sqlite-0.8.6.crate"
1005 +
CHECKSUM_SHA256_332="c2d12fe70b2c1b4401038055f90f151b78208de1f9f89a7dbfd41587a10c3eea"
1006 +
SOURCE_FILENAME_332="sqlx-sqlite-0.8.6.tar.gz"
1007 +
SOURCE_URI_333="https://static.crates.io/crates/stable_deref_trait/stable_deref_trait-1.2.1.crate"
1008 +
CHECKSUM_SHA256_333="6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
1009 +
SOURCE_FILENAME_333="stable_deref_trait-1.2.1.tar.gz"
1010 +
SOURCE_URI_334="https://static.crates.io/crates/streaming-iterator/streaming-iterator-0.1.9.crate"
1011 +
CHECKSUM_SHA256_334="2b2231b7c3057d5e4ad0156fb3dc807d900806020c5ffa3ee6ff2c8c76fb8520"
1012 +
SOURCE_FILENAME_334="streaming-iterator-0.1.9.tar.gz"
1013 +
SOURCE_URI_335="https://static.crates.io/crates/string_cache/string_cache-0.11.0.crate"
1014 +
CHECKSUM_SHA256_335="ffa8a5dbe8b3f0bbe29d4c3225daafaeead63afdc1b65fc4c01a1384166038e6"
1015 +
SOURCE_FILENAME_335="string_cache-0.11.0.tar.gz"
1016 +
SOURCE_URI_336="https://static.crates.io/crates/string_cache_codegen/string_cache_codegen-0.11.2.crate"
1017 +
CHECKSUM_SHA256_336="928dcdf75e47626b3617a976ec205d9f057584c371c1f23b782129268d0e6edc"
1018 +
SOURCE_FILENAME_336="string_cache_codegen-0.11.2.tar.gz"
1019 +
SOURCE_URI_337="https://static.crates.io/crates/stringprep/stringprep-0.1.5.crate"
1020 +
CHECKSUM_SHA256_337="7b4df3d392d81bd458a8a621b8bffbd2302a12ffe288a9d931670948749463b1"
1021 +
SOURCE_FILENAME_337="stringprep-0.1.5.tar.gz"
1022 +
SOURCE_URI_338="https://static.crates.io/crates/strsim/strsim-0.11.1.crate"
1023 +
CHECKSUM_SHA256_338="7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
1024 +
SOURCE_FILENAME_338="strsim-0.11.1.tar.gz"
1025 +
SOURCE_URI_339="https://static.crates.io/crates/subtle/subtle-2.6.1.crate"
1026 +
CHECKSUM_SHA256_339="13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
1027 +
SOURCE_FILENAME_339="subtle-2.6.1.tar.gz"
1028 +
SOURCE_URI_340="https://static.crates.io/crates/syn/syn-2.0.119.crate"
1029 +
CHECKSUM_SHA256_340="872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
1030 +
SOURCE_FILENAME_340="syn-2.0.119.tar.gz"
1031 +
SOURCE_URI_341="https://static.crates.io/crates/syn/syn-3.0.6.crate"
1032 +
CHECKSUM_SHA256_341="8593e8e72159ed2257d083c7a454a85cbf854f37a0966d8d483aff8c8a3ebcee"
1033 +
SOURCE_FILENAME_341="syn-3.0.6.tar.gz"
1034 +
SOURCE_URI_342="https://static.crates.io/crates/sync_wrapper/sync_wrapper-1.0.2.crate"
1035 +
CHECKSUM_SHA256_342="0bf256ce5efdfa370213c1dabab5935a12e49f2c58d15e9eac2870d3b4f27263"
1036 +
SOURCE_FILENAME_342="sync_wrapper-1.0.2.tar.gz"
1037 +
SOURCE_URI_343="https://static.crates.io/crates/synstructure/synstructure-0.13.2.crate"
1038 +
CHECKSUM_SHA256_343="728a70f3dbaf5bab7f0c4b1ac8d7ae5ea60a4b5549c8a5914361c99147a709d2"
1039 +
SOURCE_FILENAME_343="synstructure-0.13.2.tar.gz"
1040 +
SOURCE_URI_344="https://static.crates.io/crates/synstructure/synstructure-0.14.0.crate"
1041 +
CHECKSUM_SHA256_344="901704edd0dfe137f1987838ee4f259e4e063c31371bdb423f7ae38ec6f77f02"
1042 +
SOURCE_FILENAME_344="synstructure-0.14.0.tar.gz"
1043 +
SOURCE_URI_345="https://static.crates.io/crates/syntect/syntect-5.3.0.crate"
1044 +
CHECKSUM_SHA256_345="656b45c05d95a5704399aeef6bd0ddec7b2b3531b7c9e900abbf7c4d2190c925"
1045 +
SOURCE_FILENAME_345="syntect-5.3.0.tar.gz"
1046 +
SOURCE_URI_346="https://static.crates.io/crates/tar/tar-0.4.46.crate"
1047 +
CHECKSUM_SHA256_346="3f6221d9a6003c78398e3b239969f352578258df48c8eb051caadae0015bc840"
1048 +
SOURCE_FILENAME_346="tar-0.4.46.tar.gz"
1049 +
SOURCE_URI_347="https://static.crates.io/crates/tempfile/tempfile-3.27.0.crate"
1050 +
CHECKSUM_SHA256_347="32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
1051 +
SOURCE_FILENAME_347="tempfile-3.27.0.tar.gz"
1052 +
SOURCE_URI_348="https://static.crates.io/crates/tendril/tendril-0.5.1.crate"
1053 +
CHECKSUM_SHA256_348="5fed54709c5b3a53d09bb1c113ea4f5ceafd1e772ddcb0030a82e1d56c087b08"
1054 +
SOURCE_FILENAME_348="tendril-0.5.1.tar.gz"
1055 +
SOURCE_URI_349="https://static.crates.io/crates/thiserror/thiserror-1.0.69.crate"
1056 +
CHECKSUM_SHA256_349="b6aaf5339b578ea85b50e080feb250a3e8ae8cfcdff9a461c9ec2904bc923f52"
1057 +
SOURCE_FILENAME_349="thiserror-1.0.69.tar.gz"
1058 +
SOURCE_URI_350="https://static.crates.io/crates/thiserror/thiserror-2.0.21.crate"
1059 +
CHECKSUM_SHA256_350="09e52cb86a36cede5cb101bf8908837b3e4c6e5e59fe7fd85c23fb56200d189e"
1060 +
SOURCE_FILENAME_350="thiserror-2.0.21.tar.gz"
1061 +
SOURCE_URI_351="https://static.crates.io/crates/thiserror-impl/thiserror-impl-1.0.69.crate"
1062 +
CHECKSUM_SHA256_351="4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1"
1063 +
SOURCE_FILENAME_351="thiserror-impl-1.0.69.tar.gz"
1064 +
SOURCE_URI_352="https://static.crates.io/crates/thiserror-impl/thiserror-impl-2.0.21.crate"
1065 +
CHECKSUM_SHA256_352="fe5197923287db20a58125f0bc85c062f7f2c892de97b18c356f9efb14b28524"
1066 +
SOURCE_FILENAME_352="thiserror-impl-2.0.21.tar.gz"
1067 +
SOURCE_URI_353="https://static.crates.io/crates/thread_local/thread_local-1.1.10.crate"
1068 +
CHECKSUM_SHA256_353="1ad99c4c6d32803332c548b1af0540b357b3f5fc0be8f6c6bfe8b2e6ae784070"
1069 +
SOURCE_FILENAME_353="thread_local-1.1.10.tar.gz"
1070 +
SOURCE_URI_354="https://static.crates.io/crates/time/time-0.3.55.crate"
1071 +
CHECKSUM_SHA256_354="cdb87b95ec50ddfa440816d227a17b2ccbdda963a316a727fda0fc4334f7d134"
1072 +
SOURCE_FILENAME_354="time-0.3.55.tar.gz"
1073 +
SOURCE_URI_355="https://static.crates.io/crates/time-core/time-core-0.1.9.crate"
1074 +
CHECKSUM_SHA256_355="9e1c906769ad99c88eaa54e728060edef082f8e358ff32030cb7c7d315e81109"
1075 +
SOURCE_FILENAME_355="time-core-0.1.9.tar.gz"
1076 +
SOURCE_URI_356="https://static.crates.io/crates/time-macros/time-macros-0.2.32.crate"
1077 +
CHECKSUM_SHA256_356="7e689342a48d2ea927c87ea50cabf8594854bf940e9310208848d680d668ed85"
1078 +
SOURCE_FILENAME_356="time-macros-0.2.32.tar.gz"
1079 +
SOURCE_URI_357="https://static.crates.io/crates/tinystr/tinystr-0.8.4.crate"
1080 +
CHECKSUM_SHA256_357="b1e27c91459209c2986af3dcf603a5a74a4368754ce37414f59acc971167f643"
1081 +
SOURCE_FILENAME_357="tinystr-0.8.4.tar.gz"
1082 +
SOURCE_URI_358="https://static.crates.io/crates/tinyvec/tinyvec-1.13.3.crate"
1083 +
CHECKSUM_SHA256_358="fd3ca314f692efd6c868f8408f53fe444634a845f96c028b97d35f6a1f79f0ee"
1084 +
SOURCE_FILENAME_358="tinyvec-1.13.3.tar.gz"
1085 +
SOURCE_URI_359="https://static.crates.io/crates/tokio/tokio-1.53.1.crate"
1086 +
CHECKSUM_SHA256_359="202caea871b69668250d242070849eb495be178ed697a3e98aebce5bc81a0bed"
1087 +
SOURCE_FILENAME_359="tokio-1.53.1.tar.gz"
1088 +
SOURCE_URI_360="https://static.crates.io/crates/tokio-macros/tokio-macros-2.7.2.crate"
1089 +
CHECKSUM_SHA256_360="78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e"
1090 +
SOURCE_FILENAME_360="tokio-macros-2.7.2.tar.gz"
1091 +
SOURCE_URI_361="https://static.crates.io/crates/tokio-rustls/tokio-rustls-0.26.5.crate"
1092 +
CHECKSUM_SHA256_361="b0c85f2c3ef0b1cd58b36682f4b17aaa995f0e5db534d85692b4903abce21f67"
1093 +
SOURCE_FILENAME_361="tokio-rustls-0.26.5.tar.gz"
1094 +
SOURCE_URI_362="https://static.crates.io/crates/tokio-stream/tokio-stream-0.1.19.crate"
1095 +
CHECKSUM_SHA256_362="a3d06f0b082ba57c26b79407372e57cf2a1e28124f78e9479fe80322cf53420b"
1096 +
SOURCE_FILENAME_362="tokio-stream-0.1.19.tar.gz"
1097 +
SOURCE_URI_363="https://static.crates.io/crates/tokio-util/tokio-util-0.7.19.crate"
1098 +
CHECKSUM_SHA256_363="494815d09bf52b5548659851081238f0ca39ff638363907596da739561c62c52"
1099 +
SOURCE_FILENAME_363="tokio-util-0.7.19.tar.gz"
1100 +
SOURCE_URI_364="https://static.crates.io/crates/toml/toml-0.8.23.crate"
1101 +
CHECKSUM_SHA256_364="dc1beb996b9d83529a9e75c17a1686767d148d70663143c7854d8b4a09ced362"
1102 +
SOURCE_FILENAME_364="toml-0.8.23.tar.gz"
1103 +
SOURCE_URI_365="https://static.crates.io/crates/toml_datetime/toml_datetime-0.6.11.crate"
1104 +
CHECKSUM_SHA256_365="22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c"
1105 +
SOURCE_FILENAME_365="toml_datetime-0.6.11.tar.gz"
1106 +
SOURCE_URI_366="https://static.crates.io/crates/toml_edit/toml_edit-0.22.27.crate"
1107 +
CHECKSUM_SHA256_366="41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a"
1108 +
SOURCE_FILENAME_366="toml_edit-0.22.27.tar.gz"
1109 +
SOURCE_URI_367="https://static.crates.io/crates/toml_write/toml_write-0.1.2.crate"
1110 +
CHECKSUM_SHA256_367="5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801"
1111 +
SOURCE_FILENAME_367="toml_write-0.1.2.tar.gz"
1112 +
SOURCE_URI_368="https://static.crates.io/crates/tower/tower-0.5.3.crate"
1113 +
CHECKSUM_SHA256_368="ebe5ef63511595f1344e2d5cfa636d973292adc0eec1f0ad45fae9f0851ab1d4"
1114 +
SOURCE_FILENAME_368="tower-0.5.3.tar.gz"
1115 +
SOURCE_URI_369="https://static.crates.io/crates/tower-http/tower-http-0.6.11.crate"
1116 +
CHECKSUM_SHA256_369="4cfcf7e2740e6fc6d4d688b4ef00650406bb94adf4731e43c096c3a19fe40840"
1117 +
SOURCE_FILENAME_369="tower-http-0.6.11.tar.gz"
1118 +
SOURCE_URI_370="https://static.crates.io/crates/tower-layer/tower-layer-0.3.3.crate"
1119 +
CHECKSUM_SHA256_370="121c2a6cda46980bb0fcd1647ffaf6cd3fc79a013de288782836f6df9c48780e"
1120 +
SOURCE_FILENAME_370="tower-layer-0.3.3.tar.gz"
1121 +
SOURCE_URI_371="https://static.crates.io/crates/tower-service/tower-service-0.3.3.crate"
1122 +
CHECKSUM_SHA256_371="8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
1123 +
SOURCE_FILENAME_371="tower-service-0.3.3.tar.gz"
1124 +
SOURCE_URI_372="https://static.crates.io/crates/tracing/tracing-0.1.44.crate"
1125 +
CHECKSUM_SHA256_372="63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
1126 +
SOURCE_FILENAME_372="tracing-0.1.44.tar.gz"
1127 +
SOURCE_URI_373="https://static.crates.io/crates/tracing-attributes/tracing-attributes-0.1.31.crate"
1128 +
CHECKSUM_SHA256_373="7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da"
1129 +
SOURCE_FILENAME_373="tracing-attributes-0.1.31.tar.gz"
1130 +
SOURCE_URI_374="https://static.crates.io/crates/tracing-core/tracing-core-0.1.36.crate"
1131 +
CHECKSUM_SHA256_374="db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
1132 +
SOURCE_FILENAME_374="tracing-core-0.1.36.tar.gz"
1133 +
SOURCE_URI_375="https://static.crates.io/crates/tracing-log/tracing-log-0.2.0.crate"
1134 +
CHECKSUM_SHA256_375="ee855f1f400bd0e5c02d150ae5de3840039a3f54b025156404e34c23c03f47c3"
1135 +
SOURCE_FILENAME_375="tracing-log-0.2.0.tar.gz"
1136 +
SOURCE_URI_376="https://static.crates.io/crates/tracing-subscriber/tracing-subscriber-0.3.23.crate"
1137 +
CHECKSUM_SHA256_376="cb7f578e5945fb242538965c2d0b04418d38ec25c79d160cd279bf0731c8d319"
1138 +
SOURCE_FILENAME_376="tracing-subscriber-0.3.23.tar.gz"
1139 +
SOURCE_URI_377="https://static.crates.io/crates/tree-sitter/tree-sitter-0.27.0.crate"
1140 +
CHECKSUM_SHA256_377="2038684e0058edba0d17302619f62eabce4a8e11c6ac59506996a8d79848851d"
1141 +
SOURCE_FILENAME_377="tree-sitter-0.27.0.tar.gz"
1142 +
SOURCE_URI_378="https://static.crates.io/crates/tree-sitter-bash/tree-sitter-bash-0.25.1.crate"
1143 +
CHECKSUM_SHA256_378="9e5ec769279cc91b561d3df0d8a5deb26b0ad40d183127f409494d6d8fc53062"
1144 +
SOURCE_FILENAME_378="tree-sitter-bash-0.25.1.tar.gz"
1145 +
SOURCE_URI_379="https://static.crates.io/crates/tree-sitter-language/tree-sitter-language-0.1.8.crate"
1146 +
CHECKSUM_SHA256_379="ca0d1bf6fdd806e43ae5198f82f527056d359def39e54e67a0f478ac09dac081"
1147 +
SOURCE_FILENAME_379="tree-sitter-language-0.1.8.tar.gz"
1148 +
SOURCE_URI_380="https://static.crates.io/crates/tree-sitter-python/tree-sitter-python-0.25.0.crate"
1149 +
CHECKSUM_SHA256_380="6bf85fd39652e740bf60f46f4cda9492c3a9ad75880575bf14960f775cb74a1c"
1150 +
SOURCE_FILENAME_380="tree-sitter-python-0.25.0.tar.gz"
1151 +
SOURCE_URI_381="https://static.crates.io/crates/tree-sitter-rust/tree-sitter-rust-0.24.2.crate"
1152 +
CHECKSUM_SHA256_381="439e577dbe07423ec2582ac62c7531120dbfccfa6e5f92406f93dd271a120e45"
1153 +
SOURCE_FILENAME_381="tree-sitter-rust-0.24.2.tar.gz"
1154 +
SOURCE_URI_382="https://static.crates.io/crates/tree-sitter-typescript/tree-sitter-typescript-0.23.2.crate"
1155 +
CHECKSUM_SHA256_382="6c5f76ed8d947a75cc446d5fccd8b602ebf0cde64ccf2ffa434d873d7a575eff"
1156 +
SOURCE_FILENAME_382="tree-sitter-typescript-0.23.2.tar.gz"
1157 +
SOURCE_URI_383="https://static.crates.io/crates/try-lock/try-lock-0.2.5.crate"
1158 +
CHECKSUM_SHA256_383="e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
1159 +
SOURCE_FILENAME_383="try-lock-0.2.5.tar.gz"
1160 +
SOURCE_URI_384="https://static.crates.io/crates/typenum/typenum-1.20.1.crate"
1161 +
CHECKSUM_SHA256_384="b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20"
1162 +
SOURCE_FILENAME_384="typenum-1.20.1.tar.gz"
1163 +
SOURCE_URI_385="https://static.crates.io/crates/unicase/unicase-2.9.0.crate"
1164 +
CHECKSUM_SHA256_385="dbc4bc3a9f746d862c45cb89d705aa10f187bb96c76001afab07a0d35ce60142"
1165 +
SOURCE_FILENAME_385="unicase-2.9.0.tar.gz"
1166 +
SOURCE_URI_386="https://static.crates.io/crates/unicode-bidi/unicode-bidi-0.3.18.crate"
1167 +
CHECKSUM_SHA256_386="5c1cb5db39152898a79168971543b1cb5020dff7fe43c8dc468b0885f5e29df5"
1168 +
SOURCE_FILENAME_386="unicode-bidi-0.3.18.tar.gz"
1169 +
SOURCE_URI_387="https://static.crates.io/crates/unicode-ident/unicode-ident-1.0.26.crate"
1170 +
CHECKSUM_SHA256_387="d245f478577f809a851594d02313b640fb437e0bb33866753cff937863096954"
1171 +
SOURCE_FILENAME_387="unicode-ident-1.0.26.tar.gz"
1172 +
SOURCE_URI_388="https://static.crates.io/crates/unicode-normalization/unicode-normalization-0.1.25.crate"
1173 +
CHECKSUM_SHA256_388="5fd4f6878c9cb28d874b009da9e8d183b5abc80117c40bbd187a1fde336be6e8"
1174 +
SOURCE_FILENAME_388="unicode-normalization-0.1.25.tar.gz"
1175 +
SOURCE_URI_389="https://static.crates.io/crates/unicode-properties/unicode-properties-0.1.4.crate"
1176 +
CHECKSUM_SHA256_389="7df058c713841ad818f1dc5d3fd88063241cc61f49f5fbea4b951e8cf5a8d71d"
1177 +
SOURCE_FILENAME_389="unicode-properties-0.1.4.tar.gz"
1178 +
SOURCE_URI_390="https://static.crates.io/crates/unicode-width/unicode-width-0.1.14.crate"
1179 +
CHECKSUM_SHA256_390="7dd6e30e90baa6f72411720665d41d89b9a3d039dc45b8faea1ddd07f617f6af"
1180 +
SOURCE_FILENAME_390="unicode-width-0.1.14.tar.gz"
1181 +
SOURCE_URI_391="https://static.crates.io/crates/untrusted/untrusted-0.9.0.crate"
1182 +
CHECKSUM_SHA256_391="8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1"
1183 +
SOURCE_FILENAME_391="untrusted-0.9.0.tar.gz"
1184 +
SOURCE_URI_392="https://static.crates.io/crates/ureq/ureq-2.12.1.crate"
1185 +
CHECKSUM_SHA256_392="02d1a66277ed75f640d608235660df48c8e3c19f3b4edb6a263315626cc3c01d"
1186 +
SOURCE_FILENAME_392="ureq-2.12.1.tar.gz"
1187 +
SOURCE_URI_393="https://static.crates.io/crates/url/url-2.5.8.crate"
1188 +
CHECKSUM_SHA256_393="ff67a8a4397373c3ef660812acab3268222035010ab8680ec4215f38ba3d0eed"
1189 +
SOURCE_FILENAME_393="url-2.5.8.tar.gz"
1190 +
SOURCE_URI_394="https://static.crates.io/crates/utf8_iter/utf8_iter-1.0.4.crate"
1191 +
CHECKSUM_SHA256_394="b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"
1192 +
SOURCE_FILENAME_394="utf8_iter-1.0.4.tar.gz"
1193 +
SOURCE_URI_395="https://static.crates.io/crates/utf8parse/utf8parse-0.2.2.crate"
1194 +
CHECKSUM_SHA256_395="06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
1195 +
SOURCE_FILENAME_395="utf8parse-0.2.2.tar.gz"
1196 +
SOURCE_URI_396="https://static.crates.io/crates/valuable/valuable-0.1.1.crate"
1197 +
CHECKSUM_SHA256_396="ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65"
1198 +
SOURCE_FILENAME_396="valuable-0.1.1.tar.gz"
1199 +
SOURCE_URI_397="https://static.crates.io/crates/vcpkg/vcpkg-0.2.15.crate"
1200 +
CHECKSUM_SHA256_397="accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
1201 +
SOURCE_FILENAME_397="vcpkg-0.2.15.tar.gz"
1202 +
SOURCE_URI_398="https://static.crates.io/crates/version_check/version_check-0.9.5.crate"
1203 +
CHECKSUM_SHA256_398="0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
1204 +
SOURCE_FILENAME_398="version_check-0.9.5.tar.gz"
1205 +
SOURCE_URI_399="https://static.crates.io/crates/walkdir/walkdir-2.5.0.crate"
1206 +
CHECKSUM_SHA256_399="29790946404f91d9c5d06f9874efddea1dc06c5efe94541a7d6863108e3a5e4b"
1207 +
SOURCE_FILENAME_399="walkdir-2.5.0.tar.gz"
1208 +
SOURCE_URI_400="https://static.crates.io/crates/want/want-0.3.1.crate"
1209 +
CHECKSUM_SHA256_400="bfa7760aed19e106de2c7c0b581b509f2f25d3dacaf737cb82ac61bc6d760b0e"
1210 +
SOURCE_FILENAME_400="want-0.3.1.tar.gz"
1211 +
SOURCE_URI_401="https://static.crates.io/crates/wasi/wasi-0.11.1+wasi-snapshot-preview1.crate"
1212 +
CHECKSUM_SHA256_401="ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
1213 +
SOURCE_FILENAME_401="wasi-0.11.1+wasi-snapshot-preview1.tar.gz"
1214 +
SOURCE_URI_402="https://static.crates.io/crates/wasite/wasite-0.1.0.crate"
1215 +
CHECKSUM_SHA256_402="b8dad83b4f25e74f184f64c43b150b91efe7647395b42289f38e50566d82855b"
1216 +
SOURCE_FILENAME_402="wasite-0.1.0.tar.gz"
1217 +
SOURCE_URI_403="https://static.crates.io/crates/wasm-bindgen/wasm-bindgen-0.2.129.crate"
1218 +
CHECKSUM_SHA256_403="9bb54f33acc68fd454578d9820b0bde1a1a3d17aa17bb7b6595806d02886d409"
1219 +
SOURCE_FILENAME_403="wasm-bindgen-0.2.129.tar.gz"
1220 +
SOURCE_URI_404="https://static.crates.io/crates/wasm-bindgen-futures/wasm-bindgen-futures-0.4.79.crate"
1221 +
CHECKSUM_SHA256_404="3cbab34de2d982e9b48e18d216d04c4a6f641066ff19ffb699980f591ee3610e"
1222 +
SOURCE_FILENAME_404="wasm-bindgen-futures-0.4.79.tar.gz"
1223 +
SOURCE_URI_405="https://static.crates.io/crates/wasm-bindgen-macro/wasm-bindgen-macro-0.2.129.crate"
1224 +
CHECKSUM_SHA256_405="2e29d0c35b16e224a7eeb5cd2d25e3e1968fbd65604117b44d3b789d00ee8535"
1225 +
SOURCE_FILENAME_405="wasm-bindgen-macro-0.2.129.tar.gz"
1226 +
SOURCE_URI_406="https://static.crates.io/crates/wasm-bindgen-macro-support/wasm-bindgen-macro-support-0.2.129.crate"
1227 +
CHECKSUM_SHA256_406="6f501a8bc3719dba86ef8ae4728879c08001bea749eb1333ac5b91e040e2a6b7"
1228 +
SOURCE_FILENAME_406="wasm-bindgen-macro-support-0.2.129.tar.gz"
1229 +
SOURCE_URI_407="https://static.crates.io/crates/wasm-bindgen-shared/wasm-bindgen-shared-0.2.129.crate"
1230 +
CHECKSUM_SHA256_407="23f0c9c52aa7cd7d77769a4cfe2a9adb1b331f489a41d912ce14513d5ab995c6"
1231 +
SOURCE_FILENAME_407="wasm-bindgen-shared-0.2.129.tar.gz"
1232 +
SOURCE_URI_408="https://static.crates.io/crates/web-sys/web-sys-0.3.106.crate"
1233 +
CHECKSUM_SHA256_408="88261b9deccee56594c11a3460c462c41f58d148598fe70ad77070126a68aba4"
1234 +
SOURCE_FILENAME_408="web-sys-0.3.106.tar.gz"
1235 +
SOURCE_URI_409="https://static.crates.io/crates/web-time/web-time-1.1.0.crate"
1236 +
CHECKSUM_SHA256_409="5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb"
1237 +
SOURCE_FILENAME_409="web-time-1.1.0.tar.gz"
1238 +
SOURCE_URI_410="https://static.crates.io/crates/web_atoms/web_atoms-0.3.0.crate"
1239 +
CHECKSUM_SHA256_410="7572660c8890448ba236b7376f27e389c6a7e1c70195622faced601f855c0ada"
1240 +
SOURCE_FILENAME_410="web_atoms-0.3.0.tar.gz"
1241 +
SOURCE_URI_411="https://static.crates.io/crates/webpki-roots/webpki-roots-0.26.11.crate"
1242 +
CHECKSUM_SHA256_411="521bc38abb08001b01866da9f51eb7c5d647a19260e00054a8c7fd5f9e57f7a9"
1243 +
SOURCE_FILENAME_411="webpki-roots-0.26.11.tar.gz"
1244 +
SOURCE_URI_412="https://static.crates.io/crates/webpki-roots/webpki-roots-1.0.9.crate"
1245 +
CHECKSUM_SHA256_412="7dcd9d09a39985f5344844e66b0c530a33843579125f23e21e9f0f220850f22a"
1246 +
SOURCE_FILENAME_412="webpki-roots-1.0.9.tar.gz"
1247 +
SOURCE_URI_413="https://static.crates.io/crates/whoami/whoami-1.6.1.crate"
1248 +
CHECKSUM_SHA256_413="5d4a4db5077702ca3015d3d02d74974948aba2ad9e12ab7df718ee64ccd7e97d"
1249 +
SOURCE_FILENAME_413="whoami-1.6.1.tar.gz"
1250 +
SOURCE_URI_414="https://static.crates.io/crates/winapi-util/winapi-util-0.1.11.crate"
1251 +
CHECKSUM_SHA256_414="c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
1252 +
SOURCE_FILENAME_414="winapi-util-0.1.11.tar.gz"
1253 +
SOURCE_URI_415="https://static.crates.io/crates/windows-core/windows-core-0.62.2.crate"
1254 +
CHECKSUM_SHA256_415="b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb"
1255 +
SOURCE_FILENAME_415="windows-core-0.62.2.tar.gz"
1256 +
SOURCE_URI_416="https://static.crates.io/crates/windows-implement/windows-implement-0.60.2.crate"
1257 +
CHECKSUM_SHA256_416="053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
1258 +
SOURCE_FILENAME_416="windows-implement-0.60.2.tar.gz"
1259 +
SOURCE_URI_417="https://static.crates.io/crates/windows-interface/windows-interface-0.59.3.crate"
1260 +
CHECKSUM_SHA256_417="3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
1261 +
SOURCE_FILENAME_417="windows-interface-0.59.3.tar.gz"
1262 +
SOURCE_URI_418="https://static.crates.io/crates/windows-link/windows-link-0.2.1.crate"
1263 +
CHECKSUM_SHA256_418="f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
1264 +
SOURCE_FILENAME_418="windows-link-0.2.1.tar.gz"
1265 +
SOURCE_URI_419="https://static.crates.io/crates/windows-result/windows-result-0.4.1.crate"
1266 +
CHECKSUM_SHA256_419="7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
1267 +
SOURCE_FILENAME_419="windows-result-0.4.1.tar.gz"
1268 +
SOURCE_URI_420="https://static.crates.io/crates/windows-strings/windows-strings-0.5.1.crate"
1269 +
CHECKSUM_SHA256_420="7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091"
1270 +
SOURCE_FILENAME_420="windows-strings-0.5.1.tar.gz"
1271 +
SOURCE_URI_421="https://static.crates.io/crates/windows-sys/windows-sys-0.48.0.crate"
1272 +
CHECKSUM_SHA256_421="677d2418bec65e3338edb076e806bc1ec15693c5d0104683f2efe857f61056a9"
1273 +
SOURCE_FILENAME_421="windows-sys-0.48.0.tar.gz"
1274 +
SOURCE_URI_422="https://static.crates.io/crates/windows-sys/windows-sys-0.52.0.crate"
1275 +
CHECKSUM_SHA256_422="282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d"
1276 +
SOURCE_FILENAME_422="windows-sys-0.52.0.tar.gz"
1277 +
SOURCE_URI_423="https://static.crates.io/crates/windows-sys/windows-sys-0.61.2.crate"
1278 +
CHECKSUM_SHA256_423="ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
1279 +
SOURCE_FILENAME_423="windows-sys-0.61.2.tar.gz"
1280 +
SOURCE_URI_424="https://static.crates.io/crates/windows-targets/windows-targets-0.48.5.crate"
1281 +
CHECKSUM_SHA256_424="9a2fa6e2155d7247be68c096456083145c183cbbbc2764150dda45a87197940c"
1282 +
SOURCE_FILENAME_424="windows-targets-0.48.5.tar.gz"
1283 +
SOURCE_URI_425="https://static.crates.io/crates/windows-targets/windows-targets-0.52.6.crate"
1284 +
CHECKSUM_SHA256_425="9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973"
1285 +
SOURCE_FILENAME_425="windows-targets-0.52.6.tar.gz"
1286 +
SOURCE_URI_426="https://static.crates.io/crates/windows_aarch64_gnullvm/windows_aarch64_gnullvm-0.48.5.crate"
1287 +
CHECKSUM_SHA256_426="2b38e32f0abccf9987a4e3079dfb67dcd799fb61361e53e2882c3cbaf0d905d8"
1288 +
SOURCE_FILENAME_426="windows_aarch64_gnullvm-0.48.5.tar.gz"
1289 +
SOURCE_URI_427="https://static.crates.io/crates/windows_aarch64_gnullvm/windows_aarch64_gnullvm-0.52.6.crate"
1290 +
CHECKSUM_SHA256_427="32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3"
1291 +
SOURCE_FILENAME_427="windows_aarch64_gnullvm-0.52.6.tar.gz"
1292 +
SOURCE_URI_428="https://static.crates.io/crates/windows_aarch64_msvc/windows_aarch64_msvc-0.48.5.crate"
1293 +
CHECKSUM_SHA256_428="dc35310971f3b2dbbf3f0690a219f40e2d9afcf64f9ab7cc1be722937c26b4bc"
1294 +
SOURCE_FILENAME_428="windows_aarch64_msvc-0.48.5.tar.gz"
1295 +
SOURCE_URI_429="https://static.crates.io/crates/windows_aarch64_msvc/windows_aarch64_msvc-0.52.6.crate"
1296 +
CHECKSUM_SHA256_429="09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469"
1297 +
SOURCE_FILENAME_429="windows_aarch64_msvc-0.52.6.tar.gz"
1298 +
SOURCE_URI_430="https://static.crates.io/crates/windows_i686_gnu/windows_i686_gnu-0.48.5.crate"
1299 +
CHECKSUM_SHA256_430="a75915e7def60c94dcef72200b9a8e58e5091744960da64ec734a6c6e9b3743e"
1300 +
SOURCE_FILENAME_430="windows_i686_gnu-0.48.5.tar.gz"
1301 +
SOURCE_URI_431="https://static.crates.io/crates/windows_i686_gnu/windows_i686_gnu-0.52.6.crate"
1302 +
CHECKSUM_SHA256_431="8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b"
1303 +
SOURCE_FILENAME_431="windows_i686_gnu-0.52.6.tar.gz"
1304 +
SOURCE_URI_432="https://static.crates.io/crates/windows_i686_gnullvm/windows_i686_gnullvm-0.52.6.crate"
1305 +
CHECKSUM_SHA256_432="0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66"
1306 +
SOURCE_FILENAME_432="windows_i686_gnullvm-0.52.6.tar.gz"
1307 +
SOURCE_URI_433="https://static.crates.io/crates/windows_i686_msvc/windows_i686_msvc-0.48.5.crate"
1308 +
CHECKSUM_SHA256_433="8f55c233f70c4b27f66c523580f78f1004e8b5a8b659e05a4eb49d4166cca406"
1309 +
SOURCE_FILENAME_433="windows_i686_msvc-0.48.5.tar.gz"
1310 +
SOURCE_URI_434="https://static.crates.io/crates/windows_i686_msvc/windows_i686_msvc-0.52.6.crate"
1311 +
CHECKSUM_SHA256_434="240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66"
1312 +
SOURCE_FILENAME_434="windows_i686_msvc-0.52.6.tar.gz"
1313 +
SOURCE_URI_435="https://static.crates.io/crates/windows_x86_64_gnu/windows_x86_64_gnu-0.48.5.crate"
1314 +
CHECKSUM_SHA256_435="53d40abd2583d23e4718fddf1ebec84dbff8381c07cae67ff7768bbf19c6718e"
1315 +
SOURCE_FILENAME_435="windows_x86_64_gnu-0.48.5.tar.gz"
1316 +
SOURCE_URI_436="https://static.crates.io/crates/windows_x86_64_gnu/windows_x86_64_gnu-0.52.6.crate"
1317 +
CHECKSUM_SHA256_436="147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78"
1318 +
SOURCE_FILENAME_436="windows_x86_64_gnu-0.52.6.tar.gz"
1319 +
SOURCE_URI_437="https://static.crates.io/crates/windows_x86_64_gnullvm/windows_x86_64_gnullvm-0.48.5.crate"
1320 +
CHECKSUM_SHA256_437="0b7b52767868a23d5bab768e390dc5f5c55825b6d30b86c844ff2dc7414044cc"
1321 +
SOURCE_FILENAME_437="windows_x86_64_gnullvm-0.48.5.tar.gz"
1322 +
SOURCE_URI_438="https://static.crates.io/crates/windows_x86_64_gnullvm/windows_x86_64_gnullvm-0.52.6.crate"
1323 +
CHECKSUM_SHA256_438="24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d"
1324 +
SOURCE_FILENAME_438="windows_x86_64_gnullvm-0.52.6.tar.gz"
1325 +
SOURCE_URI_439="https://static.crates.io/crates/windows_x86_64_msvc/windows_x86_64_msvc-0.48.5.crate"
1326 +
CHECKSUM_SHA256_439="ed94fce61571a4006852b7389a063ab983c02eb1bb37b47f8272ce92d06d9538"
1327 +
SOURCE_FILENAME_439="windows_x86_64_msvc-0.48.5.tar.gz"
1328 +
SOURCE_URI_440="https://static.crates.io/crates/windows_x86_64_msvc/windows_x86_64_msvc-0.52.6.crate"
1329 +
CHECKSUM_SHA256_440="589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
1330 +
SOURCE_FILENAME_440="windows_x86_64_msvc-0.52.6.tar.gz"
1331 +
SOURCE_URI_441="https://static.crates.io/crates/winnow/winnow-0.7.15.crate"
1332 +
CHECKSUM_SHA256_441="df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945"
1333 +
SOURCE_FILENAME_441="winnow-0.7.15.tar.gz"
1334 +
SOURCE_URI_442="https://static.crates.io/crates/writeable/writeable-0.6.4.crate"
1335 +
CHECKSUM_SHA256_442="3ad82d2a33cdc9674dc7465672f271e096168fcdbe0f799d9e6db8c5892679dc"
1336 +
SOURCE_FILENAME_442="writeable-0.6.4.tar.gz"
1337 +
SOURCE_URI_443="https://static.crates.io/crates/x509-parser/x509-parser-0.18.1.crate"
1338 +
CHECKSUM_SHA256_443="d43b0f71ce057da06bc0851b23ee24f3f86190b07203dd8f567d0b706a185202"
1339 +
SOURCE_FILENAME_443="x509-parser-0.18.1.tar.gz"
1340 +
SOURCE_URI_444="https://static.crates.io/crates/xattr/xattr-1.6.1.crate"
1341 +
CHECKSUM_SHA256_444="32e45ad4206f6d2479085147f02bc2ef834ac85886624a23575ae137c8aa8156"
1342 +
SOURCE_FILENAME_444="xattr-1.6.1.tar.gz"
1343 +
SOURCE_URI_445="https://static.crates.io/crates/yaml-rust/yaml-rust-0.4.5.crate"
1344 +
CHECKSUM_SHA256_445="56c1936c4cc7a1c9ab21a1ebb602eb942ba868cbd44a99cb7cdc5892335e1c85"
1345 +
SOURCE_FILENAME_445="yaml-rust-0.4.5.tar.gz"
1346 +
SOURCE_URI_446="https://static.crates.io/crates/yasna/yasna-0.6.0.crate"
1347 +
CHECKSUM_SHA256_446="b5f6765e852b9b4dc8e2a76843e4d64d1cea8e79bcde0b6901aea8e7c7f08282"
1348 +
SOURCE_FILENAME_446="yasna-0.6.0.tar.gz"
1349 +
SOURCE_URI_447="https://static.crates.io/crates/yoke/yoke-0.8.3.crate"
1350 +
CHECKSUM_SHA256_447="709fe23a0424b6a435d82152b1bd3fdfb0833487d5fa90d05d42762a9891fef5"
1351 +
SOURCE_FILENAME_447="yoke-0.8.3.tar.gz"
1352 +
SOURCE_URI_448="https://static.crates.io/crates/yoke-derive/yoke-derive-0.8.3.crate"
1353 +
CHECKSUM_SHA256_448="33811428bee40dbceb6d545e95754741d17a6aef9a4849f0fd62e2ba4f412a78"
1354 +
SOURCE_FILENAME_448="yoke-derive-0.8.3.tar.gz"
1355 +
SOURCE_URI_449="https://static.crates.io/crates/zerocopy/zerocopy-0.8.59.crate"
1356 +
CHECKSUM_SHA256_449="6df92bf3d9227be3d53173901ddbffac2babc27ae50f397776ffd6dc33f800cb"
1357 +
SOURCE_FILENAME_449="zerocopy-0.8.59.tar.gz"
1358 +
SOURCE_URI_450="https://static.crates.io/crates/zerocopy-derive/zerocopy-derive-0.8.59.crate"
1359 +
CHECKSUM_SHA256_450="ac4f328cf2f05d084e496c3e9c3f33ed0a183656a16e1fcec4d464d8373aec82"
1360 +
SOURCE_FILENAME_450="zerocopy-derive-0.8.59.tar.gz"
1361 +
SOURCE_URI_451="https://static.crates.io/crates/zerofrom/zerofrom-0.1.8.crate"
1362 +
CHECKSUM_SHA256_451="0ec05a11813ea801ff6d75110ad09cd0824ddba17dfe17128ea0d5f68e6c5272"
1363 +
SOURCE_FILENAME_451="zerofrom-0.1.8.tar.gz"
1364 +
SOURCE_URI_452="https://static.crates.io/crates/zerofrom-derive/zerofrom-derive-0.1.8.crate"
1365 +
CHECKSUM_SHA256_452="f75b4683f6c7f45248d4d64056a24298c6281e0993356d7d1b4a1a962ef10d4a"
1366 +
SOURCE_FILENAME_452="zerofrom-derive-0.1.8.tar.gz"
1367 +
SOURCE_URI_453="https://static.crates.io/crates/zeroize/zeroize-1.9.0.crate"
1368 +
CHECKSUM_SHA256_453="e13c156562582aa81c60cb29407084cdb54c4164760106ab78e6c5b0858cf64e"
1369 +
SOURCE_FILENAME_453="zeroize-1.9.0.tar.gz"
1370 +
SOURCE_URI_454="https://static.crates.io/crates/zerotrie/zerotrie-0.2.5.crate"
1371 +
CHECKSUM_SHA256_454="4ea269c3bd32f0a32c321907a2ae912ba6f4649bb0fc764a15627e99a7095a3f"
1372 +
SOURCE_FILENAME_454="zerotrie-0.2.5.tar.gz"
1373 +
SOURCE_URI_455="https://static.crates.io/crates/zerovec/zerovec-0.11.8.crate"
1374 +
CHECKSUM_SHA256_455="bb0464e17806c1d976d5cba29399c7f08e516e279e2ba493f63123b5fca67dd8"
1375 +
SOURCE_FILENAME_455="zerovec-0.11.8.tar.gz"
1376 +
SOURCE_URI_456="https://static.crates.io/crates/zerovec-derive/zerovec-derive-0.11.6.crate"
1377 +
CHECKSUM_SHA256_456="34df6fc39dbd26ddc9c10e6a2984476e13acce22e64e4487636ef494369225da"
1378 +
SOURCE_FILENAME_456="zerovec-derive-0.11.6.tar.gz"
1379 +
SOURCE_URI_457="https://static.crates.io/crates/zip/zip-2.4.2.crate"
1380 +
CHECKSUM_SHA256_457="fabe6324e908f85a1c52063ce7aa26b68dcb7eb6dbc83a2d148403c9bc3eba50"
1381 +
SOURCE_FILENAME_457="zip-2.4.2.tar.gz"
1382 +
SOURCE_URI_458="https://static.crates.io/crates/zlib-rs/zlib-rs-0.6.8.crate"
1383 +
CHECKSUM_SHA256_458="b268e58e7c693d7c271f93ffc4ba3b380412554231c85bf61ca7af91042a4112"
1384 +
SOURCE_FILENAME_458="zlib-rs-0.6.8.tar.gz"
1385 +
SOURCE_URI_459="https://static.crates.io/crates/zmij/zmij-1.0.23.crate"
1386 +
CHECKSUM_SHA256_459="29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
1387 +
SOURCE_FILENAME_459="zmij-1.0.23.tar.gz"
1388 +
SOURCE_URI_460="https://static.crates.io/crates/zopfli/zopfli-0.8.3.crate"
1389 +
CHECKSUM_SHA256_460="f05cd8797d63865425ff89b5c4a48804f35ba0ce8d125800027ad6017d2b5249"
1390 +
SOURCE_FILENAME_460="zopfli-0.8.3.tar.gz"
1391 +
SOURCE_URI_461="https://static.crates.io/crates/zstd/zstd-0.13.3.crate"
1392 +
CHECKSUM_SHA256_461="e91ee311a569c327171651566e07972200e76fcfe2242a4fa446149a3881c08a"
1393 +
SOURCE_FILENAME_461="zstd-0.13.3.tar.gz"
1394 +
SOURCE_URI_462="https://static.crates.io/crates/zstd-safe/zstd-safe-7.3.0.crate"
1395 +
CHECKSUM_SHA256_462="64d80649ab6db9d9f6f9c80a40becd948eda4714a0a5ac8c4d157a32231c7882"
1396 +
SOURCE_FILENAME_462="zstd-safe-7.3.0.tar.gz"
1397 +
SOURCE_URI_463="https://static.crates.io/crates/zstd-sys/zstd-sys-2.1.0+zstd.1.5.7.crate"
1398 +
CHECKSUM_SHA256_463="0ef0a8027ec3ee71300ab3bcbcd0393f434aa72b91ca6d635a39941deae8eea0"
1399 +
SOURCE_FILENAME_463="zstd-sys-2.1.0+zstd.1.5.7.tar.gz"
1400 +
SOURCE_URI_464="https://static.crates.io/crates/zune-core/zune-core-0.5.3.crate"
1401 +
CHECKSUM_SHA256_464="d56377fd46368984a170bc5aac5567e52ca5da874caa60bea39fcbca78fb658b"
1402 +
SOURCE_FILENAME_464="zune-core-0.5.3.tar.gz"
1403 +
SOURCE_URI_465="https://static.crates.io/crates/zune-jpeg/zune-jpeg-0.5.15.crate"
1404 +
CHECKSUM_SHA256_465="27bc9d5b815bc103f142aa054f561d9187d191692ec7c2d1e2b4737f8dbd7296"
1405 +
SOURCE_FILENAME_465="zune-jpeg-0.5.15.tar.gz"
1406 +
1407 +
ARCHITECTURES="!all x86_64"
1408 +
1409 +
PROVIDES="
1410 +
	oh$secondaryArchSuffix = $portVersion
1411 +
	cmd:oh
1412 +
	"
1413 +
REQUIRES="
1414 +
	haiku$secondaryArchSuffix
1415 +
	lib:libzstd$secondaryArchSuffix
1416 +
	"
1417 +
1418 +
BUILD_REQUIRES="
1419 +
	haiku${secondaryArchSuffix}_devel
1420 +
	devel:libzstd$secondaryArchSuffix
1421 +
	"
1422 +
BUILD_PREREQUIRES="
1423 +
	cmd:gcc$secondaryArchSuffix
1424 +
	cmd:pkg_config$secondaryArchSuffix
1425 +
	cmd:rustc$secondaryArchSuffix
1426 +
	"
1427 +
1428 +
BUILD()
1429 +
{
1430 +
	export CARGO_HOME=$sourceDir/../cargo
1431 +
	vendor=$CARGO_HOME/haiku
1432 +
	mkdir -p "$vendor"
1433 +
	for i in $(seq 2 465); do
1434 +
		eval "srcDir=\$sourceDir$i"
1435 +
		eval "sha256sum=\$CHECKSUM_SHA256_$i"
1436 +
		set -- "$srcDir"/*
1437 +
		ln -sf "$1" "$vendor"
1438 +
		cat <<-EOF >"$vendor/${1##*/}/.cargo-checksum.json"
1439 +
		{
1440 +
		  "package": "$sha256sum",
1441 +
		  "files": {}
1442 +
		}
1443 +
		EOF
1444 +
	done
1445 +
1446 +
	cat <<-EOF >"$CARGO_HOME"/config.toml
1447 +
	[source.haiku]
1448 +
	directory = "$vendor"
1449 +
1450 +
	[source.crates-io]
1451 +
	replace-with = "haiku"
1452 +
	EOF
1453 +
1454 +
	# The bundled zstd's hidden symbols break the link on Haiku: use the system's.
1455 +
	export ZSTD_SYS_USE_PKG_CONFIG=1
1456 +
	export OXY_BUILD=v$portVersion
1457 +
	cargo build --release --frozen -p oh
1458 +
}
1459 +
1460 +
INSTALL()
1461 +
{
1462 +
	install -v -m 755 -d "$prefix"/bin "$docDir"
1463 +
	install -v -m 755 ./target/release/oh "$prefix"/bin
1464 +
	install -v -m 644 README.md LICENSE docs/oh.md "$docDir"
1465 +
}
1466 +
1467 +
TEST()
1468 +
{
1469 +
	export CARGO_HOME=$sourceDir/../cargo
1470 +
	export ZSTD_SYS_USE_PKG_CONFIG=1
1471 +
	cargo test --release --frozen -p hub-api
1472 +
}
added packaging/haikuports/dev-vcs/oxy/oxy-0.9.0.recipe +990 −0
1 +
SUMMARY="A version control system of signed, commuting changes, with Git interop"
2 +
DESCRIPTION="oxy is the command line of Oxygen, a version control system built on \
3 +
patch theory: changes commute, conflicts are recorded rather than blocking, \
4 +
every change can be signed, and a trust policy says who may move what. It \
5 +
imports and exports Git history, clones from OxygenHub and other Oxygen \
6 +
hosts, and keeps repositories on disk or in S3, GCS or Azure buckets."
7 +
HOMEPAGE="https://oxygenhub.io/andymccall/oxygen"
8 +
COPYRIGHT="2026 Andy McCall"
9 +
LICENSE="MIT
10 +
	Apache v2"
11 +
REVISION="1"
12 +
SOURCE_URI="$HOMEPAGE/releases/download/v$portVersion/oxy-$portVersion-src.tar.gz"
13 +
CHECKSUM_SHA256="b284392c53677177429b7db7f47b0b3585c766a910654a24ab47a6a75555bb8a"
14 +
SOURCE_DIR="oxygen-$portVersion"
15 +
SOURCE_URI_2="https://static.crates.io/crates/adler2/adler2-2.0.1.crate"
16 +
CHECKSUM_SHA256_2="320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
17 +
SOURCE_FILENAME_2="adler2-2.0.1.tar.gz"
18 +
SOURCE_URI_3="https://static.crates.io/crates/aho-corasick/aho-corasick-1.1.5.crate"
19 +
CHECKSUM_SHA256_3="c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba"
20 +
SOURCE_FILENAME_3="aho-corasick-1.1.5.tar.gz"
21 +
SOURCE_URI_4="https://static.crates.io/crates/android_system_properties/android_system_properties-0.1.6.crate"
22 +
CHECKSUM_SHA256_4="ae221649c9976a6f6c56ae1facf410f3ddb33cc661c4b7b61020a912d4237fbc"
23 +
SOURCE_FILENAME_4="android_system_properties-0.1.6.tar.gz"
24 +
SOURCE_URI_5="https://static.crates.io/crates/anstream/anstream-1.0.0.crate"
25 +
CHECKSUM_SHA256_5="824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d"
26 +
SOURCE_FILENAME_5="anstream-1.0.0.tar.gz"
27 +
SOURCE_URI_6="https://static.crates.io/crates/anstyle/anstyle-1.0.14.crate"
28 +
CHECKSUM_SHA256_6="940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000"
29 +
SOURCE_FILENAME_6="anstyle-1.0.14.tar.gz"
30 +
SOURCE_URI_7="https://static.crates.io/crates/anstyle-parse/anstyle-parse-1.0.0.crate"
31 +
CHECKSUM_SHA256_7="52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e"
32 +
SOURCE_FILENAME_7="anstyle-parse-1.0.0.tar.gz"
33 +
SOURCE_URI_8="https://static.crates.io/crates/anstyle-query/anstyle-query-1.1.5.crate"
34 +
CHECKSUM_SHA256_8="40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
35 +
SOURCE_FILENAME_8="anstyle-query-1.1.5.tar.gz"
36 +
SOURCE_URI_9="https://static.crates.io/crates/anstyle-wincon/anstyle-wincon-3.0.11.crate"
37 +
CHECKSUM_SHA256_9="291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
38 +
SOURCE_FILENAME_9="anstyle-wincon-3.0.11.tar.gz"
39 +
SOURCE_URI_10="https://static.crates.io/crates/anyhow/anyhow-1.0.104.crate"
40 +
CHECKSUM_SHA256_10="330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470"
41 +
SOURCE_FILENAME_10="anyhow-1.0.104.tar.gz"
42 +
SOURCE_URI_11="https://static.crates.io/crates/arc-swap/arc-swap-1.9.2.crate"
43 +
CHECKSUM_SHA256_11="c049c0be4daef0b145cb3555416b3b8ef5b7888a38aea1a3a155801fe7b0810b"
44 +
SOURCE_FILENAME_11="arc-swap-1.9.2.tar.gz"
45 +
SOURCE_URI_12="https://static.crates.io/crates/arrayvec/arrayvec-0.7.8.crate"
46 +
CHECKSUM_SHA256_12="d3fb67a6e08acf24fdeccbac2cb6ac4305825bd1f117462e0e6f2f193345ad56"
47 +
SOURCE_FILENAME_12="arrayvec-0.7.8.tar.gz"
48 +
SOURCE_URI_13="https://static.crates.io/crates/asn1-rs/asn1-rs-0.7.2.crate"
49 +
CHECKSUM_SHA256_13="b7f43a50ac4fdca5df8e885c21b835997f0a1cdee65494a6847694a98652d9d8"
50 +
SOURCE_FILENAME_13="asn1-rs-0.7.2.tar.gz"
51 +
SOURCE_URI_14="https://static.crates.io/crates/asn1-rs-derive/asn1-rs-derive-0.6.0.crate"
52 +
CHECKSUM_SHA256_14="3109e49b1e4909e9db6515a30c633684d68cdeaa252f215214cb4fa1a5bfee2c"
53 +
SOURCE_FILENAME_14="asn1-rs-derive-0.6.0.tar.gz"
54 +
SOURCE_URI_15="https://static.crates.io/crates/asn1-rs-impl/asn1-rs-impl-0.2.0.crate"
55 +
CHECKSUM_SHA256_15="7b18050c2cd6fe86c3a76584ef5e0baf286d038cda203eb6223df2cc413565f7"
56 +
SOURCE_FILENAME_15="asn1-rs-impl-0.2.0.tar.gz"
57 +
SOURCE_URI_16="https://static.crates.io/crates/async-trait/async-trait-0.1.92.crate"
58 +
CHECKSUM_SHA256_16="82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667"
59 +
SOURCE_FILENAME_16="async-trait-0.1.92.tar.gz"
60 +
SOURCE_URI_17="https://static.crates.io/crates/atomic-waker/atomic-waker-1.1.2.crate"
61 +
CHECKSUM_SHA256_17="1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0"
62 +
SOURCE_FILENAME_17="atomic-waker-1.1.2.tar.gz"
63 +
SOURCE_URI_18="https://static.crates.io/crates/autocfg/autocfg-1.5.1.crate"
64 +
CHECKSUM_SHA256_18="f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
65 +
SOURCE_FILENAME_18="autocfg-1.5.1.tar.gz"
66 +
SOURCE_URI_19="https://static.crates.io/crates/axum/axum-0.8.9.crate"
67 +
CHECKSUM_SHA256_19="31b698c5f9a010f6573133b09e0de5408834d0c82f8d7475a89fc1867a71cd90"
68 +
SOURCE_FILENAME_19="axum-0.8.9.tar.gz"
69 +
SOURCE_URI_20="https://static.crates.io/crates/axum-core/axum-core-0.5.6.crate"
70 +
CHECKSUM_SHA256_20="08c78f31d7b1291f7ee735c1c6780ccde7785daae9a9206026862dab7d8792d1"
71 +
SOURCE_FILENAME_20="axum-core-0.5.6.tar.gz"
72 +
SOURCE_URI_21="https://static.crates.io/crates/axum-server/axum-server-0.8.0.crate"
73 +
CHECKSUM_SHA256_21="b1df331683d982a0b9492b38127151e6453639cd34926eb9c07d4cd8c6d22bfc"
74 +
SOURCE_FILENAME_21="axum-server-0.8.0.tar.gz"
75 +
SOURCE_URI_22="https://static.crates.io/crates/base64/base64-0.22.1.crate"
76 +
CHECKSUM_SHA256_22="72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
77 +
SOURCE_FILENAME_22="base64-0.22.1.tar.gz"
78 +
SOURCE_URI_23="https://static.crates.io/crates/base64/base64-0.23.1.crate"
79 +
CHECKSUM_SHA256_23="ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
80 +
SOURCE_FILENAME_23="base64-0.23.1.tar.gz"
81 +
SOURCE_URI_24="https://static.crates.io/crates/base64ct/base64ct-1.8.3.crate"
82 +
CHECKSUM_SHA256_24="2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
83 +
SOURCE_FILENAME_24="base64ct-1.8.3.tar.gz"
84 +
SOURCE_URI_25="https://static.crates.io/crates/bit-set/bit-set-0.8.0.crate"
85 +
CHECKSUM_SHA256_25="08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3"
86 +
SOURCE_FILENAME_25="bit-set-0.8.0.tar.gz"
87 +
SOURCE_URI_26="https://static.crates.io/crates/bit-vec/bit-vec-0.8.0.crate"
88 +
CHECKSUM_SHA256_26="5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7"
89 +
SOURCE_FILENAME_26="bit-vec-0.8.0.tar.gz"
90 +
SOURCE_URI_27="https://static.crates.io/crates/bit-vec/bit-vec-0.9.1.crate"
91 +
CHECKSUM_SHA256_27="b71798fca2c1fe1086445a7258a4bc81e6e49dcd24c8d0dd9a1e57395b603f51"
92 +
SOURCE_FILENAME_27="bit-vec-0.9.1.tar.gz"
93 +
SOURCE_URI_28="https://static.crates.io/crates/bitflags/bitflags-2.13.2.crate"
94 +
CHECKSUM_SHA256_28="3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06"
95 +
SOURCE_FILENAME_28="bitflags-2.13.2.tar.gz"
96 +
SOURCE_URI_29="https://static.crates.io/crates/blake3/blake3-1.8.7.crate"
97 +
CHECKSUM_SHA256_29="6d9e454fc11f76977dc803893aff6304ed33d6a26efae8696573bea74baa27ae"
98 +
SOURCE_FILENAME_29="blake3-1.8.7.tar.gz"
99 +
SOURCE_URI_30="https://static.crates.io/crates/block-buffer/block-buffer-0.12.1.crate"
100 +
CHECKSUM_SHA256_30="d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa"
101 +
SOURCE_FILENAME_30="block-buffer-0.12.1.tar.gz"
102 +
SOURCE_URI_31="https://static.crates.io/crates/bstr/bstr-1.13.1.crate"
103 +
CHECKSUM_SHA256_31="6bb31b46c14244e20ee9984b11bf5c992b91fb6939fea616e3512c8baecdbe5f"
104 +
SOURCE_FILENAME_31="bstr-1.13.1.tar.gz"
105 +
SOURCE_URI_32="https://static.crates.io/crates/bumpalo/bumpalo-3.20.3.crate"
106 +
CHECKSUM_SHA256_32="72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
107 +
SOURCE_FILENAME_32="bumpalo-3.20.3.tar.gz"
108 +
SOURCE_URI_33="https://static.crates.io/crates/bytes/bytes-1.12.1.crate"
109 +
CHECKSUM_SHA256_33="fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04"
110 +
SOURCE_FILENAME_33="bytes-1.12.1.tar.gz"
111 +
SOURCE_URI_34="https://static.crates.io/crates/cc/cc-1.5.1.crate"
112 +
CHECKSUM_SHA256_34="f360145194ee8e21db5ee7f3fcd4fe52210864c75c985dae33218202c8bbe040"
113 +
SOURCE_FILENAME_34="cc-1.5.1.tar.gz"
114 +
SOURCE_URI_35="https://static.crates.io/crates/cfg-if/cfg-if-1.0.5.crate"
115 +
CHECKSUM_SHA256_35="4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600"
116 +
SOURCE_FILENAME_35="cfg-if-1.0.5.tar.gz"
117 +
SOURCE_URI_36="https://static.crates.io/crates/chacha20/chacha20-0.10.2.crate"
118 +
CHECKSUM_SHA256_36="65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06"
119 +
SOURCE_FILENAME_36="chacha20-0.10.2.tar.gz"
120 +
SOURCE_URI_37="https://static.crates.io/crates/chrono/chrono-0.4.45.crate"
121 +
CHECKSUM_SHA256_37="1aa79e62e7697b8e29b513a68abacf485adcd1fe8284a4316c5ae868e6633327"
122 +
SOURCE_FILENAME_37="chrono-0.4.45.tar.gz"
123 +
SOURCE_URI_38="https://static.crates.io/crates/clap/clap-4.6.7.crate"
124 +
CHECKSUM_SHA256_38="aa8876b300ab35ba921adea3dfd70157a46249b33f95c9084ae5709785478946"
125 +
SOURCE_FILENAME_38="clap-4.6.7.tar.gz"
126 +
SOURCE_URI_39="https://static.crates.io/crates/clap_builder/clap_builder-4.6.7.crate"
127 +
CHECKSUM_SHA256_39="ec0797fb7aeb1406c84efac526901f7ec3ead2124f946b494e72879d4b54704d"
128 +
SOURCE_FILENAME_39="clap_builder-4.6.7.tar.gz"
129 +
SOURCE_URI_40="https://static.crates.io/crates/clap_complete/clap_complete-4.6.11.crate"
130 +
CHECKSUM_SHA256_40="037e2a1a92236d0aff7e845093f64661d6df4c02c9fcc61a60e9e1d736fa392f"
131 +
SOURCE_FILENAME_40="clap_complete-4.6.11.tar.gz"
132 +
SOURCE_URI_41="https://static.crates.io/crates/clap_derive/clap_derive-4.6.7.crate"
133 +
CHECKSUM_SHA256_41="f9c751b79415d4e559e3d1fcf128e09e720eb673a06d26cf6f392d37d75b66e0"
134 +
SOURCE_FILENAME_41="clap_derive-4.6.7.tar.gz"
135 +
SOURCE_URI_42="https://static.crates.io/crates/clap_lex/clap_lex-1.1.1.crate"
136 +
CHECKSUM_SHA256_42="1c133bc6a41be0d194c306b5506d15e6feeea7b1d6604bd3f8310dfb2ca96486"
137 +
SOURCE_FILENAME_42="clap_lex-1.1.1.tar.gz"
138 +
SOURCE_URI_43="https://static.crates.io/crates/clap_mangen/clap_mangen-0.3.3.crate"
139 +
CHECKSUM_SHA256_43="211d617eaa4b735c96c9e0228fcbdb5120ef623f2b8cb67ffb84c3e02dbc28a4"
140 +
SOURCE_FILENAME_43="clap_mangen-0.3.3.tar.gz"
141 +
SOURCE_URI_44="https://static.crates.io/crates/cmov/cmov-0.5.4.crate"
142 +
CHECKSUM_SHA256_44="0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a"
143 +
SOURCE_FILENAME_44="cmov-0.5.4.tar.gz"
144 +
SOURCE_URI_45="https://static.crates.io/crates/colorchoice/colorchoice-1.0.5.crate"
145 +
CHECKSUM_SHA256_45="1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
146 +
SOURCE_FILENAME_45="colorchoice-1.0.5.tar.gz"
147 +
SOURCE_URI_46="https://static.crates.io/crates/console/console-0.15.11.crate"
148 +
CHECKSUM_SHA256_46="054ccb5b10f9f2cbf51eb355ca1d05c2d279ce1804688d0db74b4733a5aeafd8"
149 +
SOURCE_FILENAME_46="console-0.15.11.tar.gz"
150 +
SOURCE_URI_47="https://static.crates.io/crates/const-oid/const-oid-0.10.2.crate"
151 +
CHECKSUM_SHA256_47="a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c"
152 +
SOURCE_FILENAME_47="const-oid-0.10.2.tar.gz"
153 +
SOURCE_URI_48="https://static.crates.io/crates/constant_time_eq/constant_time_eq-0.4.2.crate"
154 +
CHECKSUM_SHA256_48="3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b"
155 +
SOURCE_FILENAME_48="constant_time_eq-0.4.2.tar.gz"
156 +
SOURCE_URI_49="https://static.crates.io/crates/cpufeatures/cpufeatures-0.3.1.crate"
157 +
CHECKSUM_SHA256_49="5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566"
158 +
SOURCE_FILENAME_49="cpufeatures-0.3.1.tar.gz"
159 +
SOURCE_URI_50="https://static.crates.io/crates/crc32fast/crc32fast-1.5.2.crate"
160 +
CHECKSUM_SHA256_50="01a7799fd6b852db0e61728dde9a204c423b44d689dbd432522543614b490e78"
161 +
SOURCE_FILENAME_50="crc32fast-1.5.2.tar.gz"
162 +
SOURCE_URI_51="https://static.crates.io/crates/crossbeam-deque/crossbeam-deque-0.8.8.crate"
163 +
CHECKSUM_SHA256_51="622f3fc73690be383c7214310406f28a90e6edeadc3cea882f9d71e495b9711a"
164 +
SOURCE_FILENAME_51="crossbeam-deque-0.8.8.tar.gz"
165 +
SOURCE_URI_52="https://static.crates.io/crates/crossbeam-epoch/crossbeam-epoch-0.9.21.crate"
166 +
CHECKSUM_SHA256_52="dc74980687109a3b14c72fd458107bf0baa1da1a1a805e178d15501ba9b86d9d"
167 +
SOURCE_FILENAME_52="crossbeam-epoch-0.9.21.tar.gz"
168 +
SOURCE_URI_53="https://static.crates.io/crates/crossbeam-utils/crossbeam-utils-0.8.23.crate"
169 +
CHECKSUM_SHA256_53="a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6"
170 +
SOURCE_FILENAME_53="crossbeam-utils-0.8.23.tar.gz"
171 +
SOURCE_URI_54="https://static.crates.io/crates/crypto-common/crypto-common-0.1.7.crate"
172 +
CHECKSUM_SHA256_54="78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
173 +
SOURCE_FILENAME_54="crypto-common-0.1.7.tar.gz"
174 +
SOURCE_URI_55="https://static.crates.io/crates/crypto-common/crypto-common-0.2.2.crate"
175 +
CHECKSUM_SHA256_55="ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453"
176 +
SOURCE_FILENAME_55="crypto-common-0.2.2.tar.gz"
177 +
SOURCE_URI_56="https://static.crates.io/crates/ctutils/ctutils-0.4.2.crate"
178 +
CHECKSUM_SHA256_56="7d5515a3834141de9eafb9717ad39eea8247b5674e6066c404e8c4b365d2a29e"
179 +
SOURCE_FILENAME_56="ctutils-0.4.2.tar.gz"
180 +
SOURCE_URI_57="https://static.crates.io/crates/curve25519-dalek/curve25519-dalek-5.0.0.crate"
181 +
CHECKSUM_SHA256_57="b5eed333089e2e1c1ac8c6c0398e5e2497b4c9926ca6d0365ed1e099afa5bc23"
182 +
SOURCE_FILENAME_57="curve25519-dalek-5.0.0.tar.gz"
183 +
SOURCE_URI_58="https://static.crates.io/crates/curve25519-dalek-derive/curve25519-dalek-derive-0.1.1.crate"
184 +
CHECKSUM_SHA256_58="f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3"
185 +
SOURCE_FILENAME_58="curve25519-dalek-derive-0.1.1.tar.gz"
186 +
SOURCE_URI_59="https://static.crates.io/crates/data-encoding/data-encoding-2.11.1.crate"
187 +
CHECKSUM_SHA256_59="4583a4551df46e2792f82ceeac45e850d2e2d5debba0b91f102385cda5b11f06"
188 +
SOURCE_FILENAME_59="data-encoding-2.11.1.tar.gz"
189 +
SOURCE_URI_60="https://static.crates.io/crates/der/der-0.8.2.crate"
190 +
CHECKSUM_SHA256_60="a878c850e9e421b20262e9b41f9c860e4785fa07541c266b62ff9d1ef998a80a"
191 +
SOURCE_FILENAME_60="der-0.8.2.tar.gz"
192 +
SOURCE_URI_61="https://static.crates.io/crates/der-parser/der-parser-10.0.0.crate"
193 +
CHECKSUM_SHA256_61="07da5016415d5a3c4dd39b11ed26f915f52fc4e0dc197d87908bc916e51bc1a6"
194 +
SOURCE_FILENAME_61="der-parser-10.0.0.tar.gz"
195 +
SOURCE_URI_62="https://static.crates.io/crates/deranged/deranged-0.5.8.crate"
196 +
CHECKSUM_SHA256_62="7cd812cc2bc1d69d4764bd80df88b4317eaef9e773c75226407d9bc0876b211c"
197 +
SOURCE_FILENAME_62="deranged-0.5.8.tar.gz"
198 +
SOURCE_URI_63="https://static.crates.io/crates/digest/digest-0.10.7.crate"
199 +
CHECKSUM_SHA256_63="9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
200 +
SOURCE_FILENAME_63="digest-0.10.7.tar.gz"
201 +
SOURCE_URI_64="https://static.crates.io/crates/digest/digest-0.11.3.crate"
202 +
CHECKSUM_SHA256_64="f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
203 +
SOURCE_FILENAME_64="digest-0.11.3.tar.gz"
204 +
SOURCE_URI_65="https://static.crates.io/crates/displaydoc/displaydoc-0.2.7.crate"
205 +
CHECKSUM_SHA256_65="c6232dd377dcc64799954cbd3a9bb882e9cdc1308ccd87b1c098f1fb2eaf82a8"
206 +
SOURCE_FILENAME_65="displaydoc-0.2.7.tar.gz"
207 +
SOURCE_URI_66="https://static.crates.io/crates/ed25519/ed25519-3.0.0.crate"
208 +
CHECKSUM_SHA256_66="29fcf32e6c73d1079f83ab4d782de2d81620346a5f38c6237a86a22f8368980a"
209 +
SOURCE_FILENAME_66="ed25519-3.0.0.tar.gz"
210 +
SOURCE_URI_67="https://static.crates.io/crates/ed25519-dalek/ed25519-dalek-3.0.0.crate"
211 +
CHECKSUM_SHA256_67="6ebaa1a2bf1290ab3bfe5a7b771d050ebffab2711c19a81691c683a5144a25de"
212 +
SOURCE_FILENAME_67="ed25519-dalek-3.0.0.tar.gz"
213 +
SOURCE_URI_68="https://static.crates.io/crates/either/either-1.18.0.crate"
214 +
CHECKSUM_SHA256_68="252afb9ae5eaa683babdc6a068b3f5726eb19e05070c731f9b2a23a7c3e8ed34"
215 +
SOURCE_FILENAME_68="either-1.18.0.tar.gz"
216 +
SOURCE_URI_69="https://static.crates.io/crates/encode_unicode/encode_unicode-1.0.0.crate"
217 +
CHECKSUM_SHA256_69="34aa73646ffb006b8f5147f3dc182bd4bcb190227ce861fc4a4844bf8e3cb2c0"
218 +
SOURCE_FILENAME_69="encode_unicode-1.0.0.tar.gz"
219 +
SOURCE_URI_70="https://static.crates.io/crates/equivalent/equivalent-1.0.2.crate"
220 +
CHECKSUM_SHA256_70="877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
221 +
SOURCE_FILENAME_70="equivalent-1.0.2.tar.gz"
222 +
SOURCE_URI_71="https://static.crates.io/crates/errno/errno-0.3.14.crate"
223 +
CHECKSUM_SHA256_71="39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
224 +
SOURCE_FILENAME_71="errno-0.3.14.tar.gz"
225 +
SOURCE_URI_72="https://static.crates.io/crates/fastrand/fastrand-2.5.0.crate"
226 +
CHECKSUM_SHA256_72="da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223"
227 +
SOURCE_FILENAME_72="fastrand-2.5.0.tar.gz"
228 +
SOURCE_URI_73="https://static.crates.io/crates/fiat-crypto/fiat-crypto-0.3.0.crate"
229 +
CHECKSUM_SHA256_73="64cd1e32ddd350061ae6edb1b082d7c54915b5c672c389143b9a63403a109f24"
230 +
SOURCE_FILENAME_73="fiat-crypto-0.3.0.tar.gz"
231 +
SOURCE_URI_74="https://static.crates.io/crates/find-msvc-tools/find-msvc-tools-0.1.14.crate"
232 +
CHECKSUM_SHA256_74="aedcfb3409746eddb02b9e19ebda1c3394f759a152e48ee875a0844d1b955484"
233 +
SOURCE_FILENAME_74="find-msvc-tools-0.1.14.tar.gz"
234 +
SOURCE_URI_75="https://static.crates.io/crates/flate2/flate2-1.1.10.crate"
235 +
CHECKSUM_SHA256_75="6e634e2e0ebac1ee034020da1ca582e17ffe4e0f5e985823721e168928136dcb"
236 +
SOURCE_FILENAME_75="flate2-1.1.10.tar.gz"
237 +
SOURCE_URI_76="https://static.crates.io/crates/fnv/fnv-1.0.7.crate"
238 +
CHECKSUM_SHA256_76="3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
239 +
SOURCE_FILENAME_76="fnv-1.0.7.tar.gz"
240 +
SOURCE_URI_77="https://static.crates.io/crates/form_urlencoded/form_urlencoded-1.2.2.crate"
241 +
CHECKSUM_SHA256_77="cb4cb245038516f5f85277875cdaa4f7d2c9a0fa0468de06ed190163b1581fcf"
242 +
SOURCE_FILENAME_77="form_urlencoded-1.2.2.tar.gz"
243 +
SOURCE_URI_78="https://static.crates.io/crates/fs-err/fs-err-3.3.1.crate"
244 +
CHECKSUM_SHA256_78="b91aa448ca50d7e79433bdf3ee8d99215430d2ec02ade5aefab2a073a1822e8a"
245 +
SOURCE_FILENAME_78="fs-err-3.3.1.tar.gz"
246 +
SOURCE_URI_79="https://static.crates.io/crates/futures/futures-0.3.34.crate"
247 +
CHECKSUM_SHA256_79="9a31d2a3fbaaeb2af2368bbdd904aa8e812d3c04a1ee10d3171f52d556e5d0a3"
248 +
SOURCE_FILENAME_79="futures-0.3.34.tar.gz"
249 +
SOURCE_URI_80="https://static.crates.io/crates/futures-channel/futures-channel-0.3.34.crate"
250 +
CHECKSUM_SHA256_80="b1f9e3d69d39e4862ffed03ed071a76f9a13ba1d9109d355b0f0aa6b15e393c4"
251 +
SOURCE_FILENAME_80="futures-channel-0.3.34.tar.gz"
252 +
SOURCE_URI_81="https://static.crates.io/crates/futures-core/futures-core-0.3.34.crate"
253 +
CHECKSUM_SHA256_81="92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e"
254 +
SOURCE_FILENAME_81="futures-core-0.3.34.tar.gz"
255 +
SOURCE_URI_82="https://static.crates.io/crates/futures-executor/futures-executor-0.3.34.crate"
256 +
CHECKSUM_SHA256_82="031b47cf1a3c6cc8bc2fc76cd437f521619387907d469316e7c0bc278f1f5432"
257 +
SOURCE_FILENAME_82="futures-executor-0.3.34.tar.gz"
258 +
SOURCE_URI_83="https://static.crates.io/crates/futures-io/futures-io-0.3.34.crate"
259 +
CHECKSUM_SHA256_83="53c0fa8157de1303bfffdaa1cc2a673bfffb60102f76b0ef4441659124373fed"
260 +
SOURCE_FILENAME_83="futures-io-0.3.34.tar.gz"
261 +
SOURCE_URI_84="https://static.crates.io/crates/futures-macro/futures-macro-0.3.34.crate"
262 +
CHECKSUM_SHA256_84="9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44"
263 +
SOURCE_FILENAME_84="futures-macro-0.3.34.tar.gz"
264 +
SOURCE_URI_85="https://static.crates.io/crates/futures-sink/futures-sink-0.3.34.crate"
265 +
CHECKSUM_SHA256_85="1944426bf7d03f1d14f708785e4b33efd750b36d48a157b836b3efc15ede8e1d"
266 +
SOURCE_FILENAME_85="futures-sink-0.3.34.tar.gz"
267 +
SOURCE_URI_86="https://static.crates.io/crates/futures-task/futures-task-0.3.34.crate"
268 +
CHECKSUM_SHA256_86="cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd"
269 +
SOURCE_FILENAME_86="futures-task-0.3.34.tar.gz"
270 +
SOURCE_URI_87="https://static.crates.io/crates/futures-util/futures-util-0.3.34.crate"
271 +
CHECKSUM_SHA256_87="0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc"
272 +
SOURCE_FILENAME_87="futures-util-0.3.34.tar.gz"
273 +
SOURCE_URI_88="https://static.crates.io/crates/generic-array/generic-array-0.14.7.crate"
274 +
CHECKSUM_SHA256_88="85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
275 +
SOURCE_FILENAME_88="generic-array-0.14.7.tar.gz"
276 +
SOURCE_URI_89="https://static.crates.io/crates/getrandom/getrandom-0.2.17.crate"
277 +
CHECKSUM_SHA256_89="ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0"
278 +
SOURCE_FILENAME_89="getrandom-0.2.17.tar.gz"
279 +
SOURCE_URI_90="https://static.crates.io/crates/getrandom/getrandom-0.3.4.crate"
280 +
CHECKSUM_SHA256_90="899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd"
281 +
SOURCE_FILENAME_90="getrandom-0.3.4.tar.gz"
282 +
SOURCE_URI_91="https://static.crates.io/crates/getrandom/getrandom-0.4.3.crate"
283 +
CHECKSUM_SHA256_91="300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099"
284 +
SOURCE_FILENAME_91="getrandom-0.4.3.tar.gz"
285 +
SOURCE_URI_92="https://static.crates.io/crates/globset/globset-0.4.20.crate"
286 +
CHECKSUM_SHA256_92="07c34a9410465b45bd9787443bc7370f37735bad04b0f0cd57ff1a3186c98988"
287 +
SOURCE_FILENAME_92="globset-0.4.20.tar.gz"
288 +
SOURCE_URI_93="https://static.crates.io/crates/h2/h2-0.4.19.crate"
289 +
CHECKSUM_SHA256_93="ef8e5e5a340588f4452631496976cf8636d4a7ecf600239fdc27615d2530bc16"
290 +
SOURCE_FILENAME_93="h2-0.4.19.tar.gz"
291 +
SOURCE_URI_94="https://static.crates.io/crates/hashbrown/hashbrown-0.17.1.crate"
292 +
CHECKSUM_SHA256_94="ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
293 +
SOURCE_FILENAME_94="hashbrown-0.17.1.tar.gz"
294 +
SOURCE_URI_95="https://static.crates.io/crates/heck/heck-0.5.0.crate"
295 +
CHECKSUM_SHA256_95="2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
296 +
SOURCE_FILENAME_95="heck-0.5.0.tar.gz"
297 +
SOURCE_URI_96="https://static.crates.io/crates/hex/hex-0.4.3.crate"
298 +
CHECKSUM_SHA256_96="7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
299 +
SOURCE_FILENAME_96="hex-0.4.3.tar.gz"
300 +
SOURCE_URI_97="https://static.crates.io/crates/http/http-1.5.0.crate"
301 +
CHECKSUM_SHA256_97="918d3568bebf352712bc2ef3d46a8bcf1a75b373be6539de198e9105cbbf9ce0"
302 +
SOURCE_FILENAME_97="http-1.5.0.tar.gz"
303 +
SOURCE_URI_98="https://static.crates.io/crates/http-body/http-body-1.1.0.crate"
304 +
CHECKSUM_SHA256_98="ca2a8f2913ee65f60facd6a5905613afaa448497a0230cc41ce022d93290bc2c"
305 +
SOURCE_FILENAME_98="http-body-1.1.0.tar.gz"
306 +
SOURCE_URI_99="https://static.crates.io/crates/http-body-util/http-body-util-0.1.5.crate"
307 +
CHECKSUM_SHA256_99="23169fe34a5fbcdd3f3862e78fb9b6fccd5f02a6dc6f732547005d45631ce71c"
308 +
SOURCE_FILENAME_99="http-body-util-0.1.5.tar.gz"
309 +
SOURCE_URI_100="https://static.crates.io/crates/httparse/httparse-1.10.1.crate"
310 +
CHECKSUM_SHA256_100="6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87"
311 +
SOURCE_FILENAME_100="httparse-1.10.1.tar.gz"
312 +
SOURCE_URI_101="https://static.crates.io/crates/httpdate/httpdate-1.0.3.crate"
313 +
CHECKSUM_SHA256_101="df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
314 +
SOURCE_FILENAME_101="httpdate-1.0.3.tar.gz"
315 +
SOURCE_URI_102="https://static.crates.io/crates/humantime/humantime-2.4.0.crate"
316 +
CHECKSUM_SHA256_102="15cdd26707701c53297e2fa6afb323d55fbc1d0810c3aec078ae3ef0424c3c15"
317 +
SOURCE_FILENAME_102="humantime-2.4.0.tar.gz"
318 +
SOURCE_URI_103="https://static.crates.io/crates/hybrid-array/hybrid-array-0.4.15.crate"
319 +
CHECKSUM_SHA256_103="27f864f10dfb56725ce5ce5472bc52252c8f93a4ab86327122cebf62c5f59a17"
320 +
SOURCE_FILENAME_103="hybrid-array-0.4.15.tar.gz"
321 +
SOURCE_URI_104="https://static.crates.io/crates/hyper/hyper-1.11.1.crate"
322 +
CHECKSUM_SHA256_104="27b501faa50e7a26c3d3560ca625132f4078a17771f4810baf70475ae48cbe43"
323 +
SOURCE_FILENAME_104="hyper-1.11.1.tar.gz"
324 +
SOURCE_URI_105="https://static.crates.io/crates/hyper-util/hyper-util-0.1.21.crate"
325 +
CHECKSUM_SHA256_105="ddc03d96684f9226b8a787cdb71488417b53ab5ea8fdb1dac946cb9431cc8bff"
326 +
SOURCE_FILENAME_105="hyper-util-0.1.21.tar.gz"
327 +
SOURCE_URI_106="https://static.crates.io/crates/iana-time-zone-haiku/iana-time-zone-haiku-0.1.2.crate"
328 +
CHECKSUM_SHA256_106="f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f"
329 +
SOURCE_FILENAME_106="iana-time-zone-haiku-0.1.2.tar.gz"
330 +
SOURCE_URI_107="https://static.crates.io/crates/icu_collections/icu_collections-2.3.0.crate"
331 +
CHECKSUM_SHA256_107="fa68d21081c4a05d5a901a1c62add574c77048b6a1c67be3b50ce0b60d4ca513"
332 +
SOURCE_FILENAME_107="icu_collections-2.3.0.tar.gz"
333 +
SOURCE_URI_108="https://static.crates.io/crates/icu_locale_core/icu_locale_core-2.3.0.crate"
334 +
CHECKSUM_SHA256_108="d56e28588da92eee5c3201a6eff33fabdd49b62269c8938d4ff050ce4d900deb"
335 +
SOURCE_FILENAME_108="icu_locale_core-2.3.0.tar.gz"
336 +
SOURCE_URI_109="https://static.crates.io/crates/icu_normalizer/icu_normalizer-2.3.0.crate"
337 +
CHECKSUM_SHA256_109="12f9cf5f235641ed274641dd81c3f28d870e276763d0797aeeab72317b1c646f"
338 +
SOURCE_FILENAME_109="icu_normalizer-2.3.0.tar.gz"
339 +
SOURCE_URI_110="https://static.crates.io/crates/icu_normalizer_data/icu_normalizer_data-2.3.0.crate"
340 +
CHECKSUM_SHA256_110="1563da1ed3e0b3bf3d74c9b85917ac9c56464d2f57242270c09c9e752f8021a0"
341 +
SOURCE_FILENAME_110="icu_normalizer_data-2.3.0.tar.gz"
342 +
SOURCE_URI_111="https://static.crates.io/crates/icu_properties/icu_properties-2.3.0.crate"
343 +
CHECKSUM_SHA256_111="7e7ca276ad3145661a65914e6daf131ca5120cd3dcee8f8f3214b8875184a148"
344 +
SOURCE_FILENAME_111="icu_properties-2.3.0.tar.gz"
345 +
SOURCE_URI_112="https://static.crates.io/crates/icu_properties_data/icu_properties_data-2.3.0.crate"
346 +
CHECKSUM_SHA256_112="e590f038c1464a96894fd6d10127e90a8be4509f56ff7ecef851b15cee0b7caa"
347 +
SOURCE_FILENAME_112="icu_properties_data-2.3.0.tar.gz"
348 +
SOURCE_URI_113="https://static.crates.io/crates/icu_provider/icu_provider-2.3.1.crate"
349 +
CHECKSUM_SHA256_113="d27bbb9d3abbefac45d55f647c9de1d44aafcd1186eb91879afef17c396c3e73"
350 +
SOURCE_FILENAME_113="icu_provider-2.3.1.tar.gz"
351 +
SOURCE_URI_114="https://static.crates.io/crates/idna/idna-1.1.0.crate"
352 +
CHECKSUM_SHA256_114="3b0875f23caa03898994f6ddc501886a45c7d3d62d04d2d90788d47be1b1e4de"
353 +
SOURCE_FILENAME_114="idna-1.1.0.tar.gz"
354 +
SOURCE_URI_115="https://static.crates.io/crates/idna_adapter/idna_adapter-1.2.2.crate"
355 +
CHECKSUM_SHA256_115="cb68373c0d6620ef8105e855e7745e18b0d00d3bdb07fb532e434244cdb9a714"
356 +
SOURCE_FILENAME_115="idna_adapter-1.2.2.tar.gz"
357 +
SOURCE_URI_116="https://static.crates.io/crates/ignore/ignore-0.4.33.crate"
358 +
CHECKSUM_SHA256_116="00b69833ed729dc5aa7d19541d96d6cf8e9137194207a04916d658e43168402f"
359 +
SOURCE_FILENAME_116="ignore-0.4.33.tar.gz"
360 +
SOURCE_URI_117="https://static.crates.io/crates/indexmap/indexmap-2.14.2.crate"
361 +
CHECKSUM_SHA256_117="cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855"
362 +
SOURCE_FILENAME_117="indexmap-2.14.2.tar.gz"
363 +
SOURCE_URI_118="https://static.crates.io/crates/indicatif/indicatif-0.17.11.crate"
364 +
CHECKSUM_SHA256_118="183b3088984b400f4cfac3620d5e076c84da5364016b4f49473de574b2586235"
365 +
SOURCE_FILENAME_118="indicatif-0.17.11.tar.gz"
366 +
SOURCE_URI_119="https://static.crates.io/crates/is_terminal_polyfill/is_terminal_polyfill-1.70.2.crate"
367 +
CHECKSUM_SHA256_119="a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695"
368 +
SOURCE_FILENAME_119="is_terminal_polyfill-1.70.2.tar.gz"
369 +
SOURCE_URI_120="https://static.crates.io/crates/itertools/itertools-0.15.0.crate"
370 +
CHECKSUM_SHA256_120="8b4baf93f58d4425749ca49a51c50ebab072c5df6994d08fed93541c331481dc"
371 +
SOURCE_FILENAME_120="itertools-0.15.0.tar.gz"
372 +
SOURCE_URI_121="https://static.crates.io/crates/itoa/itoa-1.0.18.crate"
373 +
CHECKSUM_SHA256_121="8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
374 +
SOURCE_FILENAME_121="itoa-1.0.18.tar.gz"
375 +
SOURCE_URI_122="https://static.crates.io/crates/jobserver/jobserver-0.1.35.crate"
376 +
CHECKSUM_SHA256_122="1c00acbd29eabad4a2392fa0e921c874934dbbf4194312ad20f04a0ed67a3cb3"
377 +
SOURCE_FILENAME_122="jobserver-0.1.35.tar.gz"
378 +
SOURCE_URI_123="https://static.crates.io/crates/js-sys/js-sys-0.3.106.crate"
379 +
CHECKSUM_SHA256_123="7883d941dae510fb2d978fc3fe018c71c9e2892fd38854de3e8b92c2e5ad9cc5"
380 +
SOURCE_FILENAME_123="js-sys-0.3.106.tar.gz"
381 +
SOURCE_URI_124="https://static.crates.io/crates/keccak/keccak-0.2.2.crate"
382 +
CHECKSUM_SHA256_124="d8f198d1db720e4940b5a493201d199d9f24f568f8f746bd13706243a2f71598"
383 +
SOURCE_FILENAME_124="keccak-0.2.2.tar.gz"
384 +
SOURCE_URI_125="https://static.crates.io/crates/lazy_static/lazy_static-1.5.0.crate"
385 +
CHECKSUM_SHA256_125="bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
386 +
SOURCE_FILENAME_125="lazy_static-1.5.0.tar.gz"
387 +
SOURCE_URI_126="https://static.crates.io/crates/libc/libc-0.2.189.crate"
388 +
CHECKSUM_SHA256_126="3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
389 +
SOURCE_FILENAME_126="libc-0.2.189.tar.gz"
390 +
SOURCE_URI_127="https://static.crates.io/crates/libmimalloc-sys/libmimalloc-sys-0.1.49.crate"
391 +
CHECKSUM_SHA256_127="6a45a52f43e1c16f667ccfe4dd8c85b7f7c204fd5e3bf46c5b0db9a5c3c0b8e9"
392 +
SOURCE_FILENAME_127="libmimalloc-sys-0.1.49.tar.gz"
393 +
SOURCE_URI_128="https://static.crates.io/crates/linux-raw-sys/linux-raw-sys-0.12.1.crate"
394 +
CHECKSUM_SHA256_128="32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
395 +
SOURCE_FILENAME_128="linux-raw-sys-0.12.1.tar.gz"
396 +
SOURCE_URI_129="https://static.crates.io/crates/litemap/litemap-0.8.3.crate"
397 +
CHECKSUM_SHA256_129="47d9d19d1d6efa0109d2f65ff4c85cddd50bd572e5a00127ab10987290bcefae"
398 +
SOURCE_FILENAME_129="litemap-0.8.3.tar.gz"
399 +
SOURCE_URI_130="https://static.crates.io/crates/lock_api/lock_api-0.4.14.crate"
400 +
CHECKSUM_SHA256_130="224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965"
401 +
SOURCE_FILENAME_130="lock_api-0.4.14.tar.gz"
402 +
SOURCE_URI_131="https://static.crates.io/crates/log/log-0.4.34.crate"
403 +
CHECKSUM_SHA256_131="f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
404 +
SOURCE_FILENAME_131="log-0.4.34.tar.gz"
405 +
SOURCE_URI_132="https://static.crates.io/crates/matchit/matchit-0.8.4.crate"
406 +
CHECKSUM_SHA256_132="47e1ffaa40ddd1f3ed91f717a33c8c0ee23fff369e3aa8772b9605cc1d22f4c3"
407 +
SOURCE_FILENAME_132="matchit-0.8.4.tar.gz"
408 +
SOURCE_URI_133="https://static.crates.io/crates/md-5/md-5-0.11.0.crate"
409 +
CHECKSUM_SHA256_133="69b6441f590336821bb897fb28fc622898ccceb1d6cea3fde5ea86b090c4de98"
410 +
SOURCE_FILENAME_133="md-5-0.11.0.tar.gz"
411 +
SOURCE_URI_134="https://static.crates.io/crates/memchr/memchr-2.8.3.crate"
412 +
CHECKSUM_SHA256_134="cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
413 +
SOURCE_FILENAME_134="memchr-2.8.3.tar.gz"
414 +
SOURCE_URI_135="https://static.crates.io/crates/mimalloc/mimalloc-0.1.52.crate"
415 +
CHECKSUM_SHA256_135="2d4139bb28d14ad1facf21d5eb8825051b326e172d216b39f6d31df53cc97862"
416 +
SOURCE_FILENAME_135="mimalloc-0.1.52.tar.gz"
417 +
SOURCE_URI_136="https://static.crates.io/crates/mime/mime-0.3.17.crate"
418 +
CHECKSUM_SHA256_136="6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a"
419 +
SOURCE_FILENAME_136="mime-0.3.17.tar.gz"
420 +
SOURCE_URI_137="https://static.crates.io/crates/minimal-lexical/minimal-lexical-0.2.1.crate"
421 +
CHECKSUM_SHA256_137="68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
422 +
SOURCE_FILENAME_137="minimal-lexical-0.2.1.tar.gz"
423 +
SOURCE_URI_138="https://static.crates.io/crates/miniz_oxide/miniz_oxide-0.9.1.crate"
424 +
CHECKSUM_SHA256_138="b63fbc4a50860e98e7b2aa7804ded1db5cbc3aff9193adaff57a6931bf7c4b4c"
425 +
SOURCE_FILENAME_138="miniz_oxide-0.9.1.tar.gz"
426 +
SOURCE_URI_139="https://static.crates.io/crates/mio/mio-1.2.3.crate"
427 +
CHECKSUM_SHA256_139="4b18443e9c262bfe8fa82f51666e2642c53393f7e5c27b3e1aeab922cff5b9d8"
428 +
SOURCE_FILENAME_139="mio-1.2.3.tar.gz"
429 +
SOURCE_URI_140="https://static.crates.io/crates/ml-dsa/ml-dsa-0.1.1.crate"
430 +
CHECKSUM_SHA256_140="add6b9d92e496f16f4526d68ff29da1483aba4b119baeab8bed3b9e3544a6f3d"
431 +
SOURCE_FILENAME_140="ml-dsa-0.1.1.tar.gz"
432 +
SOURCE_URI_141="https://static.crates.io/crates/module-lattice/module-lattice-0.2.3.crate"
433 +
CHECKSUM_SHA256_141="0c61b87c9683ab7cb1c6871d261ad5479b6b10ceb52c4352aaca3b5d35a8febe"
434 +
SOURCE_FILENAME_141="module-lattice-0.2.3.tar.gz"
435 +
SOURCE_URI_142="https://static.crates.io/crates/nom/nom-7.1.3.crate"
436 +
CHECKSUM_SHA256_142="d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
437 +
SOURCE_FILENAME_142="nom-7.1.3.tar.gz"
438 +
SOURCE_URI_143="https://static.crates.io/crates/num-bigint/num-bigint-0.4.8.crate"
439 +
CHECKSUM_SHA256_143="c89e69e7e0f03bea5ef08013795c25018e101932225a656383bd384495ecc367"
440 +
SOURCE_FILENAME_143="num-bigint-0.4.8.tar.gz"
441 +
SOURCE_URI_144="https://static.crates.io/crates/num-conv/num-conv-0.2.2.crate"
442 +
CHECKSUM_SHA256_144="521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441"
443 +
SOURCE_FILENAME_144="num-conv-0.2.2.tar.gz"
444 +
SOURCE_URI_145="https://static.crates.io/crates/num-integer/num-integer-0.1.47.crate"
445 +
CHECKSUM_SHA256_145="7ce2d95d4b3734dc35aa2f45e1aa22cd416814592a4f9d9205e11affd5b8e10b"
446 +
SOURCE_FILENAME_145="num-integer-0.1.47.tar.gz"
447 +
SOURCE_URI_146="https://static.crates.io/crates/num-traits/num-traits-0.2.19.crate"
448 +
CHECKSUM_SHA256_146="071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
449 +
SOURCE_FILENAME_146="num-traits-0.2.19.tar.gz"
450 +
SOURCE_URI_147="https://static.crates.io/crates/number_prefix/number_prefix-0.4.0.crate"
451 +
CHECKSUM_SHA256_147="830b246a0e5f20af87141b25c173cd1b609bd7779a4617d6ec582abaf90870f3"
452 +
SOURCE_FILENAME_147="number_prefix-0.4.0.tar.gz"
453 +
SOURCE_URI_148="https://static.crates.io/crates/object_store/object_store-0.14.2.crate"
454 +
CHECKSUM_SHA256_148="f1796bc93603f78c5760a69f2d58badc9618d22adade0a95385bb2adbae4eb94"
455 +
SOURCE_FILENAME_148="object_store-0.14.2.tar.gz"
456 +
SOURCE_URI_149="https://static.crates.io/crates/oid-registry/oid-registry-0.8.1.crate"
457 +
CHECKSUM_SHA256_149="12f40cff3dde1b6087cc5d5f5d4d65712f34016a03ed60e9c08dcc392736b5b7"
458 +
SOURCE_FILENAME_149="oid-registry-0.8.1.tar.gz"
459 +
SOURCE_URI_150="https://static.crates.io/crates/once_cell/once_cell-1.21.4.crate"
460 +
CHECKSUM_SHA256_150="9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
461 +
SOURCE_FILENAME_150="once_cell-1.21.4.tar.gz"
462 +
SOURCE_URI_151="https://static.crates.io/crates/once_cell_polyfill/once_cell_polyfill-1.70.2.crate"
463 +
CHECKSUM_SHA256_151="384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
464 +
SOURCE_FILENAME_151="once_cell_polyfill-1.70.2.tar.gz"
465 +
SOURCE_URI_152="https://static.crates.io/crates/parking_lot/parking_lot-0.12.5.crate"
466 +
CHECKSUM_SHA256_152="93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a"
467 +
SOURCE_FILENAME_152="parking_lot-0.12.5.tar.gz"
468 +
SOURCE_URI_153="https://static.crates.io/crates/parking_lot_core/parking_lot_core-0.9.12.crate"
469 +
CHECKSUM_SHA256_153="2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1"
470 +
SOURCE_FILENAME_153="parking_lot_core-0.9.12.tar.gz"
471 +
SOURCE_URI_154="https://static.crates.io/crates/pem/pem-4.0.0.crate"
472 +
CHECKSUM_SHA256_154="d354a98a3d1251555de99e8fdd8afda05573c31b82f59063a7b0a29b5527f120"
473 +
SOURCE_FILENAME_154="pem-4.0.0.tar.gz"
474 +
SOURCE_URI_155="https://static.crates.io/crates/percent-encoding/percent-encoding-2.3.2.crate"
475 +
CHECKSUM_SHA256_155="9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
476 +
SOURCE_FILENAME_155="percent-encoding-2.3.2.tar.gz"
477 +
SOURCE_URI_156="https://static.crates.io/crates/pin-project-lite/pin-project-lite-0.2.17.crate"
478 +
CHECKSUM_SHA256_156="a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
479 +
SOURCE_FILENAME_156="pin-project-lite-0.2.17.tar.gz"
480 +
SOURCE_URI_157="https://static.crates.io/crates/pkcs8/pkcs8-0.11.0.crate"
481 +
CHECKSUM_SHA256_157="451913da69c775a56034ea8d9003d27ee8948e12443eae7c038ba100a4f21cb7"
482 +
SOURCE_FILENAME_157="pkcs8-0.11.0.tar.gz"
483 +
SOURCE_URI_158="https://static.crates.io/crates/pkg-config/pkg-config-0.3.34.crate"
484 +
CHECKSUM_SHA256_158="f6b464fbc74e149a392436b17d523f769e057cb6877f6a5c4618bc6f11800548"
485 +
SOURCE_FILENAME_158="pkg-config-0.3.34.tar.gz"
486 +
SOURCE_URI_159="https://static.crates.io/crates/portable-atomic/portable-atomic-1.15.0.crate"
487 +
CHECKSUM_SHA256_159="05c8b63e8d9609db387f0324918f81d68fe27748f084ef092fb35954d0539a85"
488 +
SOURCE_FILENAME_159="portable-atomic-1.15.0.tar.gz"
489 +
SOURCE_URI_160="https://static.crates.io/crates/potential_utf/potential_utf-0.1.6.crate"
490 +
CHECKSUM_SHA256_160="d83eb9bc6d8e5cf568e7a1101d60ee05e81ed50ea106026f3d18deeb046d7661"
491 +
SOURCE_FILENAME_160="potential_utf-0.1.6.tar.gz"
492 +
SOURCE_URI_161="https://static.crates.io/crates/powerfmt/powerfmt-0.2.0.crate"
493 +
CHECKSUM_SHA256_161="439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391"
494 +
SOURCE_FILENAME_161="powerfmt-0.2.0.tar.gz"
495 +
SOURCE_URI_162="https://static.crates.io/crates/ppv-lite86/ppv-lite86-0.2.21.crate"
496 +
CHECKSUM_SHA256_162="85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9"
497 +
SOURCE_FILENAME_162="ppv-lite86-0.2.21.tar.gz"
498 +
SOURCE_URI_163="https://static.crates.io/crates/proc-macro2/proc-macro2-1.0.107.crate"
499 +
CHECKSUM_SHA256_163="985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
500 +
SOURCE_FILENAME_163="proc-macro2-1.0.107.tar.gz"
501 +
SOURCE_URI_164="https://static.crates.io/crates/proptest/proptest-1.11.0.crate"
502 +
CHECKSUM_SHA256_164="4b45fcc2344c680f5025fe57779faef368840d0bd1f42f216291f0dc4ace4744"
503 +
SOURCE_FILENAME_164="proptest-1.11.0.tar.gz"
504 +
SOURCE_URI_165="https://static.crates.io/crates/quick-error/quick-error-1.2.3.crate"
505 +
CHECKSUM_SHA256_165="a1d01941d82fa2ab50be1e79e6714289dd7cde78eba4c074bc5a4374f650dfe0"
506 +
SOURCE_FILENAME_165="quick-error-1.2.3.tar.gz"
507 +
SOURCE_URI_166="https://static.crates.io/crates/quick-xml/quick-xml-0.41.0.crate"
508 +
CHECKSUM_SHA256_166="e660451e55124f798a69a5af3f49ccfbefbd41910eefd25caf2393e1f3473ec1"
509 +
SOURCE_FILENAME_166="quick-xml-0.41.0.tar.gz"
510 +
SOURCE_URI_167="https://static.crates.io/crates/quote/quote-1.0.47.crate"
511 +
CHECKSUM_SHA256_167="1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
512 +
SOURCE_FILENAME_167="quote-1.0.47.tar.gz"
513 +
SOURCE_URI_168="https://static.crates.io/crates/r-efi/r-efi-5.3.0.crate"
514 +
CHECKSUM_SHA256_168="69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
515 +
SOURCE_FILENAME_168="r-efi-5.3.0.tar.gz"
516 +
SOURCE_URI_169="https://static.crates.io/crates/r-efi/r-efi-6.0.0.crate"
517 +
CHECKSUM_SHA256_169="f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
518 +
SOURCE_FILENAME_169="r-efi-6.0.0.tar.gz"
519 +
SOURCE_URI_170="https://static.crates.io/crates/rand/rand-0.10.3.crate"
520 +
CHECKSUM_SHA256_170="65c9fb96cbc91e3478eaae79a69fcd3f1ae4ad052e471fe6732fff548984b4af"
521 +
SOURCE_FILENAME_170="rand-0.10.3.tar.gz"
522 +
SOURCE_URI_171="https://static.crates.io/crates/rand/rand-0.8.8.crate"
523 +
CHECKSUM_SHA256_171="e058c7de0b26af77780c769414d6257830bb240f3c38477dbc2c16e5f54d6d4c"
524 +
SOURCE_FILENAME_171="rand-0.8.8.tar.gz"
525 +
SOURCE_URI_172="https://static.crates.io/crates/rand/rand-0.9.5.crate"
526 +
CHECKSUM_SHA256_172="b9ef1d0d795eb7d84685bca4f72f3649f064e6641543d3a8c415898726a57b41"
527 +
SOURCE_FILENAME_172="rand-0.9.5.tar.gz"
528 +
SOURCE_URI_173="https://static.crates.io/crates/rand_chacha/rand_chacha-0.3.1.crate"
529 +
CHECKSUM_SHA256_173="e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88"
530 +
SOURCE_FILENAME_173="rand_chacha-0.3.1.tar.gz"
531 +
SOURCE_URI_174="https://static.crates.io/crates/rand_chacha/rand_chacha-0.9.0.crate"
532 +
CHECKSUM_SHA256_174="d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb"
533 +
SOURCE_FILENAME_174="rand_chacha-0.9.0.tar.gz"
534 +
SOURCE_URI_175="https://static.crates.io/crates/rand_core/rand_core-0.10.1.crate"
535 +
CHECKSUM_SHA256_175="63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69"
536 +
SOURCE_FILENAME_175="rand_core-0.10.1.tar.gz"
537 +
SOURCE_URI_176="https://static.crates.io/crates/rand_core/rand_core-0.6.4.crate"
538 +
CHECKSUM_SHA256_176="ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c"
539 +
SOURCE_FILENAME_176="rand_core-0.6.4.tar.gz"
540 +
SOURCE_URI_177="https://static.crates.io/crates/rand_core/rand_core-0.9.5.crate"
541 +
CHECKSUM_SHA256_177="76afc826de14238e6e8c374ddcc1fa19e374fd8dd986b0d2af0d02377261d83c"
542 +
SOURCE_FILENAME_177="rand_core-0.9.5.tar.gz"
543 +
SOURCE_URI_178="https://static.crates.io/crates/rand_xorshift/rand_xorshift-0.4.0.crate"
544 +
CHECKSUM_SHA256_178="513962919efc330f829edb2535844d1b912b0fbe2ca165d613e4e8788bb05a5a"
545 +
SOURCE_FILENAME_178="rand_xorshift-0.4.0.tar.gz"
546 +
SOURCE_URI_179="https://static.crates.io/crates/rayon/rayon-1.12.0.crate"
547 +
CHECKSUM_SHA256_179="fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d"
548 +
SOURCE_FILENAME_179="rayon-1.12.0.tar.gz"
549 +
SOURCE_URI_180="https://static.crates.io/crates/rayon-core/rayon-core-1.13.0.crate"
550 +
CHECKSUM_SHA256_180="22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91"
551 +
SOURCE_FILENAME_180="rayon-core-1.13.0.tar.gz"
552 +
SOURCE_URI_181="https://static.crates.io/crates/rcgen/rcgen-0.14.10.crate"
553 +
CHECKSUM_SHA256_181="8774e05a7d0de114588e6a28fe7e71694b82614ed569d86d8b389dfbc98b8ad8"
554 +
SOURCE_FILENAME_181="rcgen-0.14.10.tar.gz"
555 +
SOURCE_URI_182="https://static.crates.io/crates/redox_syscall/redox_syscall-0.5.18.crate"
556 +
CHECKSUM_SHA256_182="ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
557 +
SOURCE_FILENAME_182="redox_syscall-0.5.18.tar.gz"
558 +
SOURCE_URI_183="https://static.crates.io/crates/regex-automata/regex-automata-0.4.18.crate"
559 +
CHECKSUM_SHA256_183="ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
560 +
SOURCE_FILENAME_183="regex-automata-0.4.18.tar.gz"
561 +
SOURCE_URI_184="https://static.crates.io/crates/regex-syntax/regex-syntax-0.8.11.crate"
562 +
CHECKSUM_SHA256_184="d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
563 +
SOURCE_FILENAME_184="regex-syntax-0.8.11.tar.gz"
564 +
SOURCE_URI_185="https://static.crates.io/crates/ring/ring-0.17.14.crate"
565 +
CHECKSUM_SHA256_185="a4689e6c2294d81e88dc6261c768b63bc4fcdb852be6d1352498b114f61383b7"
566 +
SOURCE_FILENAME_185="ring-0.17.14.tar.gz"
567 +
SOURCE_URI_186="https://static.crates.io/crates/roff/roff-1.1.1.crate"
568 +
CHECKSUM_SHA256_186="323c417e1d9665a65b263ec744ba09030cfb277e9daa0b018a4ab62e57bc8189"
569 +
SOURCE_FILENAME_186="roff-1.1.1.tar.gz"
570 +
SOURCE_URI_187="https://static.crates.io/crates/rustc_version/rustc_version-0.4.1.crate"
571 +
CHECKSUM_SHA256_187="cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92"
572 +
SOURCE_FILENAME_187="rustc_version-0.4.1.tar.gz"
573 +
SOURCE_URI_188="https://static.crates.io/crates/rusticata-macros/rusticata-macros-4.1.0.crate"
574 +
CHECKSUM_SHA256_188="faf0c4a6ece9950b9abdb62b1cfcf2a68b3b67a10ba445b3bb85be2a293d0632"
575 +
SOURCE_FILENAME_188="rusticata-macros-4.1.0.tar.gz"
576 +
SOURCE_URI_189="https://static.crates.io/crates/rustix/rustix-1.1.5.crate"
577 +
CHECKSUM_SHA256_189="891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
578 +
SOURCE_FILENAME_189="rustix-1.1.5.tar.gz"
579 +
SOURCE_URI_190="https://static.crates.io/crates/rustls/rustls-0.23.45.crate"
580 +
CHECKSUM_SHA256_190="0d41d731c7d2f962d1ccc364cec258de3c0e93b38c2fb3ba97ac74513048d634"
581 +
SOURCE_FILENAME_190="rustls-0.23.45.tar.gz"
582 +
SOURCE_URI_191="https://static.crates.io/crates/rustls-pemfile/rustls-pemfile-2.2.0.crate"
583 +
CHECKSUM_SHA256_191="dce314e5fee3f39953d46bb63bb8a46d40c2f8fb7cc5a3b6cab2bde9721d6e50"
584 +
SOURCE_FILENAME_191="rustls-pemfile-2.2.0.tar.gz"
585 +
SOURCE_URI_192="https://static.crates.io/crates/rustls-pki-types/rustls-pki-types-1.15.1.crate"
586 +
CHECKSUM_SHA256_192="2f4925028c7eb5d1fcdaf196971378ed9d2c1c4efc7dc5d011256f76c99c0a96"
587 +
SOURCE_FILENAME_192="rustls-pki-types-1.15.1.tar.gz"
588 +
SOURCE_URI_193="https://static.crates.io/crates/rustls-webpki/rustls-webpki-0.103.15.crate"
589 +
CHECKSUM_SHA256_193="f3c3cf1d8b1e7d4927e2d154c3fcb02979afb9939629c62cd9048d4f07b60ac2"
590 +
SOURCE_FILENAME_193="rustls-webpki-0.103.15.tar.gz"
591 +
SOURCE_URI_194="https://static.crates.io/crates/rustversion/rustversion-1.0.23.crate"
592 +
CHECKSUM_SHA256_194="cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f"
593 +
SOURCE_FILENAME_194="rustversion-1.0.23.tar.gz"
594 +
SOURCE_URI_195="https://static.crates.io/crates/rusty-fork/rusty-fork-0.3.1.crate"
595 +
CHECKSUM_SHA256_195="cc6bf79ff24e648f6da1f8d1f011e9cac26491b619e6b9280f2b47f1774e6ee2"
596 +
SOURCE_FILENAME_195="rusty-fork-0.3.1.tar.gz"
597 +
SOURCE_URI_196="https://static.crates.io/crates/ryu/ryu-1.0.23.crate"
598 +
CHECKSUM_SHA256_196="9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f"
599 +
SOURCE_FILENAME_196="ryu-1.0.23.tar.gz"
600 +
SOURCE_URI_197="https://static.crates.io/crates/same-file/same-file-1.0.6.crate"
601 +
CHECKSUM_SHA256_197="93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502"
602 +
SOURCE_FILENAME_197="same-file-1.0.6.tar.gz"
603 +
SOURCE_URI_198="https://static.crates.io/crates/scopeguard/scopeguard-1.2.0.crate"
604 +
CHECKSUM_SHA256_198="94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
605 +
SOURCE_FILENAME_198="scopeguard-1.2.0.tar.gz"
606 +
SOURCE_URI_199="https://static.crates.io/crates/semver/semver-1.0.28.crate"
607 +
CHECKSUM_SHA256_199="8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd"
608 +
SOURCE_FILENAME_199="semver-1.0.28.tar.gz"
609 +
SOURCE_URI_200="https://static.crates.io/crates/serde/serde-1.0.229.crate"
610 +
CHECKSUM_SHA256_200="4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
611 +
SOURCE_FILENAME_200="serde-1.0.229.tar.gz"
612 +
SOURCE_URI_201="https://static.crates.io/crates/serde_core/serde_core-1.0.229.crate"
613 +
CHECKSUM_SHA256_201="67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
614 +
SOURCE_FILENAME_201="serde_core-1.0.229.tar.gz"
615 +
SOURCE_URI_202="https://static.crates.io/crates/serde_derive/serde_derive-1.0.229.crate"
616 +
CHECKSUM_SHA256_202="e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
617 +
SOURCE_FILENAME_202="serde_derive-1.0.229.tar.gz"
618 +
SOURCE_URI_203="https://static.crates.io/crates/serde_json/serde_json-1.0.151.crate"
619 +
CHECKSUM_SHA256_203="c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
620 +
SOURCE_FILENAME_203="serde_json-1.0.151.tar.gz"
621 +
SOURCE_URI_204="https://static.crates.io/crates/serde_path_to_error/serde_path_to_error-0.1.20.crate"
622 +
CHECKSUM_SHA256_204="10a9ff822e371bb5403e391ecd83e182e0e77ba7f6fe0160b795797109d1b457"
623 +
SOURCE_FILENAME_204="serde_path_to_error-0.1.20.tar.gz"
624 +
SOURCE_URI_205="https://static.crates.io/crates/serde_urlencoded/serde_urlencoded-0.7.1.crate"
625 +
CHECKSUM_SHA256_205="d3491c14715ca2294c4d6a88f15e84739788c1d030eed8c110436aafdaa2f3fd"
626 +
SOURCE_FILENAME_205="serde_urlencoded-0.7.1.tar.gz"
627 +
SOURCE_URI_206="https://static.crates.io/crates/sha2/sha2-0.11.0.crate"
628 +
CHECKSUM_SHA256_206="446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
629 +
SOURCE_FILENAME_206="sha2-0.11.0.tar.gz"
630 +
SOURCE_URI_207="https://static.crates.io/crates/shake/shake-0.1.0.crate"
631 +
CHECKSUM_SHA256_207="09057cb2149ad4cbd2da1e26b351f9a4c354219421229c69c3063e6f61947c4a"
632 +
SOURCE_FILENAME_207="shake-0.1.0.tar.gz"
633 +
SOURCE_URI_208="https://static.crates.io/crates/shlex/shlex-2.0.1.crate"
634 +
CHECKSUM_SHA256_208="f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
635 +
SOURCE_FILENAME_208="shlex-2.0.1.tar.gz"
636 +
SOURCE_URI_209="https://static.crates.io/crates/signature/signature-3.0.0.crate"
637 +
CHECKSUM_SHA256_209="28d567dcbaf0049cb8ac2608a76cd95ff9e4412e1899d389ee400918ca7537f5"
638 +
SOURCE_FILENAME_209="signature-3.0.0.tar.gz"
639 +
SOURCE_URI_210="https://static.crates.io/crates/simd-adler32/simd-adler32-0.3.10.crate"
640 +
CHECKSUM_SHA256_210="3a219298ac11a56ea9a6d2120044824d6f01aeb034955e7af7bc16858527deea"
641 +
SOURCE_FILENAME_210="simd-adler32-0.3.10.tar.gz"
642 +
SOURCE_URI_211="https://static.crates.io/crates/similar/similar-2.7.0.crate"
643 +
CHECKSUM_SHA256_211="bbbb5d9659141646ae647b42fe094daf6c6192d1620870b449d9557f748b2daa"
644 +
SOURCE_FILENAME_211="similar-2.7.0.tar.gz"
645 +
SOURCE_URI_212="https://static.crates.io/crates/slab/slab-0.4.12.crate"
646 +
CHECKSUM_SHA256_212="0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
647 +
SOURCE_FILENAME_212="slab-0.4.12.tar.gz"
648 +
SOURCE_URI_213="https://static.crates.io/crates/smallvec/smallvec-1.16.2.crate"
649 +
CHECKSUM_SHA256_213="f9395f0f0eee849a9b707b2f06bb92a6a422090e2123bb2ef8e87a0e61892a8e"
650 +
SOURCE_FILENAME_213="smallvec-1.16.2.tar.gz"
651 +
SOURCE_URI_214="https://static.crates.io/crates/socket2/socket2-0.6.5.crate"
652 +
CHECKSUM_SHA256_214="c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4"
653 +
SOURCE_FILENAME_214="socket2-0.6.5.tar.gz"
654 +
SOURCE_URI_215="https://static.crates.io/crates/spin/spin-0.10.1.crate"
655 +
CHECKSUM_SHA256_215="023a211cb3138dbc438680b32560ad89f699977624c9f8dbb95a47d5b4c07dd3"
656 +
SOURCE_FILENAME_215="spin-0.10.1.tar.gz"
657 +
SOURCE_URI_216="https://static.crates.io/crates/spki/spki-0.8.0.crate"
658 +
CHECKSUM_SHA256_216="1d9efca8738c78ee9484207732f728b1ef517bbb1833d6fc0879ca898a522f6f"
659 +
SOURCE_FILENAME_216="spki-0.8.0.tar.gz"
660 +
SOURCE_URI_217="https://static.crates.io/crates/sponge-cursor/sponge-cursor-0.1.0.crate"
661 +
CHECKSUM_SHA256_217="3a0219bd7d979d58245a4f41f695e1ac9f8befdffadd7f61f1bae9e39abc6620"
662 +
SOURCE_FILENAME_217="sponge-cursor-0.1.0.tar.gz"
663 +
SOURCE_URI_218="https://static.crates.io/crates/stable_deref_trait/stable_deref_trait-1.2.1.crate"
664 +
CHECKSUM_SHA256_218="6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
665 +
SOURCE_FILENAME_218="stable_deref_trait-1.2.1.tar.gz"
666 +
SOURCE_URI_219="https://static.crates.io/crates/strsim/strsim-0.11.1.crate"
667 +
CHECKSUM_SHA256_219="7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
668 +
SOURCE_FILENAME_219="strsim-0.11.1.tar.gz"
669 +
SOURCE_URI_220="https://static.crates.io/crates/subtle/subtle-2.6.1.crate"
670 +
CHECKSUM_SHA256_220="13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
671 +
SOURCE_FILENAME_220="subtle-2.6.1.tar.gz"
672 +
SOURCE_URI_221="https://static.crates.io/crates/syn/syn-2.0.119.crate"
673 +
CHECKSUM_SHA256_221="872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
674 +
SOURCE_FILENAME_221="syn-2.0.119.tar.gz"
675 +
SOURCE_URI_222="https://static.crates.io/crates/syn/syn-3.0.6.crate"
676 +
CHECKSUM_SHA256_222="8593e8e72159ed2257d083c7a454a85cbf854f37a0966d8d483aff8c8a3ebcee"
677 +
SOURCE_FILENAME_222="syn-3.0.6.tar.gz"
678 +
SOURCE_URI_223="https://static.crates.io/crates/sync_wrapper/sync_wrapper-1.0.2.crate"
679 +
CHECKSUM_SHA256_223="0bf256ce5efdfa370213c1dabab5935a12e49f2c58d15e9eac2870d3b4f27263"
680 +
SOURCE_FILENAME_223="sync_wrapper-1.0.2.tar.gz"
681 +
SOURCE_URI_224="https://static.crates.io/crates/synstructure/synstructure-0.13.2.crate"
682 +
CHECKSUM_SHA256_224="728a70f3dbaf5bab7f0c4b1ac8d7ae5ea60a4b5549c8a5914361c99147a709d2"
683 +
SOURCE_FILENAME_224="synstructure-0.13.2.tar.gz"
684 +
SOURCE_URI_225="https://static.crates.io/crates/synstructure/synstructure-0.14.0.crate"
685 +
CHECKSUM_SHA256_225="901704edd0dfe137f1987838ee4f259e4e063c31371bdb423f7ae38ec6f77f02"
686 +
SOURCE_FILENAME_225="synstructure-0.14.0.tar.gz"
687 +
SOURCE_URI_226="https://static.crates.io/crates/tempfile/tempfile-3.27.0.crate"
688 +
CHECKSUM_SHA256_226="32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
689 +
SOURCE_FILENAME_226="tempfile-3.27.0.tar.gz"
690 +
SOURCE_URI_227="https://static.crates.io/crates/thiserror/thiserror-1.0.69.crate"
691 +
CHECKSUM_SHA256_227="b6aaf5339b578ea85b50e080feb250a3e8ae8cfcdff9a461c9ec2904bc923f52"
692 +
SOURCE_FILENAME_227="thiserror-1.0.69.tar.gz"
693 +
SOURCE_URI_228="https://static.crates.io/crates/thiserror/thiserror-2.0.21.crate"
694 +
CHECKSUM_SHA256_228="09e52cb86a36cede5cb101bf8908837b3e4c6e5e59fe7fd85c23fb56200d189e"
695 +
SOURCE_FILENAME_228="thiserror-2.0.21.tar.gz"
696 +
SOURCE_URI_229="https://static.crates.io/crates/thiserror-impl/thiserror-impl-1.0.69.crate"
697 +
CHECKSUM_SHA256_229="4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1"
698 +
SOURCE_FILENAME_229="thiserror-impl-1.0.69.tar.gz"
699 +
SOURCE_URI_230="https://static.crates.io/crates/thiserror-impl/thiserror-impl-2.0.21.crate"
700 +
CHECKSUM_SHA256_230="fe5197923287db20a58125f0bc85c062f7f2c892de97b18c356f9efb14b28524"
701 +
SOURCE_FILENAME_230="thiserror-impl-2.0.21.tar.gz"
702 +
SOURCE_URI_231="https://static.crates.io/crates/time/time-0.3.55.crate"
703 +
CHECKSUM_SHA256_231="cdb87b95ec50ddfa440816d227a17b2ccbdda963a316a727fda0fc4334f7d134"
704 +
SOURCE_FILENAME_231="time-0.3.55.tar.gz"
705 +
SOURCE_URI_232="https://static.crates.io/crates/time-core/time-core-0.1.9.crate"
706 +
CHECKSUM_SHA256_232="9e1c906769ad99c88eaa54e728060edef082f8e358ff32030cb7c7d315e81109"
707 +
SOURCE_FILENAME_232="time-core-0.1.9.tar.gz"
708 +
SOURCE_URI_233="https://static.crates.io/crates/time-macros/time-macros-0.2.32.crate"
709 +
CHECKSUM_SHA256_233="7e689342a48d2ea927c87ea50cabf8594854bf940e9310208848d680d668ed85"
710 +
SOURCE_FILENAME_233="time-macros-0.2.32.tar.gz"
711 +
SOURCE_URI_234="https://static.crates.io/crates/tinystr/tinystr-0.8.4.crate"
712 +
CHECKSUM_SHA256_234="b1e27c91459209c2986af3dcf603a5a74a4368754ce37414f59acc971167f643"
713 +
SOURCE_FILENAME_234="tinystr-0.8.4.tar.gz"
714 +
SOURCE_URI_235="https://static.crates.io/crates/tokio/tokio-1.53.1.crate"
715 +
CHECKSUM_SHA256_235="202caea871b69668250d242070849eb495be178ed697a3e98aebce5bc81a0bed"
716 +
SOURCE_FILENAME_235="tokio-1.53.1.tar.gz"
717 +
SOURCE_URI_236="https://static.crates.io/crates/tokio-macros/tokio-macros-2.7.2.crate"
718 +
CHECKSUM_SHA256_236="78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e"
719 +
SOURCE_FILENAME_236="tokio-macros-2.7.2.tar.gz"
720 +
SOURCE_URI_237="https://static.crates.io/crates/tokio-rustls/tokio-rustls-0.26.5.crate"
721 +
CHECKSUM_SHA256_237="b0c85f2c3ef0b1cd58b36682f4b17aaa995f0e5db534d85692b4903abce21f67"
722 +
SOURCE_FILENAME_237="tokio-rustls-0.26.5.tar.gz"
723 +
SOURCE_URI_238="https://static.crates.io/crates/tokio-util/tokio-util-0.7.19.crate"
724 +
CHECKSUM_SHA256_238="494815d09bf52b5548659851081238f0ca39ff638363907596da739561c62c52"
725 +
SOURCE_FILENAME_238="tokio-util-0.7.19.tar.gz"
726 +
SOURCE_URI_239="https://static.crates.io/crates/tower/tower-0.5.3.crate"
727 +
CHECKSUM_SHA256_239="ebe5ef63511595f1344e2d5cfa636d973292adc0eec1f0ad45fae9f0851ab1d4"
728 +
SOURCE_FILENAME_239="tower-0.5.3.tar.gz"
729 +
SOURCE_URI_240="https://static.crates.io/crates/tower-layer/tower-layer-0.3.3.crate"
730 +
CHECKSUM_SHA256_240="121c2a6cda46980bb0fcd1647ffaf6cd3fc79a013de288782836f6df9c48780e"
731 +
SOURCE_FILENAME_240="tower-layer-0.3.3.tar.gz"
732 +
SOURCE_URI_241="https://static.crates.io/crates/tower-service/tower-service-0.3.3.crate"
733 +
CHECKSUM_SHA256_241="8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
734 +
SOURCE_FILENAME_241="tower-service-0.3.3.tar.gz"
735 +
SOURCE_URI_242="https://static.crates.io/crates/tracing/tracing-0.1.44.crate"
736 +
CHECKSUM_SHA256_242="63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
737 +
SOURCE_FILENAME_242="tracing-0.1.44.tar.gz"
738 +
SOURCE_URI_243="https://static.crates.io/crates/tracing-attributes/tracing-attributes-0.1.31.crate"
739 +
CHECKSUM_SHA256_243="7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da"
740 +
SOURCE_FILENAME_243="tracing-attributes-0.1.31.tar.gz"
741 +
SOURCE_URI_244="https://static.crates.io/crates/tracing-core/tracing-core-0.1.36.crate"
742 +
CHECKSUM_SHA256_244="db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
743 +
SOURCE_FILENAME_244="tracing-core-0.1.36.tar.gz"
744 +
SOURCE_URI_245="https://static.crates.io/crates/typenum/typenum-1.20.1.crate"
745 +
CHECKSUM_SHA256_245="b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20"
746 +
SOURCE_FILENAME_245="typenum-1.20.1.tar.gz"
747 +
SOURCE_URI_246="https://static.crates.io/crates/unarray/unarray-0.1.4.crate"
748 +
CHECKSUM_SHA256_246="eaea85b334db583fe3274d12b4cd1880032beab409c0d774be044d4480ab9a94"
749 +
SOURCE_FILENAME_246="unarray-0.1.4.tar.gz"
750 +
SOURCE_URI_247="https://static.crates.io/crates/unicode-ident/unicode-ident-1.0.26.crate"
751 +
CHECKSUM_SHA256_247="d245f478577f809a851594d02313b640fb437e0bb33866753cff937863096954"
752 +
SOURCE_FILENAME_247="unicode-ident-1.0.26.tar.gz"
753 +
SOURCE_URI_248="https://static.crates.io/crates/unicode-width/unicode-width-0.2.2.crate"
754 +
CHECKSUM_SHA256_248="b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254"
755 +
SOURCE_FILENAME_248="unicode-width-0.2.2.tar.gz"
756 +
SOURCE_URI_249="https://static.crates.io/crates/untrusted/untrusted-0.9.0.crate"
757 +
CHECKSUM_SHA256_249="8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1"
758 +
SOURCE_FILENAME_249="untrusted-0.9.0.tar.gz"
759 +
SOURCE_URI_250="https://static.crates.io/crates/ureq/ureq-2.12.1.crate"
760 +
CHECKSUM_SHA256_250="02d1a66277ed75f640d608235660df48c8e3c19f3b4edb6a263315626cc3c01d"
761 +
SOURCE_FILENAME_250="ureq-2.12.1.tar.gz"
762 +
SOURCE_URI_251="https://static.crates.io/crates/url/url-2.5.8.crate"
763 +
CHECKSUM_SHA256_251="ff67a8a4397373c3ef660812acab3268222035010ab8680ec4215f38ba3d0eed"
764 +
SOURCE_FILENAME_251="url-2.5.8.tar.gz"
765 +
SOURCE_URI_252="https://static.crates.io/crates/utf8_iter/utf8_iter-1.0.4.crate"
766 +
CHECKSUM_SHA256_252="b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"
767 +
SOURCE_FILENAME_252="utf8_iter-1.0.4.tar.gz"
768 +
SOURCE_URI_253="https://static.crates.io/crates/utf8parse/utf8parse-0.2.2.crate"
769 +
CHECKSUM_SHA256_253="06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
770 +
SOURCE_FILENAME_253="utf8parse-0.2.2.tar.gz"
771 +
SOURCE_URI_254="https://static.crates.io/crates/version_check/version_check-0.9.5.crate"
772 +
CHECKSUM_SHA256_254="0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
773 +
SOURCE_FILENAME_254="version_check-0.9.5.tar.gz"
774 +
SOURCE_URI_255="https://static.crates.io/crates/wait-timeout/wait-timeout-0.2.1.crate"
775 +
CHECKSUM_SHA256_255="09ac3b126d3914f9849036f826e054cbabdc8519970b8998ddaf3b5bd3c65f11"
776 +
SOURCE_FILENAME_255="wait-timeout-0.2.1.tar.gz"
777 +
SOURCE_URI_256="https://static.crates.io/crates/walkdir/walkdir-2.5.0.crate"
778 +
CHECKSUM_SHA256_256="29790946404f91d9c5d06f9874efddea1dc06c5efe94541a7d6863108e3a5e4b"
779 +
SOURCE_FILENAME_256="walkdir-2.5.0.tar.gz"
780 +
SOURCE_URI_257="https://static.crates.io/crates/wasi/wasi-0.11.1+wasi-snapshot-preview1.crate"
781 +
CHECKSUM_SHA256_257="ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
782 +
SOURCE_FILENAME_257="wasi-0.11.1+wasi-snapshot-preview1.tar.gz"
783 +
SOURCE_URI_258="https://static.crates.io/crates/wasip2/wasip2-1.0.4+wasi-0.2.12.crate"
784 +
CHECKSUM_SHA256_258="b67efb37e106e55ce722a510d6b5f9c17f083e5fc79afc2badeb12cc313d9487"
785 +
SOURCE_FILENAME_258="wasip2-1.0.4+wasi-0.2.12.tar.gz"
786 +
SOURCE_URI_259="https://static.crates.io/crates/wasm-bindgen/wasm-bindgen-0.2.129.crate"
787 +
CHECKSUM_SHA256_259="9bb54f33acc68fd454578d9820b0bde1a1a3d17aa17bb7b6595806d02886d409"
788 +
SOURCE_FILENAME_259="wasm-bindgen-0.2.129.tar.gz"
789 +
SOURCE_URI_260="https://static.crates.io/crates/wasm-bindgen-futures/wasm-bindgen-futures-0.4.79.crate"
790 +
CHECKSUM_SHA256_260="3cbab34de2d982e9b48e18d216d04c4a6f641066ff19ffb699980f591ee3610e"
791 +
SOURCE_FILENAME_260="wasm-bindgen-futures-0.4.79.tar.gz"
792 +
SOURCE_URI_261="https://static.crates.io/crates/wasm-bindgen-macro/wasm-bindgen-macro-0.2.129.crate"
793 +
CHECKSUM_SHA256_261="2e29d0c35b16e224a7eeb5cd2d25e3e1968fbd65604117b44d3b789d00ee8535"
794 +
SOURCE_FILENAME_261="wasm-bindgen-macro-0.2.129.tar.gz"
795 +
SOURCE_URI_262="https://static.crates.io/crates/wasm-bindgen-macro-support/wasm-bindgen-macro-support-0.2.129.crate"
796 +
CHECKSUM_SHA256_262="6f501a8bc3719dba86ef8ae4728879c08001bea749eb1333ac5b91e040e2a6b7"
797 +
SOURCE_FILENAME_262="wasm-bindgen-macro-support-0.2.129.tar.gz"
798 +
SOURCE_URI_263="https://static.crates.io/crates/wasm-bindgen-shared/wasm-bindgen-shared-0.2.129.crate"
799 +
CHECKSUM_SHA256_263="23f0c9c52aa7cd7d77769a4cfe2a9adb1b331f489a41d912ce14513d5ab995c6"
800 +
SOURCE_FILENAME_263="wasm-bindgen-shared-0.2.129.tar.gz"
801 +
SOURCE_URI_264="https://static.crates.io/crates/web-time/web-time-1.1.0.crate"
802 +
CHECKSUM_SHA256_264="5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb"
803 +
SOURCE_FILENAME_264="web-time-1.1.0.tar.gz"
804 +
SOURCE_URI_265="https://static.crates.io/crates/webpki-roots/webpki-roots-0.26.11.crate"
805 +
CHECKSUM_SHA256_265="521bc38abb08001b01866da9f51eb7c5d647a19260e00054a8c7fd5f9e57f7a9"
806 +
SOURCE_FILENAME_265="webpki-roots-0.26.11.tar.gz"
807 +
SOURCE_URI_266="https://static.crates.io/crates/webpki-roots/webpki-roots-1.0.9.crate"
808 +
CHECKSUM_SHA256_266="7dcd9d09a39985f5344844e66b0c530a33843579125f23e21e9f0f220850f22a"
809 +
SOURCE_FILENAME_266="webpki-roots-1.0.9.tar.gz"
810 +
SOURCE_URI_267="https://static.crates.io/crates/winapi-util/winapi-util-0.1.11.crate"
811 +
CHECKSUM_SHA256_267="c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
812 +
SOURCE_FILENAME_267="winapi-util-0.1.11.tar.gz"
813 +
SOURCE_URI_268="https://static.crates.io/crates/windows-core/windows-core-0.62.2.crate"
814 +
CHECKSUM_SHA256_268="b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb"
815 +
SOURCE_FILENAME_268="windows-core-0.62.2.tar.gz"
816 +
SOURCE_URI_269="https://static.crates.io/crates/windows-implement/windows-implement-0.60.2.crate"
817 +
CHECKSUM_SHA256_269="053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
818 +
SOURCE_FILENAME_269="windows-implement-0.60.2.tar.gz"
819 +
SOURCE_URI_270="https://static.crates.io/crates/windows-interface/windows-interface-0.59.3.crate"
820 +
CHECKSUM_SHA256_270="3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
821 +
SOURCE_FILENAME_270="windows-interface-0.59.3.tar.gz"
822 +
SOURCE_URI_271="https://static.crates.io/crates/windows-link/windows-link-0.2.1.crate"
823 +
CHECKSUM_SHA256_271="f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
824 +
SOURCE_FILENAME_271="windows-link-0.2.1.tar.gz"
825 +
SOURCE_URI_272="https://static.crates.io/crates/windows-result/windows-result-0.4.1.crate"
826 +
CHECKSUM_SHA256_272="7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
827 +
SOURCE_FILENAME_272="windows-result-0.4.1.tar.gz"
828 +
SOURCE_URI_273="https://static.crates.io/crates/windows-strings/windows-strings-0.5.1.crate"
829 +
CHECKSUM_SHA256_273="7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091"
830 +
SOURCE_FILENAME_273="windows-strings-0.5.1.tar.gz"
831 +
SOURCE_URI_274="https://static.crates.io/crates/windows-sys/windows-sys-0.52.0.crate"
832 +
CHECKSUM_SHA256_274="282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d"
833 +
SOURCE_FILENAME_274="windows-sys-0.52.0.tar.gz"
834 +
SOURCE_URI_275="https://static.crates.io/crates/windows-sys/windows-sys-0.59.0.crate"
835 +
CHECKSUM_SHA256_275="1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b"
836 +
SOURCE_FILENAME_275="windows-sys-0.59.0.tar.gz"
837 +
SOURCE_URI_276="https://static.crates.io/crates/windows-sys/windows-sys-0.61.2.crate"
838 +
CHECKSUM_SHA256_276="ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
839 +
SOURCE_FILENAME_276="windows-sys-0.61.2.tar.gz"
840 +
SOURCE_URI_277="https://static.crates.io/crates/windows-targets/windows-targets-0.52.6.crate"
841 +
CHECKSUM_SHA256_277="9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973"
842 +
SOURCE_FILENAME_277="windows-targets-0.52.6.tar.gz"
843 +
SOURCE_URI_278="https://static.crates.io/crates/windows_aarch64_gnullvm/windows_aarch64_gnullvm-0.52.6.crate"
844 +
CHECKSUM_SHA256_278="32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3"
845 +
SOURCE_FILENAME_278="windows_aarch64_gnullvm-0.52.6.tar.gz"
846 +
SOURCE_URI_279="https://static.crates.io/crates/windows_aarch64_msvc/windows_aarch64_msvc-0.52.6.crate"
847 +
CHECKSUM_SHA256_279="09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469"
848 +
SOURCE_FILENAME_279="windows_aarch64_msvc-0.52.6.tar.gz"
849 +
SOURCE_URI_280="https://static.crates.io/crates/windows_i686_gnu/windows_i686_gnu-0.52.6.crate"
850 +
CHECKSUM_SHA256_280="8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b"
851 +
SOURCE_FILENAME_280="windows_i686_gnu-0.52.6.tar.gz"
852 +
SOURCE_URI_281="https://static.crates.io/crates/windows_i686_gnullvm/windows_i686_gnullvm-0.52.6.crate"
853 +
CHECKSUM_SHA256_281="0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66"
854 +
SOURCE_FILENAME_281="windows_i686_gnullvm-0.52.6.tar.gz"
855 +
SOURCE_URI_282="https://static.crates.io/crates/windows_i686_msvc/windows_i686_msvc-0.52.6.crate"
856 +
CHECKSUM_SHA256_282="240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66"
857 +
SOURCE_FILENAME_282="windows_i686_msvc-0.52.6.tar.gz"
858 +
SOURCE_URI_283="https://static.crates.io/crates/windows_x86_64_gnu/windows_x86_64_gnu-0.52.6.crate"
859 +
CHECKSUM_SHA256_283="147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78"
860 +
SOURCE_FILENAME_283="windows_x86_64_gnu-0.52.6.tar.gz"
861 +
SOURCE_URI_284="https://static.crates.io/crates/windows_x86_64_gnullvm/windows_x86_64_gnullvm-0.52.6.crate"
862 +
CHECKSUM_SHA256_284="24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d"
863 +
SOURCE_FILENAME_284="windows_x86_64_gnullvm-0.52.6.tar.gz"
864 +
SOURCE_URI_285="https://static.crates.io/crates/windows_x86_64_msvc/windows_x86_64_msvc-0.52.6.crate"
865 +
CHECKSUM_SHA256_285="589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
866 +
SOURCE_FILENAME_285="windows_x86_64_msvc-0.52.6.tar.gz"
867 +
SOURCE_URI_286="https://static.crates.io/crates/wit-bindgen/wit-bindgen-0.57.1.crate"
868 +
CHECKSUM_SHA256_286="1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e"
869 +
SOURCE_FILENAME_286="wit-bindgen-0.57.1.tar.gz"
870 +
SOURCE_URI_287="https://static.crates.io/crates/writeable/writeable-0.6.4.crate"
871 +
CHECKSUM_SHA256_287="3ad82d2a33cdc9674dc7465672f271e096168fcdbe0f799d9e6db8c5892679dc"
872 +
SOURCE_FILENAME_287="writeable-0.6.4.tar.gz"
873 +
SOURCE_URI_288="https://static.crates.io/crates/x509-parser/x509-parser-0.18.1.crate"
874 +
CHECKSUM_SHA256_288="d43b0f71ce057da06bc0851b23ee24f3f86190b07203dd8f567d0b706a185202"
875 +
SOURCE_FILENAME_288="x509-parser-0.18.1.tar.gz"
876 +
SOURCE_URI_289="https://static.crates.io/crates/yasna/yasna-0.6.0.crate"
877 +
CHECKSUM_SHA256_289="b5f6765e852b9b4dc8e2a76843e4d64d1cea8e79bcde0b6901aea8e7c7f08282"
878 +
SOURCE_FILENAME_289="yasna-0.6.0.tar.gz"
879 +
SOURCE_URI_290="https://static.crates.io/crates/yoke/yoke-0.8.3.crate"
880 +
CHECKSUM_SHA256_290="709fe23a0424b6a435d82152b1bd3fdfb0833487d5fa90d05d42762a9891fef5"
881 +
SOURCE_FILENAME_290="yoke-0.8.3.tar.gz"
882 +
SOURCE_URI_291="https://static.crates.io/crates/yoke-derive/yoke-derive-0.8.3.crate"
883 +
CHECKSUM_SHA256_291="33811428bee40dbceb6d545e95754741d17a6aef9a4849f0fd62e2ba4f412a78"
884 +
SOURCE_FILENAME_291="yoke-derive-0.8.3.tar.gz"
885 +
SOURCE_URI_292="https://static.crates.io/crates/zerocopy/zerocopy-0.8.59.crate"
886 +
CHECKSUM_SHA256_292="6df92bf3d9227be3d53173901ddbffac2babc27ae50f397776ffd6dc33f800cb"
887 +
SOURCE_FILENAME_292="zerocopy-0.8.59.tar.gz"
888 +
SOURCE_URI_293="https://static.crates.io/crates/zerocopy-derive/zerocopy-derive-0.8.59.crate"
889 +
CHECKSUM_SHA256_293="ac4f328cf2f05d084e496c3e9c3f33ed0a183656a16e1fcec4d464d8373aec82"
890 +
SOURCE_FILENAME_293="zerocopy-derive-0.8.59.tar.gz"
891 +
SOURCE_URI_294="https://static.crates.io/crates/zerofrom/zerofrom-0.1.8.crate"
892 +
CHECKSUM_SHA256_294="0ec05a11813ea801ff6d75110ad09cd0824ddba17dfe17128ea0d5f68e6c5272"
893 +
SOURCE_FILENAME_294="zerofrom-0.1.8.tar.gz"
894 +
SOURCE_URI_295="https://static.crates.io/crates/zerofrom-derive/zerofrom-derive-0.1.8.crate"
895 +
CHECKSUM_SHA256_295="f75b4683f6c7f45248d4d64056a24298c6281e0993356d7d1b4a1a962ef10d4a"
896 +
SOURCE_FILENAME_295="zerofrom-derive-0.1.8.tar.gz"
897 +
SOURCE_URI_296="https://static.crates.io/crates/zeroize/zeroize-1.9.0.crate"
898 +
CHECKSUM_SHA256_296="e13c156562582aa81c60cb29407084cdb54c4164760106ab78e6c5b0858cf64e"
899 +
SOURCE_FILENAME_296="zeroize-1.9.0.tar.gz"
900 +
SOURCE_URI_297="https://static.crates.io/crates/zerotrie/zerotrie-0.2.5.crate"
901 +
CHECKSUM_SHA256_297="4ea269c3bd32f0a32c321907a2ae912ba6f4649bb0fc764a15627e99a7095a3f"
902 +
SOURCE_FILENAME_297="zerotrie-0.2.5.tar.gz"
903 +
SOURCE_URI_298="https://static.crates.io/crates/zerovec/zerovec-0.11.8.crate"
904 +
CHECKSUM_SHA256_298="bb0464e17806c1d976d5cba29399c7f08e516e279e2ba493f63123b5fca67dd8"
905 +
SOURCE_FILENAME_298="zerovec-0.11.8.tar.gz"
906 +
SOURCE_URI_299="https://static.crates.io/crates/zerovec-derive/zerovec-derive-0.11.6.crate"
907 +
CHECKSUM_SHA256_299="34df6fc39dbd26ddc9c10e6a2984476e13acce22e64e4487636ef494369225da"
908 +
SOURCE_FILENAME_299="zerovec-derive-0.11.6.tar.gz"
909 +
SOURCE_URI_300="https://static.crates.io/crates/zlib-rs/zlib-rs-0.6.8.crate"
910 +
CHECKSUM_SHA256_300="b268e58e7c693d7c271f93ffc4ba3b380412554231c85bf61ca7af91042a4112"
911 +
SOURCE_FILENAME_300="zlib-rs-0.6.8.tar.gz"
912 +
SOURCE_URI_301="https://static.crates.io/crates/zmij/zmij-1.0.23.crate"
913 +
CHECKSUM_SHA256_301="29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
914 +
SOURCE_FILENAME_301="zmij-1.0.23.tar.gz"
915 +
SOURCE_URI_302="https://static.crates.io/crates/zstd/zstd-0.13.3.crate"
916 +
CHECKSUM_SHA256_302="e91ee311a569c327171651566e07972200e76fcfe2242a4fa446149a3881c08a"
917 +
SOURCE_FILENAME_302="zstd-0.13.3.tar.gz"
918 +
SOURCE_URI_303="https://static.crates.io/crates/zstd-safe/zstd-safe-7.3.0.crate"
919 +
CHECKSUM_SHA256_303="64d80649ab6db9d9f6f9c80a40becd948eda4714a0a5ac8c4d157a32231c7882"
920 +
SOURCE_FILENAME_303="zstd-safe-7.3.0.tar.gz"
921 +
SOURCE_URI_304="https://static.crates.io/crates/zstd-sys/zstd-sys-2.1.0+zstd.1.5.7.crate"
922 +
CHECKSUM_SHA256_304="0ef0a8027ec3ee71300ab3bcbcd0393f434aa72b91ca6d635a39941deae8eea0"
923 +
SOURCE_FILENAME_304="zstd-sys-2.1.0+zstd.1.5.7.tar.gz"
924 +
925 +
ARCHITECTURES="!all x86_64"
926 +
927 +
PROVIDES="
928 +
	oxy$secondaryArchSuffix = $portVersion
929 +
	cmd:oxy
930 +
	"
931 +
REQUIRES="
932 +
	haiku$secondaryArchSuffix
933 +
	lib:libzstd$secondaryArchSuffix
934 +
	"
935 +
936 +
BUILD_REQUIRES="
937 +
	haiku${secondaryArchSuffix}_devel
938 +
	devel:libzstd$secondaryArchSuffix
939 +
	"
940 +
BUILD_PREREQUIRES="
941 +
	cmd:gcc$secondaryArchSuffix
942 +
	cmd:pkg_config$secondaryArchSuffix
943 +
	cmd:rustc$secondaryArchSuffix
944 +
	"
945 +
946 +
BUILD()
947 +
{
948 +
	export CARGO_HOME=$sourceDir/../cargo
949 +
	vendor=$CARGO_HOME/haiku
950 +
	mkdir -p "$vendor"
951 +
	for i in $(seq 2 304); do
952 +
		eval "srcDir=\$sourceDir$i"
953 +
		eval "sha256sum=\$CHECKSUM_SHA256_$i"
954 +
		set -- "$srcDir"/*
955 +
		ln -sf "$1" "$vendor"
956 +
		cat <<-EOF >"$vendor/${1##*/}/.cargo-checksum.json"
957 +
		{
958 +
		  "package": "$sha256sum",
959 +
		  "files": {}
960 +
		}
961 +
		EOF
962 +
	done
963 +
964 +
	cat <<-EOF >"$CARGO_HOME"/config.toml
965 +
	[source.haiku]
966 +
	directory = "$vendor"
967 +
968 +
	[source.crates-io]
969 +
	replace-with = "haiku"
970 +
	EOF
971 +
972 +
	# The bundled zstd's hidden symbols break the link on Haiku: use the system's.
973 +
	export ZSTD_SYS_USE_PKG_CONFIG=1
974 +
	export OXY_BUILD=v$portVersion
975 +
	cargo build --release --frozen -p oxy
976 +
}
977 +
978 +
INSTALL()
979 +
{
980 +
	install -v -m 755 -d "$prefix"/bin "$docDir"
981 +
	install -v -m 755 ./target/release/oxy "$prefix"/bin
982 +
	install -v -m 644 README.md LICENSE* "$docDir"
983 +
}
984 +
985 +
TEST()
986 +
{
987 +
	export CARGO_HOME=$sourceDir/../cargo
988 +
	export ZSTD_SYS_USE_PKG_CONFIG=1
989 +
	cargo test --release --frozen -p oxy-core
990 +
}
added packaging/haikuports/icons/oh-16.png +0 −0

No line changes.

added packaging/haikuports/icons/oh-32.png +0 −0

No line changes.

added packaging/haikuports/icons/oh-64.png +0 −0

No line changes.

added packaging/haikuports/icons/oh.hvif +0 −0

No line changes.

added packaging/haikuports/icons/oh.svg +17 −0
1 +
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" width="64" height="64">
2 +
  <defs>
3 +
    <linearGradient id="g" gradientUnits="userSpaceOnUse" x1="-22" y1="-11" x2="22" y2="11">
4 +
      <stop offset="0%" stop-color="#00F0FF"/>
5 +
      <stop offset="100%" stop-color="#0072FF"/>
6 +
    </linearGradient>
7 +
  </defs>
8 +
  <rect width="64" height="64" rx="14" fill="#070c18"/>
9 +
  <g transform="translate(32 32)">
10 +
    <path d="M -11 -6 L 11 -6" stroke="url(#g)" stroke-width="3" stroke-linecap="round"/>
11 +
    <path d="M -11 6 L 11 6" stroke="url(#g)" stroke-width="3" stroke-linecap="round"/>
12 +
    <circle cx="-11" cy="0" r="10" fill="none" stroke="url(#g)" stroke-width="3"/>
13 +
    <circle cx="-11" cy="0" r="3.5" fill="#00F0FF"/>
14 +
    <circle cx="11" cy="0" r="10" fill="none" stroke="url(#g)" stroke-width="3"/>
15 +
    <circle cx="11" cy="0" r="3.5" fill="#FFFFFF"/>
16 +
  </g>
17 +
</svg>
added packaging/haikuports/icons/oxy-16.png +0 −0

No line changes.

added packaging/haikuports/icons/oxy-32.png +0 −0

No line changes.

added packaging/haikuports/icons/oxy-64.png +0 −0

No line changes.

added packaging/haikuports/icons/oxy.hvif +0 −0

No line changes.

added packaging/haikuports/icons/oxy.svg +17 −0
1 +
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" width="64" height="64">
2 +
  <defs>
3 +
    <linearGradient id="g" gradientUnits="userSpaceOnUse" x1="-22" y1="-11" x2="22" y2="11">
4 +
      <stop offset="0%" stop-color="#00F0FF"/>
5 +
      <stop offset="100%" stop-color="#0072FF"/>
6 +
    </linearGradient>
7 +
  </defs>
8 +
  <rect width="64" height="64" rx="14" fill="#C9DCF5"/><rect x="1.5" y="1.5" width="61" height="61" rx="12.5" fill="#F2F7FF"/>
9 +
  <g transform="translate(32 32)">
10 +
    <path d="M -11 -6 L 11 -6" stroke="url(#g)" stroke-width="3" stroke-linecap="round"/>
11 +
    <path d="M -11 6 L 11 6" stroke="url(#g)" stroke-width="3" stroke-linecap="round"/>
12 +
    <circle cx="-11" cy="0" r="10" fill="none" stroke="url(#g)" stroke-width="3"/>
13 +
    <circle cx="-11" cy="0" r="3.5" fill="#00F0FF"/>
14 +
    <circle cx="11" cy="0" r="10" fill="none" stroke="url(#g)" stroke-width="3"/>
15 +
    <circle cx="11" cy="0" r="3.5" fill="#0072FF"/>
16 +
  </g>
17 +
</svg>
added packaging/haikuports/make-icons.py +163 −0
1 +
#!/usr/bin/env python3
2 +
"""HaikuDepot icons for oxy and oh (oxygen#157): the Oxygen mark as Haiku's
3 +
vector icon format (HVIF), and as SVG to render the 16, 32 and 64 px PNGs.
4 +
5 +
A family of two (Andy, 2026-10-29): Oxygen (oxy, the Oxygen VCS extension)
6 +
is the mark on a pale rounded tile with a blue right dot; OxygenHub (oh, the
7 +
OxygenHub extension) is the mark on the hub's dark tile with a white one, as
8 +
oxygenhub/assets/icons/oxygenhub.svg. Each keeps its edge on light and dark
9 +
backgrounds alike.
10 +
11 +
    make-icons.py OUTDIR    writes oxy.hvif, oh.hvif, oxy.svg, oh.svg
12 +
13 +
HVIF as Haiku's FlatIconImporter reads it: "ncif", then styles, paths and
14 +
shapes, each a count byte and its items; coordinates in a 64-unit canvas.
15 +
"""
16 +
import math
17 +
import struct
18 +
import sys
19 +
20 +
CYAN, BLUE, WHITE, TILE = (0x00, 0xF0, 0xFF), (0x00, 0x72, 0xFF), (0xFF, 0xFF, 0xFF), (0x07, 0x0C, 0x18)
21 +
LIGHT, EDGE = (0xF2, 0xF7, 0xFF), (0xC9, 0xDC, 0xF5)
22 +
K = 0.5522847498  # a circle's quarter as one cubic Bézier
23 +
24 +
25 +
def coord(v):
26 +
    if v == int(v) and -32 <= v <= 95:
27 +
        return bytes([int(v) + 32])
28 +
    n = int(round((v + 128.0) * 102.0))
29 +
    return bytes([(n >> 8) | 0x80, n & 0xFF])
30 +
31 +
32 +
def float24(v):
33 +
    if v == 0:
34 +
        return b"\0\0\0"
35 +
    bits = struct.unpack(">I", struct.pack(">f", v))[0]
36 +
    sign, exp, man = bits >> 31, ((bits >> 23) & 0xFF) - 127, bits & 0x7FFFFF
37 +
    if not -32 <= exp < 32:
38 +
        raise ValueError(f"{v} does not fit a float24")
39 +
    n = (sign << 23) | ((exp + 32) << 17) | (man >> 6)
40 +
    return n.to_bytes(3, "big")
41 +
42 +
43 +
def solid(rgb):
44 +
    return bytes([3, *rgb])  # STYLE_TYPE_SOLID_COLOR_NO_ALPHA
45 +
46 +
47 +
def linear(p1, p2, stops):
48 +
    """A linear gradient from p1 to p2: gradient space runs x from -64 to 64."""
49 +
    dx, dy = (p2[0] - p1[0]) / 128.0, (p2[1] - p1[1]) / 128.0
50 +
    mx, my = (p1[0] + p2[0]) / 2, (p1[1] + p2[1]) / 2
51 +
    out = bytes([2, 0, (1 << 1) | (1 << 2), len(stops)])  # gradient, linear, transform | no alpha
52 +
    for v in (dx, dy, -dy, dx, mx, my):
53 +
        out += float24(v)
54 +
    for off, rgb in stops:
55 +
        out += bytes([off, *rgb])
56 +
    return out
57 +
58 +
59 +
def line(a, b):
60 +
    return bytes([1 << 3, 2]) + coord(a[0]) + coord(a[1]) + coord(b[0]) + coord(b[1])  # no curves, open
61 +
62 +
63 +
def curve_path(points, closed=True):
64 +
    """points: (x, y, in_x, in_y, out_x, out_y)."""
65 +
    out = bytes([(1 << 1) if closed else 0, len(points)])
66 +
    for p in points:
67 +
        for v in p:
68 +
            out += coord(v)
69 +
    return out
70 +
71 +
72 +
def circle(cx, cy, r):
73 +
    k = r * K
74 +
    pts = [(cx, cy - r, cx - k, cy - r, cx + k, cy - r), (cx + r, cy, cx + r, cy - k, cx + r, cy + k),
75 +
           (cx, cy + r, cx + k, cy + r, cx - k, cy + r), (cx - r, cy, cx - r, cy + k, cx - r, cy - k)]
76 +
    return curve_path(pts)
77 +
78 +
79 +
def rounded(x, y, w, h, r):
80 +
    k = r * (1 - K)
81 +
    x2, y2 = x + w, y + h
82 +
    pts = [
83 +
        (x + r, y, x + k, y, x + r, y), (x2 - r, y, x2 - r, y, x2 - k, y),
84 +
        (x2, y + r, x2, y + k, x2, y + r), (x2, y2 - r, x2, y2 - r, x2, y2 - k),
85 +
        (x2 - r, y2, x2 - k, y2, x2 - r, y2), (x + r, y2, x + r, y2, x + k, y2),
86 +
        (x, y2 - r, x, y2 - k, x, y2 - r), (x, y + r, x, y + r, x, y + k),
87 +
    ]
88 +
    return curve_path(pts)
89 +
90 +
91 +
def shape(style, paths, stroke=None, cap=0):
92 +
    out = bytes([10, style, len(paths), *paths])
93 +
    if stroke is None:
94 +
        return out + bytes([0])
95 +
    # has transformers: one stroke (width + 128, cap << 4 | round join, miter limit)
96 +
    return out + bytes([1 << 4, 1, 23, int(round(stroke)) + 128, (cap << 4) | 2, 4])
97 +
98 +
99 +
def mark(scale, ox, oy):
100 +
    """The mark's geometry, scaled about its centre and moved to (ox, oy)."""
101 +
    p = lambda x, y: (ox + x * scale, oy + y * scale)
102 +
    return {
103 +
        "top": (p(-11, -6), p(11, -6)), "bottom": (p(-11, 6), p(11, 6)),
104 +
        "left": (p(-11, 0), 10 * scale), "right": (p(11, 0), 10 * scale),
105 +
        "dot": 3.5 * scale, "stroke": 3 * scale, "grad": (p(-22, -11), p(22, 11)),
106 +
    }
107 +
108 +
109 +
def hvif(dark):
110 +
    m = mark(1.0, 32, 32)
111 +
    styles = [linear(*m["grad"], [(0, CYAN), (255, BLUE)]), solid(CYAN), solid(WHITE if dark else BLUE)]
112 +
    paths = [line(*m["top"]), line(*m["bottom"]), circle(*m["left"][0], m["left"][1]),
113 +
             circle(*m["right"][0], m["right"][1]), circle(*m["left"][0], m["dot"]), circle(*m["right"][0], m["dot"])]
114 +
    if dark:
115 +
        styles.append(solid(TILE))
116 +
        paths.append(rounded(0, 0, 64, 64, 14))
117 +
        tile = [shape(3, [6])]
118 +
    else:
119 +
        # A pale tile with a thin edge: the edge's colour, then the tile inset by it.
120 +
        styles += [solid(EDGE), solid(LIGHT)]
121 +
        paths += [rounded(0, 0, 64, 64, 14), rounded(1.5, 1.5, 61, 61, 12.5)]
122 +
        tile = [shape(3, [6]), shape(4, [7])]
123 +
    shapes = tile + [
124 +
        shape(0, [0], m["stroke"], cap=2), shape(0, [1], m["stroke"], cap=2),
125 +
        shape(0, [2], m["stroke"]), shape(0, [3], m["stroke"]),
126 +
        shape(1, [4]), shape(2, [5]),
127 +
    ]
128 +
    out = b"ncif"
129 +
    for section in (styles, paths, shapes):
130 +
        out += bytes([len(section)]) + b"".join(section)
131 +
    return out
132 +
133 +
134 +
def svg(dark):
135 +
    if dark:
136 +
        bg, dot = '<rect width="64" height="64" rx="14" fill="#070c18"/>', "#FFFFFF"
137 +
    else:
138 +
        bg = '<rect width="64" height="64" rx="14" fill="#C9DCF5"/><rect x="1.5" y="1.5" width="61" height="61" rx="12.5" fill="#F2F7FF"/>'
139 +
        dot = "#0072FF"
140 +
    return f'''<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" width="64" height="64">
141 +
  <defs>
142 +
    <linearGradient id="g" gradientUnits="userSpaceOnUse" x1="-22" y1="-11" x2="22" y2="11">
143 +
      <stop offset="0%" stop-color="#00F0FF"/>
144 +
      <stop offset="100%" stop-color="#0072FF"/>
145 +
    </linearGradient>
146 +
  </defs>
147 +
  {bg}
148 +
  <g transform="translate(32 32)">
149 +
    <path d="M -11 -6 L 11 -6" stroke="url(#g)" stroke-width="3" stroke-linecap="round"/>
150 +
    <path d="M -11 6 L 11 6" stroke="url(#g)" stroke-width="3" stroke-linecap="round"/>
151 +
    <circle cx="-11" cy="0" r="10" fill="none" stroke="url(#g)" stroke-width="3"/>
152 +
    <circle cx="-11" cy="0" r="3.5" fill="#00F0FF"/>
153 +
    <circle cx="11" cy="0" r="10" fill="none" stroke="url(#g)" stroke-width="3"/>
154 +
    <circle cx="11" cy="0" r="3.5" fill="{dot}"/>
155 +
  </g>
156 +
</svg>
157 +
'''
158 +
159 +
160 +
out = sys.argv[1]
161 +
for name, dark in (("oxy", False), ("oh", True)):
162 +
    open(f"{out}/{name}.hvif", "wb").write(hvif(dark))
163 +
    open(f"{out}/{name}.svg", "w").write(svg(dark))
added packaging/haikuports/make-recipe.py +140 −0
1 +
#!/usr/bin/env python3
2 +
"""Write a haikuports recipe for oxy or oh (oxygen#156, #157) from a release's
3 +
source tarball, NAME-VERSION-src.tar.gz, as the release job attaches it: the
4 +
tarball itself from the release's stable download address, and every
5 +
crates.io crate in its Cargo.lock as a source of its own with its checksum,
6 +
vendored offline in BUILD() as haikuports' Rust recipes (ripgrep) do.
7 +
8 +
    make-recipe.py oxy|oh SRC_TARBALL VERSION > NAME-VERSION.recipe
9 +
"""
10 +
import hashlib
11 +
import sys
12 +
import tarfile
13 +
import tomllib
14 +
15 +
PORTS = {
16 +
    "oxy": {
17 +
        "repo": "oxygen",
18 +
        "package": "oxy",
19 +
        "summary": "A version control system of signed, commuting changes, with Git interop",
20 +
        "description": [
21 +
            "oxy is the command line of Oxygen, a version control system built on",
22 +
            "patch theory: changes commute, conflicts are recorded rather than blocking,",
23 +
            "every change can be signed, and a trust policy says who may move what. It",
24 +
            "imports and exports Git history, clones from OxygenHub and other Oxygen",
25 +
            "hosts, and keeps repositories on disk or in S3, GCS or Azure buckets.",
26 +
        ],
27 +
        "docs": "README.md LICENSE*",
28 +
        "test": "oxy-core",
29 +
    },
30 +
    "oh": {
31 +
        "repo": "oxygenhub",
32 +
        "package": "oh",
33 +
        "summary": "The command line of OxygenHub, the forge for Oxygen repositories",
34 +
        "description": [
35 +
            "oh talks to an OxygenHub (such as https://oxygenhub.io): issues, proposals",
36 +
            "and their reviews, releases, Flow runs, their logs and reports, and checks,",
37 +
            "from the terminal. It works alongside oxy, the Oxygen version control",
38 +
            "client.",
39 +
        ],
40 +
        "docs": "README.md LICENSE docs/oh.md",
41 +
        "test": "hub-api",
42 +
    },
43 +
}
44 +
45 +
name, tarball, version = sys.argv[1], sys.argv[2], sys.argv[3]
46 +
port = PORTS[name]
47 +
prefix = f"{port['repo']}-{version}"
48 +
with tarfile.open(tarball) as t:
49 +
    lock = tomllib.load(t.extractfile(f"{prefix}/Cargo.lock"))
50 +
crates = sorted(
51 +
    (p["name"], p["version"], p["checksum"])
52 +
    for p in lock["package"]
53 +
    if p.get("source", "").startswith("registry+")
54 +
)
55 +
sha = hashlib.sha256(open(tarball, "rb").read()).hexdigest()
56 +
57 +
out = []
58 +
w = out.append
59 +
w(f'SUMMARY="{port["summary"]}"')
60 +
w('DESCRIPTION="' + " \\\n".join(port["description"]) + '"')
61 +
w(f'HOMEPAGE="https://oxygenhub.io/andymccall/{port["repo"]}"')
62 +
w('COPYRIGHT="2026 Andy McCall"')
63 +
w('LICENSE="MIT\n\tApache v2"')
64 +
w('REVISION="1"')
65 +
w(f'SOURCE_URI="$HOMEPAGE/releases/download/v$portVersion/{name}-$portVersion-src.tar.gz"')
66 +
w(f'CHECKSUM_SHA256="{sha}"')
67 +
w(f'SOURCE_DIR="{port["repo"]}-$portVersion"')
68 +
for i, (crate, ver, ck) in enumerate(crates, start=2):
69 +
    w(f'SOURCE_URI_{i}="https://static.crates.io/crates/{crate}/{crate}-{ver}.crate"')
70 +
    w(f'CHECKSUM_SHA256_{i}="{ck}"')
71 +
    w(f'SOURCE_FILENAME_{i}="{crate}-{ver}.tar.gz"')
72 +
last = len(crates) + 1
73 +
w(f'''
74 +
ARCHITECTURES="!all x86_64"
75 +
76 +
PROVIDES="
77 +
	{name}$secondaryArchSuffix = $portVersion
78 +
	cmd:{name}
79 +
	"
80 +
REQUIRES="
81 +
	haiku$secondaryArchSuffix
82 +
	lib:libzstd$secondaryArchSuffix
83 +
	"
84 +
85 +
BUILD_REQUIRES="
86 +
	haiku${{secondaryArchSuffix}}_devel
87 +
	devel:libzstd$secondaryArchSuffix
88 +
	"
89 +
BUILD_PREREQUIRES="
90 +
	cmd:gcc$secondaryArchSuffix
91 +
	cmd:pkg_config$secondaryArchSuffix
92 +
	cmd:rustc$secondaryArchSuffix
93 +
	"
94 +
95 +
BUILD()
96 +
{{
97 +
	export CARGO_HOME=$sourceDir/../cargo
98 +
	vendor=$CARGO_HOME/haiku
99 +
	mkdir -p "$vendor"
100 +
	for i in $(seq 2 {last}); do
101 +
		eval "srcDir=\\$sourceDir$i"
102 +
		eval "sha256sum=\\$CHECKSUM_SHA256_$i"
103 +
		set -- "$srcDir"/*
104 +
		ln -sf "$1" "$vendor"
105 +
		cat <<-EOF >"$vendor/${{1##*/}}/.cargo-checksum.json"
106 +
		{{
107 +
		  "package": "$sha256sum",
108 +
		  "files": {{}}
109 +
		}}
110 +
		EOF
111 +
	done
112 +
113 +
	cat <<-EOF >"$CARGO_HOME"/config.toml
114 +
	[source.haiku]
115 +
	directory = "$vendor"
116 +
117 +
	[source.crates-io]
118 +
	replace-with = "haiku"
119 +
	EOF
120 +
121 +
	# The bundled zstd's hidden symbols break the link on Haiku: use the system's.
122 +
	export ZSTD_SYS_USE_PKG_CONFIG=1
123 +
	export OXY_BUILD=v$portVersion
124 +
	cargo build --release --frozen -p {port["package"]}
125 +
}}
126 +
127 +
INSTALL()
128 +
{{
129 +
	install -v -m 755 -d "$prefix"/bin "$docDir"
130 +
	install -v -m 755 ./target/release/{name} "$prefix"/bin
131 +
	install -v -m 644 {port["docs"]} "$docDir"
132 +
}}
133 +
134 +
TEST()
135 +
{{
136 +
	export CARGO_HOME=$sourceDir/../cargo
137 +
	export ZSTD_SYS_USE_PKG_CONFIG=1
138 +
	cargo test --release --frozen -p {port["test"]}
139 +
}}''')
140 +
print("\n".join(out))
added scripts/changelog-check.sh +27 −0
1 +
#!/bin/sh
2 +
# The changelog check (2026-10-19): a commit that changes what users get
3 +
# adds a line under "## Unreleased" in CHANGELOG.md, so a release's notes
4 +
# are written as the work is done and versions say what changed.
5 +
#
6 +
#   scripts/changelog-check.sh [PATH...]
7 +
#
8 +
# PATHs are what the commit will take (as given to `oxy commit`); without
9 +
# any, the working copy's changes. Code is anything under src/, crates/,
10 +
# templates/, assets/, packaging/, images/ or scripts/, except tests.
11 +
# Docs, pipelines, pins (deps/) and lock files need no line. For a code
12 +
# change users won't notice (a refactor, a test helper), set NO_CHANGELOG=1.
13 +
set -eu
14 +
[ "${NO_CHANGELOG:-}" = 1 ] && exit 0
15 +
cd "$(dirname "$0")/.."
16 +
if [ "$#" -gt 0 ]; then
17 +
    paths=$(printf '%s\n' "$@")
18 +
else
19 +
    paths=$(oxy --plain status --json | python3 -c 'import json,sys; print("\n".join(p["path"] for p in json.load(sys.stdin)["paths"]))')
20 +
fi
21 +
code=$(printf '%s\n' "$paths" | grep -E '^(src|crates|templates|assets|packaging|images|scripts)/' | grep -vE '(^|/)tests?/|_test\.rs$' || true)
22 +
[ -z "$code" ] && exit 0
23 +
printf '%s\n' "$paths" | grep -qx 'CHANGELOG.md' && exit 0
24 +
echo "changelog: these changes touch code but CHANGELOG.md has no new line under ## Unreleased:" >&2
25 +
printf '%s\n' "$code" | sed 's/^/  /' >&2
26 +
echo "add one (Added / Changed / Fixed / Security, with the issue number), or set NO_CHANGELOG=1 if users won't notice" >&2
27 +
exit 1
modified scripts/dist.sh +19 −0
…
98 unchanged lines
99 99
URIs: https://oxygenhub.io/apt
100 100
Suites: stable
101 101
Components: main
102 +
Architectures: $(dpkg --print-architecture)
102 103
Signed-By: /usr/share/keyrings/oxygenhub-archive-keyring.gpg
103 104
SRC
104 105
  fi
106 +
  # Only this machine's architecture: with foreign ones enabled (i386 on
107 +
  # most amd64 systems) apt otherwise asks for packages the repository
108 +
  # has not got, and says so on every update. Sources written before
109 +
  # this get the line too.
110 +
  if [ -e "$L" ] && ! grep -q "^Architectures:" "$L"; then
111 +
    sed -i "/^Components:/a Architectures: $(dpkg --print-architecture)" "$L"
112 +
  fi
105 113
fi
106 114
EOF
107 115
    sed "s/@OTHER@/$OTHER_DEB/" >"$root/DEBIAN/postrm" <<'EOF'
…
21 unchanged lines
129 137
  fi
130 138
done
131 139
140 +
# The source as released (oxygen#157): the oxygen tree,
141 +
# under oxygen-VERSION/, for packagers (haikuports) to fetch from the
142 +
# release at a fixed checksum. Reproducible: sorted, fixed times and
143 +
# owners, gzip without a timestamp.
144 +
if [ -z "${ONLY:-}" ] || [[ src == $ONLY ]]; then
145 +
  echo "== src"
146 +
  tar --sort=name --mtime='1970-01-01 00:00:00Z' --owner=0 --group=0 --numeric-owner \
147 +
    --mode='u+rwX,go+rX,go-w' --exclude=./target --exclude=./lost+found --exclude=.oxy \
148 +
    --transform "s|^\.|oxygen-$VERSION|" -cf - . | gzip -9n >"$OUT/$BIN-$VERSION-src.tar.gz"
149 +
fi
150 +
132 151
rm -rf "$OUT/stage"
133 152
(cd "$OUT" && shopt -s nullglob && sha256sum -- *.tar.gz *.zip *.deb >SHA256SUMS)
134 153
# DIST_FLAT=DIR: the same files again under names without the version, for
…
12 unchanged lines
added src/cli/approve.rs +151 −0
1 +
//! `oxy seal --flow RUN_URL --job NAME [--reject]` (oxygenhub#235): sign an
2 +
//! approval of one job of one run with your key, so the hub can show, and
3 +
//! anyone can check, who released it. The hub says what the job is (its
4 +
//! hash with its sandbox, the state, the plan it takes, the environment);
5 +
//! the signature covers exactly that and nothing else.
6 +
7 +
use super::{open, ui};
8 +
use anyhow::{anyhow, bail, Context, Result};
9 +
use oxy_core::approval::{SignedApproval, Statement};
10 +
use oxy_core::hash::ObjectRef;
11 +
use serde_json::Value;
12 +
13 +
/// `https://hub/OWNER/NAME/flow/N` to the hub's base, `OWNER/NAME`, N and
14 +
/// the repository's URL.
15 +
fn run_url(url: &str) -> Result<(String, String, i64, String)> {
16 +
    let url = url.trim_end_matches('/');
17 +
    let (repo_url, n) = url
18 +
        .rsplit_once("/flow/")
19 +
        .ok_or_else(|| anyhow!("{url} is not a run URL (…/OWNER/NAME/flow/N)"))?;
20 +
    let number: i64 = n
21 +
        .parse()
22 +
        .map_err(|_| anyhow!("{url} is not a run URL (…/OWNER/NAME/flow/N)"))?;
23 +
    let (rest, name) = repo_url
24 +
        .rsplit_once('/')
25 +
        .ok_or_else(|| anyhow!("{url} is not a run URL"))?;
26 +
    let (base, owner) = rest
27 +
        .rsplit_once('/')
28 +
        .ok_or_else(|| anyhow!("{url} is not a run URL"))?;
29 +
    Ok((
30 +
        base.to_string(),
31 +
        format!("{owner}/{name}"),
32 +
        number,
33 +
        repo_url.to_string(),
34 +
    ))
35 +
}
36 +
37 +
fn to_hex(b: &[u8]) -> String {
38 +
    b.iter().map(|x| format!("{x:02x}")).collect()
39 +
}
40 +
41 +
fn oref(v: &Value, key: &str) -> Result<ObjectRef> {
42 +
    v.get(key)
43 +
        .and_then(Value::as_str)
44 +
        .and_then(ObjectRef::from_hex)
45 +
        .ok_or_else(|| anyhow!("the hub's statement has no valid {key}"))
46 +
}
47 +
48 +
pub fn seal(url: &str, job: &str, reject: bool, as_json: bool) -> Result<i32> {
49 +
    let repo = open()?;
50 +
    let (base, slug, number, repo_url) = run_url(url)?;
51 +
    let options = repo.options_for_url(&repo_url);
52 +
    let api = format!("{base}/api/v1/repos/{slug}/flow/runs/{number}/jobs/{job}/approval");
53 +
    let body = oxy_core::protocol::http_get(&api, &options).map_err(|e| anyhow!("{e}"))?;
54 +
    let v: Value = serde_json::from_slice(&body).context("the hub did not answer with JSON")?;
55 +
    let repo_hex = v.get("repo").and_then(Value::as_str).unwrap_or("");
56 +
    let mine = to_hex(&repo.id()?);
57 +
    if repo_hex != mine {
58 +
        bail!("run #{number} belongs to repository {repo_hex}, not this checkout's ({mine}): run oxy seal --flow from a checkout of {slug}");
59 +
    }
60 +
    let mut repo_id = [0u8; 16];
61 +
    for (i, b) in repo_id.iter_mut().enumerate() {
62 +
        *b = u8::from_str_radix(&repo_hex[i * 2..i * 2 + 2], 16)?;
63 +
    }
64 +
    let inputs = v
65 +
        .get("inputs")
66 +
        .and_then(Value::as_array)
67 +
        .map(|a| {
68 +
            a.iter()
69 +
                .filter_map(|x| x.as_str().and_then(ObjectRef::from_hex))
70 +
                .collect::<Vec<_>>()
71 +
        })
72 +
        .unwrap_or_default();
73 +
    let statement = Statement {
74 +
        repo: repo_id,
75 +
        run: v
76 +
            .get("run")
77 +
            .and_then(Value::as_str)
78 +
            .unwrap_or(url)
79 +
            .to_string(),
80 +
        job: job.to_string(),
81 +
        job_hash: oref(&v, "job_hash")?,
82 +
        state: oref(&v, "state")?,
83 +
        inputs,
84 +
        environment: v
85 +
            .get("environment")
86 +
            .and_then(Value::as_str)
87 +
            .unwrap_or("")
88 +
            .to_string(),
89 +
        approve: !reject,
90 +
    };
91 +
    let (key, sk) = repo
92 +
        .secret_key()?
93 +
        .ok_or_else(|| anyhow!("no signing key: create one with `oxy identity new`"))?;
94 +
    let me = repo.identity(&key)?;
95 +
    let signed = SignedApproval::sign(&statement, &me, &sk);
96 +
    let post = format!("{base}/api/v1/repos/{slug}/flow/runs/{number}/jobs/{job}/seal");
97 +
    let payload = serde_json::json!({ "signed": to_hex(&signed.encode()) }).to_string();
98 +
    let (code, answer) = oxy_core::protocol::http_json("POST", &post, Some(&payload), &options)
99 +
        .map_err(|e| anyhow!("{e}"))?;
100 +
    if !(200..300).contains(&code) {
101 +
        let why = serde_json::from_str::<Value>(&answer)
102 +
            .ok()
103 +
            .and_then(|v| v.get("error").and_then(Value::as_str).map(String::from));
104 +
        bail!(
105 +
            "the hub refused the seal (HTTP {code}): {}",
106 +
            why.unwrap_or(answer)
107 +
        );
108 +
    }
109 +
    if as_json {
110 +
        println!("{answer}");
111 +
    } else {
112 +
        let verb = if reject { "Rejected" } else { "Approved" };
113 +
        println!(
114 +
            "{} {job} of run #{number} with key {}",
115 +
            ui::done(verb),
116 +
            &key.to_hex()[..12]
117 +
        );
118 +
        println!(
119 +
            "  state {} · job {}",
120 +
            &statement.state.to_hex()[..12],
121 +
            &statement.job_hash.to_hex()[..12]
122 +
        );
123 +
        for i in &statement.inputs {
124 +
            println!("  takes {}", &i.to_hex()[..12]);
125 +
        }
126 +
        if !statement.environment.is_empty() {
127 +
            println!("  for environment {}", statement.environment);
128 +
        }
129 +
    }
130 +
    Ok(0)
131 +
}
132 +
133 +
#[cfg(test)]
134 +
mod tests {
135 +
    #[test]
136 +
    fn run_urls_are_read() {
137 +
        let (base, slug, n, repo) =
138 +
            super::run_url("https://oxygenhub.io/andymccall/oxygenhub-gcp/flow/8/").unwrap();
139 +
        assert_eq!(
140 +
            (base.as_str(), slug.as_str(), n, repo.as_str()),
141 +
            (
142 +
                "https://oxygenhub.io",
143 +
                "andymccall/oxygenhub-gcp",
144 +
                8,
145 +
                "https://oxygenhub.io/andymccall/oxygenhub-gcp"
146 +
            )
147 +
        );
148 +
        assert!(super::run_url("https://oxygenhub.io/andymccall/x/proposal/3").is_err());
149 +
        assert!(super::run_url("https://oxygenhub.io/andymccall/x/flow/abc").is_err());
150 +
    }
151 +
}
added src/cli/artifact.rs +180 −0
1 +
//! `oxy artifact`: Cylinder's artifacts (docs/cylinder.md §4.1, step C1):
2 +
//! a directory packed into a tree of the repository's own chunks, named by
3 +
//! the BLAKE3 of its canonical encoding.
4 +
5 +
use super::json;
6 +
use super::open;
7 +
use super::ui;
8 +
use anyhow::Result;
9 +
use cylinder::{Kind, Spec};
10 +
use std::path::Path;
11 +
12 +
/// `KEY=VALUE` pairs, refused when one has no `=`.
13 +
fn pairs(list: &[String], what: &str) -> Result<std::collections::BTreeMap<String, String>> {
14 +
    list.iter()
15 +
        .map(|p| {
16 +
            p.split_once('=')
17 +
                .map(|(k, v)| (k.to_string(), v.to_string()))
18 +
                .ok_or_else(|| anyhow::anyhow!("{what} {p:?}: write it KEY=VALUE"))
19 +
        })
20 +
        .collect()
21 +
}
22 +
23 +
fn summary(k: &cylinder::Kept) -> String {
24 +
    let a = &k.artifact;
25 +
    let mut parts = vec![a.kind.as_str().to_string()];
26 +
    parts.extend(a.os.clone());
27 +
    parts.extend(a.arch.clone());
28 +
    format!(
29 +
        "{} {} {}",
30 +
        ui::hash(&k.name[..12]),
31 +
        parts.join(" "),
32 +
        ui::dim(format!("{} bytes", a.total_size))
33 +
    )
34 +
}
35 +
36 +
fn as_json(k: &cylinder::Kept, files: usize) -> String {
37 +
    let a = &k.artifact;
38 +
    let text = |v: &Option<String>| v.as_deref().map(json::s).unwrap_or_else(|| "null".into());
39 +
    let map = |m: &std::collections::BTreeMap<String, String>| {
40 +
        let inner: Vec<String> = m
41 +
            .iter()
42 +
            .map(|(k, v)| format!("{}:{}", json::s(k), json::s(v)))
43 +
            .collect();
44 +
        format!("{{{}}}", inner.join(","))
45 +
    };
46 +
    json::obj([
47 +
        ("name", json::s(&k.name)),
48 +
        ("kind", json::s(a.kind.as_str())),
49 +
        ("arch", text(&a.arch)),
50 +
        ("os", text(&a.os)),
51 +
        ("tree", json::s(&a.tree.to_hex())),
52 +
        ("files", json::n(files as u64)),
53 +
        ("total_size", json::n(a.total_size)),
54 +
        ("env", map(&a.env)),
55 +
        ("labels", map(&a.labels)),
56 +
    ])
57 +
}
58 +
59 +
#[allow(clippy::too_many_arguments)]
60 +
pub fn pack(
61 +
    dir: &Path,
62 +
    kind: &str,
63 +
    arch: Option<String>,
64 +
    os: Option<String>,
65 +
    entrypoint: Vec<String>,
66 +
    env: &[String],
67 +
    labels: &[String],
68 +
    as_json_out: bool,
69 +
) -> Result<i32> {
70 +
    let repo = open()?;
71 +
    let mut spec = Spec::new(Kind::parse(kind)?);
72 +
    spec.arch = arch;
73 +
    spec.os = os;
74 +
    spec.entrypoint = (!entrypoint.is_empty()).then_some(entrypoint);
75 +
    spec.env = pairs(env, "env")?;
76 +
    spec.labels = pairs(labels, "label")?;
77 +
    let sp = ui::spinner("Packing…");
78 +
    let k = cylinder::pack(&repo, dir, &spec)?;
79 +
    sp.clear();
80 +
    let files = cylinder::tree(&repo, &k.artifact)?.entries.len();
81 +
    if as_json_out {
82 +
        println!("{}", as_json(&k, files));
83 +
    } else {
84 +
        println!(
85 +
            "{} {} as {}",
86 +
            ui::done("Packed"),
87 +
            ui::path(dir.display()),
88 +
            summary(&k)
89 +
        );
90 +
        println!("{}", ui::kv("name", &k.name));
91 +
        println!("{}", ui::kv("files", files.to_string()));
92 +
    }
93 +
    Ok(0)
94 +
}
95 +
96 +
pub fn list(as_json_out: bool) -> Result<i32> {
97 +
    let repo = open()?;
98 +
    let names = cylinder::list(&repo)?;
99 +
    let kept: Vec<cylinder::Kept> = names
100 +
        .iter()
101 +
        .map(|n| cylinder::open(&repo, n))
102 +
        .collect::<std::result::Result<_, _>>()?;
103 +
    if as_json_out {
104 +
        let items: Vec<String> = kept
105 +
            .iter()
106 +
            .map(|k| {
107 +
                let files = cylinder::tree(&repo, &k.artifact)
108 +
                    .map(|t| t.entries.len())
109 +
                    .unwrap_or(0);
110 +
                as_json(k, files)
111 +
            })
112 +
            .collect();
113 +
        println!("[{}]", items.join(","));
114 +
        return Ok(0);
115 +
    }
116 +
    if kept.is_empty() {
117 +
        println!("No artifacts yet: oxy artifact pack DIR --kind archive");
118 +
    }
119 +
    for k in &kept {
120 +
        println!("{}", summary(k));
121 +
    }
122 +
    Ok(0)
123 +
}
124 +
125 +
pub fn show(name: &str, as_json_out: bool) -> Result<i32> {
126 +
    let repo = open()?;
127 +
    let k = cylinder::open(&repo, name)?;
128 +
    let t = cylinder::tree(&repo, &k.artifact)?;
129 +
    if as_json_out {
130 +
        println!("{}", as_json(&k, t.entries.len()));
131 +
        return Ok(0);
132 +
    }
133 +
    println!("{}", summary(&k));
134 +
    println!("{}", ui::kv("name", &k.name));
135 +
    if let Some(e) = &k.artifact.entrypoint {
136 +
        println!("{}", ui::kv("entrypoint", e.join(" ")));
137 +
    }
138 +
    for (key, v) in &k.artifact.labels {
139 +
        println!("{}", ui::kv("label", format!("{key}={v}")));
140 +
    }
141 +
    for (key, v) in &k.artifact.env {
142 +
        println!("{}", ui::kv("env", format!("{key}={v}")));
143 +
    }
144 +
    for e in &t.entries {
145 +
        let mode = match e.mode {
146 +
            cylinder::MODE_EXEC => "x",
147 +
            cylinder::MODE_LINK => "l",
148 +
            _ => " ",
149 +
        };
150 +
        println!("  {mode} {:>10}  {}", e.size, e.path);
151 +
    }
152 +
    Ok(0)
153 +
}
154 +
155 +
pub fn unpack(name: &str, dest: &Path) -> Result<i32> {
156 +
    let repo = open()?;
157 +
    let k = cylinder::open(&repo, name)?;
158 +
    let n = cylinder::unpack(&repo, &k, dest)?;
159 +
    println!(
160 +
        "{} {} files of {} into {}",
161 +
        ui::done("Unpacked"),
162 +
        n,
163 +
        ui::hash(&k.name[..12]),
164 +
        ui::path(dest.display())
165 +
    );
166 +
    Ok(0)
167 +
}
168 +
169 +
pub fn verify(name: &str) -> Result<i32> {
170 +
    let repo = open()?;
171 +
    let k = cylinder::open(&repo, name)?;
172 +
    let n = cylinder::verify(&repo, &k)?;
173 +
    println!(
174 +
        "{} {}: its name, tree and {} files check out",
175 +
        ui::done("Verified"),
176 +
        ui::hash(&k.name[..12]),
177 +
        n
178 +
    );
179 +
    Ok(0)
180 +
}
modified src/cli/auth.rs +42 −6
…
169 unchanged lines
170 170
    let anonymous = RemoteOptions {
171 171
        token: None,
172 172
        ca: ca.clone(),
173 -
        repo_id: None,
174 -
        cache_dir: None,
173 +
        ..RemoteOptions::default()
175 174
    };
176 175
    let token = if with_token {
177 176
        let mut line = String::new();
…
35 unchanged lines
213 212
    Ok(0)
214 213
}
215 214
215 +
/// `oxy auth grant`: keep an agent's grant token (oxygenhub#241) for
216 +
/// keyless signing, read from standard input.
217 +
pub fn grant(hub: Option<&str>) -> Result<i32> {
218 +
    use std::io::IsTerminal;
219 +
    let hub = resolve_hub(hub)?;
220 +
    if !token_allowed(&hub) {
221 +
        bail!("{hub} is plain http: a token would cross the network unencrypted; use its https address");
222 +
    }
223 +
    if std::io::stdin().is_terminal() {
224 +
        eprint!("Agent grant token for {hub}: ");
225 +
    }
226 +
    let mut line = String::new();
227 +
    std::io::stdin()
228 +
        .lock()
229 +
        .read_line(&mut line)
230 +
        .context("reading the token from standard input")?;
231 +
    let token = line.trim();
232 +
    if !token.starts_with("oxyagent_") {
233 +
        bail!("that is not an agent grant token (they start oxyagent_; `oh agent grant NAME` makes one)");
234 +
    }
235 +
    let stored = Credentials::user()
236 +
        .save_grant(&hub, token)
237 +
        .map_err(|e| anyhow::anyhow!("{e}"))?;
238 +
    println!(
239 +
        "{} for {}",
240 +
        ui::done("Kept the agent grant"),
241 +
        ui::path(&hub)
242 +
    );
243 +
    println!(
244 +
        "  {} {}",
245 +
        ui::note(),
246 +
        ui::dim(format!(
247 +
            "token in {}; sign with it: oxy config author.signer keyless",
248 +
            where_stored(&stored)
249 +
        ))
250 +
    );
251 +
    Ok(0)
252 +
}
253 +
216 254
/// The start of a token, enough to tell tokens apart and no more.
217 255
fn masked(token: &str) -> String {
218 256
    let shown: String = token.chars().take(10).collect();
…
42 unchanged lines
261 299
        let options = RemoteOptions {
262 300
            token: Some(token.clone()),
263 301
            ca: e.ca.clone(),
264 -
            repo_id: None,
265 -
            cache_dir: None,
302 +
            ..RemoteOptions::default()
266 303
        };
267 304
        let check = match whoami(&e.hub, &options) {
268 305
            Ok(Ok(a)) => ui::ok(format!("valid, signed in as {a}")),
…
46 unchanged lines
315 352
            let options = RemoteOptions {
316 353
                token: Some(t.clone()),
317 354
                ca: entry.ca.clone(),
318 -
                repo_id: None,
319 -
                cache_dir: None,
355 +
                ..RemoteOptions::default()
320 356
            };
321 357
            matches!(
322 358
                http_json(
…
26 unchanged lines
modified src/cli/format.rs +29 −4
…
41 unchanged lines
42 42
43 43
/// One line describing how a change is authenticated.
44 44
pub fn signature_line(repo: &Repo, h: &ChangeHash) -> Result<String> {
45 -
    Ok(match repo.authenticate_change(h)? {
45 +
    let status = repo.authenticate_change(h)?;
46 +
    let awaiting = repo
47 +
        .store()
48 +
        .get_change(h)
49 +
        .is_ok_and(|c| awaits_seal(repo, &status, &c.author));
50 +
    Ok(match status {
51 +
        SignatureStatus::Unsigned if awaiting => {
52 +
            "awaiting seal (the next push seals it under the agent's grant)".to_string()
53 +
        }
46 54
        SignatureStatus::Unsigned => "unsigned".to_string(),
47 55
        SignatureStatus::Valid {
48 56
            key,
…
65 unchanged lines
114 122
                .identity(author)
115 123
                .ok()
116 124
                .and_then(|a| a.delegate(key).copied());
125 +
            // A short-lived key acting under the author's grant (§14.7)
126 +
            // is no delegate, and not a revoked one either.
127 +
            let certified = entry.is_none() && repo.certified_subject(key).is_some();
117 128
            Some(Via {
118 129
                name: name.clone(),
119 130
                email: email.clone(),
120 131
                key: *key,
121 132
                scope: entry.map(|d| d.scope.to_string()),
122 133
                expires: entry.and_then(|d| d.expires),
123 -
                revoked: *revoked || entry.is_none(),
134 +
                revoked: *revoked || (entry.is_none() && !certified),
124 135
            })
125 136
        }
126 137
        _ => None,
127 138
    }
128 139
}
129 140
141 +
/// Whether an unsigned change is one the next push will seal: this
142 +
/// checkout signs keyless (an agent under a grant) and the change is its
143 +
/// configured author's (oxygen#165). Read cold, "unsigned" looks like an
144 +
/// error; it is not one until the push.
145 +
pub fn awaits_seal(repo: &Repo, status: &SignatureStatus, author: &oxy_core::hash::KeyId) -> bool {
146 +
    matches!(status, SignatureStatus::Unsigned)
147 +
        && repo.config.signer_keyless
148 +
        && repo.config.author_key == Some(*author)
149 +
}
150 +
130 151
/// A compact, styled authentication marker for log lines, from a status
131 -
/// already in hand (the caller also reads who signed, see [`via`]).
132 -
pub fn signature_mark_of(status: &SignatureStatus) -> String {
152 +
/// already in hand (the caller also reads who signed, see [`via`]);
153 +
/// `awaiting` from [`awaits_seal`].
154 +
pub fn signature_mark_of(status: &SignatureStatus, awaiting: bool) -> String {
133 155
    match status {
156 +
        SignatureStatus::Unsigned if awaiting => {
157 +
            ui::dim(format!("{} awaiting seal", ui::glyph("◌", "o")))
158 +
        }
134 159
        SignatureStatus::Unsigned => ui::dim(format!("{} unsigned", ui::glyph("·", "-"))),
135 160
        SignatureStatus::Valid {
136 161
            sealed,
…
64 unchanged lines
modified src/cli/identity.rs +201 −8
…
131 unchanged lines
132 132
        ui::author(&r.identity.name),
133 133
        r.identity.email
134 134
    );
135 -
    println!("{}", ui::kv("old key", ui::dim(r.old.to_hex())));
135 +
    // Which key it started from, and where that came from (oxygen#163).
136 +
    let from = if repo.config.is_repo_set("author.key") {
137 +
        "this repository's author.key"
138 +
    } else {
139 +
        "the user config's author.key"
140 +
    };
141 +
    println!(
142 +
        "{}",
143 +
        ui::kv(
144 +
            "old key",
145 +
            format!(
146 +
                "{} {}",
147 +
                ui::dim(r.old.to_hex()),
148 +
                ui::dim(format!("(from {from})"))
149 +
            )
150 +
        )
151 +
    );
136 152
    println!("{}", ui::kv("new key", r.new.to_hex()));
137 153
    print_delegates(repo, &r.identity.delegates);
138 154
    if let Some(e) = r.identity.expires {
…
16 unchanged lines
155 171
    local: bool,
156 172
    expires: Option<u64>,
157 173
) -> Result<i32> {
158 -
    let mut repo = open()?;
174 +
    // A first identity comes before any repository (oxygen#144): only
175 +
    // --local needs one; inside one, it is kept there too.
176 +
    let mut repo = match open() {
177 +
        Ok(r) => r,
178 +
        Err(e) if local => return Err(e),
179 +
        Err(_) => {
180 +
            let (key, id) = repo::create_user_identity(
181 +
                name.as_deref(),
182 +
                email.as_deref(),
183 +
                expires.and_then(expiry_from_days),
184 +
            )?;
185 +
            println!(
186 +
                "{} identity {} <{}>",
187 +
                ui::done("Created"),
188 +
                ui::author(&id.name),
189 +
                id.email
190 +
            );
191 +
            println!("{}", ui::kv("key", key));
192 +
            println!(
193 +
                "{}",
194 +
                ui::kv("secret", ui::path(repo::default_keys_dir().display()))
195 +
            );
196 +
            println!(
197 +
                "{}",
198 +
                ui::kv(
199 +
                    "scope",
200 +
                    format!(
201 +
                        "default author in {}",
202 +
                        ui::path(repo::default_user_config().display())
203 +
                    )
204 +
                )
205 +
            );
206 +
            println!("  {} {}", ui::note(), ui::dim("changes in every repository carry this author; run oxy config --user core.sign change to sign each commit"));
207 +
            return Ok(0);
208 +
        }
209 +
    };
159 210
    let name = name.unwrap_or_else(|| repo.config.author_name.clone());
160 211
    let email = email.unwrap_or_else(|| repo.config.author_email.clone());
161 212
    let scope = if local {
…
121 unchanged lines
283 334
            );
284 335
        }
285 336
    }
337 +
    // A key with a successor is stale, and a key with two is a fork: both
338 +
    // are how an identity splits without anyone noticing (oxygen#163).
339 +
    let next = repo.successors(&key)?;
340 +
    if let Some(n) = next.first() {
341 +
        println!(
342 +
            "  {} {}",
343 +
            ui::caution(),
344 +
            ui::warn(format!(
345 +
                "this key is not your identity's newest: {} succeeds it, {} is the newest. Rotating or delegating from it would fork the identity; point author.key at the newest",
346 +
                n.short(12),
347 +
                repo.newest(n)?.short(12)
348 +
            ))
349 +
        );
350 +
    }
351 +
    let mut cur = Some(key);
352 +
    for _ in 0..256 {
353 +
        let Some(k) = cur else { break };
354 +
        let after = repo.successors(&k)?;
355 +
        if after.len() > 1 {
356 +
            println!(
357 +
                "  {} {}",
358 +
                ui::caution(),
359 +
                ui::err(format!(
360 +
                    "the identity forks at {}: {} each succeed it. Keep one branch: revoke the others' keys (oxy identity revoke KEY) and point every home's author.key at the one you keep",
361 +
                    k.short(12),
362 +
                    after.iter().map(|a| a.short(12)).collect::<Vec<_>>().join(" and ")
363 +
                ))
364 +
            );
365 +
            break;
366 +
        }
367 +
        cur = repo.identity(&k).ok().and_then(|i| i.previous);
368 +
    }
286 369
    print_delegates(&repo, &id.delegates);
287 370
    if repo.is_revoked(&key)? {
288 371
        println!(
…
118 unchanged lines
407 490
    Ok(if bad > 0 { 1 } else { 0 })
408 491
}
409 492
493 +
/// Whether a configuration key holds a secret, never to be printed.
494 +
fn secret_key(key: &str) -> bool {
495 +
    let k = key.to_ascii_lowercase();
496 +
    k.ends_with(".token") || k.contains("secret") || k.contains("password")
497 +
}
498 +
499 +
/// A secret as a confirmation may show it (oxygen#113): enough of its start
500 +
/// to tell which one it is, never the rest.
501 +
fn mask(value: &str) -> String {
502 +
    let shown: String = value.chars().take(10).collect();
503 +
    let rest = value.chars().count().saturating_sub(10);
504 +
    if rest == 0 {
505 +
        "…".to_string()
506 +
    } else {
507 +
        format!("{shown}…({rest} more)")
508 +
    }
509 +
}
510 +
410 511
pub fn config(key: &str, value: &str, user: bool, unset: bool) -> Result<i32> {
411 512
    // An empty value removes the key, which is what --unset says plainly.
412 -
    let value = if unset { "" } else { value };
513 +
    // `-` reads the value from standard input, so a token never sits in
514 +
    // the shell's history (oxygen#113).
515 +
    let read;
516 +
    let value = if unset {
517 +
        ""
518 +
    } else if value == "-" {
519 +
        let mut line = String::new();
520 +
        std::io::stdin().read_line(&mut line)?;
521 +
        read = line.trim().to_string();
522 +
        read.as_str()
523 +
    } else {
524 +
        value
525 +
    };
413 526
    let said = if unset {
414 527
        format!("{} {key}", ui::done("Unset"))
528 +
    } else if secret_key(key) {
529 +
        format!("{} {key} = {}", ui::done("Set"), mask(value))
415 530
    } else {
416 531
        format!("{} {key} = {value}", ui::done("Set"))
417 532
    };
…
10 unchanged lines
428 543
    Ok(0)
429 544
}
430 545
431 -
pub fn export(path: Option<PathBuf>) -> Result<i32> {
546 +
pub fn export(path: Option<PathBuf>, pem: bool) -> Result<i32> {
432 547
    let repo = open()?;
433 -
    let Some(key) = repo.config.author_key else {
548 +
    // The PEM is for checking what this home signs, so it is the signing
549 +
    // key's: a delegate's when `author.signer` names one.
550 +
    let key = if pem {
551 +
        repo.signing_key()
552 +
    } else {
553 +
        repo.config.author_key
554 +
    };
555 +
    let Some(key) = key else {
434 556
        anyhow::bail!("no identity configured; run `oxy identity create`");
435 557
    };
436 558
    let id = repo.identity(&key)?;
437 -
    let path = path.unwrap_or_else(|| PathBuf::from(format!("{}.identity", key.short(16))));
438 -
    std::fs::write(&path, id.encode())?;
559 +
    let ext = if pem { "pem" } else { "identity" };
560 +
    let path = path.unwrap_or_else(|| PathBuf::from(format!("{}.{ext}", key.short(16))));
561 +
    if pem {
562 +
        std::fs::write(&path, oxy_core::dsse::public_pem(&id))?;
563 +
    } else {
564 +
        std::fs::write(&path, id.encode())?;
565 +
    }
439 566
    println!(
440 -
        "{} identity {} <{}> ({}) to {}",
567 +
        "{} {} {} <{}> ({}) to {}",
441 568
        ui::done("Wrote"),
569 +
        if pem { "the public key of" } else { "identity" },
442 570
        ui::author(&id.name),
443 571
        id.email,
444 572
        key.short(12),
…
238 unchanged lines
683 811
                            ("threshold", json::n(rule.threshold)),
684 812
                            ("require", json::s(rule.require.name())),
685 813
                            ("allow_conflicts", json::b(rule.allow_conflicts)),
814 +
                            (
815 +
                                "subjects",
816 +
                                json::arr(rule.subjects.iter().map(|g| {
817 +
                                    json::obj([
818 +
                                        ("subject", json::s(&g.pattern)),
819 +
                                        ("issuer", ident(&g.issuer)),
820 +
                                        ("scope", json::s(&g.scope.to_string())),
821 +
                                    ])
822 +
                                })),
823 +
                            ),
686 824
                        ])
687 825
                    })),
688 826
                ),
…
57 unchanged lines
746 884
        for k in &rule.keys {
747 885
            println!("      {}  {}", ui::hash(k.short(12)), ui::author(name(k)));
748 886
        }
887 +
        for g in &rule.subjects {
888 +
            println!(
889 +
                "      {}  {} {}",
890 +
                ui::author(&g.pattern),
891 +
                ui::dim(format!(
892 +
                    "certified by {} ({}),",
893 +
                    name(&g.issuer),
894 +
                    g.issuer.short(12)
895 +
                )),
896 +
                ui::dim(format!("scope {}", g.scope))
897 +
            );
898 +
        }
749 899
    }
750 900
    if !p.revoked.is_empty() {
751 901
        println!("{}", ui::section("revoked"));
…
30 unchanged lines
782 932
    Ok(0)
783 933
}
784 934
935 +
pub fn policy_grant_subject(
936 +
    subject: &str,
937 +
    issuer_file: &Path,
938 +
    scope: &str,
939 +
    pattern: Option<&str>,
940 +
    out: Option<PathBuf>,
941 +
) -> Result<i32> {
942 +
    let repo = open()?;
943 +
    let issuer = Identity::decode(&std::fs::read(issuer_file)?)
944 +
        .map_err(|e| anyhow::anyhow!("{}: not an identity file: {e}", issuer_file.display()))?;
945 +
    let scope = oxy_core::identity::Scope::parse(scope).map_err(|e| anyhow::anyhow!(e))?;
946 +
    let key = issuer.hash();
947 +
    let u = repo.policy_grant_subject(subject, issuer.clone(), scope, pattern)?;
948 +
    println!(
949 +
        "{} {} certified by {} ({}) on {}, scope {}",
950 +
        ui::done("Granted"),
951 +
        ui::author(subject),
952 +
        issuer.name,
953 +
        key.short(12),
954 +
        ui::bookmark(pattern.unwrap_or("*")),
955 +
        scope
956 +
    );
957 +
    report_update(&u, out)?;
958 +
    Ok(0)
959 +
}
960 +
785 961
pub fn policy_revoke(key: &str, out: Option<PathBuf>) -> Result<i32> {
786 962
    let repo = open()?;
787 963
    if repo.current_policy()?.is_none() {
…
102 unchanged lines
890 1066
fn hex_string(b: &[u8]) -> String {
891 1067
    b.iter().map(|x| format!("{x:02x}")).collect()
892 1068
}
1069 +
1070 +
#[cfg(test)]
1071 +
mod tests {
1072 +
    #[test]
1073 +
    fn secrets_are_never_printed_whole() {
1074 +
        assert!(
1075 +
            super::secret_key("remote.earth.token")
1076 +
                && super::secret_key("auth.secret")
1077 +
                && !super::secret_key("remote.earth.url")
1078 +
        );
1079 +
        assert_eq!(
1080 +
            super::mask("oxyhub_6b7dce13409b634d"),
1081 +
            "oxyhub_6b7…(13 more)"
1082 +
        );
1083 +
        assert_eq!(super::mask("short"), "…");
1084 +
    }
1085 +
}
added src/cli/mcp.rs +543 −0
1 +
//! `oxy mcp` (oxygen#159): a Model Context Protocol server over stdio for
2 +
//! the repository it starts in, so an agent works through typed tools
3 +
//! instead of parsing a shell's output.
4 +
//!
5 +
//! Each tool runs this same oxy binary with the command's `--json`, in the
6 +
//! repository, with the server's environment: so a tool returns exactly
7 +
//! what the command does, and signs as whatever `OXY_HOME` configures. In
8 +
//! an agent home every change is sealed via the agent, with no environment
9 +
//! variable to forget per call. Tools that can lose work (undo, a forced
10 +
//! push) are offered only with `--allow-destructive`.
11 +
12 +
use anyhow::{bail, Result};
13 +
use oxy_core::mcp::{self, arg_str, arg_strings, ServerInfo, Tool, ToolOutput, Tools};
14 +
use serde_json::{json, Value};
15 +
use std::path::{Path, PathBuf};
16 +
use std::process::{Command, Stdio};
17 +
18 +
/// A tool's text is cut here; its structured result is then left out.
19 +
const MAX_TEXT: usize = 256 * 1024;
20 +
21 +
type Build = fn(&Value, bool) -> Result<Vec<String>, String>;
22 +
23 +
struct Spec {
24 +
    name: &'static str,
25 +
    description: &'static str,
26 +
    schema: fn() -> Value,
27 +
    build: Build,
28 +
    json: bool,
29 +
    read_only: bool,
30 +
    destructive: bool,
31 +
}
32 +
33 +
fn obj(props: Value, required: &[&str]) -> Value {
34 +
    json!({ "type": "object", "properties": props, "required": required, "additionalProperties": false })
35 +
}
36 +
37 +
fn none() -> Value {
38 +
    obj(json!({}), &[])
39 +
}
40 +
41 +
/// An option's value as `--flag=value`, so a value starting with `-` is
42 +
/// never read as another flag.
43 +
fn opt(flag: &str, value: &str) -> String {
44 +
    format!("{flag}={value}")
45 +
}
46 +
47 +
/// Positional values after `--`, for the same reason.
48 +
fn positional(out: &mut Vec<String>, values: Vec<String>) {
49 +
    if !values.is_empty() {
50 +
        out.push("--".into());
51 +
        out.extend(values);
52 +
    }
53 +
}
54 +
55 +
fn required<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
56 +
    arg_str(args, key)
57 +
        .filter(|s| !s.trim().is_empty())
58 +
        .ok_or_else(|| format!("{key} is required"))
59 +
}
60 +
61 +
/// Paths to act on, or every path when `all` is true; never neither, so a
62 +
/// commit or amend is always a deliberate choice of what goes in.
63 +
fn paths_or_all(args: &Value) -> Result<Vec<String>, String> {
64 +
    let paths = arg_strings(args, "paths")?;
65 +
    let all = args.get("all").and_then(Value::as_bool).unwrap_or(false);
66 +
    match (paths.is_empty(), all) {
67 +
        (false, false) => Ok(paths),
68 +
        (true, true) => Ok(vec![]),
69 +
        (false, true) => Err("give paths or all: true, not both".into()),
70 +
        (true, false) => {
71 +
            Err("give the paths to include, or all: true for every changed path".into())
72 +
        }
73 +
    }
74 +
}
75 +
76 +
fn limit(args: &Value, default: u64) -> String {
77 +
    args.get("limit")
78 +
        .and_then(Value::as_u64)
79 +
        .unwrap_or(default)
80 +
        .clamp(1, 1000)
81 +
        .to_string()
82 +
}
83 +
84 +
fn specs() -> Vec<Spec> {
85 +
    vec![
86 +
        Spec {
87 +
            name: "status",
88 +
            description: "The working copy: the bookmark, changed paths, and every conflict with its kind and the changes behind it.",
89 +
            schema: none,
90 +
            build: |_, _| Ok(vec!["status".into(), "--json".into()]),
91 +
            json: true,
92 +
            read_only: true,
93 +
            destructive: false,
94 +
        },
95 +
        Spec {
96 +
            name: "diff",
97 +
            description: "Uncommitted edits in the working copy, or one change's edits when `change` is given (a ChangeId or #hash prefix).",
98 +
            schema: || obj(json!({ "change": { "type": "string" } }), &[]),
99 +
            build: |a, _| {
100 +
                let mut v = vec!["diff".into(), "--json".into()];
101 +
                positional(&mut v, arg_str(a, "change").map(|c| vec![c.to_string()]).unwrap_or_default());
102 +
                Ok(v)
103 +
            },
104 +
            json: true,
105 +
            read_only: true,
106 +
            destructive: false,
107 +
        },
108 +
        Spec {
109 +
            name: "log",
110 +
            description: "Changes, newest first: a revset (`main`, `@`, …) or the current bookmark, optionally only those touching `path`; `limit` defaults to 20.",
111 +
            schema: || obj(json!({ "revset": { "type": "string" }, "path": { "type": "string" }, "limit": { "type": "integer", "minimum": 1 } }), &[]),
112 +
            build: |a, _| {
113 +
                let mut v = vec!["log".into(), "--json".into(), opt("--limit", &limit(a, 20))];
114 +
                if let Some(r) = arg_str(a, "revset") {
115 +
                    v.push(opt("--revset", r));
116 +
                }
117 +
                if let Some(p) = arg_str(a, "path") {
118 +
                    v.push(opt("--path", p));
119 +
                }
120 +
                Ok(v)
121 +
            },
122 +
            json: true,
123 +
            read_only: true,
124 +
            destructive: false,
125 +
        },
126 +
        Spec {
127 +
            name: "show",
128 +
            description: "One change: its message, author, signatures and edits (`stat: true` for files and line counts only). `change` defaults to @, the working copy.",
129 +
            schema: || obj(json!({ "change": { "type": "string" }, "stat": { "type": "boolean" } }), &[]),
130 +
            build: |a, _| {
131 +
                let mut v = vec!["show".into(), "--json".into()];
132 +
                if a.get("stat").and_then(Value::as_bool).unwrap_or(false) {
133 +
                    v.push("--stat".into());
134 +
                }
135 +
                positional(&mut v, vec![arg_str(a, "change").unwrap_or("@").to_string()]);
136 +
                Ok(v)
137 +
            },
138 +
            json: true,
139 +
            read_only: true,
140 +
            destructive: false,
141 +
        },
142 +
        Spec {
143 +
            name: "cat",
144 +
            description: "A file's content as a bookmark, tag or change has it (`change`), or as committed on the working copy's base.",
145 +
            schema: || obj(json!({ "path": { "type": "string" }, "change": { "type": "string" } }), &["path"]),
146 +
            build: |a, _| {
147 +
                let mut v = vec!["cat".into()];
148 +
                if let Some(c) = arg_str(a, "change") {
149 +
                    v.push(opt("--change", c));
150 +
                }
151 +
                positional(&mut v, vec![required(a, "path")?.to_string()]);
152 +
                Ok(v)
153 +
            },
154 +
            json: false,
155 +
            read_only: true,
156 +
            destructive: false,
157 +
        },
158 +
        Spec {
159 +
            name: "blame",
160 +
            description: "Which change wrote each line of a file, with its line atoms.",
161 +
            schema: || obj(json!({ "path": { "type": "string" } }), &["path"]),
162 +
            build: |a, _| {
163 +
                let mut v = vec!["blame".into(), "--json".into()];
164 +
                positional(&mut v, vec![required(a, "path")?.to_string()]);
165 +
                Ok(v)
166 +
            },
167 +
            json: true,
168 +
            read_only: true,
169 +
            destructive: false,
170 +
        },
171 +
        Spec {
172 +
            name: "bookmarks",
173 +
            description: "Bookmarks with their heads and how far each is ahead of or behind its remote; `all: true` adds remotes' bookmarks.",
174 +
            schema: || obj(json!({ "all": { "type": "boolean" } }), &[]),
175 +
            build: |a, _| {
176 +
                let mut v = vec!["bookmark".into(), "list".into(), "--json".into()];
177 +
                if a.get("all").and_then(Value::as_bool).unwrap_or(false) {
178 +
                    v.push("--all".into());
179 +
                }
180 +
                Ok(v)
181 +
            },
182 +
            json: true,
183 +
            read_only: true,
184 +
            destructive: false,
185 +
        },
186 +
        Spec {
187 +
            name: "tags",
188 +
            description: "Tags with their states, messages and signers.",
189 +
            schema: none,
190 +
            build: |_, _| Ok(vec!["tag".into(), "list".into(), "--json".into()]),
191 +
            json: true,
192 +
            read_only: true,
193 +
            destructive: false,
194 +
        },
195 +
        Spec {
196 +
            name: "conflicts",
197 +
            description: "Which conflicts of the working copy's base the current files resolve, and which remain.",
198 +
            schema: none,
199 +
            build: |_, _| Ok(vec!["resolve".into(), "--json".into()]),
200 +
            json: true,
201 +
            read_only: true,
202 +
            destructive: false,
203 +
        },
204 +
        Spec {
205 +
            name: "op_log",
206 +
            description: "The operation log, newest first (what undo would take back); `limit` defaults to 20.",
207 +
            schema: || obj(json!({ "limit": { "type": "integer", "minimum": 1 } }), &[]),
208 +
            build: |a, _| Ok(vec!["op".into(), "log".into(), "--json".into(), opt("--limit", &limit(a, 20))]),
209 +
            json: true,
210 +
            read_only: true,
211 +
            destructive: false,
212 +
        },
213 +
        Spec {
214 +
            name: "identity",
215 +
            description: "Who changes are signed as here: the author, the key, delegates and the signer (an agent home signs keyless, via the agent).",
216 +
            schema: none,
217 +
            build: |_, _| Ok(vec!["identity".into(), "show".into()]),
218 +
            json: false,
219 +
            read_only: true,
220 +
            destructive: false,
221 +
        },
222 +
        Spec {
223 +
            name: "commit",
224 +
            description: "Record a change with a message: only `paths`, or every changed path with `all: true`. Signed as this server's identity.",
225 +
            schema: || obj(json!({ "message": { "type": "string" }, "paths": { "type": "array", "items": { "type": "string" } }, "all": { "type": "boolean" } }), &["message"]),
226 +
            build: |a, _| {
227 +
                let mut v = vec!["commit".into(), opt("--message", required(a, "message")?)];
228 +
                positional(&mut v, paths_or_all(a)?);
229 +
                Ok(v)
230 +
            },
231 +
            json: false,
232 +
            read_only: false,
233 +
            destructive: false,
234 +
        },
235 +
        Spec {
236 +
            name: "amend",
237 +
            description: "Fold edits into the change the working copy sits on: only `paths`, or all with `all: true`; `message` replaces its message too.",
238 +
            schema: || obj(json!({ "message": { "type": "string" }, "paths": { "type": "array", "items": { "type": "string" } }, "all": { "type": "boolean" } }), &[]),
239 +
            build: |a, _| {
240 +
                let mut v = vec!["amend".into()];
241 +
                if let Some(m) = arg_str(a, "message") {
242 +
                    v.push(opt("--message", m));
243 +
                }
244 +
                positional(&mut v, paths_or_all(a)?);
245 +
                Ok(v)
246 +
            },
247 +
            json: false,
248 +
            read_only: false,
249 +
            destructive: false,
250 +
        },
251 +
        Spec {
252 +
            name: "describe",
253 +
            description: "Change a head's message (`change` defaults to the one the working copy sits on).",
254 +
            schema: || obj(json!({ "message": { "type": "string" }, "change": { "type": "string" } }), &["message"]),
255 +
            build: |a, _| {
256 +
                let mut v = vec!["describe".into(), "--json".into(), opt("--message", required(a, "message")?)];
257 +
                positional(&mut v, arg_str(a, "change").map(|c| vec![c.to_string()]).unwrap_or_default());
258 +
                Ok(v)
259 +
            },
260 +
            json: true,
261 +
            read_only: false,
262 +
            destructive: false,
263 +
        },
264 +
        Spec {
265 +
            name: "fetch",
266 +
            description: "Fetch from a remote (default origin) without moving local bookmarks.",
267 +
            schema: || obj(json!({ "remote": { "type": "string" } }), &[]),
268 +
            build: |a, _| remote_cmd("fetch", a),
269 +
            json: true,
270 +
            read_only: false,
271 +
            destructive: false,
272 +
        },
273 +
        Spec {
274 +
            name: "pull",
275 +
            description: "Fetch from a remote (default origin) and bring the current bookmark and working copy up to date.",
276 +
            schema: || obj(json!({ "remote": { "type": "string" } }), &[]),
277 +
            build: |a, _| remote_cmd("pull", a),
278 +
            json: true,
279 +
            read_only: false,
280 +
            destructive: false,
281 +
        },
282 +
        Spec {
283 +
            name: "push",
284 +
            description: "Push the current bookmark (or `bookmark`) to a remote (default origin). Changes are sealed at push as this server's identity. `force` (drop heads the remote has) needs the server started with --allow-destructive.",
285 +
            schema: || obj(json!({ "remote": { "type": "string" }, "bookmark": { "type": "string" }, "force": { "type": "boolean" } }), &[]),
286 +
            build: |a, destructive| {
287 +
                let mut v = vec!["push".to_string(), "--json".into()];
288 +
                if let Some(b) = arg_str(a, "bookmark") {
289 +
                    v.push(opt("--bookmark", b));
290 +
                }
291 +
                if a.get("force").and_then(Value::as_bool).unwrap_or(false) {
292 +
                    if !destructive {
293 +
                        return Err("a forced push can drop the remote's heads; start oxy mcp with --allow-destructive to allow it".into());
294 +
                    }
295 +
                    v.push("--force".into());
296 +
                }
297 +
                positional(&mut v, arg_str(a, "remote").map(|r| vec![r.to_string()]).unwrap_or_default());
298 +
                Ok(v)
299 +
            },
300 +
            json: true,
301 +
            read_only: false,
302 +
            destructive: false,
303 +
        },
304 +
        Spec {
305 +
            name: "tlog_verify",
306 +
            description: "Check the hub's transparency log: that it shows everyone the same history, with this repository's entries and each witness's cosignature.",
307 +
            schema: none,
308 +
            build: |_, _| Ok(vec!["tlog".into(), "verify".into(), "--json".into()]),
309 +
            json: true,
310 +
            read_only: true,
311 +
            destructive: false,
312 +
        },
313 +
        Spec {
314 +
            name: "undo",
315 +
            description: "Take back the last operation (see op_log). Offered only when the server was started with --allow-destructive.",
316 +
            schema: none,
317 +
            build: |_, _| Ok(vec!["undo".into()]),
318 +
            json: false,
319 +
            read_only: false,
320 +
            destructive: true,
321 +
        },
322 +
    ]
323 +
}
324 +
325 +
fn remote_cmd(command: &str, a: &Value) -> Result<Vec<String>, String> {
326 +
    let mut v = vec![command.to_string(), "--json".into()];
327 +
    positional(
328 +
        &mut v,
329 +
        arg_str(a, "remote")
330 +
            .map(|r| vec![r.to_string()])
331 +
            .unwrap_or_default(),
332 +
    );
333 +
    Ok(v)
334 +
}
335 +
336 +
struct OxyTools {
337 +
    exe: PathBuf,
338 +
    dir: PathBuf,
339 +
    allow_destructive: bool,
340 +
}
341 +
342 +
impl OxyTools {
343 +
    fn offered(&self) -> impl Iterator<Item = Spec> + '_ {
344 +
        specs()
345 +
            .into_iter()
346 +
            .filter(|s| self.allow_destructive || !s.destructive)
347 +
    }
348 +
349 +
    fn run(&self, spec: &Spec, argv: Vec<String>) -> ToolOutput {
350 +
        let out = Command::new(&self.exe)
351 +
            .arg("--plain")
352 +
            .args(&argv)
353 +
            .current_dir(&self.dir)
354 +
            .stdin(Stdio::null())
355 +
            .output();
356 +
        let out = match out {
357 +
            Ok(o) => o,
358 +
            Err(e) => return ToolOutput::error(format!("could not run oxy: {e}")),
359 +
        };
360 +
        let stdout = String::from_utf8_lossy(&out.stdout).into_owned();
361 +
        let stderr = String::from_utf8_lossy(&out.stderr).into_owned();
362 +
        if !out.status.success() {
363 +
            let why = if stderr.trim().is_empty() {
364 +
                stdout
365 +
            } else {
366 +
                stderr
367 +
            };
368 +
            return ToolOutput::error(format!("oxy {} failed: {}", argv.join(" "), why.trim()));
369 +
        }
370 +
        let mut result = ToolOutput::ok(stdout.clone());
371 +
        if spec.json {
372 +
            result.structured = serde_json::from_str(&stdout).ok();
373 +
        } else if !stderr.trim().is_empty() && stdout.trim().is_empty() {
374 +
            // Commands that report on stderr (commit's summary) still say what they did.
375 +
            result.text = stderr;
376 +
        }
377 +
        if result.text.len() > MAX_TEXT {
378 +
            let mut cut = MAX_TEXT;
379 +
            while !result.text.is_char_boundary(cut) {
380 +
                cut -= 1;
381 +
            }
382 +
            result.text.truncate(cut);
383 +
            result.text.push_str(
384 +
                "\n… (cut at 256 KiB: narrow the request, e.g. a smaller limit or one path)",
385 +
            );
386 +
            result.structured = None;
387 +
        }
388 +
        result
389 +
    }
390 +
}
391 +
392 +
impl Tools for OxyTools {
393 +
    fn list(&self) -> Vec<Tool> {
394 +
        self.offered()
395 +
            .map(|s| Tool {
396 +
                name: s.name.into(),
397 +
                description: s.description.into(),
398 +
                input_schema: (s.schema)(),
399 +
                read_only: s.read_only,
400 +
                destructive: s.destructive,
401 +
            })
402 +
            .collect()
403 +
    }
404 +
405 +
    fn call(&self, name: &str, args: &Value) -> Option<ToolOutput> {
406 +
        let spec = self.offered().find(|s| s.name == name)?;
407 +
        Some(match (spec.build)(args, self.allow_destructive) {
408 +
            Ok(argv) => self.run(&spec, argv),
409 +
            Err(why) => ToolOutput::error(why),
410 +
        })
411 +
    }
412 +
}
413 +
414 +
/// The first line of `oxy identity show`, said in `initialize` so the client
415 +
/// knows who its changes will be signed as.
416 +
fn signer_line(exe: &Path, dir: &Path) -> String {
417 +
    let out = Command::new(exe)
418 +
        .args(["--plain", "identity", "show"])
419 +
        .current_dir(dir)
420 +
        .stdin(Stdio::null())
421 +
        .output();
422 +
    match out {
423 +
        Ok(o) if o.status.success() => {
424 +
            let text = String::from_utf8_lossy(&o.stdout);
425 +
            let who = text.lines().next().unwrap_or("").trim_start_matches(['#', '▍', ' ']).trim().to_string();
426 +
            let signing = text.lines().find(|l| l.trim_start().starts_with("signing")).map(|l| l.trim().to_string());
427 +
            match signing {
428 +
                Some(s) => format!("{who}; {s}"),
429 +
                None => who,
430 +
            }
431 +
        }
432 +
        _ => "no identity is configured: commits will be refused until one is (oxy identity create, or an agent home with OXY_HOME)".into(),
433 +
    }
434 +
}
435 +
436 +
pub fn serve(dir: Option<PathBuf>, allow_destructive: bool) -> Result<i32> {
437 +
    let dir = match dir {
438 +
        Some(d) => d,
439 +
        None => std::env::current_dir()?,
440 +
    };
441 +
    if !dir.join(oxy_core::repo::OXY_DIR).is_dir() {
442 +
        bail!("{} is not the root of an Oxygen repository (no {}): start oxy mcp in a checkout's root, or name it", dir.display(), oxy_core::repo::OXY_DIR);
443 +
    }
444 +
    let exe = std::env::current_exe()?;
445 +
    let signer = signer_line(&exe, &dir);
446 +
    let info = ServerInfo {
447 +
        name: "oxy".into(),
448 +
        version: env!("CARGO_PKG_VERSION").into(),
449 +
        instructions: format!(
450 +
            "Oxygen version control for the repository at {}. Changes are signed as: {signer}. Prefer these tools to running oxy in a shell. \
451 +
             commit and amend take explicit paths (or all: true). {}",
452 +
            dir.display(),
453 +
            if allow_destructive { "Destructive tools (undo, forced push) are enabled: use them only when asked." } else { "Destructive tools (undo, forced push) are off." }
454 +
        ),
455 +
    };
456 +
    let tools = OxyTools {
457 +
        exe,
458 +
        dir,
459 +
        allow_destructive,
460 +
    };
461 +
    let stdin = std::io::stdin();
462 +
    mcp::serve(&info, &tools, stdin.lock(), std::io::stdout().lock())?;
463 +
    Ok(0)
464 +
}
465 +
466 +
#[cfg(test)]
467 +
mod tests {
468 +
    use super::*;
469 +
470 +
    fn build(name: &str, args: Value, destructive: bool) -> Result<Vec<String>, String> {
471 +
        let spec = specs().into_iter().find(|s| s.name == name).unwrap();
472 +
        (spec.build)(&args, destructive)
473 +
    }
474 +
475 +
    #[test]
476 +
    fn values_never_read_as_flags() {
477 +
        assert_eq!(
478 +
            build(
479 +
                "commit",
480 +
                json!({"message": "-m oops", "paths": ["-rf"]}),
481 +
                false
482 +
            )
483 +
            .unwrap(),
484 +
            ["commit", "--message=-m oops", "--", "-rf"]
485 +
        );
486 +
        assert_eq!(
487 +
            build("cat", json!({"path": "--help", "change": "main"}), false).unwrap(),
488 +
            ["cat", "--change=main", "--", "--help"]
489 +
        );
490 +
        assert_eq!(
491 +
            build("log", json!({"limit": 5000}), false).unwrap(),
492 +
            ["log", "--json", "--limit=1000"],
493 +
            "limit is capped"
494 +
        );
495 +
    }
496 +
497 +
    #[test]
498 +
    fn commits_name_what_goes_in() {
499 +
        assert!(build("commit", json!({"message": "x"}), false)
500 +
            .unwrap_err()
501 +
            .contains("all: true"));
502 +
        assert!(build(
503 +
            "commit",
504 +
            json!({"message": "x", "paths": ["a"], "all": true}),
505 +
            false
506 +
        )
507 +
        .is_err());
508 +
        assert_eq!(
509 +
            build("commit", json!({"message": "x", "all": true}), false).unwrap(),
510 +
            ["commit", "--message=x"]
511 +
        );
512 +
        assert!(build("commit", json!({"message": " ", "all": true}), false)
513 +
            .unwrap_err()
514 +
            .contains("message"));
515 +
    }
516 +
517 +
    #[test]
518 +
    fn destructive_tools_need_the_flag() {
519 +
        let off = OxyTools {
520 +
            exe: "oxy".into(),
521 +
            dir: ".".into(),
522 +
            allow_destructive: false,
523 +
        };
524 +
        let on = OxyTools {
525 +
            exe: "oxy".into(),
526 +
            dir: ".".into(),
527 +
            allow_destructive: true,
528 +
        };
529 +
        assert!(!off.list().iter().any(|t| t.name == "undo"));
530 +
        assert!(
531 +
            off.call("undo", &json!({})).is_none(),
532 +
            "an unoffered tool is unknown"
533 +
        );
534 +
        assert!(on.list().iter().any(|t| t.name == "undo" && t.destructive));
535 +
        assert!(build("push", json!({"force": true}), false)
536 +
            .unwrap_err()
537 +
            .contains("--allow-destructive"));
538 +
        assert_eq!(
539 +
            build("push", json!({"force": true, "remote": "earth"}), true).unwrap(),
540 +
            ["push", "--json", "--force", "--", "earth"]
541 +
        );
542 +
    }
543 +
}
modified src/cli/mod.rs +394 −26
1 1
//! Command-line definitions and dispatch.
2 2
3 +
mod approve;
4 +
mod artifact;
3 5
mod auth;
4 6
mod format;
5 7
mod identity;
6 8
mod json;
9 +
mod mcp;
7 10
mod proposal;
8 11
mod receipt;
9 12
mod sync;
13 +
mod tlog;
10 14
pub mod ui;
11 15
mod workspace;
12 16
…
31 unchanged lines
44 48
        /// No working copy: an object store and references only, for hosting
45 49
        #[arg(long)]
46 50
        bare: bool,
51 +
        /// Keep the objects and the operation log on this store
52 +
        /// (dir://PATH, s3://BUCKET/PREFIX, gs://…, az://…) instead of
53 +
        /// in .oxy, with a local cache in front (storage.cache, 2 GiB)
54 +
        #[arg(long, value_name = "URL")]
55 +
        store: Option<String>,
47 56
    },
48 57
    /// Clone a repository from a local path or an http(s) URL
49 58
    Clone {
…
102 unchanged lines
152 161
    },
153 162
    /// List changes reachable from the bookmark and the working copy
154 163
    Log {
155 -
        /// Show the changes a revset selects: `main..feature & author(andy)`,
156 -
        /// `touches("src/**") & date(-7d..)` (see docs/revset_query_language.md)
164 +
        /// Show the changes a revset selects: `main` (a bookmark name means its
165 +
        /// heads; `bookmark(main)` only ever the bookmark), `main..feature &
166 +
        /// author(andy)`, `touches("src/**") & date(-7d..)` (see
167 +
        /// docs/revset_query_language.md)
157 168
        #[arg(short = 'r', long, value_name = "REVSET", conflicts_with_all = ["path", "lines"])]
158 169
        revset: Option<String>,
159 170
        /// Show at most this many changes
…
26 unchanged lines
186 197
    Cat {
187 198
        /// File to print, relative to the repository root
188 199
        path: String,
189 -
        /// A change, bookmark, or tag
200 +
        /// A bookmark or tag (its state), or a change: a change means its
201 +
        /// dependency closure, the change and what it depends on, so files
202 +
        /// only later changes touched are absent or at older versions
190 203
        #[arg(short = 'r', long)]
191 204
        change: Option<String>,
192 205
    },
…
3 unchanged lines
196 209
        #[arg(long)]
197 210
        json: bool,
198 211
    },
212 +
    /// Serve this repository to an AI agent over the Model Context Protocol (stdio), signing as this checkout's identity
213 +
    Mcp {
214 +
        /// The repository's root (default: the current directory)
215 +
        dir: Option<PathBuf>,
216 +
        /// Also offer tools that can lose work: undo, and a forced push
217 +
        #[arg(long)]
218 +
        allow_destructive: bool,
219 +
    },
199 220
    /// Print the version of oxy, its repository format, and protocol
200 221
    Version {
201 222
        /// Machine-readable output: one JSON document on stdout
…
33 unchanged lines
235 256
        #[arg(short, long)]
236 257
        message: Option<String>,
237 258
        /// Amend an older change; the edits must touch only lines it already knows
238 -
        #[arg(long)]
259 +
        #[arg(long, conflicts_with = "paths")]
239 260
        into: Option<String>,
261 +
        /// Fold in only these paths; the rest stays as uncommitted work
262 +
        paths: Vec<String>,
240 263
    },
241 264
    /// Split a head: the named paths become a separate first change
242 265
    Split {
…
22 unchanged lines
265 288
        /// Show every change a revset selects
266 289
        #[arg(short = 'r', long, value_name = "REVSET", conflicts_with = "change")]
267 290
        revset: Option<String>,
291 +
        /// The files it touched with lines added and removed, instead of the diff
292 +
        #[arg(long)]
293 +
        stat: bool,
268 294
        /// Machine-readable output: one JSON document on stdout
269 295
        #[arg(long)]
270 296
        json: bool,
…
10 unchanged lines
281 307
        #[arg(long, conflicts_with = "discard")]
282 308
        carry: bool,
283 309
    },
284 -
    /// Report which conflicts of the working copy's base are resolved by the current files
310 +
    /// Report which conflicts of the working copy's base are resolved by the current files;
311 +
    /// with paths, settle their replace conflicts (binary files, subrepo pins) with what the
312 +
    /// working copy holds there
285 313
    Resolve {
314 +
        /// Opaque files or subrepos whose replace conflict to settle with their current content;
315 +
        /// with --both, the text files whose conflicts to settle (all of them when none)
316 +
        paths: Vec<String>,
317 +
        /// Settle each adjacent insertion (two changes adding lines at the same place) by keeping
318 +
        /// every side, earliest change first; refused where the sides changed the same lines
319 +
        #[arg(long)]
320 +
        both: bool,
321 +
        /// With --both, the latest change first
322 +
        #[arg(long, requires = "both")]
323 +
        reverse: bool,
286 324
        /// Machine-readable output: one JSON document on stdout
287 325
        #[arg(long)]
288 326
        json: bool,
…
33 unchanged lines
322 360
        /// Ignore the grace period (core.gc-grace) and prune immediately
323 361
        #[arg(long)]
324 362
        now: bool,
363 +
        /// Keep no past working-copy snapshots, only each workspace's current one, so files only they held go (core.snapshot-retention, default 7 days)
364 +
        #[arg(long)]
365 +
        expire_snapshots: bool,
325 366
        /// Instead: fold the bundles of a remote kept on a store into one, and remove the ones a gc superseded 7 days ago (or now, with --now)
326 367
        #[arg(long, value_name = "NAME")]
327 368
        remote: Option<String>,
…
24 unchanged lines
352 393
        #[command(subcommand)]
353 394
        cmd: BookmarkCmd,
354 395
    },
355 -
    /// Union another bookmark's heads into the current one (no merge change)
396 +
    /// Union another bookmark's heads into the current one (no merge change):
397 +
    /// a bookmark here, or a remote's (`origin/develop`) as last fetched
356 398
    Merge {
357 399
        /// The bookmark whose heads to add
358 400
        bookmark: String,
…
8 unchanged lines
367 409
        #[command(subcommand)]
368 410
        cmd: SubmoduleCmd,
369 411
    },
412 +
    /// Witness hubs' transparency logs: cosign each checkpoint that extends
413 +
    /// the last one cosigned (spec §14.8)
414 +
    Witness {
415 +
        #[command(subcommand)]
416 +
        cmd: WitnessCmd,
417 +
    },
418 +
    /// Hubs' transparency logs: check that a hub shows everyone the same
419 +
    /// history (spec §14.8)
420 +
    Tlog {
421 +
        #[command(subcommand)]
422 +
        cmd: TlogCmd,
423 +
    },
424 +
    /// Artifacts: a directory packed into a tree of this repository's chunks (Cylinder, docs/cylinder.md)
425 +
    Artifact {
426 +
        #[command(subcommand)]
427 +
        cmd: ArtifactCmd,
428 +
    },
370 429
    /// Execution receipts: signed records of what a runner ran on a state
371 430
    Receipt {
372 431
        #[command(subcommand)]
…
36 unchanged lines
409 468
        #[arg(short = 'b', long)]
410 469
        bookmark: Option<String>,
411 470
        /// Delete this bookmark on the remote (never its default bookmark);
412 -
        /// the local bookmark is kept
471 +
        /// the local bookmark is kept. `tags/NAME` withdraws a tag there
472 +
        /// (and here, as `oxy tag delete` does)
413 473
        #[arg(long, value_name = "NAME", conflicts_with_all = ["bookmark", "force"])]
414 474
        delete: Option<String>,
415 475
        /// Machine-readable output: one JSON document on stdout
…
30 unchanged lines
446 506
        /// Seal every change of a hub proposal and push the seal (your approval)
447 507
        #[arg(long, conflicts_with = "changes")]
448 508
        proposal: Option<String>,
509 +
        /// Approve a job of a hub run that waits for a sealed approval: `--flow https://hub/OWNER/NAME/flow/N --job apply`
510 +
        #[arg(long, conflicts_with_all = ["changes", "proposal"], requires = "job")]
511 +
        flow: Option<String>,
512 +
        /// The job to approve, with --flow
513 +
        #[arg(long, requires = "flow")]
514 +
        job: Option<String>,
515 +
        /// Reject the job instead of approving it, with --flow
516 +
        #[arg(long, requires = "flow")]
517 +
        reject: bool,
449 518
        /// Machine-readable output: one JSON document on stdout
450 519
        #[arg(long)]
451 520
        json: bool,
…
24 unchanged lines
476 545
        #[command(subcommand)]
477 546
        cmd: PolicyCmd,
478 547
    },
479 -
    /// Set a configuration key (author.*, core.sign, core.op-retention, core.gc-grace, core.checkpoint-interval, filter.max-blob, filter.paths, workspace.bookmark, remote.<name>.url|token|ca, auth.helper with --user)
548 +
    /// Set a configuration key (author.*, core.sign, core.op-retention, core.gc-grace, core.checkpoint-interval, filter.max-blob, filter.paths, storage.cache, workspace.bookmark, remote.<name>.url|token|ca, git.claim, git.colocate, auth.helper with --user)
480 549
    Config {
481 550
        /// The key, such as author.name or remote.origin.token
482 551
        key: String,
483 -
        /// The value to set (none with --unset)
552 +
        /// The value to set (none with --unset); `-` reads it from standard input, out of the shell's history
484 553
        #[arg(required_unless_present = "unset")]
485 554
        value: Option<String>,
486 555
        /// Write to the user file (~/.config/oxy/config.toml) instead of this repository
…
100 unchanged lines
587 656
        /// branch at a commit would hold
588 657
        #[arg(long, conflicts_with = "only")]
589 658
        upto: bool,
590 -
        /// Just CHANGE... and what they depend on
659 +
        /// Just CHANGE... and what they depend on. Other changes to the
660 +
        /// same files are left out, so a file can come out partly edited;
661 +
        /// to add one change to a bookmark's work, give the bookmark's
662 +
        /// heads with it instead: `bookmark set NAME <its heads> CHANGE`
591 663
        #[arg(long)]
592 664
        only: bool,
593 665
    },
…
9 unchanged lines
603 675
        /// moved to a commit would hold
604 676
        #[arg(long, conflicts_with = "only")]
605 677
        upto: bool,
606 -
        /// Just CHANGE... and what they depend on
678 +
        /// Just CHANGE... and what they depend on. Other changes to the
679 +
        /// same files are left out, so a file can come out partly edited;
680 +
        /// to add one change to a bookmark's work, give the bookmark's
681 +
        /// heads with it instead: `bookmark set NAME <its heads> CHANGE`
607 682
        #[arg(long)]
608 683
        only: bool,
609 684
    },
…
7 unchanged lines
617 692
        /// Also list remotes' bookmarks that have no local bookmark
618 693
        #[arg(long, short)]
619 694
        all: bool,
695 +
        /// Each head of every bookmark, as `bookmark show` prints them
696 +
        #[arg(long, short)]
697 +
        verbose: bool,
698 +
        /// Machine-readable output: one JSON document on stdout
699 +
        #[arg(long)]
700 +
        json: bool,
701 +
    },
702 +
    /// A bookmark's heads: each change nothing else on it depends on,
703 +
    /// newest first. A bookmark name is also a revset for its heads
704 +
    /// (`oxy log -r NAME`, or `bookmark(NAME)` to mean only the bookmark)
705 +
    Show {
706 +
        /// The bookmark, or REMOTE/NAME (default: the current one)
707 +
        name: Option<String>,
620 708
        /// Machine-readable output: one JSON document on stdout
621 709
        #[arg(long)]
622 710
        json: bool,
…
13 unchanged lines
636 724
}
637 725
638 726
#[derive(Subcommand)]
727 +
enum WitnessCmd {
728 +
    /// Serve a witness (C2SP tlog-witness: POST /add-checkpoint), signing
729 +
    /// with this oxy home's key (author.key)
730 +
    Serve {
731 +
        /// The witness's name in cosignatures: a host name
732 +
        #[arg(long)]
733 +
        name: String,
734 +
        /// A log to witness, as ORIGIN=FILE: its origin (HOST/log) and its
735 +
        /// key's identity file (a hub's /.well-known/oxyhub-key); repeatable
736 +
        #[arg(long = "log", required = true)]
737 +
        logs: Vec<String>,
738 +
        /// Where to keep what it cosigned
739 +
        #[arg(long, default_value = "witness-state")]
740 +
        state: PathBuf,
741 +
        /// Address and port to listen on
742 +
        #[arg(long, default_value = "127.0.0.1:8891")]
743 +
        bind: String,
744 +
        /// PEM certificate chain; enables HTTPS together with --tls-key
745 +
        #[arg(long, requires = "tls_key")]
746 +
        tls_cert: Option<PathBuf>,
747 +
        /// PEM private key for --tls-cert
748 +
        #[arg(long, requires = "tls_cert")]
749 +
        tls_key: Option<PathBuf>,
750 +
    },
751 +
}
752 +
753 +
#[derive(Subcommand)]
754 +
enum TlogCmd {
755 +
    /// Check a hub's whole log: its checkpoint against the log key and the
756 +
    /// last checkpoint seen, and every entry against the root it signed;
757 +
    /// then list this repository's entries (or a key's, or all)
758 +
    Verify {
759 +
        /// The hub, as https://oxygenhub.io (default: this checkout's remote)
760 +
        hub: Option<String>,
761 +
        /// List the entries a key signed (a KeyId) instead
762 +
        #[arg(long, conflicts_with = "all")]
763 +
        key: Option<String>,
764 +
        /// List every entry
765 +
        #[arg(long)]
766 +
        all: bool,
767 +
        /// Machine-readable output
768 +
        #[arg(long)]
769 +
        json: bool,
770 +
    },
771 +
}
772 +
773 +
#[derive(Subcommand)]
639 774
enum AuthCmd {
640 775
    /// Sign in to HUB in the browser (device sign-in) and keep the token for
641 776
    /// every repository there
…
14 unchanged lines
656 791
        /// The hub (default: this checkout's origin, or the only one)
657 792
        hub: Option<String>,
658 793
    },
794 +
    /// Keep an agent grant token (from `oh agent grant`) for keyless
795 +
    /// signing: with `author.signer = keyless`, commits and pushes are
796 +
    /// signed by fresh keys the hub certifies, in the grantor's name
797 +
    /// (the token is read from standard input, so it stays out of the
798 +
    /// shell's history)
799 +
    Grant {
800 +
        /// The hub (default: this checkout's origin)
801 +
        hub: Option<String>,
802 +
    },
659 803
}
660 804
661 805
#[derive(Subcommand)]
…
26 unchanged lines
688 832
    Remove {
689 833
        /// The subrepo to unregister
690 834
        path: String,
835 +
        /// Delete its directory too (#164); refused while it has uncommitted edits
836 +
        #[arg(long)]
837 +
        delete: bool,
838 +
        /// With --delete: delete it even with uncommitted edits
839 +
        #[arg(long, requires = "delete")]
840 +
        force: bool,
841 +
    },
842 +
}
843 +
844 +
#[derive(Subcommand)]
845 +
enum ArtifactCmd {
846 +
    /// Pack DIR into an artifact: each file a blob of this repository's store, so files and chunks it holds cost nothing
847 +
    Pack {
848 +
        /// The directory whose files make the artifact
849 +
        dir: std::path::PathBuf,
850 +
        /// container, runner-image, package, archive or report
851 +
        #[arg(long, default_value = "archive")]
852 +
        kind: String,
853 +
        /// x86_64, riscv64, aarch64…
854 +
        #[arg(long)]
855 +
        arch: Option<String>,
856 +
        /// linux, darwin, none…
857 +
        #[arg(long)]
858 +
        os: Option<String>,
859 +
        /// The default command, one word per flag
860 +
        #[arg(long = "entrypoint", value_name = "WORD")]
861 +
        entrypoint: Vec<String>,
862 +
        /// KEY=VALUE for the environment (repeatable)
863 +
        #[arg(long = "env", value_name = "KEY=VALUE")]
864 +
        env: Vec<String>,
865 +
        /// KEY=VALUE metadata (repeatable)
866 +
        #[arg(long = "label", value_name = "KEY=VALUE")]
867 +
        labels: Vec<String>,
868 +
        /// Machine-readable output: one JSON document on stdout
869 +
        #[arg(long)]
870 +
        json: bool,
871 +
    },
872 +
    /// The artifacts kept here
873 +
    List {
874 +
        /// Machine-readable output: one JSON document on stdout
875 +
        #[arg(long)]
876 +
        json: bool,
877 +
    },
878 +
    /// One artifact and its files
879 +
    Show {
880 +
        /// Its name, or a unique prefix of at least six characters
881 +
        name: String,
882 +
        /// Machine-readable output: one JSON document on stdout
883 +
        #[arg(long)]
884 +
        json: bool,
885 +
    },
886 +
    /// Write an artifact's files into DEST (missing or empty)
887 +
    Unpack {
888 +
        /// Its name, or a unique prefix of at least six characters
889 +
        name: String,
890 +
        /// Where to write its files
891 +
        dest: std::path::PathBuf,
691 892
    },
893 +
    /// Check an artifact's name, tree and every file against the store
894 +
    Verify {
895 +
        /// Its name, or a unique prefix of at least six characters
896 +
        name: String,
897 +
    },
692 898
}
693 899
694 900
#[derive(Subcommand)]
…
26 unchanged lines
721 927
        /// Change, bookmark or tag whose state holds it
722 928
        #[arg(long)]
723 929
        rev: Option<String>,
930 +
        /// `oxygen`: the runner's signed receipt, as `verify` and `import`
931 +
        /// read it. `slsa`: SLSA provenance in a DSSE envelope, signed with
932 +
        /// your key, for in-toto and cosign (`oxy identity export --pem`
933 +
        /// gives the key they check it with)
934 +
        #[arg(long, default_value = "oxygen", value_parser = ["oxygen", "slsa"])]
935 +
        format: String,
724 936
    },
725 937
    /// Check a signed receipt in FILE without keeping it
726 938
    Verify {
…
22 unchanged lines
749 961
        #[arg(short, long, default_value = "")]
750 962
        message: String,
751 963
    },
964 +
    /// Withdraw a tag: a signed withdrawal, kept beside the tag's creation;
965 +
    /// `oxy push` sends it to a remote that has the tag (the name is never
966 +
    /// used again)
967 +
    Delete {
968 +
        /// Name of the tag
969 +
        name: String,
970 +
        /// Why it is withdrawn, recorded with the withdrawal
971 +
        #[arg(short, long, default_value = "")]
972 +
        reason: String,
973 +
    },
752 974
    /// List tags and the states they name
753 975
    List {
754 976
        /// Machine-readable output: one JSON document on stdout
…
82 unchanged lines
837 1059
    Show,
838 1060
    /// Write your identity object to a file, to hand to a policy holder
839 1061
    Export {
840 -
        /// File to write (default: <key>.identity in the current directory)
1062 +
        /// File to write (default: <key>.identity, or <key>.pem with --pem, in the current directory)
841 1063
        path: Option<PathBuf>,
1064 +
        /// Write the Ed25519 public key as PEM instead, as cosign's --key and
1065 +
        /// other DSSE verifiers take it
1066 +
        #[arg(long)]
1067 +
        pem: bool,
842 1068
    },
843 1069
    /// Sign a host's challenge (hex, at least 16 bytes) to prove you hold your key; prints the signature as hex
844 1070
    SignChallenge {
…
52 unchanged lines
897 1123
898 1124
#[derive(Subcommand)]
899 1125
enum GitCmd {
1126 +
    /// Keep a .git beside .oxy, up to date after every oxy command, so editors' Git features work here (one way: oxy to Git)
1127 +
    Colocate {
1128 +
        /// Stop updating it (the .git is left as it is)
1129 +
        #[arg(long)]
1130 +
        off: bool,
1131 +
    },
900 1132
    /// Export the current bookmark to a Git repository (created bare if absent)
901 1133
    Export {
902 1134
        /// Git repository to write to (created bare if absent)
…
9 unchanged lines
912 1144
        /// Branch to fetch
913 1145
        #[arg(long, default_value = "main")]
914 1146
        branch: String,
1147 +
        /// Author your own commits (Git author email = your identity's) with your key; sets git.claim
1148 +
        #[arg(long)]
1149 +
        claim: bool,
915 1150
    },
916 1151
    /// Push the current bookmark to a branch of a Git remote (fetch first)
917 1152
    Push {
…
3 unchanged lines
921 1156
        #[arg(long, default_value = "main")]
922 1157
        branch: String,
923 1158
    },
1159 +
    /// Recover an import that stopped before it moved the bookmark: point the bookmark at every imported commit's change in the store, so oxy git fetch carries on from there
1160 +
    Recover,
924 1161
    /// Check this repository's import of a Git branch: every commit's tree must be reproduced exactly
925 1162
    Verify {
926 1163
        /// The Git repository the import came from
…
4 unchanged lines
931 1168
        /// Rebuild only the last N commits (the others are followed, not checked)
932 1169
        #[arg(long, value_name = "N")]
933 1170
        last: Option<usize>,
1171 +
        /// Stop after N differences (0: check every commit whatever it finds)
1172 +
        #[arg(long, value_name = "N", default_value_t = 50)]
1173 +
        max_differences: usize,
934 1174
    },
935 1175
    /// Import a Git branch into a new Oxygen repository at DEST
936 1176
    Import {
…
4 unchanged lines
941 1181
        /// Branch to import
942 1182
        #[arg(long, default_value = "main")]
943 1183
        branch: String,
1184 +
        /// Author your own commits (Git author email = your identity's) with your key; sets git.claim
1185 +
        #[arg(long)]
1186 +
        claim: bool,
944 1187
    },
945 1188
}
946 1189
…
18 unchanged lines
965 1208
        #[arg(long)]
966 1209
        out: Option<PathBuf>,
967 1210
    },
1211 +
    /// Allow holders of an issuer's certificates for SUBJECT to sign (keyless)
1212 +
    GrantSubject {
1213 +
        /// Who: `repo:OWNER/NAME:bookmark:main`, `agent:NAME@ACCOUNT`; `*` matches any run of characters
1214 +
        subject: String,
1215 +
        /// The issuer's identity file, e.g. a hub's /.well-known/oxyhub-key
1216 +
        #[arg(long)]
1217 +
        issuer: PathBuf,
1218 +
        /// What its keys may sign: changes, refs, seal
1219 +
        #[arg(long, default_value = "changes,refs")]
1220 +
        scope: String,
1221 +
        /// Reference pattern the grant applies to (default: every bookmark)
1222 +
        #[arg(long)]
1223 +
        pattern: Option<String>,
1224 +
        /// Where to write the proposal when more root signatures are needed
1225 +
        #[arg(long)]
1226 +
        out: Option<PathBuf>,
1227 +
    },
968 1228
    /// Revoke a key (KeyId prefix of an identity in the policy)
969 1229
    Revoke {
970 1230
        /// KeyId prefix of the identity
…
45 unchanged lines
1016 1276
    },
1017 1277
}
1018 1278
1279 +
/// After a command, bring the side-by-side `.git` up to date when this
1280 +
/// checkout keeps one (`git.colocate`). It never fails the command: a
1281 +
/// problem is a warning, and the next command tries again.
1282 +
pub fn after_command() {
1283 +
    if AT_OP.get().and_then(|o| o.as_deref()).is_some() {
1284 +
        return;
1285 +
    }
1286 +
    let Ok(dir) = std::env::current_dir() else {
1287 +
        return;
1288 +
    };
1289 +
    let Ok(repo) = Repo::open(&dir) else { return };
1290 +
    if !repo.config.git_colocate || repo.is_bare() {
1291 +
        return;
1292 +
    }
1293 +
    if let Err(e) = repo.git_colocate_sync() {
1294 +
        eprintln!(
1295 +
            "{} the .git beside this checkout was not updated: {e}",
1296 +
            ui::warn("!")
1297 +
        );
1298 +
    }
1299 +
}
1300 +
1019 1301
/// Open the repository containing the current directory.
1020 1302
fn open() -> Result<Repo> {
1021 1303
    let repo = Repo::open(&std::env::current_dir()?).context("opening repository")?;
…
41 unchanged lines
1063 1345
            Ok(0)
1064 1346
        }
1065 1347
        Cmd::Man { dir } => man(dir),
1066 -
        Cmd::Init { path, bare } => workspace::init(path, bare),
1348 +
        Cmd::Init { path, bare, store } => workspace::init(path, bare, store.as_deref()),
1067 1349
        Cmd::Status { json } => workspace::status(json),
1068 1350
        Cmd::Prompt {
1069 1351
            format,
…
24 unchanged lines
1094 1376
        }
1095 1377
        Cmd::Cat { path, change } => workspace::cat(&path, change.as_deref()),
1096 1378
        Cmd::Root { json } => workspace::root(json),
1379 +
        Cmd::Mcp {
1380 +
            dir,
1381 +
            allow_destructive,
1382 +
        } => mcp::serve(dir, allow_destructive),
1097 1383
        Cmd::Version { json } => workspace::version(json),
1098 1384
        Cmd::Discard { paths, hunks } => workspace::discard(&paths, &hunks),
1099 1385
        Cmd::Hunks { json } => workspace::hunks(json),
…
42 unchanged lines
1142 1428
            }
1143 1429
            None => workspace::describe_name(json),
1144 1430
        },
1145 -
        Cmd::Amend { message, into } => workspace::amend(message.as_deref(), into.as_deref()),
1431 +
        Cmd::Amend {
1432 +
            message,
1433 +
            into,
1434 +
            paths,
1435 +
        } => workspace::amend(message.as_deref(), into.as_deref(), &paths),
1146 1436
        Cmd::Split { change, paths } => workspace::split(&change, &paths),
1147 1437
        Cmd::Abandon {
1148 1438
            change,
…
3 unchanged lines
1152 1442
        Cmd::Show {
1153 1443
            change,
1154 1444
            revset,
1445 +
            stat,
1155 1446
            json,
1156 -
        } => workspace::show(change.as_deref(), revset.as_deref(), json),
1447 +
        } => workspace::show(change.as_deref(), revset.as_deref(), stat, json),
1157 1448
        Cmd::Checkout {
1158 1449
            changes,
1159 1450
            discard,
1160 1451
            carry,
1161 1452
        } => workspace::checkout(&changes, discard, carry),
1162 -
        Cmd::Resolve { json } => workspace::resolve(json),
1453 +
        Cmd::Resolve {
1454 +
            paths,
1455 +
            both,
1456 +
            reverse,
1457 +
            json,
1458 +
        } => workspace::resolve(&paths, both, reverse, json),
1163 1459
        Cmd::Undo => workspace::undo(),
1164 -
        Cmd::Gc { now, remote } => match remote {
1460 +
        Cmd::Gc {
1461 +
            now,
1462 +
            expire_snapshots,
1463 +
            remote,
1464 +
        } => match remote {
1165 1465
            Some(name) => workspace::gc_remote(&name, now),
1166 -
            None => workspace::gc(now),
1466 +
            None => workspace::gc(now, expire_snapshots),
1167 1467
        },
1168 1468
        Cmd::Fsck { json } => workspace::fsck(json),
1169 1469
        Cmd::Doctor { offline, json } => workspace::doctor(offline, json),
…
34 unchanged lines
1204 1504
                only,
1205 1505
            } => workspace::bookmark_set(&name, &changes, upto, only),
1206 1506
            BookmarkCmd::Delete { name } => workspace::bookmark_delete(&name),
1207 -
            BookmarkCmd::List { all, json } => workspace::bookmark_list(all, json),
1507 +
            BookmarkCmd::List { all, verbose, json } => {
1508 +
                workspace::bookmark_list(all, verbose, json)
1509 +
            }
1510 +
            BookmarkCmd::Show { name, json } => workspace::bookmark_show(name.as_deref(), json),
1208 1511
            BookmarkCmd::Switch {
1209 1512
                name,
1210 1513
                discard,
…
9 unchanged lines
1220 1523
            } => auth::login(hub.as_deref(), with_token, ca),
1221 1524
            AuthCmd::Status => auth::status(),
1222 1525
            AuthCmd::Logout { hub } => auth::logout(hub.as_deref()),
1526 +
            AuthCmd::Grant { hub } => auth::grant(hub.as_deref()),
1223 1527
        },
1224 1528
        Cmd::Submodule { cmd } => match cmd {
1225 1529
            SubmoduleCmd::Add {
…
3 unchanged lines
1229 1533
            } => sync::submodule_add(&url, &path, bookmark.as_deref()),
1230 1534
            SubmoduleCmd::Update { init, paths } => sync::submodule_update(init, &paths),
1231 1535
            SubmoduleCmd::Status { json } => sync::submodule_status(json),
1232 -
            SubmoduleCmd::Remove { path } => sync::submodule_remove(&path),
1536 +
            SubmoduleCmd::Remove {
1537 +
                path,
1538 +
                delete,
1539 +
                force,
1540 +
            } => sync::submodule_remove(&path, delete, force),
1541 +
        },
1542 +
        Cmd::Witness {
1543 +
            cmd:
1544 +
                WitnessCmd::Serve {
1545 +
                    name,
1546 +
                    logs,
1547 +
                    state,
1548 +
                    bind,
1549 +
                    tls_cert,
1550 +
                    tls_key,
1551 +
                },
1552 +
        } => tlog::witness_serve(&name, &logs, state, &bind, tls_cert.zip(tls_key)),
1553 +
        Cmd::Tlog { cmd } => match cmd {
1554 +
            TlogCmd::Verify {
1555 +
                hub,
1556 +
                key,
1557 +
                all,
1558 +
                json,
1559 +
            } => tlog::verify(hub.as_deref(), key.as_deref(), all, json),
1233 1560
        },
1561 +
        Cmd::Artifact { cmd } => match cmd {
1562 +
            ArtifactCmd::Pack {
1563 +
                dir,
1564 +
                kind,
1565 +
                arch,
1566 +
                os,
1567 +
                entrypoint,
1568 +
                env,
1569 +
                labels,
1570 +
                json,
1571 +
            } => artifact::pack(&dir, &kind, arch, os, entrypoint, &env, &labels, json),
1572 +
            ArtifactCmd::List { json } => artifact::list(json),
1573 +
            ArtifactCmd::Show { name, json } => artifact::show(&name, json),
1574 +
            ArtifactCmd::Unpack { name, dest } => artifact::unpack(&name, &dest),
1575 +
            ArtifactCmd::Verify { name } => artifact::verify(&name),
1576 +
        },
1234 1577
        Cmd::Receipt { cmd } => match cmd {
1235 1578
            ReceiptCmd::List { rev, json } => receipt::list(rev.as_deref(), json),
1236 1579
            ReceiptCmd::Show {
…
5 unchanged lines
1242 1585
                receipt: r,
1243 1586
                file,
1244 1587
                rev,
1245 -
            } => receipt::export(&r, &file, rev.as_deref()),
1588 +
                format,
1589 +
            } => receipt::export(&r, &file, rev.as_deref(), &format),
1246 1590
            ReceiptCmd::Verify { file, json } => receipt::verify(&file, json),
1247 1591
            ReceiptCmd::Import { file } => receipt::import(&file),
1248 1592
        },
…
3 unchanged lines
1252 1596
                changes,
1253 1597
                message,
1254 1598
            } => workspace::tag_create(&name, &changes, &message),
1599 +
            TagCmd::Delete { name, reason } => workspace::tag_delete(&name, &reason),
1255 1600
            TagCmd::List { json } => workspace::tag_list(json),
1256 1601
        },
1257 1602
        Cmd::Remote { cmd } => match cmd {
…
45 unchanged lines
1303 1648
                expires,
1304 1649
            } => identity::create(name, email, local, expires),
1305 1650
            IdentityCmd::Show => identity::show(),
1306 -
            IdentityCmd::Export { path } => identity::export(path),
1651 +
            IdentityCmd::Export { path, pem } => identity::export(path, pem),
1307 1652
            IdentityCmd::SignChallenge { challenge, json } => {
1308 1653
                identity::sign_challenge(&challenge, json)
1309 1654
            }
…
11 unchanged lines
1321 1666
            },
1322 1667
        },
1323 1668
        Cmd::Git {
1669 +
            cmd: GitCmd::Colocate { off },
1670 +
        } => sync::git_colocate(off),
1671 +
        Cmd::Git {
1324 1672
            cmd: GitCmd::Export { git_dir, branch },
1325 1673
        } => sync::git_export(&git_dir, branch.as_deref()),
1326 1674
        Cmd::Git {
1327 -
            cmd: GitCmd::Fetch { url, branch },
1328 -
        } => sync::git_fetch(&url, &branch),
1675 +
            cmd: GitCmd::Fetch { url, branch, claim },
1676 +
        } => sync::git_fetch(&url, &branch, claim),
1329 1677
        Cmd::Git {
1330 1678
            cmd: GitCmd::Push { url, branch },
1331 1679
        } => sync::git_push(&url, &branch),
1332 1680
        Cmd::Git {
1681 +
            cmd: GitCmd::Recover,
1682 +
        } => sync::git_recover(),
1683 +
        Cmd::Git {
1333 1684
            cmd:
1334 1685
                GitCmd::Import {
1335 1686
                    git_dir,
1336 1687
                    dest,
1337 1688
                    branch,
1689 +
                    claim,
1338 1690
                },
1339 -
        } => sync::git_import(&git_dir, &dest, &branch),
1691 +
        } => sync::git_import(&git_dir, &dest, &branch, claim),
1340 1692
        Cmd::Git {
1341 1693
            cmd:
1342 1694
                GitCmd::Verify {
1343 1695
                    git_dir,
1344 1696
                    branch,
1345 1697
                    last,
1698 +
                    max_differences,
1346 1699
                },
1347 -
        } => sync::git_verify(&git_dir, &branch, last),
1700 +
        } => sync::git_verify(&git_dir, &branch, last, max_differences),
1348 1701
        Cmd::Policy { cmd } => match cmd {
1349 1702
            PolicyCmd::Init => identity::policy_init(),
1350 1703
            PolicyCmd::Show { json } => identity::policy_show(json),
…
2 unchanged lines
1353 1706
                pattern,
1354 1707
                out,
1355 1708
            } => identity::policy_grant(&identity_file, pattern.as_deref(), out),
1709 +
            PolicyCmd::GrantSubject {
1710 +
                subject,
1711 +
                issuer,
1712 +
                scope,
1713 +
                pattern,
1714 +
                out,
1715 +
            } => identity::policy_grant_subject(&subject, &issuer, &scope, pattern.as_deref(), out),
1356 1716
            PolicyCmd::Revoke { key, out } => identity::policy_revoke(&key, out),
1357 1717
            PolicyCmd::Root { cmd } => match cmd {
1358 1718
                RootCmd::Add { identity_file, out } => {
…
5 unchanged lines
1364 1724
            PolicyCmd::Sign { proposal, out } => identity::policy_sign(&proposal, out),
1365 1725
        },
1366 1726
        Cmd::Seal {
1727 +
            flow: Some(url),
1728 +
            job: Some(job),
1729 +
            reject,
1730 +
            json,
1731 +
            ..
1732 +
        } => approve::seal(&url, &job, reject, json),
1733 +
        Cmd::Seal {
1367 1734
            changes,
1368 1735
            proposal: Some(url),
1369 1736
            json,
1737 +
            ..
1370 1738
        } if changes.is_empty() => proposal::seal(&url, json),
1371 1739
        Cmd::Seal { changes, json, .. } => identity::seal(&changes, json),
1372 1740
        Cmd::Proposal { cmd } => match cmd {
…
54 unchanged lines
modified src/cli/receipt.rs +108 −2
…
172 unchanged lines
173 173
    Ok(if i.problem.is_some() { 1 } else { 0 })
174 174
}
175 175
176 -
/// `oxy receipt export HASH FILE`: the signed receipt, to hand to anyone.
177 -
pub fn export(prefix: &str, file: &str, rev: Option<&str>) -> Result<i32> {
176 +
/// `oxy receipt export HASH FILE [--format slsa]`: the signed receipt, to
177 +
/// hand to anyone, or SLSA provenance of it for tools outside Oxygen.
178 +
pub fn export(prefix: &str, file: &str, rev: Option<&str>, format: &str) -> Result<i32> {
178 179
    let repo = open()?;
179 180
    let i = find(&repo, prefix, rev)?;
181 +
    if format == "slsa" {
182 +
        return export_slsa(&repo, &i, file);
183 +
    }
180 184
    std::fs::write(file, i.signed.encode())?;
181 185
    println!(
182 186
        "{} {} {}",
…
4 unchanged lines
187 191
    Ok(0)
188 192
}
189 193
194 +
/// What SLSA calls the build type of a Flow job.
195 +
pub const BUILD_TYPE: &str = "https://oxygenhub.io/oxygenflow/job/v1";
196 +
197 +
/// The in-toto Statement v1, with an SLSA provenance v1 predicate, of what
198 +
/// a receipt says (spec §14.6). Its subjects are the job's outputs (the
199 +
/// state it ran on when it made none), named by their Oxygen object hashes;
200 +
/// the signed receipt itself rides along as a byproduct, so an Oxygen
201 +
/// verifier can still check the runner's own signature.
202 +
pub fn statement(sr: &SignedReceipt) -> serde_json::Value {
203 +
    use serde_json::json;
204 +
    let r = &sr.receipt;
205 +
    let subject: Vec<_> = if r.outputs.is_empty() {
206 +
        vec![json!({ "name": "state", "digest": { "oxygen-state": sr.state().to_hex() } })]
207 +
    } else {
208 +
        r.outputs
209 +
            .iter()
210 +
            .map(|o| json!({ "name": o.to_hex(), "digest": { "oxygen-blob": o.to_hex() } }))
211 +
            .collect()
212 +
    };
213 +
    let rfc3339 = |ms: u64| super::format::when(ms).replacen(' ', "T", 1);
214 +
    let made = sr.attest.hlc.wall_ms;
215 +
    let mut deps = vec![
216 +
        json!({ "name": "state", "digest": { "oxygen-state": sr.state().to_hex() } }),
217 +
        json!({ "name": "scope", "digest": { "oxygen-scope": r.scope_tree_hash.to_hex() } }),
218 +
    ];
219 +
    deps.extend(
220 +
        r.inputs
221 +
            .iter()
222 +
            .map(|h| json!({ "name": h.to_hex(), "digest": { "oxygen-blob": h.to_hex() } })),
223 +
    );
224 +
    json!({
225 +
        "_type": "https://in-toto.io/Statement/v1",
226 +
        "subject": subject,
227 +
        "predicateType": "https://slsa.dev/provenance/v1",
228 +
        "predicate": {
229 +
            "buildDefinition": {
230 +
                "buildType": BUILD_TYPE,
231 +
                "externalParameters": {
232 +
                    "job": r.job_name,
233 +
                    "pipeline": r.pipeline_hash.to_hex(),
234 +
                },
235 +
                "internalParameters": {
236 +
                    "environment": r.environment.to_hex(),
237 +
                },
238 +
                "resolvedDependencies": deps,
239 +
            },
240 +
            "runDetails": {
241 +
                "builder": {
242 +
                    "id": format!("urn:oxygen:runner:{}", sr.runner.hash().to_hex()),
243 +
                    "version": { "name": sr.runner.name },
244 +
                },
245 +
                "metadata": {
246 +
                    "invocationId": sr.run(),
247 +
                    "startedOn": rfc3339(made.saturating_sub(r.duration_ms)),
248 +
                    "finishedOn": rfc3339(made),
249 +
                },
250 +
                "byproducts": [{
251 +
                    "name": "oxygen-receipt",
252 +
                    "mediaType": "application/vnd.oxygen.receipt+cbor",
253 +
                    "content": oxy_core::dsse::base64(&sr.encode()),
254 +
                    "annotations": {
255 +
                        "exitCode": r.exit_code,
256 +
                        "durationMs": r.duration_ms,
257 +
                        "runner": sr.runner.name,
258 +
                    },
259 +
                }],
260 +
            },
261 +
        },
262 +
    })
263 +
}
264 +
265 +
fn export_slsa(repo: &oxy_core::repo::Repo, i: &ReceiptInfo, file: &str) -> Result<i32> {
266 +
    if let Some(p) = &i.problem {
267 +
        bail!(
268 +
            "receipt #{} does not verify ({p}); provenance is only made of a valid one",
269 +
            i.attest.short(12)
270 +
        );
271 +
    }
272 +
    let Some((key, sk)) = repo.secret_key()? else {
273 +
        bail!("SLSA provenance is signed with your key, and none is set up here; run `oxy identity create`");
274 +
    };
275 +
    let payload = serde_json::to_vec(&statement(&i.signed))?;
276 +
    std::fs::write(
277 +
        file,
278 +
        oxy_core::dsse::envelope(oxy_core::dsse::IN_TOTO, &payload, &key.to_hex(), &sk),
279 +
    )?;
280 +
    println!(
281 +
        "{} {} {} {}",
282 +
        ui::done("Exported SLSA provenance of receipt"),
283 +
        ui::hash(format!("#{}", i.attest.short(12))),
284 +
        ui::path(file),
285 +
        ui::dim(format!("signed by {}", key.short(12)))
286 +
    );
287 +
    println!(
288 +
        "{}",
289 +
        ui::dim(format!(
290 +
            "  check it: oxy identity export --pem key.pem, then cosign verify-blob-attestation --key key.pem --type slsaprovenance1 --insecure-ignore-tlog=true --check-claims=false --signature {file}"
291 +
        ))
292 +
    );
293 +
    Ok(0)
294 +
}
295 +
190 296
fn read(file: &str) -> Result<SignedReceipt> {
191 297
    let bytes = std::fs::read(file).map_err(|e| anyhow::anyhow!("{file}: {e}"))?;
192 298
    SignedReceipt::decode(&bytes).map_err(|e| anyhow::anyhow!("{file}: not a signed receipt: {e}"))
…
29 unchanged lines
modified src/cli/sync.rs +312 −45
…
25 unchanged lines
26 26
    });
27 27
    let max_blob = max_blob.map(oxy_core::repo::parse_size).transpose()?;
28 28
    let sp = ui::spinner(&format!("Cloning {url}…"));
29 -
    let repo = match depth {
30 -
        Some(d) => Repo::clone_shallow(url, &dest, d, max_blob, sparse)?,
31 -
        None => Repo::clone_with(url, &dest, max_blob, sparse)?,
32 -
    };
29 +
    let (repo, skipped) = Repo::clone_reporting(url, &dest, max_blob, sparse, depth)?;
33 30
    let heads = repo.bookmark_heads()?;
34 31
    sp.done(&format!(
35 32
        "{} into {} {}",
…
5 unchanged lines
41 38
            plural(heads.len(), "head")
42 39
        ))
43 40
    ));
41 +
    warn_skipped(&skipped);
44 42
    let others = repo.remote_only_bookmarks()?.len();
45 43
    if others > 0 {
46 44
        println!(
…
249 unchanged lines
296 294
297 295
pub fn remote_remove(name: &str) -> Result<i32> {
298 296
    let mut repo = open()?;
299 -
    repo.config.set(&format!("remote.{name}.url"), "")?;
300 -
    repo.save_config()?;
297 +
    repo.remove_remote(name)?;
301 298
    println!("{} remote {}", ui::done("Removed"), ui::bookmark(name));
302 299
    Ok(0)
303 300
}
…
71 unchanged lines
375 372
    let repo = open()?;
376 373
    let remote = remote_name(&repo, remote);
377 374
    let sp = ui::spinner(&format!("Fetching from {remote}…"));
378 -
    let report = repo.fetch(&remote)?;
375 +
    let report = noted(&repo, &remote, repo.fetch(&remote))?;
379 376
    sp.clear();
380 377
    if as_json {
381 378
        println!(
…
20 unchanged lines
402 399
                    }))
403 400
                ),
404 401
                ("new_tags", json::n(report.new_tags)),
402 +
                (
403 +
                    "withdrawn_tags",
404 +
                    json::arr(report.withdrawn_tags.iter().map(|n| json::s(n))),
405 +
                ),
405 406
                ("unverified", json::b(report.unverified)),
407 +
                (
408 +
                    "log_notes",
409 +
                    json::arr(report.log_notes.iter().map(|n| json::s(n))),
410 +
                ),
411 +
                ("skipped", skipped_json(&report.skipped)),
406 412
            ])
407 413
        );
408 414
        return Ok(0);
…
50 unchanged lines
459 465
            ui::dim(format!("{} adopted", plural(report.new_tags, "tag")))
460 466
        );
461 467
    }
468 +
    for t in &report.withdrawn_tags {
469 +
        println!(
470 +
            "  {} {}",
471 +
            ui::note(),
472 +
            ui::dim(format!("tag {t} was withdrawn on {remote}, and here"))
473 +
        );
474 +
    }
462 475
    if report.unverified {
463 476
        eprintln!(
464 477
            "{} {remote} has no trust policy; nothing fetched from it was verified",
465 478
            ui::caution()
466 479
        );
467 480
    }
481 +
    for n in &report.log_notes {
482 +
        eprintln!("{} {n}", ui::caution());
483 +
    }
484 +
    warn_skipped(&report.skipped);
468 485
    Ok(0)
469 486
}
470 487
488 +
/// Bookmarks a fetch left as they were, their attestation unverifiable
489 +
/// (oxygenhub#266): one warning each, naming it and why.
490 +
fn warn_skipped(skipped: &[(String, String)]) {
491 +
    for (name, why) in skipped {
492 +
        eprintln!(
493 +
            "{} {} {}",
494 +
            ui::caution(),
495 +
            ui::warn(format!("{name} not updated:")),
496 +
            ui::dim(why)
497 +
        );
498 +
    }
499 +
}
500 +
501 +
fn skipped_json(skipped: &[(String, String)]) -> String {
502 +
    json::arr(
503 +
        skipped
504 +
            .iter()
505 +
            .map(|(name, why)| json::obj([("name", json::s(name)), ("reason", json::s(why))])),
506 +
    )
507 +
}
508 +
471 509
/// Pull acts on the current bookmark only: name the others the fetch
472 510
/// found behind their remote, which is easy to take for the current one
473 511
/// (oxygen#29).
…
27 unchanged lines
501 539
    let repo = open()?;
502 540
    let remote = remote_name(&repo, remote);
503 541
    let sp = ui::spinner(&format!("Pulling from {remote}…"));
504 -
    let r = repo.pull(&remote)?;
542 +
    let r = noted(&repo, &remote, repo.pull(&remote))?;
505 543
    if as_json {
506 544
        sp.clear();
507 545
        let conflicted = repo
…
9 unchanged lines
517 555
                ("changed", json::b(r.changed)),
518 556
                ("checked_out", json::b(r.checked_out)),
519 557
                ("conflicted", json::b(conflicted)),
520 -
                ("others", json::arr(r.others.iter().map(|o| json::s(o))))
558 +
                ("others", json::arr(r.others.iter().map(|o| json::s(o)))),
559 +
                ("skipped", skipped_json(&r.skipped))
521 560
            ])
522 561
        );
523 562
        return Ok(0);
524 563
    }
525 564
    if !r.changed {
526 565
        sp.done(&format!(
527 -
            "{} {}",
566 +
            "{} {}{}",
528 567
            ui::ok(ui::glyph("✔", "*")),
529 568
            ui::dim(format!(
530 569
                "{} is up to date with {remote}.",
531 570
                repo.config.bookmark
532 -
            ))
571 +
            )),
572 +
            if r.checked_out {
573 +
                ui::dim(" The working copy had fallen behind it and is updated.")
574 +
            } else {
575 +
                String::new()
576 +
            }
533 577
        ));
534 578
        others_pulled(&r.others);
535 579
        others_behind(&repo, &remote);
580 +
        warn_skipped(&r.skipped);
581 +
        if r.checked_out {
582 +
            super::workspace::where_now(&repo);
583 +
        }
536 584
        return Ok(0);
537 585
    }
538 586
    sp.done(&format!(
…
17 unchanged lines
556 604
    }
557 605
    others_pulled(&r.others);
558 606
    others_behind(&repo, &remote);
607 +
    warn_skipped(&r.skipped);
559 608
    subrepos_behind(&repo);
560 609
    super::workspace::where_now(&repo);
561 610
    Ok(0)
…
42 unchanged lines
604 653
    let mut code = 0;
605 654
    for r in results {
606 655
        if r.ok {
656 +
            let done = if r.name.starts_with("tags/") {
657 +
                "Withdrew"
658 +
            } else {
659 +
                "Deleted"
660 +
            };
607 661
            println!(
608 662
                "{} {}",
609 -
                ui::done("Deleted"),
663 +
                ui::done(done),
610 664
                ui::id(format!("{remote}/{}", r.name))
611 665
            );
612 666
        } else {
…
20 unchanged lines
633 687
    let repo = open()?;
634 688
    let remote = remote_name(&repo, remote);
635 689
    let sp = ui::spinner(&format!("Pushing to {remote}…"));
636 -
    let results = match bookmark {
637 -
        Some(b) => repo.push_bookmark(&remote, b, force)?,
638 -
        None => repo.push(&remote, force)?,
639 -
    };
690 +
    let bookmark_name = bookmark.map_or_else(|| repo.config.bookmark.clone(), String::from);
691 +
    // A long push goes in parts: the spinner says which.
692 +
    let results = noted(
693 +
        &repo,
694 +
        &remote,
695 +
        repo.push_bookmark_with(&remote, &bookmark_name, force, &mut |m| {
696 +
            sp.set(&format!("Pushing to {remote}… {m}"))
697 +
        }),
698 +
    )?;
640 699
    sp.clear();
641 700
    if as_json {
642 701
        // The state each accepted bookmark now holds on the remote (its local
…
55 unchanged lines
698 757
    tls: Option<(PathBuf, PathBuf)>,
699 758
) -> Result<i32> {
700 759
    let listener = std::net::TcpListener::bind(bind).with_context(|| format!("binding {bind}"))?;
701 -
    let root = root.canonicalize()?;
760 +
    let root = oxy_core::portable::canonical(&root)?;
702 761
    let tls = tls.map(|(cert, key)| oxy_server::TlsFiles { cert, key });
703 762
    println!(
704 763
        "Serving repositories under {} on {}://{bind}/<repo>/oxy/v1/ ({})",
…
16 unchanged lines
721 780
    Ok(0)
722 781
}
723 782
783 +
/// `oxy git colocate [--off]`: keep a `.git` beside `.oxy` (spec §23).
784 +
pub fn git_colocate(off: bool) -> Result<i32> {
785 +
    let mut repo = open()?;
786 +
    if repo.is_bare() {
787 +
        anyhow::bail!("a bare repository has no working copy for a .git beside it");
788 +
    }
789 +
    repo.config
790 +
        .set("git.colocate", if off { "false" } else { "true" })?;
791 +
    repo.save_config()?;
792 +
    if off {
793 +
        println!(
794 +
            "{} the .git beside this checkout is no longer updated {}",
795 +
            ui::done("Stopped:"),
796 +
            ui::dim("(it is left as it is: delete .git yourself if you want it gone)")
797 +
        );
798 +
        return Ok(0);
799 +
    }
800 +
    let sp = ui::spinner("Exporting to .git…");
801 +
    repo.git_colocate_sync()?;
802 +
    sp.clear();
803 +
    println!(
804 +
        "{} .git beside .oxy holds {} {}",
805 +
        ui::done("Colocated:"),
806 +
        ui::bookmark(&repo.config.bookmark),
807 +
        ui::dim("(updated after every oxy command; Git's own commits are not read back: commit with oxy)")
808 +
    );
809 +
    Ok(0)
810 +
}
811 +
724 812
pub fn git_export(git_dir: &std::path::Path, branch: Option<&str>) -> Result<i32> {
725 813
    let repo = open()?;
726 814
    let sp = ui::spinner("Exporting to Git…");
…
17 unchanged lines
744 832
    Ok(0)
745 833
}
746 834
747 -
pub fn git_fetch(url: &str, branch: &str) -> Result<i32> {
835 +
pub fn git_recover() -> Result<i32> {
748 836
    let repo = open()?;
837 +
    let sp = ui::spinner("Reading the store…");
838 +
    let (changes, heads) = repo.recover_git_import(&|m| sp.set(m))?;
839 +
    sp.done(&format!(
840 +
        "{} {} from imported Git commits: {} now has {}",
841 +
        ui::done("Recovered"),
842 +
        plural(changes, "change"),
843 +
        ui::bookmark(&repo.config.bookmark),
844 +
        plural(heads, "head")
845 +
    ));
846 +
    println!(
847 +
        "  {} {}",
848 +
        ui::note(),
849 +
        ui::dim("carry on with oxy git fetch GIT_DIR --branch BRANCH")
850 +
    );
851 +
    Ok(0)
852 +
}
853 +
854 +
/// `--claim` (oxygen#97): record `git.claim`, so this import and every
855 +
/// later fetch author the identity's own commits with its key.
856 +
fn claim_commits(repo: &mut Repo) -> Result<()> {
857 +
    repo.config.set("git.claim", "true")?;
858 +
    repo.save_config()?;
859 +
    Ok(())
860 +
}
861 +
862 +
pub fn git_fetch(url: &str, branch: &str, claim: bool) -> Result<i32> {
863 +
    let mut repo = open()?;
864 +
    if claim {
865 +
        claim_commits(&mut repo)?;
866 +
    }
749 867
    let sp = ui::spinner(&format!("Fetching {url} {branch}…"));
750 868
    let r = repo.git_fetch(url, branch)?;
751 869
    sp.clear();
…
60 unchanged lines
812 930
}
813 931
814 932
/// `oxy git verify`: exit 1 if any commit's tree is not reproduced.
815 -
pub fn git_verify(git_dir: &std::path::Path, branch: &str, last: Option<usize>) -> Result<i32> {
933 +
pub fn git_verify(
934 +
    git_dir: &std::path::Path,
935 +
    branch: &str,
936 +
    last: Option<usize>,
937 +
    max: usize,
938 +
) -> Result<i32> {
816 939
    let repo = open()?;
817 940
    let started = std::time::Instant::now();
818 941
    let sp = ui::spinner(&format!(
819 942
        "Rebuilding every commit of {} {branch}…",
820 943
        git_dir.display()
821 944
    ));
822 -
    let r = repo.verify_git_import_last(git_dir, branch, last)?;
945 +
    let r = repo.verify_git_import_last(git_dir, branch, last, max)?;
823 946
    sp.clear();
824 947
    for (_, what) in &r.mismatches {
825 948
        println!("{} {what}", ui::err(ui::glyph("✖", "x")));
826 949
    }
950 +
    if r.not_yet > 0 {
951 +
        println!(
952 +
            "  {} {}",
953 +
            ui::note(),
954 +
            ui::dim(format!(
955 +
                "{} not imported yet, after the last imported commit: not checked",
956 +
                plural(r.not_yet, "commit")
957 +
            ))
958 +
        );
959 +
    }
827 960
    if r.mismatches.is_empty() {
828 961
        println!(
829 962
            "{}",
…
16 unchanged lines
846 979
        );
847 980
        Ok(0)
848 981
    } else {
982 +
        // What differs, by kind, the commit taken off: one difference
983 +
        // carried through history reads as one line.
984 +
        let mut kinds: std::collections::BTreeMap<String, usize> = Default::default();
985 +
        for (_, what) in &r.mismatches {
986 +
            let kind = what.split_once(": ").map_or(what.as_str(), |(_, k)| k);
987 +
            *kinds.entry(kind.to_string()).or_default() += 1;
988 +
        }
989 +
        let mut kinds: Vec<_> = kinds.into_iter().collect();
990 +
        kinds.sort_by_key(|k| std::cmp::Reverse(k.1));
991 +
        for (kind, n) in kinds.iter().take(10) {
992 +
            println!("  {} {}", ui::dim(format!("{n:>6} ×")), kind);
993 +
        }
849 994
        println!(
850 -
            "{} differences in {} commits checked",
995 +
            "{} differences in {} commits checked{}",
851 996
            r.mismatches.len(),
852 -
            r.commits
997 +
            r.commits,
998 +
            if max > 0 && r.mismatches.len() >= max {
999 +
                format!(" (stopped at {max}; --max-differences 0 checks every commit)")
1000 +
            } else {
1001 +
                String::new()
1002 +
            }
853 1003
        );
854 1004
        Ok(1)
855 1005
    }
856 1006
}
857 1007
858 -
pub fn git_import(git_dir: &std::path::Path, dest: &std::path::Path, branch: &str) -> Result<i32> {
859 -
    let repo = Repo::init(dest)?;
1008 +
pub fn git_import(
1009 +
    git_dir: &std::path::Path,
1010 +
    dest: &std::path::Path,
1011 +
    branch: &str,
1012 +
    claim: bool,
1013 +
) -> Result<i32> {
1014 +
    let mut repo = Repo::init(dest)?;
1015 +
    if claim {
1016 +
        claim_commits(&mut repo)?;
1017 +
    }
860 1018
    let started = std::time::Instant::now();
861 1019
    let sp = ui::spinner(&format!("Importing {} {branch}…", git_dir.display()));
862 1020
    let r = repo.import_git_with(git_dir, branch, &mut |where_| sp.set(where_))?;
…
54 unchanged lines
917 1075
    let repo = open()?;
918 1076
    let sp = ui::spinner("Updating subrepos…");
919 1077
    let results = repo.submodule_update(init, paths)?;
1078 +
    let unpinned: Vec<_> = repo
1079 +
        .unpinned_modules()?
1080 +
        .into_iter()
1081 +
        .filter(|(p, _)| paths.is_empty() || paths.iter().any(|q| q.trim_end_matches('/') == p))
1082 +
        .collect();
920 1083
    sp.clear();
921 -
    Ok(print_subrepo_results(results))
1084 +
    if results.is_empty() && unpinned.is_empty() {
1085 +
        println!("{}", ui::dim("No subrepos."));
1086 +
    }
1087 +
    let code = print_subrepo_results(results);
1088 +
    print_unpinned(&unpinned);
1089 +
    Ok(if unpinned.is_empty() { code } else { 1 })
922 1090
}
923 1091
924 -
fn print_subrepo_results(results: Vec<(String, oxy_core::repo::SubrepoSync)>) -> i32 {
925 -
    if results.is_empty() {
926 -
        println!("{}", ui::dim("No subrepos."));
1092 +
/// `.oxymodules` entries never added: nothing to clone, so say how to add
1093 +
/// them (oxygen#72).
1094 +
fn print_unpinned(unpinned: &[(String, oxy_core::subrepo::Module)]) {
1095 +
    for (path, m) in unpinned {
1096 +
        let url = if m.url.is_empty() { "<url>" } else { &m.url };
1097 +
        println!(
1098 +
            "{} {}",
1099 +
            ui::warn(format!(
1100 +
                "{} in .oxymodules but never added, so no pin",
1101 +
                ui::glyph("⚠", "!")
1102 +
            )),
1103 +
            ui::path(path)
1104 +
        );
1105 +
        println!(
1106 +
            "  {}",
1107 +
            ui::dim(format!("add it with: oxy submodule add {url} {path}"))
1108 +
        );
927 1109
    }
1110 +
}
1111 +
1112 +
fn print_subrepo_results(results: Vec<(String, oxy_core::repo::SubrepoSync)>) -> i32 {
928 1113
    let mut code = 0;
929 1114
    for (path, r) in results {
930 1115
        use oxy_core::repo::SubrepoSync::*;
…
55 unchanged lines
986 1171
pub fn submodule_status(as_json: bool) -> Result<i32> {
987 1172
    let repo = open()?;
988 1173
    let subs = repo.subrepos()?;
1174 +
    let unpinned = repo.unpinned_modules()?;
989 1175
    if as_json {
990 -
        println!(
991 -
            "{}",
992 -
            json::arr(subs.iter().map(|s| {
993 -
                json::obj([
994 -
                    ("path", json::s(&s.path)),
995 -
                    ("pin", pin_json(&s.pin)),
996 -
                    ("actual", json::opt(s.actual.as_ref(), pin_json)),
997 -
                    ("at_pin", json::b(s.actual.as_ref() == Some(&s.pin))),
998 -
                    ("dirty", json::b(s.dirty)),
999 -
                    ("committed", json::b(s.committed)),
1000 -
                    ("url", json::opt(s.url.as_deref(), json::s)),
1001 -
                ])
1002 -
            }))
1003 -
        );
1176 +
        let pinned = subs.iter().map(|s| {
1177 +
            json::obj([
1178 +
                ("path", json::s(&s.path)),
1179 +
                ("pin", pin_json(&s.pin)),
1180 +
                ("actual", json::opt(s.actual.as_ref(), pin_json)),
1181 +
                ("at_pin", json::b(s.actual.as_ref() == Some(&s.pin))),
1182 +
                ("dirty", json::b(s.dirty)),
1183 +
                ("committed", json::b(s.committed)),
1184 +
                ("url", json::opt(s.url.as_deref(), json::s)),
1185 +
            ])
1186 +
        });
1187 +
        // Listed in .oxymodules but never added: no pin at all.
1188 +
        let unpinned_json = unpinned.iter().map(|(path, m)| {
1189 +
            json::obj([
1190 +
                ("path", json::s(path)),
1191 +
                ("pin", "null".to_string()),
1192 +
                ("actual", "null".to_string()),
1193 +
                ("at_pin", json::b(false)),
1194 +
                ("dirty", json::b(false)),
1195 +
                ("committed", json::b(false)),
1196 +
                ("url", json::s(&m.url)),
1197 +
            ])
1198 +
        });
1199 +
        println!("{}", json::arr(pinned.chain(unpinned_json)));
1004 1200
        return Ok(0);
1005 1201
    }
1006 -
    if subs.is_empty() {
1202 +
    if subs.is_empty() && unpinned.is_empty() {
1007 1203
        println!("{}", ui::dim("No subrepos."));
1008 1204
    }
1009 1205
    for s in subs {
…
26 unchanged lines
1036 1232
            println!("    {} {}", ui::dim(ui::glyph("↳", "->")), ui::dim(u));
1037 1233
        }
1038 1234
    }
1235 +
    print_unpinned(&unpinned);
1039 1236
    Ok(0)
1040 1237
}
1041 1238
1042 -
pub fn submodule_remove(path: &str) -> Result<i32> {
1239 +
pub fn submodule_remove(path: &str, delete: bool, force: bool) -> Result<i32> {
1043 1240
    let repo = open()?;
1241 +
    let dir = repo.root().join(path.trim_end_matches('/'));
1242 +
    // Look before unregistering, so a refused delete changes nothing.
1243 +
    if delete && dir.exists() && !force {
1244 +
        if let Ok(sub) = oxy_core::repo::Repo::open(&dir) {
1245 +
            let snap = sub.snapshot("oxy submodule remove --delete")?;
1246 +
            if !snap.statuses.is_empty() {
1247 +
                anyhow::bail!(
1248 +
                    "{path} has {} with uncommitted edits; commit them in the subrepo (and push them if you want them kept), or pass --force to delete them",
1249 +
                    plural(snap.statuses.len(), "file")
1250 +
                );
1251 +
            }
1252 +
        }
1253 +
    }
1044 1254
    repo.submodule_remove(path)?;
1255 +
    if !delete {
1256 +
        println!(
1257 +
            "{} subrepo {} {}",
1258 +
            ui::done("Removed"),
1259 +
            ui::path(path),
1260 +
            ui::dim("(directory left in place; commit to record the removal)")
1261 +
        );
1262 +
        println!(
1263 +
            "  {} {}",
1264 +
            ui::note(),
1265 +
            ui::dim(format!(
1266 +
                "to delete the directory too: oxy submodule remove {path} --delete"
1267 +
            ))
1268 +
        );
1269 +
        return Ok(0);
1270 +
    }
1271 +
    if dir.exists() {
1272 +
        delete_tree(&dir)?;
1273 +
    }
1045 1274
    println!(
1046 1275
        "{} subrepo {} {}",
1047 1276
        ui::done("Removed"),
1048 1277
        ui::path(path),
1049 -
        ui::dim("(directory left in place; commit to record the removal)")
1278 +
        ui::dim("and its directory (commit to record the removal)")
1050 1279
    );
1051 1280
    Ok(0)
1052 1281
}
1053 1282
1283 +
/// Delete a directory tree that holds a repository: its object files are
1284 +
/// read-only, so each is made writable first (#164).
1285 +
fn delete_tree(dir: &std::path::Path) -> Result<()> {
1286 +
    fn writable(p: &std::path::Path) -> std::io::Result<()> {
1287 +
        let meta = std::fs::symlink_metadata(p)?;
1288 +
        if meta.file_type().is_symlink() {
1289 +
            return Ok(());
1290 +
        }
1291 +
        let mut perms = meta.permissions();
1292 +
        if perms.readonly() {
1293 +
            #[allow(clippy::permissions_set_readonly_false)]
1294 +
            perms.set_readonly(false);
1295 +
            std::fs::set_permissions(p, perms)?;
1296 +
        }
1297 +
        if meta.is_dir() {
1298 +
            for e in std::fs::read_dir(p)? {
1299 +
                writable(&e?.path())?;
1300 +
            }
1301 +
        }
1302 +
        Ok(())
1303 +
    }
1304 +
    writable(dir).map_err(|e| anyhow::anyhow!("{}: {e}", dir.display()))?;
1305 +
    std::fs::remove_dir_all(dir).map_err(|e| anyhow::anyhow!("{}: {e}", dir.display()))?;
1306 +
    Ok(())
1307 +
}
1308 +
1054 1309
pub fn deepen(remote: &str, depth: u32) -> Result<i32> {
1055 1310
    let repo = open()?;
1056 1311
    if !repo.is_shallow() {
…
17 unchanged lines
1074 1329
    ));
1075 1330
    Ok(0)
1076 1331
}
1332 +
1333 +
/// `r`, with a note when the remote's own token was refused while another
1334 +
/// sign-in for its hub would do (oxygen#155).
1335 +
fn noted<T>(repo: &oxy_core::repo::Repo, remote: &str, r: oxy_core::repo::Result<T>) -> Result<T> {
1336 +
    r.map_err(|e| {
1337 +
        let m = e.to_string();
1338 +
        match repo.stale_token_note(remote, &m) {
1339 +
            Some(note) => anyhow::anyhow!("{m}\n  note: {note}"),
1340 +
            None => e.into(),
1341 +
        }
1342 +
    })
1343 +
}
added src/cli/tlog.rs +205 −0
1 +
//! `oxy tlog verify` (spec §14.8, oxygenhub#238): audit a hub's whole
2 +
//! transparency log and list the entries that concern you.
3 +
4 +
use super::json;
5 +
use super::open;
6 +
use super::ui;
7 +
use anyhow::{anyhow, Result};
8 +
use oxy_core::hash::KeyId;
9 +
use std::path::{Path, PathBuf};
10 +
11 +
pub fn verify(hub: Option<&str>, key: Option<&str>, all: bool, as_json: bool) -> Result<i32> {
12 +
    let repo = open()?;
13 +
    let remote = repo.default_remote();
14 +
    let url = match hub {
15 +
        Some(h) if h.contains("://") => h.to_string(),
16 +
        Some(h) => format!("https://{h}"),
17 +
        None => repo.config.remotes.get(&remote).cloned().ok_or_else(|| {
18 +
            anyhow!("name the hub to check (this checkout has no remote {remote})")
19 +
        })?,
20 +
    };
21 +
    let ca = repo
22 +
        .config
23 +
        .remote_cas
24 +
        .get(&remote)
25 +
        .map(Path::new)
26 +
        .filter(|_| hub.is_none());
27 +
    let audit = repo.audit_log(&url, ca).map_err(|e| anyhow!("{e}"))?;
28 +
    let key = match key {
29 +
        Some(k) => Some(
30 +
            KeyId::from_hex(k)
31 +
                .ok_or_else(|| anyhow!("--key takes a full KeyId (64 hex digits)"))?,
32 +
        ),
33 +
        None => None,
34 +
    };
35 +
    let rid = repo.id().ok();
36 +
    let listed: Vec<(usize, &oxy_core::tlog::Entry)> = audit
37 +
        .entries
38 +
        .iter()
39 +
        .enumerate()
40 +
        .filter(|(_, e)| match (&key, all) {
41 +
            (_, true) => true,
42 +
            (Some(k), _) => e.signers.contains(k),
43 +
            (None, _) => Some(e.repo) == rid,
44 +
        })
45 +
        .collect();
46 +
    let what = match (&key, all) {
47 +
        (_, true) => "every entry".to_string(),
48 +
        (Some(k), _) => format!("entries signed by {}", k.short(12)),
49 +
        (None, _) => "this repository's entries".to_string(),
50 +
    };
51 +
    if as_json {
52 +
        println!(
53 +
            "{}",
54 +
            json::obj([
55 +
                ("hub", json::s(&audit.hub)),
56 +
                ("origin", json::s(&audit.checkpoint.origin)),
57 +
                ("size", json::n(audit.checkpoint.size)),
58 +
                (
59 +
                    "root",
60 +
                    json::s(&oxy_core::hash::ObjectRef(audit.checkpoint.root).to_hex())
61 +
                ),
62 +
                (
63 +
                    "witnesses",
64 +
                    json::arr(audit.witnessed.iter().map(|(name, t)| {
65 +
                        json::obj([
66 +
                            ("name", json::s(name)),
67 +
                            ("time_ms", json::n(t.saturating_mul(1000))),
68 +
                        ])
69 +
                    })),
70 +
                ),
71 +
                (
72 +
                    "entries",
73 +
                    json::arr(listed.iter().map(|(i, e)| {
74 +
                        json::obj([
75 +
                            ("index", json::n(*i)),
76 +
                            ("kind", json::s(&e.kind)),
77 +
                            (
78 +
                                "repo",
79 +
                                json::s(
80 +
                                    &e.repo
81 +
                                        .iter()
82 +
                                        .map(|b| format!("{b:02x}"))
83 +
                                        .collect::<String>(),
84 +
                                ),
85 +
                            ),
86 +
                            ("object", json::s(&e.object.to_hex())),
87 +
                            (
88 +
                                "signers",
89 +
                                json::arr(e.signers.iter().map(|k| json::s(&k.to_hex()))),
90 +
                            ),
91 +
                            ("received_ms", json::n(e.received)),
92 +
                        ])
93 +
                    })),
94 +
                ),
95 +
            ])
96 +
        );
97 +
        return Ok(0);
98 +
    }
99 +
    for n in &audit.notes {
100 +
        eprintln!("{} {n}", ui::caution());
101 +
    }
102 +
    println!(
103 +
        "{} {}: {} entries make the root its log key signed, and extend what you saw before",
104 +
        ui::done("Verified"),
105 +
        ui::path(&audit.checkpoint.origin),
106 +
        audit.checkpoint.size
107 +
    );
108 +
    // Its cosignatures: an upper bound on when every entry in it was made
109 +
    // (spec §14.8, *Integrated time*).
110 +
    if audit.witnessed.is_empty() {
111 +
        println!(
112 +
            "  {} {}",
113 +
            ui::note(),
114 +
            ui::dim("cosigned by no witness you trust")
115 +
        );
116 +
    } else {
117 +
        let by: Vec<String> = audit
118 +
            .witnessed
119 +
            .iter()
120 +
            .map(|(name, t)| format!("{name} {}", ui::ago(t.saturating_mul(1000))))
121 +
            .collect();
122 +
        println!(
123 +
            "  {} {}",
124 +
            ui::note(),
125 +
            ui::dim(format!("cosigned by {}", by.join(", ")))
126 +
        );
127 +
    }
128 +
    if listed.is_empty() {
129 +
        println!("  {} {}", ui::note(), ui::dim(format!("no {what} in it")));
130 +
        return Ok(0);
131 +
    }
132 +
    println!(
133 +
        "  {} {}",
134 +
        ui::note(),
135 +
        ui::dim(format!("{} of {what}:", listed.len()))
136 +
    );
137 +
    for (i, e) in listed {
138 +
        let signers: Vec<String> = e.signers.iter().map(|k| k.short(12)).collect();
139 +
        println!(
140 +
            "  #{i:<6} {:<11} {} {} {}",
141 +
            e.kind,
142 +
            ui::hash(e.object.short(12)),
143 +
            ui::dim(if signers.is_empty() {
144 +
                "unsigned".to_string()
145 +
            } else {
146 +
                format!("by {}", signers.join(", "))
147 +
            }),
148 +
            ui::dim(ui::ago(e.received))
149 +
        );
150 +
    }
151 +
    Ok(0)
152 +
}
153 +
154 +
/// `oxy witness serve`: cosign the named logs' checkpoints.
155 +
pub fn witness_serve(
156 +
    name: &str,
157 +
    logs: &[String],
158 +
    state: PathBuf,
159 +
    bind: &str,
160 +
    tls: Option<(PathBuf, PathBuf)>,
161 +
) -> Result<i32> {
162 +
    let home = oxy_core::repo::default_home();
163 +
    let (identity, sk) = oxy_core::repo::home_key(&home)
164 +
        .map_err(|e| anyhow!("{e}"))?
165 +
        .ok_or_else(|| anyhow!("{} names no key (author.key) with its secret: a witness signs with it; oxy identity create", home.display()))?;
166 +
    let mut parsed = Vec::new();
167 +
    for l in logs {
168 +
        let (origin, file) = l
169 +
            .split_once('=')
170 +
            .ok_or_else(|| anyhow!("--log takes ORIGIN=FILE, as oxygenhub.io/log=hub.identity"))?;
171 +
        let bytes = std::fs::read(file).map_err(|e| anyhow!("{file}: {e}"))?;
172 +
        let key =
173 +
            oxy_core::identity::Identity::decode(&bytes).map_err(|e| anyhow!("{file}: {e}"))?;
174 +
        parsed.push((origin.to_string(), key));
175 +
    }
176 +
    let listener = std::net::TcpListener::bind(bind).map_err(|e| anyhow!("binding {bind}: {e}"))?;
177 +
    println!(
178 +
        "{} {} logs as {} with key {} on {}://{bind}/add-checkpoint",
179 +
        ui::done("Witnessing"),
180 +
        parsed.len(),
181 +
        ui::path(name),
182 +
        identity.hash().short(12),
183 +
        if tls.is_some() { "https" } else { "http" }
184 +
    );
185 +
    for (o, k) in &parsed {
186 +
        println!(
187 +
            "  {} {} {}",
188 +
            ui::note(),
189 +
            o,
190 +
            ui::dim(format!("signed by {}", k.hash().short(12)))
191 +
        );
192 +
    }
193 +
    oxy_server::witness::run_blocking(
194 +
        listener,
195 +
        oxy_server::witness::WitnessConfig {
196 +
            name: name.to_string(),
197 +
            identity,
198 +
            sk,
199 +
            logs: parsed,
200 +
            state,
201 +
            tls: tls.map(|(cert, key)| oxy_server::TlsFiles { cert, key }),
202 +
        },
203 +
    )?;
204 +
    Ok(0)
205 +
}
modified src/cli/workspace.rs +725 −47
1 1
//! Commands that act on the local repository and working copy.
2 2
3 3
use super::format::{
4 -
    author_name, deps_line, deps_of, plural, short_id, signature_line, signature_mark_of,
5 -
    subject_of, via, when,
4 +
    author_name, awaits_seal, deps_line, deps_of, plural, short_id, signature_line,
5 +
    signature_mark_of, subject_of, via, when,
6 6
};
7 7
use super::json;
8 8
use super::open;
…
5 unchanged lines
14 14
use std::io::Write;
15 15
use std::path::PathBuf;
16 16
17 -
pub fn init(path: Option<PathBuf>, bare: bool) -> Result<i32> {
17 +
pub fn init(path: Option<PathBuf>, bare: bool, store: Option<&str>) -> Result<i32> {
18 18
    let path = path.unwrap_or(std::env::current_dir()?);
19 -
    let repo = if bare {
20 -
        Repo::init_bare(&path)?
21 -
    } else {
22 -
        Repo::init(&path)?
23 -
    };
19 +
    let repo = Repo::init_new(&path, bare, store)?;
24 20
    println!(
25 21
        "{} {}oxy repository in {}",
26 22
        ui::done("Initialized"),
…
4 unchanged lines
31 27
        "{}",
32 28
        ui::kv("bookmark", ui::bookmark(&repo.config.bookmark))
33 29
    );
30 +
    if let Some(url) = &repo.config.storage_url {
31 +
        println!(
32 +
            "{}",
33 +
            ui::kv(
34 +
                "store",
35 +
                format!(
36 +
                    "{url} (cache in {}, up to {} MiB)",
37 +
                    ui::path(repo.root().join(".oxy/cache").display()),
38 +
                    repo.config.storage_cache >> 20
39 +
                )
40 +
            )
41 +
        );
42 +
    }
34 43
    println!(
35 44
        "{}",
36 45
        ui::kv(
…
20 unchanged lines
57 66
    }
58 67
}
59 68
69 +
/// New untracked files left out of the snapshot for their size (oxygen#98):
70 +
/// named, with how to keep them out quietly or take them in.
71 +
fn print_skipped_large(repo: &Repo) {
72 +
    let skipped = repo.skipped_large();
73 +
    if skipped.is_empty() {
74 +
        return;
75 +
    }
76 +
    let size = |n: u64| {
77 +
        if n >= 1 << 30 {
78 +
            format!("{:.1} GiB", n as f64 / (1u64 << 30) as f64)
79 +
        } else {
80 +
            format!("{:.0} MiB", n as f64 / (1u64 << 20) as f64)
81 +
        }
82 +
    };
83 +
    println!();
84 +
    println!(
85 +
        "{} {}",
86 +
        ui::section(ui::warn("Not tracked")),
87 +
        ui::dim(format!("new files over {}", size(repo.config.max_new_file)))
88 +
    );
89 +
    for (p, n) in &skipped {
90 +
        println!("  {} {}  {}", ui::caution(), ui::path(p), ui::dim(size(*n)));
91 +
    }
92 +
    println!(
93 +
        "  {} {}",
94 +
        ui::note(),
95 +
        ui::dim(
96 +
            "to leave them out quietly, list them in .oxyignore; to track one, \
97 +
             raise snapshot.max-new-file-size (oxy config snapshot.max-new-file-size 2G)"
98 +
        )
99 +
    );
100 +
}
101 +
102 +
/// Name what a checkout removed from the working copy (oxygen#102): a
103 +
/// switch to a bookmark lacking files takes them off disk, which an
104 +
/// editor open on the tree notices before a person reading a count does.
105 +
fn print_removed(r: &oxy_core::repo::CheckoutReport) {
106 +
    if r.removed == 0 {
107 +
        return;
108 +
    }
109 +
    let more = r.removed.saturating_sub(r.removed_paths.len());
110 +
    let names = r.removed_paths.join(", ");
111 +
    let tail = if more > 0 {
112 +
        format!(" and {more} more")
113 +
    } else {
114 +
        String::new()
115 +
    };
116 +
    let lead = if r.removed >= 20 {
117 +
        ui::caution()
118 +
    } else {
119 +
        ui::note()
120 +
    };
121 +
    println!(
122 +
        "  {} {}",
123 +
        lead,
124 +
        ui::dim(format!(
125 +
            "removed from disk (the bookmark does not have them): {names}{tail}"
126 +
        ))
127 +
    );
128 +
}
129 +
130 +
/// Name the work a switch carried (oxygen#120): it is still on disk, as it
131 +
/// was, on the new bookmark.
132 +
fn print_carried(paths: &[String]) {
133 +
    if paths.is_empty() {
134 +
        return;
135 +
    }
136 +
    let shown: Vec<&str> = paths.iter().take(10).map(String::as_str).collect();
137 +
    let more = paths.len().saturating_sub(shown.len());
138 +
    println!(
139 +
        "  {} {}",
140 +
        ui::note(),
141 +
        ui::dim(format!(
142 +
            "carried, as they were: {}{}",
143 +
            shown.join(", "),
144 +
            if more > 0 {
145 +
                format!(" and {more} more")
146 +
            } else {
147 +
                String::new()
148 +
            }
149 +
        ))
150 +
    );
151 +
}
152 +
60 153
/// Folders holding a repository of their own, which oxy leaves out (oxygen#43):
61 154
/// named, with how to track them.
62 155
fn print_untracked_nested(repo: &Repo, nested: &[String]) {
…
49 unchanged lines
112 205
    }
113 206
}
114 207
208 +
/// Say when the working copy has fallen behind its own bookmark: a
209 +
/// "clean" status there hid a pull that left it on old code (oxygen#89).
210 +
fn print_behind(repo: &Repo) {
211 +
    if let Ok(Some(n)) = repo.working_copy_behind() {
212 +
        println!(
213 +
            "  {} {}",
214 +
            ui::caution(),
215 +
            ui::warn(format!(
216 +
                "the working copy is behind {}: it lacks {} the bookmark has; oxy checkout brings it up",
217 +
                repo.config.bookmark,
218 +
                plural(n, "change")
219 +
            ))
220 +
        );
221 +
    }
222 +
}
223 +
115 224
pub fn status(json: bool) -> Result<i32> {
116 225
    let repo = open()?;
117 226
    // Answered by the manifest alone, with no history to load (oxygen#35,
…
15 unchanged lines
133 242
            ui::sep(),
134 243
            ui::dim(plural(heads.len(), "head"))
135 244
        );
245 +
        print_behind(&repo);
136 246
        print_statuses(&statuses);
247 +
        print_pin_moves(&pin_moves(&repo, None));
137 248
        print_untracked_nested(&repo, &repo.untracked_nested());
249 +
        print_skipped_large(&repo);
138 250
        return Ok(0);
139 251
    }
140 252
    let sp = ui::spinner("Snapshotting the working copy…");
…
21 unchanged lines
162 274
        ui::sep(),
163 275
        ui::dim(plural(heads.len(), "head"))
164 276
    );
277 +
    print_behind(&repo);
165 278
    print_statuses(&snap.statuses);
279 +
    print_pin_moves(&pin_moves(&repo, None));
166 280
    print_untracked_nested(&repo, &snap.untracked_nested);
281 +
    print_skipped_large(&repo);
167 282
    if !snap.base_snapshot.is_conflicted() {
168 283
        return Ok(0);
169 284
    }
…
23 unchanged lines
193 308
                _ => ui::err(format!("{} unresolved", ui::glyph("✱", "!"))),
194 309
            };
195 310
            println!("  {}  {state}", ui::path(p));
311 +
            let resolved = matches!(
312 +
                snap.statuses
313 +
                    .iter()
314 +
                    .find(|s| s.path == *p)
315 +
                    .map(|s| s.status),
316 +
                Some(ChangeStatus::Resolved)
317 +
            );
318 +
            let both = if resolved {
319 +
                0
320 +
            } else {
321 +
                snap.base.both_sides(&f.node).iter().flatten().count()
322 +
            };
323 +
            if both > 0 {
324 +
                println!(
325 +
                    "     {}",
326 +
                    ui::dim(format!(
327 +
                        "{} adjacent {}: `oxy resolve --both {p}` keeps both sides, earliest first",
328 +
                        both,
329 +
                        if both == 1 { "insertion" } else { "insertions" }
330 +
                    ))
331 +
                );
332 +
            }
196 333
        }
197 334
    }
198 335
    Ok(2)
…
87 unchanged lines
286 423
    } else {
287 424
        oxy_core::repo::Selection::Paths(paths.to_vec())
288 425
    };
426 +
    // Pins this commit records as moved, said after it (oxygen#162): a pin
427 +
    // moved by work inside a subrepo is a pin bump, easy to make unasked.
428 +
    let moves = match &selection {
429 +
        oxy_core::repo::Selection::All => pin_moves(&repo, None),
430 +
        oxy_core::repo::Selection::Paths(p) => pin_moves(&repo, Some(p)),
431 +
        oxy_core::repo::Selection::Hunks(_) => Vec::new(),
432 +
    };
289 433
    let sp = ui::spinner("Committing…");
290 -
    let h = repo.commit_selection(message, &deps, &selection)?;
434 +
    let (h, partial) = repo.commit_selection_partial(message, &deps, &selection)?;
291 435
    // The metadata only: a first commit's change holds the whole tree.
292 436
    let meta = repo.store().get_change_meta(&h)?;
293 437
    sp.done(&format!(
…
3 unchanged lines
297 441
        ui::hash(format!("#{}", h.short(12))),
298 442
        ui::subject(subject_of(message))
299 443
    ));
444 +
    for m in &moves {
445 +
        let line = match (&m.to, m.dirty) {
446 +
            (Some(to), _) => format!(
447 +
                "recorded subrepo {}'s pin moving {} {} {}: a pin bump. Meant only your own files? `oxy commit PATHS` leaves pins out",
448 +
                m.path,
449 +
                m.from,
450 +
                ui::glyph("→", "->"),
451 +
                to
452 +
            ),
453 +
            (None, true) => format!(
454 +
                "subrepo {} has uncommitted edits of its own: the pin names its last committed state; the edits stay in it",
455 +
                m.path
456 +
            ),
457 +
            (None, false) => continue,
458 +
        };
459 +
        println!("  {} {}", ui::caution(), ui::warn(line));
460 +
    }
300 461
    print_untracked_nested(&repo, &repo.untracked_nested());
301 -
    if !matches!(selection, oxy_core::repo::Selection::All) {
462 +
    print_skipped_large(&repo);
463 +
    // Only a commit that took part of the working copy leaves anything.
464 +
    if partial {
302 465
        let rest = repo.snapshot_status("oxy commit")?;
303 466
        if !rest.statuses.is_empty() {
304 467
            println!(
…
10 unchanged lines
315 478
    Ok(0)
316 479
}
317 480
481 +
/// A subrepo whose pin the next commit would move (`to`), or which has
482 +
/// uncommitted edits of its own (oxygen#162).
483 +
struct PinMove {
484 +
    path: String,
485 +
    from: String,
486 +
    to: Option<String>,
487 +
    dirty: bool,
488 +
}
489 +
490 +
/// The subrepos (among `paths`, when given) whose checkout is not at the
491 +
/// pin the base records, or holds uncommitted edits. Only a checkout with
492 +
/// subrepos (an `.oxymodules`) pays for the look.
493 +
fn pin_moves(repo: &oxy_core::repo::Repo, paths: Option<&[String]>) -> Vec<PinMove> {
494 +
    if !repo.root().join(".oxymodules").exists() {
495 +
        return Vec::new();
496 +
    }
497 +
    let Ok(subs) = repo.subrepos() else {
498 +
        return Vec::new();
499 +
    };
500 +
    let chosen = |p: &str| {
501 +
        paths.is_none_or(|ps| {
502 +
            ps.iter().any(|x| {
503 +
                let x = x.trim_end_matches('/');
504 +
                p == x || p.starts_with(&format!("{x}/")) || x.is_empty() || x == "."
505 +
            })
506 +
        })
507 +
    };
508 +
    subs.into_iter()
509 +
        .filter(|s| s.committed && chosen(&s.path))
510 +
        .filter_map(|s| {
511 +
            let to = s
512 +
                .actual
513 +
                .as_ref()
514 +
                .filter(|a| !a.same_state(&s.pin))
515 +
                .map(|a| a.summary());
516 +
            (to.is_some() || s.dirty).then(|| PinMove {
517 +
                from: s.pin.summary(),
518 +
                to,
519 +
                dirty: s.dirty,
520 +
                path: s.path,
521 +
            })
522 +
        })
523 +
        .collect()
524 +
}
525 +
526 +
/// `oxy status`'s word on subrepos: a moved pin is a change the next commit
527 +
/// records; edits inside a subrepo are not.
528 +
fn print_pin_moves(moves: &[PinMove]) {
529 +
    if moves.is_empty() {
530 +
        return;
531 +
    }
532 +
    println!("{}", ui::section("Subrepo pins"));
533 +
    for m in moves {
534 +
        if let Some(to) = &m.to {
535 +
            println!(
536 +
                "  {} {}  {} {} {}  {}",
537 +
                ui::warn(ui::glyph("↕", "<>")),
538 +
                ui::path(&m.path),
539 +
                ui::dim(&m.from),
540 +
                ui::dim(ui::glyph("→", "->")),
541 +
                to,
542 +
                ui::dim("pin moved: a bare `oxy commit` records it")
543 +
            );
544 +
        }
545 +
        if m.dirty {
546 +
            println!(
547 +
                "  {} {}  {}",
548 +
                ui::warn(ui::glyph("✎", "~")),
549 +
                ui::path(&m.path),
550 +
                ui::dim("uncommitted edits inside it: commit them there; the parent records only its pin")
551 +
            );
552 +
        }
553 +
    }
554 +
}
555 +
318 556
pub fn blame(path: &str, json: bool) -> Result<i32> {
319 557
    let repo = open()?;
320 558
    let lines = repo.blame(path)?;
…
337 unchanged lines
658 896
    } else {
659 897
        &entries
660 898
    };
661 -
    let id_prefix = unique_prefixes(all.iter().map(|e| e.change.change_id.to_hex()));
899 +
    let prefixes = oxy_core::timing::phase("log: id prefixes");
900 +
    let id_prefix = id_prefixes(
901 +
        all.iter().map(|e| e.change.change_id.0).collect(),
902 +
        entries.iter().take(shown).map(|e| e.change.change_id.0),
903 +
    );
904 +
    drop(prefixes);
662 905
    let labels = reference_labels(&repo)?;
663 906
    for (e, row) in entries.iter().take(shown).zip(&rows) {
664 907
        let c = &e.change;
…
3 unchanged lines
668 911
            e.is_working_copy && conflicted,
669 912
        );
670 913
        let id = c.change_id.to_hex();
671 -
        let n = id_prefix.get(&id).copied().unwrap_or(8).min(8);
914 +
        let n = id_prefix.get(&c.change_id.0).copied().unwrap_or(8).min(8);
672 915
        let mut head = vec![format!("{}{}", ui::id(&id[..n]), ui::dim(&id[n..8]))];
673 916
        head.push(ui::author(author_name(&repo, c)));
674 917
        head.push(ui::dim(ui::ago(c.hlc.wall_ms)));
…
9 unchanged lines
684 927
            if let Some(agent) = via(&repo, &status, &c.author) {
685 928
                head.push(ui::dim(format!("via {}", agent.name)));
686 929
            }
687 -
            head.push(signature_mark_of(&status));
930 +
            head.push(signature_mark_of(
931 +
                &status,
932 +
                awaits_seal(&repo, &status, &c.author),
933 +
            ));
688 934
        }
689 935
        println!("{}{}", node_line(row, &bullet), head.join(" "));
690 936
        let subject = if e.is_working_copy && c.message.is_empty() {
…
135 unchanged lines
826 1072
}
827 1073
828 1074
/// For each hex ID, the length of its shortest prefix no other ID shares.
1075 +
/// The shortest prefix, in hex digits, that names each of `wanted` among
1076 +
/// `all`: worked out for the few shown rather than for the whole history.
1077 +
fn id_prefixes(
1078 +
    mut all: Vec<[u8; 16]>,
1079 +
    wanted: impl Iterator<Item = [u8; 16]>,
1080 +
) -> std::collections::HashMap<[u8; 16], usize> {
1081 +
    all.sort_unstable();
1082 +
    all.dedup();
1083 +
    // Hex digits two ids share at the start.
1084 +
    let common = |a: &[u8; 16], b: &[u8; 16]| {
1085 +
        let bytes = a.iter().zip(b).take_while(|(x, y)| x == y).count();
1086 +
        match (a.get(bytes), b.get(bytes)) {
1087 +
            (Some(x), Some(y)) if x >> 4 == y >> 4 => 2 * bytes + 1,
1088 +
            _ => 2 * bytes,
1089 +
        }
1090 +
    };
1091 +
    let mut out = std::collections::HashMap::new();
1092 +
    for id in wanted {
1093 +
        let Ok(i) = all.binary_search(&id) else {
1094 +
            continue;
1095 +
        };
1096 +
        let before = i.checked_sub(1).map_or(0, |j| common(&all[j], &id));
1097 +
        let after = all.get(i + 1).map_or(0, |n| common(&id, n));
1098 +
        out.insert(id, before.max(after) + 1);
1099 +
    }
1100 +
    out
1101 +
}
1102 +
829 1103
fn unique_prefixes(ids: impl Iterator<Item = String>) -> std::collections::HashMap<String, usize> {
830 1104
    let mut v: Vec<String> = ids.collect();
831 1105
    v.sort();
…
99 unchanged lines
931 1205
            print!("{}", render_change(&c, &base, &snapshot));
932 1206
        }
933 1207
        None => {
934 -
            let (wc, base, snapshot) = repo.working_copy_with_base()?;
1208 +
            // Nothing changed: nothing to show, and no base to load.
1209 +
            let Some((wc, base, snapshot)) = repo.working_copy_with_base_unless_unchanged()? else {
1210 +
                return Ok(0);
1211 +
            };
935 1212
            let text = {
936 1213
                let _t = oxy_core::timing::phase("diff: render the change");
937 1214
                render_change(&wc, &base, &snapshot)
…
10 unchanged lines
948 1225
pub fn cat(path: &str, change: Option<&str>) -> Result<i32> {
949 1226
    let repo = open()?;
950 1227
    let heads = change.map(|c| repo.resolve_heads(c)).transpose()?;
951 -
    let bytes = repo.file_at(path, heads.as_deref())?;
1228 +
    let bytes = match repo.file_at(path, heads.as_deref()) {
1229 +
        Ok(b) => b,
1230 +
        // -r CHANGE reads that change's dependency closure, not the
1231 +
        // bookmark it is on: say so when the bookmark has the file
1232 +
        // (oxygen#76).
1233 +
        Err(e) => match change {
1234 +
            Some(c)
1235 +
                if !repo.view()?.bookmarks.contains_key(c)
1236 +
                    && repo.tag_heads(c)?.is_none()
1237 +
                    && repo.file_at(path, None).is_ok() =>
1238 +
            {
1239 +
                anyhow::bail!(
1240 +
                    "{path} is not in {c}'s dependency closure (the change and what it \
1241 +
                     depends on); it is on the bookmark: try -r {}, or --at-op OP for the \
1242 +
                     state at an operation",
1243 +
                    repo.config.bookmark
1244 +
                )
1245 +
            }
1246 +
            _ => return Err(e.into()),
1247 +
        },
1248 +
    };
952 1249
    use std::io::Write as _;
953 1250
    std::io::stdout().write_all(&bytes)?;
954 1251
    Ok(0)
…
104 unchanged lines
1059 1356
    Ok(0)
1060 1357
}
1061 1358
1062 -
pub fn show(change: Option<&str>, revset: Option<&str>, as_json: bool) -> Result<i32> {
1359 +
pub fn show(change: Option<&str>, revset: Option<&str>, stat: bool, as_json: bool) -> Result<i32> {
1063 1360
    let repo = open()?;
1064 1361
    repo.snapshot_status("oxy show")?;
1065 1362
    let Some(r) = revset else {
1066 1363
        let h = repo.resolve_change(change.unwrap_or("@"))?;
1067 -
        return show_one(&repo, h, as_json, false);
1364 +
        return show_one(&repo, h, as_json, false, stat);
1068 1365
    };
1069 1366
    // Several changes: a JSON array, or the changes one after another.
1070 1367
    let hashes = repo.revset(r)?.hashes;
…
3 unchanged lines
1074 1371
            if i > 0 {
1075 1372
                print!(",");
1076 1373
            }
1077 -
            show_one(&repo, *h, true, true)?;
1374 +
            show_one(&repo, *h, true, true, stat)?;
1078 1375
        }
1079 1376
        println!("]");
1080 1377
        return Ok(0);
…
6 unchanged lines
1087 1384
        if i > 0 {
1088 1385
            println!();
1089 1386
        }
1090 -
        show_one(&repo, *h, false, false)?;
1387 +
        show_one(&repo, *h, false, false, stat)?;
1091 1388
    }
1092 1389
    Ok(0)
1093 1390
}
1094 1391
1392 +
/// Lines added and removed in one file's diff.
1393 +
fn counts(f: &oxy_core::repo::FileDiff) -> (usize, usize) {
1394 +
    f.hunks
1395 +
        .iter()
1396 +
        .fold((0, 0), |(a, r), h| (a + h.added.len(), r + h.removed.len()))
1397 +
}
1398 +
1399 +
/// What `--stat` prints (oxygen#119): each file the change touched, its
1400 +
/// lines added and removed, then the total.
1401 +
fn print_stat(files: &[oxy_core::repo::FileDiff]) {
1402 +
    let width = files
1403 +
        .iter()
1404 +
        .map(|f| f.path.chars().count())
1405 +
        .max()
1406 +
        .unwrap_or(0)
1407 +
        .min(60);
1408 +
    let (mut added, mut removed) = (0, 0);
1409 +
    for f in files {
1410 +
        let (a, r) = counts(f);
1411 +
        added += a;
1412 +
        removed += r;
1413 +
        let path = match &f.from {
1414 +
            Some(from) => format!("{from} → {}", f.path),
1415 +
            None => f.path.clone(),
1416 +
        };
1417 +
        println!(
1418 +
            " {path:<width$}  {} {}  {}",
1419 +
            ui::ok(format!("{:>6}", format!("+{a}"))),
1420 +
            ui::err(format!("{:>6}", format!("-{r}"))),
1421 +
            ui::dim(&f.kind)
1422 +
        );
1423 +
    }
1424 +
    println!(
1425 +
        " {}",
1426 +
        ui::dim(format!(
1427 +
            "{} changed, {added} added, {removed} removed",
1428 +
            plural(files.len(), "file")
1429 +
        ))
1430 +
    );
1431 +
}
1432 +
1095 1433
/// One change, as `oxy show` prints it; `inline` JSON goes without its
1096 -
/// newline, for an array.
1097 -
fn show_one(repo: &Repo, h: ChangeHash, as_json: bool, inline: bool) -> Result<i32> {
1434 +
/// newline, for an array. `stat` lists the files instead of the diff.
1435 +
fn show_one(repo: &Repo, h: ChangeHash, as_json: bool, inline: bool, stat: bool) -> Result<i32> {
1098 1436
    let c = repo.store().get_change(&h)?;
1099 1437
    if as_json {
1100 1438
        let files = repo.diff_files(Some(&h))?;
…
9 unchanged lines
1110 1448
        body.push_str(&format!(
1111 1449
            ",{}:{}}}",
1112 1450
            json::s("files"),
1113 -
            json::arr(files.iter().map(|f| json::obj([
1114 -
                ("path", json::s(&f.path)),
1115 -
                ("kind", json::s(&f.kind)),
1116 -
                ("from", json::opt(f.from.as_deref(), json::s))
1117 -
            ])))
1451 +
            json::arr(files.iter().map(|f| {
1452 +
                let (a, r) = counts(f);
1453 +
                json::obj([
1454 +
                    ("path", json::s(&f.path)),
1455 +
                    ("kind", json::s(&f.kind)),
1456 +
                    ("from", json::opt(f.from.as_deref(), json::s)),
1457 +
                    ("added", a.to_string()),
1458 +
                    ("removed", r.to_string()),
1459 +
                ])
1460 +
            }))
1118 1461
        ));
1119 1462
        if inline {
1120 1463
            print!("{body}");
…
51 unchanged lines
1172 1515
        println!("    {line}");
1173 1516
    }
1174 1517
    println!();
1175 -
    print!("{}", repo.describe_change(&h)?);
1518 +
    if stat {
1519 +
        print_stat(&repo.diff_files(Some(&h))?);
1520 +
    } else {
1521 +
        print!("{}", repo.describe_change(&h)?);
1522 +
    }
1176 1523
    Ok(0)
1177 1524
}
1178 1525
…
41 unchanged lines
1220 1567
            format!("; carried {}", plural(carried.len(), "changed path"))
1221 1568
        }
1222 1569
    ));
1570 +
    print_removed(&r);
1223 1571
    for n in &r.notes {
1224 1572
        println!("  {} {n}", ui::note());
1225 1573
    }
1574 +
    print_kept_both(&r);
1226 1575
    if r.conflicted {
1227 1576
        println!(
1228 1577
            "  {} {}",
…
5 unchanged lines
1234 1583
    Ok(0)
1235 1584
}
1236 1585
1237 -
pub fn resolve(as_json: bool) -> Result<i32> {
1586 +
/// Union files (`merge=union`, oxygen#145) a checkout settled itself.
1587 +
fn print_kept_both(r: &oxy_core::repo::CheckoutReport) {
1588 +
    for k in &r.kept_both {
1589 +
        let how = if k.identical.len() == k.settled.len() {
1590 +
            "kept one of its identical sides"
1591 +
        } else if k.identical.is_empty() {
1592 +
            "kept both sides (merge=union)"
1593 +
        } else {
1594 +
            "kept one of identical sides and both of others (merge=union)"
1595 +
        };
1596 +
        println!(
1597 +
            "  {} settled {}: {how}; the next commit records it",
1598 +
            ui::note(),
1599 +
            ui::path(&k.path)
1600 +
        );
1601 +
    }
1602 +
}
1603 +
1604 +
pub fn resolve(paths: &[String], both: bool, reverse: bool, as_json: bool) -> Result<i32> {
1238 1605
    let repo = open()?;
1239 -
    let snap = repo.snapshot("oxy resolve")?;
1606 +
    let mut snap = repo.snapshot("oxy resolve")?;
1607 +
    let paths: Vec<String> = paths
1608 +
        .iter()
1609 +
        .map(|p| p.trim_start_matches("./").trim_end_matches('/').to_string())
1610 +
        .collect();
1611 +
    if both {
1612 +
        // Adjacent insertions (oxygen#121): every side kept, in order.
1613 +
        let kept = repo.keep_both(&snap, &paths, reverse)?;
1614 +
        if !as_json {
1615 +
            for k in &kept {
1616 +
                if !k.settled.is_empty() {
1617 +
                    let n: Vec<String> = k.settled.iter().map(|n| n.to_string()).collect();
1618 +
                    println!("kept both  {}: conflict {}", k.path, n.join(", "));
1619 +
                }
1620 +
                for (n, why) in &k.left {
1621 +
                    println!("markers    {}: conflict {n}: {why}", k.path);
1622 +
                }
1623 +
            }
1624 +
        }
1625 +
        snap = repo.snapshot("oxy resolve")?;
1626 +
    } else if !paths.is_empty() {
1627 +
        // A replace conflict has no markers to edit out: the user says which
1628 +
        // content wins (oxygen#140, spec §9.3), and the snapshot builds it.
1629 +
        repo.choose_resolutions(&snap.base_snapshot, &paths)?;
1630 +
        snap = repo.snapshot("oxy resolve")?;
1631 +
    }
1240 1632
    if as_json {
1241 1633
        println!("{}", repo.status_json(&snap)?);
1242 1634
        return Ok(if snap.base_snapshot.is_conflicted() {
…
42 unchanged lines
1285 1677
1286 1678
pub fn undo() -> Result<i32> {
1287 1679
    let repo = open()?;
1288 -
    let op = repo.undo()?;
1289 -
    println!("{} now at: {}", ui::done("Undone;"), ui::dim(&op.command));
1680 +
    let u = repo.undo()?;
1681 +
    println!("{} {}", ui::done("Undone:"), u.command);
1682 +
    for c in &u.changes {
1683 +
        println!("  {} {}", ui::note(), change_words(&repo, c));
1684 +
    }
1685 +
    if u.others_kept > 0 {
1686 +
        let s = if u.others_kept == 1 { "" } else { "s" };
1687 +
        println!(
1688 +
            "  {} {}",
1689 +
            ui::note(),
1690 +
            ui::dim(format!(
1691 +
                "{} newer operation{s} from other workspaces left as they are",
1692 +
                u.others_kept
1693 +
            ))
1694 +
        );
1695 +
    }
1696 +
    for (bookmark, remote) in &u.pushed {
1697 +
        println!(
1698 +
            "  {} {}",
1699 +
            ui::caution(),
1700 +
            ui::warn(format!(
1701 +
                "{remote}/{bookmark} already has what this undid: undone here only; the next push of {bookmark} needs --force to take it back there"
1702 +
            ))
1703 +
        );
1704 +
    }
1290 1705
    where_now(&repo);
1291 1706
    Ok(0)
1292 1707
}
…
241 unchanged lines
1534 1949
    );
1535 1950
}
1536 1951
1537 -
pub fn gc(now: bool) -> Result<i32> {
1538 -
    let repo = open()?;
1952 +
pub fn gc(now: bool, expire_snapshots: bool) -> Result<i32> {
1953 +
    let mut repo = open()?;
1954 +
    if expire_snapshots {
1955 +
        repo.config.snapshot_retention_days = 0;
1956 +
    }
1539 1957
    let sp = ui::spinner("Packing and pruning…");
1540 -
    let r = repo.gc_with(now, &mut |phase| sp.set(&format!("{phase}…")))?;
1958 +
    // Cylinder's artifacts are named from outside the history: keep them.
1959 +
    let artifacts = cylinder::roots(&repo)?;
1960 +
    let r = repo.gc_keeping(now, &artifacts, &mut |phase| sp.set(&format!("{phase}…")))?;
1541 1961
    sp.clear();
1542 1962
    if r.packed.objects > 0 {
1543 1963
        println!(
…
24 unchanged lines
1568 1988
        },
1569 1989
        plural(repo.store().pack_count(), "pack")
1570 1990
    );
1991 +
    // What earlier repacks replaced stays an hour for readers still using
1992 +
    // it (oxygenhub#249); a collection after that deletes it.
1993 +
    let kept = if r.retired.kept > 0 {
1994 +
        format!(
1995 +
            "{} retired less than an hour ago kept for readers",
1996 +
            plural(r.retired.kept, "file")
1997 +
        )
1998 +
    } else {
1999 +
        String::new()
2000 +
    };
2001 +
    if r.retired.packs + r.retired.loose > 0 {
2002 +
        println!(
2003 +
            "Deleted {} and {} retired by earlier repacks{}.",
2004 +
            plural(r.retired.packs, "pack"),
2005 +
            plural(r.retired.loose, "loose object"),
2006 +
            if kept.is_empty() {
2007 +
                String::new()
2008 +
            } else {
2009 +
                format!("; {kept}")
2010 +
            }
2011 +
        );
2012 +
    } else if !kept.is_empty() {
2013 +
        println!("{kept}.");
2014 +
    }
1571 2015
    Ok(0)
1572 2016
}
1573 2017
…
261 unchanged lines
1835 2279
    Ok(0)
1836 2280
}
1837 2281
1838 -
pub fn amend(message: Option<&str>, into: Option<&str>) -> Result<i32> {
2282 +
pub fn amend(message: Option<&str>, into: Option<&str>, paths: &[String]) -> Result<i32> {
1839 2283
    let repo = open()?;
2284 +
    let before = repo.view()?;
1840 2285
    let new = match into {
1841 2286
        Some(spec) => {
1842 2287
            let target = repo.resolve_change(spec)?;
1843 2288
            repo.amend_into(&target, message)?
1844 2289
        }
2290 +
        None if !paths.is_empty() => repo.amend_paths(message, paths)?,
1845 2291
        None => repo.amend(message)?,
1846 2292
    };
1847 2293
    let c = repo.store().get_change_meta(&new)?;
…
4 unchanged lines
1852 2298
        ui::hash(format!("#{}", new.short(12))),
1853 2299
        ui::subject(subject_of(&c.message))
1854 2300
    );
2301 +
    // Rewriting what a remote already has means the next push replaces it
2302 +
    // there (oxygen#99): say so now, not when the push is refused.
2303 +
    let bookmark = repo.config.bookmark.clone();
2304 +
    let (had, has) = (
2305 +
        repo.heads_of(&before, &bookmark)?,
2306 +
        repo.heads_of(&repo.view()?, &bookmark)?,
2307 +
    );
2308 +
    for (remote, bookmarks) in &before.remotes {
2309 +
        let Some(st) = bookmarks.get(&bookmark) else {
2310 +
            continue;
2311 +
        };
2312 +
        let theirs = repo.store().get_state(st)?.heads;
2313 +
        if theirs.iter().any(|h| had.contains(h) && !has.contains(h)) {
2314 +
            println!(
2315 +
                "  {} {}",
2316 +
                ui::caution(),
2317 +
                ui::warn(format!(
2318 +
                    "{remote}/{bookmark} has the change as it was: the next push of {bookmark} needs --force to replace it"
2319 +
                ))
2320 +
            );
2321 +
        }
2322 +
    }
1855 2323
    where_now(&repo);
1856 2324
    Ok(0)
1857 2325
}
…
170 unchanged lines
2028 2496
    Ok(0)
2029 2497
}
2030 2498
2499 +
pub fn tag_delete(name: &str, reason: &str) -> Result<i32> {
2500 +
    let repo = open()?;
2501 +
    let t = repo.tag_delete(name, reason)?;
2502 +
    println!(
2503 +
        "{} {} {} {}{}",
2504 +
        ui::done("Withdrew"),
2505 +
        ui::bookmark(&t.name),
2506 +
        ui::dim(ui::glyph("→", "->")),
2507 +
        ui::hash(format!("#{}", t.state.short(12))),
2508 +
        t.signer
2509 +
            .map(|s| format!(", signed by {s}"))
2510 +
            .unwrap_or_else(|| ui::dim(", unsigned"))
2511 +
    );
2512 +
    println!(
2513 +
        "  {} {}",
2514 +
        ui::note(),
2515 +
        ui::dim("`oxy push` sends the withdrawal to a remote that has the tag; the name is not used again")
2516 +
    );
2517 +
    Ok(0)
2518 +
}
2519 +
2031 2520
pub fn tag_list(as_json: bool) -> Result<i32> {
2032 2521
    let repo = open()?;
2033 2522
    let tags = repo.tags()?;
…
272 unchanged lines
2306 2795
        .map(|b| b.heads)
2307 2796
        .ok_or_else(|| anyhow::anyhow!("no bookmark {name}"))?;
2308 2797
    let gap = repo.state_gap(&current, &named)?;
2798 +
    // A fast-forward: the new heads hold the bookmark's newest head, so
2799 +
    // what they leave out is older work nothing since has built on, such
2800 +
    // as an early change no later one depends on (oxygen#78). Keep it, as
2801 +
    // --upto does, and say so.
2802 +
    let newest = current
2803 +
        .iter()
2804 +
        .filter_map(|h| repo.store().get_change_meta(h).ok().map(|m| (m.hlc, *h)))
2805 +
        .max()
2806 +
        .map(|(_, h)| h);
2807 +
    // Ahead, too: a head the bookmark holds already (it would only drop
2808 +
    // the others) is not a move forward.
2809 +
    let ahead = repo
2810 +
        .load_state_history(&current)
2811 +
        .is_ok_and(|st| named.iter().any(|h| !st.contains(h)));
2812 +
    let forward = !only
2813 +
        && ahead
2814 +
        && !gap.earlier.is_empty()
2815 +
        && newest.is_some_and(|n| {
2816 +
            repo.load_state_history(&named)
2817 +
                .is_ok_and(|st| st.contains(&n))
2818 +
        });
2309 2819
    let heads = if upto {
2310 2820
        gap.upto
2821 +
    } else if forward {
2822 +
        println!(
2823 +
            "  {} {}",
2824 +
            ui::note(),
2825 +
            ui::dim(format!(
2826 +
                "{} builds on {name}'s newest head; keeping {} it holds that {0} does not depend on",
2827 +
                changes.join(" "),
2828 +
                plural(gap.earlier.len(), "change")
2829 +
            ))
2830 +
        );
2831 +
        gap.upto
2311 2832
    } else if only || gap.earlier.is_empty() {
2312 2833
        named
2313 2834
    } else {
…
30 unchanged lines
2344 2865
    Ok(0)
2345 2866
}
2346 2867
2347 -
pub fn bookmark_list(all: bool, as_json: bool) -> Result<i32> {
2868 +
/// Heads with their metadata, newest first.
2869 +
fn heads_newest_first(
2870 +
    repo: &Repo,
2871 +
    heads: &[oxy_core::hash::ChangeHash],
2872 +
) -> Vec<(oxy_core::hash::ChangeHash, oxy_core::state::ChangeMeta)> {
2873 +
    let mut out: Vec<_> = heads
2874 +
        .iter()
2875 +
        .filter_map(|h| repo.store().get_change_meta(h).ok().map(|m| (*h, m)))
2876 +
        .collect();
2877 +
    out.sort_by_key(|(_, m)| std::cmp::Reverse(m.hlc));
2878 +
    out
2879 +
}
2880 +
2881 +
/// One head as a line: id, hash, author, when, subject.
2882 +
fn print_head(
2883 +
    repo: &Repo,
2884 +
    indent: &str,
2885 +
    h: &oxy_core::hash::ChangeHash,
2886 +
    m: &oxy_core::state::ChangeMeta,
2887 +
) {
2888 +
    println!(
2889 +
        "{indent}{} {} {}{}{}{}{}",
2890 +
        ui::id(m.change_id.short(8)),
2891 +
        ui::dim(format!("#{}", &h.to_hex()[..8])),
2892 +
        author_name(repo, m),
2893 +
        ui::sep(),
2894 +
        ui::dim(ui::ago(m.hlc.wall_ms)),
2895 +
        ui::sep(),
2896 +
        ui::subject(subject_of(&m.message))
2897 +
    );
2898 +
}
2899 +
2900 +
pub fn bookmark_show(name: Option<&str>, as_json: bool) -> Result<i32> {
2348 2901
    let repo = open()?;
2902 +
    let name = name.unwrap_or(&repo.config.bookmark).to_string();
2903 +
    let view = repo.view()?;
2904 +
    let st = match view.bookmarks.get(&name) {
2905 +
        Some(st) => *st,
2906 +
        None => match oxy_core::repo::remote_bookmark(&view, &name) {
2907 +
            Some((_, _, st)) => st,
2908 +
            None => anyhow::bail!("no bookmark {name}: oxy bookmark list --all lists them"),
2909 +
        },
2910 +
    };
2911 +
    let heads = repo.store().get_state(&st)?.heads;
2912 +
    let shown = heads_newest_first(&repo, &heads);
2913 +
    if as_json {
2914 +
        println!(
2915 +
            "{}",
2916 +
            json::obj([
2917 +
                ("name", json::s(&name)),
2918 +
                ("current", json::b(name == repo.config.bookmark)),
2919 +
                (
2920 +
                    "heads",
2921 +
                    json::arr(shown.iter().map(|(h, m)| {
2922 +
                        json::obj([
2923 +
                            ("hash", json::s(&h.to_hex())),
2924 +
                            ("id", json::s(&m.change_id.to_string())),
2925 +
                            ("author", json::s(&author_name(&repo, m))),
2926 +
                            ("time_ms", json::n(m.hlc.wall_ms)),
2927 +
                            ("subject", json::s(subject_of(&m.message))),
2928 +
                        ])
2929 +
                    }))
2930 +
                ),
2931 +
            ])
2932 +
        );
2933 +
        return Ok(0);
2934 +
    }
2935 +
    println!(
2936 +
        "{} {}{}{}",
2937 +
        ui::ok(ui::glyph("●", "*")),
2938 +
        ui::bookmark(&name),
2939 +
        ui::sep(),
2940 +
        ui::dim(plural(heads.len(), "head"))
2941 +
    );
2942 +
    for (h, m) in &shown {
2943 +
        print_head(&repo, "  ", h, m);
2944 +
    }
2945 +
    if heads.len() > 1 {
2946 +
        println!(
2947 +
            "  {}",
2948 +
            ui::dim("None depends on another: the bookmark is all of them and their dependencies.")
2949 +
        );
2950 +
    }
2951 +
    Ok(0)
2952 +
}
2953 +
2954 +
pub fn bookmark_list(all: bool, verbose: bool, as_json: bool) -> Result<i32> {
2955 +
    let repo = open()?;
2349 2956
    let remote_only = repo.remote_only_bookmarks()?;
2350 2957
    if as_json {
2351 2958
        let mut items: Vec<String> = repo
…
6 unchanged lines
2358 2965
                    ("current", json::b(b.current)),
2359 2966
                    ("ahead", json::opt(b.ahead, json::n)),
2360 2967
                    ("behind", json::opt(b.behind, json::n)),
2968 +
                    ("remote", json::opt(b.remote.as_deref(), json::s)),
2361 2969
                ])
2362 2970
            })
2363 2971
            .collect();
…
25 unchanged lines
2389 2997
        if let Some(n) = b.behind.filter(|n| *n > 0) {
2390 2998
            tail += &ui::warn(format!(" {}{n}", ui::glyph("↓", "-")));
2391 2999
        }
2392 -
        let head = b
2393 -
            .heads
2394 -
            .first()
2395 -
            .map(|h| {
2396 -
                repo.store()
2397 -
                    .get_change(h)
2398 -
                    .map(|c| ui::subject(subject_of(&c.message)))
2399 -
                    .unwrap_or_default()
2400 -
            })
2401 -
            .unwrap_or_else(|| ui::dim("(empty)"));
3000 +
        // Which remote, when there is a choice (oxygen#83).
3001 +
        if !tail.is_empty() && repo.config.remotes.len() > 1 {
3002 +
            if let Some(r) = &b.remote {
3003 +
                tail += &ui::dim(format!(" {r}"));
3004 +
            }
3005 +
        }
3006 +
        let shown = heads_newest_first(&repo, &b.heads);
3007 +
        // The newest head's subject, so a busy bookmark shows its latest work.
3008 +
        let head = match shown.first() {
3009 +
            Some(_) if verbose => String::new(),
3010 +
            Some((_, m)) if b.heads.len() > 1 => format!(
3011 +
                "{} {}",
3012 +
                ui::dim("newest"),
3013 +
                ui::subject(subject_of(&m.message))
3014 +
            ),
3015 +
            Some((_, m)) => ui::subject(subject_of(&m.message)),
3016 +
            None => ui::dim("(empty)"),
3017 +
        };
3018 +
        let head = if head.is_empty() {
3019 +
            head
3020 +
        } else {
3021 +
            format!("  {head}")
3022 +
        };
2402 3023
        println!(
2403 -
            "{mark} {}{}{}{tail}  {head}",
3024 +
            "{mark} {}{}{}{tail}{head}",
2404 3025
            ui::bookmark(&b.name),
2405 3026
            ui::sep(),
2406 3027
            ui::dim(plural(b.heads.len(), "head"))
2407 3028
        );
3029 +
        if verbose {
3030 +
            for (h, m) in &shown {
3031 +
                print_head(&repo, "    ", h, m);
3032 +
            }
3033 +
        }
2408 3034
    }
2409 3035
    if remote_only.is_empty() {
2410 3036
        return Ok(0);
…
38 unchanged lines
2449 3075
        );
2450 3076
    }
2451 3077
    let name = local.as_str();
3078 +
    // Only behind its remote: take the remote's state, so the switch does
3079 +
    // not check out a stale one (oxygen#74).
3080 +
    if let Some(n) = repo.catch_up_bookmark(name)? {
3081 +
        println!(
3082 +
            "{} {} with {}: {} it lacked",
3083 +
            ui::done("Caught up"),
3084 +
            ui::bookmark(name),
3085 +
            ui::dim(format!("{}/{name}", repo.default_remote())),
3086 +
            plural(n, "change")
3087 +
        );
3088 +
    }
2452 3089
    let sp = ui::spinner(&format!("Switching to {name}…"));
2453 3090
    let (r, carried) = if carry {
2454 3091
        repo.bookmark_switch_carry(name)?
…
12 unchanged lines
2467 3104
            format!("; carried {}", plural(carried.len(), "changed path"))
2468 3105
        }
2469 3106
    ));
3107 +
    print_removed(&r);
3108 +
    print_carried(&carried);
3109 +
    print_kept_both(&r);
2470 3110
    if r.conflicted {
2471 3111
        println!(
2472 3112
            "  {} {}",
…
125 unchanged lines
2598 3238
    where_now(&repo);
2599 3239
    Ok(0)
2600 3240
}
3241 +
3242 +
#[cfg(test)]
3243 +
mod tests {
3244 +
    use super::*;
3245 +
3246 +
    #[test]
3247 +
    fn id_prefixes_agree_with_the_whole_list() {
3248 +
        // Ids that share long prefixes, and some that differ only in a
3249 +
        // half byte, from a fixed sequence.
3250 +
        let mut x: u64 = 0x9e37_79b9_7f4a_7c15;
3251 +
        let mut next = || {
3252 +
            x ^= x << 13;
3253 +
            x ^= x >> 7;
3254 +
            x ^= x << 17;
3255 +
            x
3256 +
        };
3257 +
        let mut ids: Vec<[u8; 16]> = Vec::new();
3258 +
        for _ in 0..2000 {
3259 +
            let mut id = [0u8; 16];
3260 +
            id[..8].copy_from_slice(&next().to_le_bytes());
3261 +
            id[8..].copy_from_slice(&next().to_le_bytes());
3262 +
            let share = (next() % 4) as usize;
3263 +
            if let Some(prev) = ids.last() {
3264 +
                id[..share].copy_from_slice(&prev[..share]);
3265 +
                if next() % 2 == 0 {
3266 +
                    id[share] = (prev[share] & 0xf0) | (id[share] & 0x0f);
3267 +
                }
3268 +
            }
3269 +
            ids.push(id);
3270 +
        }
3271 +
        let hex = |id: &[u8; 16]| oxy_core::ids::ChangeId(*id).to_hex();
3272 +
        let whole = unique_prefixes(ids.iter().map(hex));
3273 +
        let some = id_prefixes(ids.clone(), ids.iter().copied());
3274 +
        for id in &ids {
3275 +
            assert_eq!(some[id], whole[&hex(id)], "{}", hex(id));
3276 +
        }
3277 +
    }
3278 +
}
modified src/main.rs +3 −0
…
34 unchanged lines
35 35
    }
36 36
    let command = oxy_core::timing::phase("command");
37 37
    let result = cli::run();
38 +
    if result.is_ok() {
39 +
        cli::after_command();
40 +
    }
38 41
    drop(command);
39 42
    oxy_core::timing::memory_summary();
40 43
    match result {
…
8 unchanged lines
added tests/cli_colocate.rs +110 −0
1 +
//! `oxy git colocate` (oxygenhub#355, part two): a `.git` beside `.oxy`,
2 +
//! kept up to date after every oxy command, so Git tools read an Oxygen
3 +
//! checkout; oxy never takes `.git` as content.
4 +
5 +
use std::path::Path;
6 +
use std::process::Command;
7 +
8 +
fn oxy(dir: &Path, home: &Path, args: &[&str]) -> (bool, String) {
9 +
    let out = Command::new(env!("CARGO_BIN_EXE_oxy"))
10 +
        .arg("--plain")
11 +
        .args(args)
12 +
        .current_dir(dir)
13 +
        .env("OXY_HOME", home)
14 +
        .env("NO_COLOR", "1")
15 +
        .output()
16 +
        .unwrap();
17 +
    (
18 +
        out.status.success(),
19 +
        format!(
20 +
            "{}{}",
21 +
            String::from_utf8_lossy(&out.stdout),
22 +
            String::from_utf8_lossy(&out.stderr)
23 +
        ),
24 +
    )
25 +
}
26 +
27 +
fn git(dir: &Path, args: &[&str]) -> String {
28 +
    let out = Command::new("git")
29 +
        .args(args)
30 +
        .current_dir(dir)
31 +
        .env("GIT_CONFIG_NOSYSTEM", "1")
32 +
        .env("HOME", dir)
33 +
        .output()
34 +
        .unwrap();
35 +
    assert!(
36 +
        out.status.success(),
37 +
        "git {args:?}: {}",
38 +
        String::from_utf8_lossy(&out.stderr)
39 +
    );
40 +
    String::from_utf8_lossy(&out.stdout).trim().to_string()
41 +
}
42 +
43 +
#[test]
44 +
fn a_colocated_git_follows_the_bookmark() {
45 +
    if Command::new("git").arg("--version").output().is_err() {
46 +
        eprintln!("git not installed; skipping");
47 +
        return;
48 +
    }
49 +
    let tmp = tempfile::tempdir().unwrap();
50 +
    let home = tmp.path().join("home");
51 +
    std::fs::create_dir_all(&home).unwrap();
52 +
    let r = tmp.path().join("r");
53 +
    assert!(oxy(tmp.path(), &home, &["init", r.to_str().unwrap()]).0);
54 +
    assert!(
55 +
        oxy(
56 +
            &r,
57 +
            &home,
58 +
            &[
59 +
                "identity",
60 +
                "create",
61 +
                "--name",
62 +
                "Ada",
63 +
                "--email",
64 +
                "[email protected]"
65 +
            ]
66 +
        )
67 +
        .0
68 +
    );
69 +
    std::fs::write(r.join("a.txt"), "one\n").unwrap();
70 +
    assert!(oxy(&r, &home, &["commit", "-m", "first"]).0);
71 +
72 +
    let (ok, out) = oxy(&r, &home, &["git", "colocate"]);
73 +
    assert!(ok, "{out}");
74 +
    assert!(r.join(".git/HEAD").exists());
75 +
    assert_eq!(git(&r, &["log", "--format=%s"]), "first");
76 +
    assert_eq!(git(&r, &["symbolic-ref", "HEAD"]), "refs/heads/main");
77 +
    assert_eq!(
78 +
        git(&r, &["status", "--porcelain"]),
79 +
        "",
80 +
        "Git sees no difference, and not .oxy either"
81 +
    );
82 +
    let (_, status) = oxy(&r, &home, &["status"]);
83 +
    assert!(
84 +
        !status.contains(".git"),
85 +
        "oxy never takes .git as content: {status}"
86 +
    );
87 +
88 +
    // An oxy commit shows in Git without asking.
89 +
    std::fs::write(r.join("a.txt"), "one\ntwo\n").unwrap();
90 +
    assert_eq!(
91 +
        git(&r, &["status", "--porcelain"]),
92 +
        "M a.txt",
93 +
        "uncommitted work shows as modified"
94 +
    );
95 +
    assert!(oxy(&r, &home, &["commit", "-m", "second"]).0);
96 +
    assert_eq!(git(&r, &["log", "--format=%s"]), "second\nfirst");
97 +
    assert_eq!(git(&r, &["status", "--porcelain"]), "");
98 +
    assert_eq!(git(&r, &["show", "HEAD:a.txt"]), "one\ntwo");
99 +
100 +
    // Off: no more updates, the .git stays.
101 +
    assert!(oxy(&r, &home, &["git", "colocate", "--off"]).0);
102 +
    std::fs::write(r.join("b.txt"), "b\n").unwrap();
103 +
    assert!(oxy(&r, &home, &["commit", "-m", "third"]).0);
104 +
    assert_eq!(
105 +
        git(&r, &["log", "--format=%s"]),
106 +
        "second\nfirst",
107 +
        "no longer followed"
108 +
    );
109 +
    assert!(r.join(".git/HEAD").exists());
110 +
}
modified tests/cli_json.rs +242 −1
…
163 unchanged lines
164 164
    assert_eq!(by_path["changes"].as_array().unwrap().len(), 2, "{by_path}");
165 165
    let show = json(r, &home, &["show", "--json", &format!("#{}", &hash[..12])]);
166 166
    assert_eq!(show["files"][0]["path"], "b.txt");
167 +
    // oxygen#119: lines added and removed per file, and --stat.
168 +
    assert!(show["files"][0]["added"].as_u64().unwrap() >= 1, "{show}");
169 +
    let (code, out, _) = oxy(r, &home, &["show", "--stat", &format!("#{}", &hash[..12])]);
170 +
    let out = String::from_utf8_lossy(&out);
171 +
    assert!(
172 +
        code == 0 && out.contains("b.txt") && out.contains("1 file changed"),
173 +
        "{out}"
174 +
    );
175 +
    assert!(!out.contains("@@"), "the files, not the diff: {out}");
167 176
    let verify = json(r, &home, &["verify", "--json"]);
168 177
    assert!(verify
169 178
        .as_array()
…
35 unchanged lines
205 214
    assert_eq!(oxy(r, &home, &["policy", "init"]).0, 0);
206 215
    let policy = json(r, &home, &["policy", "show", "--json"]);
207 216
    assert_eq!(policy["roots"][0]["name"], "Ada");
217 +
    // Keyless (oxygenhub#241): a hub's certified subject may sign too.
218 +
    let hub =
219 +
        oxy_core::identity::SecretKey::generate().identity("oxygenhub.io", "[email protected]", 1);
220 +
    let hub_file = r.join("hub.identity");
221 +
    std::fs::write(&hub_file, hub.encode()).unwrap();
222 +
    let grant = [
223 +
        "policy",
224 +
        "grant-subject",
225 +
        "repo:ada/demo:bookmark:main",
226 +
        "--issuer",
227 +
        hub_file.to_str().unwrap(),
228 +
        "--scope",
229 +
        "refs",
230 +
    ];
231 +
    assert_eq!(oxy(r, &home, &grant).0, 0);
232 +
    let policy = json(r, &home, &["policy", "show", "--json"]);
233 +
    let subjects = &policy["rules"][0]["subjects"][0];
208 234
    assert_eq!(
235 +
        (
236 +
            subjects["subject"].as_str(),
237 +
            subjects["issuer"]["name"].as_str(),
238 +
            subjects["scope"].as_str()
239 +
        ),
240 +
        (
241 +
            Some("repo:ada/demo:bookmark:main"),
242 +
            Some("oxygenhub.io"),
243 +
            Some("refs")
244 +
        )
245 +
    );
246 +
    std::fs::remove_file(&hub_file).unwrap();
247 +
    assert_eq!(
209 248
        oxy(r, &home, &["tag", "create", "v1", "-m", "first tag"]).0,
210 249
        0
211 250
    );
…
11 unchanged lines
223 262
224 263
/// `oxy commit -F`, `oxy describe` naming and tags as changes (feedback
225 264
/// 4 and 6).
265 +
/// Cylinder C1 (oxygen#108): a directory packs into an artifact of this
266 +
/// repository's chunks, lists and shows as JSON, survives gc, and unpacks.
267 +
#[test]
268 +
fn artifacts_pack_show_survive_gc_and_unpack() {
269 +
    let tmp = tempfile::tempdir().unwrap();
270 +
    let home = tmp.path().join("home");
271 +
    let dir = tmp.path().join("r");
272 +
    std::fs::create_dir_all(&dir).unwrap();
273 +
    assert_eq!(oxy(&dir, &home, &["init"]).0, 0);
274 +
    write(tmp.path(), "out/bin/tool", "#!/bin/sh\necho hi\n");
275 +
    write(tmp.path(), "out/README", "a tool\n");
276 +
    let out = tmp.path().join("out");
277 +
    let pack = [
278 +
        "artifact",
279 +
        "pack",
280 +
        out.to_str().unwrap(),
281 +
        "--kind",
282 +
        "package",
283 +
        "--arch",
284 +
        "x86_64",
285 +
        "--label",
286 +
        "version=0.1.0",
287 +
        "--json",
288 +
    ];
289 +
    let a = json(&dir, &home, &pack);
290 +
    let name = a["name"].as_str().unwrap().to_string();
291 +
    assert_eq!(
292 +
        (
293 +
            a["kind"].as_str(),
294 +
            a["files"].as_u64(),
295 +
            a["labels"]["version"].as_str()
296 +
        ),
297 +
        (Some("package"), Some(2), Some("0.1.0"))
298 +
    );
299 +
    let list = json(&dir, &home, &["artifact", "list", "--json"]);
300 +
    assert_eq!(list[0]["name"], name.as_str());
301 +
    let (code, _, err) = oxy(
302 +
        &dir,
303 +
        &home,
304 +
        &[
305 +
            "artifact",
306 +
            "pack",
307 +
            out.to_str().unwrap(),
308 +
            "--kind",
309 +
            "tarball",
310 +
        ],
311 +
    );
312 +
    assert_ne!(code, 0);
313 +
    assert!(err.contains("runner-image"), "{err}");
314 +
    assert_eq!(oxy(&dir, &home, &["gc", "--now"]).0, 0);
315 +
    let (code, verified, err) = oxy(&dir, &home, &["artifact", "verify", &name[..8]]);
316 +
    assert_eq!(code, 0, "{err}");
317 +
    assert!(String::from_utf8_lossy(&verified).contains("2 files check out"));
318 +
    let back = tmp.path().join("back");
319 +
    assert_eq!(
320 +
        oxy(
321 +
            &dir,
322 +
            &home,
323 +
            &["artifact", "unpack", &name, back.to_str().unwrap()]
324 +
        )
325 +
        .0,
326 +
        0
327 +
    );
328 +
    assert_eq!(
329 +
        std::fs::read_to_string(back.join("bin/tool")).unwrap(),
330 +
        "#!/bin/sh\necho hi\n"
331 +
    );
332 +
}
333 +
334 +
/// oxygen#144: a first identity needs no repository; the next repository
335 +
/// signs as it.
226 336
#[test]
337 +
fn a_first_identity_needs_no_repository() {
338 +
    let tmp = tempfile::tempdir().unwrap();
339 +
    let home = tmp.path().join("home");
340 +
    let nowhere = tmp.path().join("nowhere");
341 +
    std::fs::create_dir_all(&nowhere).unwrap();
342 +
    let local = ["identity", "create", "--local"];
343 +
    let (code, _, err) = oxy(&nowhere, &home, &local);
344 +
    assert_ne!(code, 0, "--local needs a repository");
345 +
    assert!(err.contains("no oxy repository"), "{err}");
346 +
    let ident = [
347 +
        "identity",
348 +
        "create",
349 +
        "--name",
350 +
        "Ada",
351 +
        "--email",
352 +
        "[email protected]",
353 +
    ];
354 +
    let (code, out, err) = oxy(&nowhere, &home, &ident);
355 +
    assert_eq!(code, 0, "{err}");
356 +
    let out = String::from_utf8_lossy(&out);
357 +
    assert!(
358 +
        out.contains("Created identity Ada <[email protected]>"),
359 +
        "{out}"
360 +
    );
361 +
    let user = oxy_core::repo::read_config_file(&home.join("config.toml")).unwrap();
362 +
    let key = user["author.key"].clone();
363 +
    assert_eq!(user["author.name"], "Ada");
364 +
    assert!(home.join("keys").join(format!("{key}.secret")).exists());
365 +
    assert!(!nowhere.join(".oxy").exists(), "no repository made");
366 +
367 +
    // A repository made afterwards signs its changes with it.
368 +
    let dir = tmp.path().join("r");
369 +
    std::fs::create_dir_all(&dir).unwrap();
370 +
    assert_eq!(oxy(&dir, &home, &["init"]).0, 0);
371 +
    assert_eq!(oxy(&dir, &home, &["config", "core.sign", "change"]).0, 0);
372 +
    write(&dir, "a.txt", "a\n");
373 +
    let (code, _, err) = oxy(&dir, &home, &["commit", "-m", "First"]);
374 +
    assert_eq!(code, 0, "{err}");
375 +
    let log = json(&dir, &home, &["log", "--json", "-n", "2"]);
376 +
    let first = &log["changes"][1];
377 +
    assert_eq!(first["message"], "First");
378 +
    assert_eq!(first["signature"]["status"], "signed", "{first}");
379 +
    assert_eq!(first["signature"]["key"], key.as_str());
380 +
    assert_eq!(first["signature"]["name"], "Ada");
381 +
}
382 +
383 +
#[test]
227 384
fn commit_from_a_file_and_describe_for_builds() {
228 385
    let tmp = tempfile::tempdir().unwrap();
229 386
    let home = tmp.path().join("home");
…
141 unchanged lines
371 528
    );
372 529
    assert_eq!(std::fs::read(&out).unwrap(), sr.encode());
373 530
531 +
    // SLSA provenance is signed with the exporter's key: none yet, so it is
532 +
    // refused and says what to do.
533 +
    let prov = tmp.path().join("prov.json");
534 +
    let p = prov.to_str().unwrap();
535 +
    let slsa = ["receipt", "export", &attest[..10], p, "--format", "slsa"];
536 +
    let (code, _, err) = oxy(&r, &home, &slsa);
537 +
    assert_ne!(code, 0);
538 +
    assert!(err.contains("oxy identity create"), "{err}");
539 +
    let ident = [
540 +
        "identity",
541 +
        "create",
542 +
        "--name",
543 +
        "Ada",
544 +
        "--email",
545 +
        "[email protected]",
546 +
    ];
547 +
    assert_eq!(oxy(&r, &home, &ident).0, 0);
548 +
    assert_eq!(oxy(&r, &home, &slsa).0, 0);
549 +
    let env: serde_json::Value = serde_json::from_slice(&std::fs::read(&prov).unwrap()).unwrap();
550 +
    assert_eq!(env["payloadType"], oxy_core::dsse::IN_TOTO);
551 +
    let payload = oxy_core::dsse::base64_decode(env["payload"].as_str().unwrap()).unwrap();
552 +
    let st: serde_json::Value = serde_json::from_slice(&payload).unwrap();
553 +
    assert_eq!(st["predicateType"], "https://slsa.dev/provenance/v1");
554 +
    assert_eq!(st["subject"][0]["digest"]["oxygen-state"], state.to_hex());
555 +
    let run = &st["predicate"]["runDetails"];
556 +
    assert_eq!(run["metadata"]["invocationId"], "demo#7");
557 +
    let kept = oxy_core::dsse::base64_decode(run["byproducts"][0]["content"].as_str().unwrap());
558 +
    assert_eq!(
559 +
        kept.unwrap(),
560 +
        sr.encode(),
561 +
        "the runner's receipt rides along"
562 +
    );
563 +
    // It verifies with the exporter's key and its PEM holds that key.
564 +
    let repo = oxy_core::repo::Repo::open(&r)
565 +
        .unwrap()
566 +
        .with_home(home.clone());
567 +
    let key = repo.signing_key().unwrap();
568 +
    assert_eq!(env["signatures"][0]["keyid"], key.to_hex());
569 +
    let me = repo.identity(&key).unwrap();
570 +
    let sig = oxy_core::dsse::base64_decode(env["signatures"][0]["sig"].as_str().unwrap()).unwrap();
571 +
    assert!(oxy_core::dsse::verify(&me, oxy_core::dsse::IN_TOTO, &payload, &sig).is_ok());
572 +
    assert!(oxy_core::dsse::verify(&runner, oxy_core::dsse::IN_TOTO, &payload, &sig).is_err());
573 +
    let pem = tmp.path().join("key.pem");
574 +
    let pem_args = ["identity", "export", "--pem", pem.to_str().unwrap()];
575 +
    assert_eq!(oxy(&r, &home, &pem_args).0, 0);
576 +
    assert_eq!(
577 +
        std::fs::read_to_string(&pem).unwrap(),
578 +
        oxy_core::dsse::public_pem(&me)
579 +
    );
580 +
374 581
    // A receipt whose result was changed after signing does not verify.
375 582
    let mut forged = sr.clone();
376 583
    forged.receipt.exit_code = 3;
…
77 unchanged lines
454 661
    // Moving a bookmark asks the same (metalgear, oxygen#42): main at b
455 662
    // alone would lose a, which it holds and was recorded before b.
456 663
    let (code, _, err) = oxy(&r, &home, &["bookmark", "set", "main", &b]);
457 -
    assert_ne!(code, 0);
664 +
    assert_ne!(code, 0, "b is a head main holds, not a move forward");
458 665
    assert!(
459 666
        err.contains("would lose 1 change") && err.contains("--upto"),
460 667
        "{err}"
…
10 unchanged lines
471 678
        .find(|x| x["name"] == "main")
472 679
        .unwrap();
473 680
    assert_eq!(main["heads"].as_array().unwrap().len(), 2, "a and b: {bms}");
681 +
682 +
    // A fast-forward (oxygen#78): y holds a and b; b2 is built on b, its
683 +
    // newest head. Moving y to b2 keeps a, which b2 does not depend on.
684 +
    assert_eq!(
685 +
        oxy(&r, &home, &["bookmark", "switch", "main", "--discard"]).0,
686 +
        0
687 +
    );
688 +
    assert_eq!(oxy(&r, &home, &["bookmark", "create", "y"]).0, 0);
689 +
    write(&r, "b.txt", "b2\n");
690 +
    assert_eq!(oxy(&r, &home, &["commit", "-m", "b2"]).0, 0);
691 +
    let log = json(&r, &home, &["log", "--json"]);
692 +
    let b2 = log["changes"]
693 +
        .as_array()
694 +
        .unwrap()
695 +
        .iter()
696 +
        .find(|c| c["message"] == "b2")
697 +
        .unwrap()["hash"]
698 +
        .as_str()
699 +
        .unwrap()[..12]
700 +
        .to_string();
701 +
    let (code, out, err) = oxy(&r, &home, &["bookmark", "set", "y", &format!("#{b2}")]);
702 +
    assert_eq!(code, 0, "{err}");
703 +
    assert!(
704 +
        String::from_utf8_lossy(&out).contains("builds on y's newest head"),
705 +
        "says why"
706 +
    );
707 +
    let bms = json(&r, &home, &["bookmark", "list", "--json"]);
708 +
    let y = bms
709 +
        .as_array()
710 +
        .unwrap()
711 +
        .iter()
712 +
        .find(|x| x["name"] == "y")
713 +
        .unwrap();
714 +
    assert_eq!(y["heads"].as_array().unwrap().len(), 2, "a and b2: {bms}");
474 715
}
475 716
476 717
/// oxygen#29: a pull on a feature bookmark said only "Already up to date"
…
168 unchanged lines
added tests/cli_mcp.rs +231 −0
1 +
//! `oxy mcp` (oxygen#159) end to end: an MCP client talks to the server over
2 +
//! stdio, lists its tools, reads the repository, commits by path as the
3 +
//! configured identity, and is refused what it may not do.
4 +
5 +
use serde_json::{json, Value};
6 +
use std::io::{BufRead, BufReader, Write};
7 +
use std::path::Path;
8 +
use std::process::{Command, Stdio};
9 +
10 +
fn oxy(dir: &Path, home: &Path, args: &[&str]) -> i32 {
11 +
    Command::new(env!("CARGO_BIN_EXE_oxy"))
12 +
        .arg("--plain")
13 +
        .args(args)
14 +
        .current_dir(dir)
15 +
        .env("OXY_HOME", home)
16 +
        .env("NO_COLOR", "1")
17 +
        .status()
18 +
        .unwrap()
19 +
        .code()
20 +
        .unwrap_or(-1)
21 +
}
22 +
23 +
/// Send each request in turn and collect the replies, one per request (the
24 +
/// notification gets none).
25 +
fn session(dir: &Path, home: &Path, extra: &[&str], requests: &[Value]) -> Vec<Value> {
26 +
    let mut child = Command::new(env!("CARGO_BIN_EXE_oxy"))
27 +
        .arg("mcp")
28 +
        .args(extra)
29 +
        .current_dir(dir)
30 +
        .env("OXY_HOME", home)
31 +
        .env("NO_COLOR", "1")
32 +
        .stdin(Stdio::piped())
33 +
        .stdout(Stdio::piped())
34 +
        .stderr(Stdio::null())
35 +
        .spawn()
36 +
        .unwrap();
37 +
    let mut stdin = child.stdin.take().unwrap();
38 +
    let mut stdout = BufReader::new(child.stdout.take().unwrap());
39 +
    let mut replies = vec![];
40 +
    for r in requests {
41 +
        writeln!(stdin, "{r}").unwrap();
42 +
        stdin.flush().unwrap();
43 +
        if r.get("id").is_some() {
44 +
            let mut line = String::new();
45 +
            stdout.read_line(&mut line).unwrap();
46 +
            replies.push(
47 +
                serde_json::from_str(&line).unwrap_or_else(|e| panic!("not JSON ({e}): {line}")),
48 +
            );
49 +
        }
50 +
    }
51 +
    drop(stdin);
52 +
    assert!(
53 +
        child.wait().unwrap().success(),
54 +
        "the server ends cleanly when its input does"
55 +
    );
56 +
    replies
57 +
}
58 +
59 +
fn call(id: u64, name: &str, args: Value) -> Value {
60 +
    json!({"jsonrpc": "2.0", "id": id, "method": "tools/call", "params": {"name": name, "arguments": args}})
61 +
}
62 +
63 +
#[test]
64 +
fn an_agent_reads_and_commits_through_mcp() {
65 +
    let tmp = tempfile::tempdir().unwrap();
66 +
    let home = tmp.path().join("home");
67 +
    std::fs::create_dir_all(&home).unwrap();
68 +
    let repo = tmp.path().join("repo");
69 +
    assert_eq!(oxy(tmp.path(), &home, &["init", repo.to_str().unwrap()]), 0);
70 +
    assert_eq!(
71 +
        oxy(
72 +
            &repo,
73 +
            &home,
74 +
            &[
75 +
                "identity",
76 +
                "create",
77 +
                "--name",
78 +
                "Ada",
79 +
                "--email",
80 +
                "[email protected]"
81 +
            ]
82 +
        ),
83 +
        0
84 +
    );
85 +
    assert_eq!(
86 +
        oxy(&repo, &home, &["config", "--user", "core.sign", "change"]),
87 +
        0
88 +
    );
89 +
    std::fs::write(repo.join("a.txt"), "one\n").unwrap();
90 +
    std::fs::write(repo.join("b.txt"), "two\n").unwrap();
91 +
92 +
    let r = session(
93 +
        &repo,
94 +
        &home,
95 +
        &[],
96 +
        &[
97 +
            json!({"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {"protocolVersion": "2025-06-18", "capabilities": {}, "clientInfo": {"name": "test", "version": "0"}}}),
98 +
            json!({"jsonrpc": "2.0", "method": "notifications/initialized"}),
99 +
            json!({"jsonrpc": "2.0", "id": 2, "method": "tools/list"}),
100 +
            call(3, "status", json!({})),
101 +
            call(4, "commit", json!({"message": "everything"})),
102 +
            call(5, "commit", json!({"message": "Add a", "paths": ["a.txt"]})),
103 +
            call(6, "log", json!({"limit": 5})),
104 +
            call(7, "status", json!({})),
105 +
            call(8, "undo", json!({})),
106 +
            call(9, "push", json!({"force": true})),
107 +
            call(10, "cat", json!({"path": "a.txt", "change": "@"})),
108 +
        ],
109 +
    );
110 +
111 +
    let init = &r[0]["result"];
112 +
    assert_eq!(init["serverInfo"]["name"], "oxy");
113 +
    let said = init["instructions"].as_str().unwrap();
114 +
    assert!(
115 +
        said.contains("Ada"),
116 +
        "initialize says who changes are signed as: {said}"
117 +
    );
118 +
    assert!(
119 +
        said.contains("Destructive tools (undo, forced push) are off"),
120 +
        "{said}"
121 +
    );
122 +
123 +
    let names: Vec<&str> = r[1]["result"]["tools"]
124 +
        .as_array()
125 +
        .unwrap()
126 +
        .iter()
127 +
        .map(|t| t["name"].as_str().unwrap())
128 +
        .collect();
129 +
    for t in [
130 +
        "status",
131 +
        "diff",
132 +
        "log",
133 +
        "show",
134 +
        "cat",
135 +
        "blame",
136 +
        "bookmarks",
137 +
        "commit",
138 +
        "amend",
139 +
        "push",
140 +
        "pull",
141 +
        "tlog_verify",
142 +
    ] {
143 +
        assert!(names.contains(&t), "{t} is offered: {names:?}");
144 +
    }
145 +
    assert!(!names.contains(&"undo"), "undo needs --allow-destructive");
146 +
147 +
    let before = &r[2]["result"];
148 +
    assert_eq!(before["isError"], false, "{before}");
149 +
    assert!(
150 +
        before["structuredContent"].is_object(),
151 +
        "status comes back as JSON: {before}"
152 +
    );
153 +
    assert!(before["content"][0]["text"]
154 +
        .as_str()
155 +
        .unwrap()
156 +
        .contains("b.txt"));
157 +
158 +
    assert_eq!(
159 +
        r[3]["result"]["isError"], true,
160 +
        "a commit names its paths or all: {}",
161 +
        r[3]
162 +
    );
163 +
    assert!(r[3]["result"]["content"][0]["text"]
164 +
        .as_str()
165 +
        .unwrap()
166 +
        .contains("all: true"));
167 +
168 +
    assert_eq!(r[4]["result"]["isError"], false, "commit by path: {}", r[4]);
169 +
170 +
    let log = r[5]["result"]["content"][0]["text"].as_str().unwrap();
171 +
    assert!(log.contains("Add a"), "{log}");
172 +
    assert!(
173 +
        r[5]["result"]["structuredContent"].is_object(),
174 +
        "a JSON array comes wrapped: {}",
175 +
        r[5]
176 +
    );
177 +
178 +
    let after = r[6]["result"]["content"][0]["text"].as_str().unwrap();
179 +
    assert!(
180 +
        after.contains("b.txt") && !after.contains("a.txt"),
181 +
        "only a.txt went in: {after}"
182 +
    );
183 +
184 +
    assert!(
185 +
        r[7]["error"]["message"]
186 +
            .as_str()
187 +
            .unwrap()
188 +
            .contains("no tool named undo"),
189 +
        "{}",
190 +
        r[7]
191 +
    );
192 +
    assert_eq!(r[8]["result"]["isError"], true);
193 +
    assert!(r[8]["result"]["content"][0]["text"]
194 +
        .as_str()
195 +
        .unwrap()
196 +
        .contains("--allow-destructive"));
197 +
    assert_eq!(r[9]["result"]["content"][0]["text"], "one\n");
198 +
}
199 +
200 +
#[test]
201 +
fn destructive_tools_come_with_the_flag_and_a_bad_directory_is_refused() {
202 +
    let tmp = tempfile::tempdir().unwrap();
203 +
    let home = tmp.path().join("home");
204 +
    std::fs::create_dir_all(&home).unwrap();
205 +
    let repo = tmp.path().join("repo");
206 +
    assert_eq!(oxy(tmp.path(), &home, &["init", repo.to_str().unwrap()]), 0);
207 +
    let r = session(
208 +
        &repo,
209 +
        &home,
210 +
        &["--allow-destructive"],
211 +
        &[json!({"jsonrpc": "2.0", "id": 1, "method": "tools/list"})],
212 +
    );
213 +
    let undo = r[0]["result"]["tools"]
214 +
        .as_array()
215 +
        .unwrap()
216 +
        .iter()
217 +
        .find(|t| t["name"] == "undo")
218 +
        .cloned();
219 +
    assert_eq!(
220 +
        undo.expect("undo is offered")["annotations"]["destructiveHint"],
221 +
        true
222 +
    );
223 +
224 +
    let out = Command::new(env!("CARGO_BIN_EXE_oxy"))
225 +
        .args(["mcp", tmp.path().to_str().unwrap()])
226 +
        .env("OXY_HOME", &home)
227 +
        .output()
228 +
        .unwrap();
229 +
    assert!(!out.status.success());
230 +
    assert!(String::from_utf8_lossy(&out.stderr).contains("not the root of an Oxygen repository"));
231 +
}
modified tests/cli_strata.rs +22 −0
…
140 unchanged lines
141 141
    assert_ne!(code, 0);
142 142
    assert!(err.contains("different repository"), "{err}");
143 143
}
144 +
145 +
#[test]
146 +
fn commit_with_paths_says_what_it_left() {
147 +
    let tmp = tempfile::tempdir().unwrap();
148 +
    let home = tmp.path().join("home");
149 +
    std::fs::create_dir_all(&home).unwrap();
150 +
    let a = tmp.path().join("a");
151 +
    ok(tmp.path(), &home, &["init", a.to_str().unwrap()]);
152 +
    std::fs::write(a.join("f.txt"), "one\n").unwrap();
153 +
    std::fs::write(a.join("g.txt"), "one\n").unwrap();
154 +
    ok(&a, &home, &["commit", "-m", "first"]);
155 +
156 +
    std::fs::write(a.join("f.txt"), "two\n").unwrap();
157 +
    std::fs::write(a.join("g.txt"), "two\n").unwrap();
158 +
    let part = ok(&a, &home, &["commit", "-m", "f only", "f.txt"]);
159 +
    assert!(part.contains("1 path left uncommitted"), "{part}");
160 +
161 +
    let rest = ok(&a, &home, &["commit", "-m", "g", "g.txt"]);
162 +
    assert!(!rest.contains("left uncommitted"), "{rest}");
163 +
    let status = ok(&a, &home, &["status"]);
164 +
    assert!(status.contains("working copy clean"), "{status}");
165 +
}
added tests/cli_subrepo.rs +94 −0
1 +
//! oxygen#162: a subrepo pin moved by work inside the subrepo is a change
2 +
//! the parent's next commit records; `oxy status` and `oxy commit` say so,
3 +
//! and say when a subrepo holds uncommitted edits of its own.
4 +
5 +
use std::path::Path;
6 +
use std::process::Command;
7 +
8 +
fn oxy(dir: &Path, home: &Path, args: &[&str]) -> (i32, String) {
9 +
    let out = Command::new(env!("CARGO_BIN_EXE_oxy"))
10 +
        .arg("--plain")
11 +
        .args(args)
12 +
        .current_dir(dir)
13 +
        .env("OXY_HOME", home)
14 +
        .env("NO_COLOR", "1")
15 +
        .output()
16 +
        .unwrap();
17 +
    let text = format!(
18 +
        "{}{}",
19 +
        String::from_utf8_lossy(&out.stdout),
20 +
        String::from_utf8_lossy(&out.stderr)
21 +
    );
22 +
    (out.status.code().unwrap_or(-1), text)
23 +
}
24 +
25 +
fn ok(dir: &Path, home: &Path, args: &[&str]) -> String {
26 +
    let (code, out) = oxy(dir, home, args);
27 +
    assert_eq!(code, 0, "oxy {args:?}: {out}");
28 +
    out
29 +
}
30 +
31 +
#[test]
32 +
fn moved_pins_and_edits_inside_subrepos_are_named() {
33 +
    let tmp = tempfile::tempdir().unwrap();
34 +
    let t = tmp.path();
35 +
    let home = t.join("home");
36 +
    std::fs::create_dir_all(&home).unwrap();
37 +
    ok(
38 +
        t,
39 +
        &home,
40 +
        &[
41 +
            "identity",
42 +
            "create",
43 +
            "--name",
44 +
            "Ada",
45 +
            "--email",
46 +
            "[email protected]",
47 +
        ],
48 +
    );
49 +
    // The subrepo's own repository, with one change.
50 +
    let lib = t.join("lib");
51 +
    ok(t, &home, &["init", lib.to_str().unwrap()]);
52 +
    std::fs::write(lib.join("lib.txt"), "one\n").unwrap();
53 +
    ok(&lib, &home, &["commit", "-m", "lib one"]);
54 +
    // The parent pins it.
55 +
    let app = t.join("app");
56 +
    ok(t, &home, &["init", app.to_str().unwrap()]);
57 +
    std::fs::write(app.join("app.txt"), "app\n").unwrap();
58 +
    ok(
59 +
        &app,
60 +
        &home,
61 +
        &["submodule", "add", lib.to_str().unwrap(), "deps/lib"],
62 +
    );
63 +
    ok(&app, &home, &["commit", "-m", "app with lib"]);
64 +
    let quiet = ok(&app, &home, &["status"]);
65 +
    assert!(!quiet.contains("Subrepo pins"), "{quiet}");
66 +
67 +
    // Edits inside the subrepo, not committed there: named, not a pin move.
68 +
    let inner = app.join("deps/lib");
69 +
    std::fs::write(inner.join("lib.txt"), "two\n").unwrap();
70 +
    let st = ok(&app, &home, &["status"]);
71 +
    assert!(
72 +
        st.contains("Subrepo pins") && st.contains("uncommitted edits inside it"),
73 +
        "{st}"
74 +
    );
75 +
    assert!(!st.contains("pin moved"), "{st}");
76 +
77 +
    // Committed inside: the pin has moved, and a bare commit says it records it.
78 +
    ok(&inner, &home, &["commit", "-m", "lib two"]);
79 +
    let st = ok(&app, &home, &["status"]);
80 +
    assert!(st.contains("deps/lib") && st.contains("pin moved"), "{st}");
81 +
    std::fs::write(app.join("app.txt"), "app 2\n").unwrap();
82 +
    let by_path = ok(&app, &home, &["commit", "-m", "app only", "app.txt"]);
83 +
    assert!(
84 +
        !by_path.contains("recorded subrepo"),
85 +
        "a commit by path leaves the pin out: {by_path}"
86 +
    );
87 +
    let bare = ok(&app, &home, &["commit", "-m", "everything"]);
88 +
    assert!(
89 +
        bare.contains("recorded subrepo deps/lib's pin moving") && bare.contains("a pin bump"),
90 +
        "{bare}"
91 +
    );
92 +
    let after = ok(&app, &home, &["status"]);
93 +
    assert!(!after.contains("Subrepo pins"), "recorded: {after}");
94 +
}