OXYGENHUB
# ac4ec124b9f2 oxygen / docs / plan.md
15.5 KiB · Markdown · last changed Delegate rule on fetch and tags too 1 month ago renderedsource blame history raw
96f7610f Andy McCall ↶ 1
# Overnight work plan
2
3
Each loop iteration: pick the first unchecked item, implement it with tests, run the gated checks, commit with `oxy`, tick the box here with a one-line note, and commit again. Update `docs/spec.md` (and its changelog) whenever behaviour the spec describes changes. If an item cannot be finished in two iterations, note why under it and move on.
4
5
## Rules
6
7
- Never run `git init` in this directory or push this project anywhere. Oxygen tracks itself with `target/release/oxy`.
8
- Never modify `~/development/tetriblox`; use only mirror clones in the scratch directory.
9
- Tests must stay hermetic (`hermetic()` / `OXY_HOME`); never touch `~/.config/oxy`.
10
- Gate every commit on `cargo test --workspace` exit status and a clean `cargo clippy --workspace --all-targets`.
11
- No new external services, no publishing, no artifacts.
12
13
## Items
14
7a716eb9 Andy McCall ↶ 15
- [x] Checkpoints (spec §11) — done 2026-09-27 00:05. At 5,002 changes: status 0.10 s → 0.03 s, commit 0.20 s → 0.05 s; checkpoint write 0.10 s, payload 1.7 MB; states now hold change metadata only. serialize a `State` to a checkpoint object, load the nearest checkpoint in `load_state` and replay only the difference, create one automatically every `core.checkpoint-interval` changes (default 256) and on demand (`oxy checkpoint`). Benchmark before/after on a 5,000-change synthetic repository and record the numbers here.
16
- [x] Pack merging (spec §13.4) — done 2026-09-27 00:05; folds packs into one above 16, also after auto-pack. merge packs when more than 16 exist; `oxy gc` reports it.
17
- [x] Modern CLI presentation (user request, 2026-09-26 23:20) — done 2026-09-27 00:05 via `src/cli/ui.rs` (console + indicatif): colour, spinners on long commands, `--plain`, NO_COLOR and non-TTY detection; status/log/show/commit/checkout/verify/sync/gc restyled. colour and bold for ids, paths, and statuses; a live spinner or updating status line while a command works (snapshot, fetch, push, import, gc); indented, grouped output like Claude Code's. Use `console`/`indicatif` or equivalent; detect a TTY and honour `NO_COLOR` and `--no-color`/`--plain` so pipes and scripts get plain text; keep `--json` where it exists. Touch every command's output for consistency.
18
- [x] Shell prompt integration (user request, 2026-09-26 23:30) — done 2026-09-27 00:05: `oxy prompt [--format]`, manifest-based dirty check (<10 ms), ahead/behind from tracking, `docs/prompt.md` with bash/oh-my-bash/zsh snippets. `oxy prompt` prints a compact one-line summary for shell prompts in a few milliseconds and with no side effects (no snapshot operation recorded, no locks): bookmark, working-copy ChangeId prefix, head count, a dirty marker (compare the file system against the working copy cheaply: sizes and mtimes first, content only when they differ), conflict marker, and ahead/behind counts against `remotes/origin/<bookmark>` when known. Provide `docs/prompt.md` with an oh-my-bash/bash snippet (`oxy_prompt_info` function, silent outside an oxy repository, colours via `--plain` off) and a zsh variant. Format flags: `--format` with placeholders, `--plain`.
694805cc Andy McCall ↶ 19
- [x] (Pending, for the morning) Ask Andy to complete his cut-off request ("Can we also look at ..."). (Answered 2026-09-27: it was a stray copy of the prompt request. Done: `oxy prompt --glyphs --color`, new placeholders %s %U %O %i, and an oh-my-bash powerline segment in contrib/oh-my-bash/oxy.sh, installed to ~/.oh-my-bash/custom/oxy.sh; ~/.local/bin/oxy symlinks the release binary. GPG question settled: Andy said to leave GPG out.)
13eddcc4 Andy McCall ↶ 20
- [x] Lazy blobs and the `objects` endpoint — done 2026-09-27 00:45: `clone --max-blob`, `filter.max-blob`, `.oxy/lazy/<ref>` registry, fetch on first read (checkout, export), server `objects` route; status/commit need no blobs. (spec §13.5, §16.1, §17): `clone --max-blob SIZE`, absent blobs recorded with their remote, fetched on first read; server endpoint; wire encoding in spec §16.3.
d1d1e9e8 Andy McCall ↶ 21
- [x] Sparse checkout (spec §17) — done 2026-09-27 01:05: `filter.paths`, `clone --sparse` (sets max-blob 0), `oxy sparse PATH..`, snapshot/checkout/prompt honour scope; out-of-scope blobs never fetched. `filter.paths` in config, `clone --sparse PATH..`, status/commit/checkout honour it.
717ef88f Andy McCall ↶ 22
- [x] Shallow clone (spec §17) — done 2026-09-27 01:50: `clone --depth N` takes the remote's checkpoint plus N levels of history, `.oxy/shallow` boundary, negotiation and imports tolerate it, `oxy deepen`; log works from checkpoint metadata. `clone --depth N`, shallow boundary with a checkpoint, `oxy deepen`.
af5bd4e6 Andy McCall ↶ 23
- [x] Tags (spec §10.1, §14.4) — done 2026-09-27 02:25: `oxy tag create|list`, checkout by tag, tag attestations, tags in views/refs/push, refused when existing with another state, policy rule `tags/<name>`, Git export as tags. `oxy tag create|list`, tag attestations, tags in views, wire, and export.
4eb6dd3d Andy McCall ↶ 24
- [x] Multiple workspaces (spec §12.2) — done 2026-09-27 02:55: `oxy workspace add|list|forget`, secondary workspaces via a `.oxy` file, per-workspace manifest and snapshot lock with stale takeover. `oxy workspace add|list|forget`, per-workspace directory metadata, snapshot lock.
ebef3715 Andy McCall ↶ 25
- [x] `.oxyattributes` (spec §19) — done 2026-09-27 03:25: gitattributes-style rules; `binary`/`text`, `eol=lf|crlf` (LF stored, CRLF on checkout), `nocompress` chunks packed raw, `lockable` read-only until `oxy lock`; attributes taken from the files being committed. `binary`, `eol=lf|crlf`, `nocompress`, `lockable` (advisory locally).
c177d175 Andy McCall ↶ 26
- [x] Hooks (spec §20) — done 2026-09-27 03:55: executables in `.oxy/hooks/<event>`, JSON on stdin, `pre-*` veto via exit status with stderr as the message, `semantic-deps` adds explicit deps when `hooks.semantic-deps = true`. `pre-commit`, `post-commit`, `pre-push`, `post-fetch`, `post-checkout`, and the `semantic-deps` hook with its JSON interface.
ec2cfb37 Andy McCall ↶ 27
- [x] Amend on non-head changes; `oxy split` — done 2026-09-27 04:45: `oxy amend --into CHANGE` appends the working copy's edits to an older change when they reference only its closure (identity correspondence for dependents, refused otherwise with a clear message); `oxy split CHANGE PATH..` rebuilds a head as two changes. Found and fixed a graph bug: a delete/insert conflict now requires the deleter and inserter to commute (both directions).
7bc30231 Andy McCall ↶ 28
- [x] Reify orphaned references on rewrite as conflicts instead of refusing (spec §8.4); define the conflict shape in the spec. (Done: re-anchoring to the nearest surviving line, else the chain's anchors, so order conflicts arise naturally; vanished nodes drop dependent edits; notes reported by `oxy abandon`.)
94084087 Andy McCall ↶ 29
- [x] User request (2026-09-27, mid-loop): make `oxy log` and all output look modern like `git log` with lines, spacing, and glyphs when the terminal supports them. (Done: rail-and-bullet log with `-n`/`-v`, relative times, authors, signature marks; status/show/op log/tags/workspaces/remotes/fetch/push/policy/identity restyled; ASCII fallback when piped or with `--plain`/`NO_COLOR`/non-UTF-8 locale.)
28fc23d7 Andy McCall ↶ 30
- [x] Identity lifecycle: `oxy identity rotate|revoke`, expiry, delegates (`oxy identity delegate add|remove`), multi-signer policy updates. (Done: rotation as supersession with chain-aware policy authorization, renew, delegates, revocation attestations and local markers, expiry checks, root add/remove/threshold, proposal files and `oxy policy sign`; spec 0.6n.)
31
- [x] Push and serve the chain of policies since the receiver's serial, so several offline rotations or updates publish in one push or fetch. (Done in iteration 12: `.oxy/policies/` index, `refs` history, bundle policies.)
30ea9c1e Andy McCall ↶ 32
- [x] Rename detection on Git import (similarity ≥ 50%); `oxy git fetch`/`push` against a Git remote via a mirror. (Done: detection lives in the change builder, so working-copy renames are tracked too; fetch/push through `.oxy/git/<key>/mirror.git` with fast-forward chaining; spec 0.6p.)
2b872d07 Andy McCall ↶ 33
- [x] Hardening: property tests for the CBOR decoder and pack reader against random and mutated inputs; a concurrent-push test against the server; a 1 GB single-file benchmark for chunking and blobs. (Done: random/mutated CBOR never panics and accepted input is canonical; 300 rounds of pack/index corruption fail cleanly, with new index bounds checks; 8 concurrent HTTP pushes serialise with pull-and-retry; `bench_one_gibibyte_file` is an ignored release-mode test, run with `OXY_BENCH_DIR` on a real disk.)
8f10a07a Andy McCall ↶ 34
- [x] Performance follow-up from the 1 GiB benchmark: commit takes 5.6 s on tmpfs but 63 s on the SSD, and clone 49 s, because every object is written with its own fsync (13,715 chunks). Write committed and fetched objects straight into a pack with one fsync, or batch loose writes. (Done: store write batches; small batches stay loose, large ones spill to one pack. SSD: commit 63 s → 8.8 s, clone 49 s → 15.9 s; spec 0.6r.)
b7f102fc Andy McCall ↶ 35
- [x] Performance follow-up: `oxy status` re-hashes a clean 1 GiB working copy (2.4 s). Record size and mtime in the manifest and skip hashing files whose stat is unchanged (spec allows a stat cache). (Done: manifest records blob refs for opaque files; unchanged files are not read; two-second racy window; status on a clean 1 GiB copy 2.4 s → 4 ms; spec 0.6s.)
ee06a6c5 Andy McCall ↶ 36
- [x] Documentation: `docs/guide.md` (user guide by task) and `docs/protocol.md` (for other implementers, derived from spec §16 and the wire encodings). (Done; every command and flag in the guide was checked against `--help`.)
50a046e4 Andy McCall ↶ 37
- [x] Spec review pass: resolve or re-word every "MAY ... until" allowance introduced during implementation, and close what can be closed in §27. (Done as Draft 0.7: no "MAY ... until" wording remained; `oxy status --json` implemented to meet §9.5; §12.4, §16.1 and §24 aligned with the implementation; new §26.1 status table; §27 records workspace conflicts and server locks as open.)
96f7610f Andy McCall ↶ 38
86fe29fd Andy McCall ↶ 39
## Phase 2 (2026-09-27, Andy's request: the everyday tools Git has)
40
41
Rules as above. Spec Draft 0.8 defines each item; implement in this order, each with tests, a gate, and an oxy commit.
42
fbda43e7 Andy McCall ↶ 43
- [x] Multiple bookmarks (spec §10.1): `oxy bookmark create|set|delete|list|switch`, per-workspace current bookmark, `oxy merge BOOKMARK`, `oxy checkout NAME`, bookmark names resolved before change prefixes; push/pull/log act on the current bookmark; `oxy push -b`.
4dee277c Andy McCall ↶ 44
- [x] Subrepositories (spec §7.8, §23): node kind `subrepo`, pin encoding, `.oxymodules`, snapshot stops at nested repositories and reads their pin, checkout moves or leaves nested repositories, `oxy submodule add|update|status|remove`; Git import/export of gitlinks and `.gitmodules` (test with a mirror of tetriblox, never the live repository).
957fee30 Andy McCall ↶ 45
- [x] Partial commits (spec §12.3): `oxy commit PATH..` and `oxy commit --interactive` (hunk = one edit or tree operation, dependent hunks pulled in), sharing the split machinery.
46
- [x] Blame (spec §27): `oxy blame PATH [--json]` from atom keys, with author and date; renames followed.
47
- [x] Bisect (spec §27): `oxy bisect start|good|bad|run|reset` over dependency-closed prefixes; state under `.oxy/bisect/`.
728da0f9 Andy McCall ↶ 48
- [x] Squash (spec §8.5): `oxy squash CHANGE --into DEST` generalising `amend --into`, with rewrite propagation.
49
- [x] Rebase (spec §8.5): `oxy rebase CHANGE --onto HEADS..` re-deriving edits against a new base; refused on conflicted files.
50
- [x] Guide, README and §26.1 updated for each; spec changelog per item.
86fe29fd Andy McCall ↶ 51
c6aa5bd9 Andy McCall ↶ 52
## Phase 3 (2026-09-27): machine interface for the VS Code extension (oxygen-vscode/docs/spec.md section 4)
53
54
- [x] `oxy root`, `oxy cat PATH [-r CHANGE]`, `oxy version --json`, `oxy discard PATH.. | --hunks N,M`.
55
- [x] JSON outputs: `log --json [--path P] [--bookmark B]`, `show --json`, `diff --json [CHANGE]`, `hunks --json`, `commit --hunks`, `bookmark list --json`, `op log --json`, `resolve --json`, `submodule status --json`, `bisect status --json`, `verify --json`, `policy show --json`, `remote list --json`, `fetch|pull|push --json`.
56
- [x] An integration test that runs every JSON command against a scratch repository and parses each output.
57
- [x] Spec §24 and the guide mention the machine interface; `docs/protocol.md` gets a pointer.
58
96f7610f Andy McCall ↶ 59
## Log
60
61
- 2026-09-26 23:01 plan created.
694805cc Andy McCall ↶ 62
- 2026-09-27 12:15 morning: glyph/colour prompt modes and the oh-my-bash powerline segment. 116 tests.
50a046e4 Andy McCall ↶ 63
- 2026-09-27 10:00 iteration 18: spec review pass (Draft 0.7) and oxy status --json. 113 tests.
ee06a6c5 Andy McCall ↶ 64
- 2026-09-27 09:35 iteration 17: user guide and protocol document. 112 tests.
b7f102fc Andy McCall ↶ 65
- 2026-09-27 09:10 iteration 16: manifest stat cache (status 2.4 s → 4 ms on 1 GiB). 112 tests.
8f10a07a Andy McCall ↶ 66
- 2026-09-27 08:40 iteration 15: batched object writes (commit 8.8 s, clone 15.9 s for 1 GiB on SSD). 111 tests.
2b872d07 Andy McCall ↶ 67
- 2026-09-27 08:05 iteration 14: hardening (CBOR and pack fuzz-style tests, concurrent pushes, 1 GiB benchmark: write 0.7 s, commit 63 s on SSD / 5.6 s on tmpfs, status 2.4 s, clone 49 s, read back 1.0 s, store 1025.6 MiB, 13,715 objects). 110 tests, 1 ignored benchmark.
30ea9c1e Andy McCall ↶ 68
- 2026-09-27 07:20 iteration 13: rename detection (snapshots and Git import) and oxy git fetch/push via a mirror (spec 0.6p). 108 tests.
28fc23d7 Andy McCall ↶ 69
- 2026-09-27 06:30 iteration 12: identity lifecycle and multi-signer policy updates; policy chains on fetch and push (spec 0.6n). 106 tests.
94084087 Andy McCall ↶ 70
- 2026-09-27 05:45 iteration 11: modern CLI output across all commands (user request). 105 tests.
04546979 Andy McCall ↶ 71
- 2026-09-27 05:15 iteration 10: orphaned references reified on rewrite (spec 0.6m); gc fixed to tolerate old views whose working copies were pruned under a zero op-retention during development (this repo's oldest 20 operations). 104 tests.
ec2cfb37 Andy McCall ↶ 72
- 2026-09-27 04:45 iteration 9: amend --into and split; graph delete/insert fix. 104 tests.
c177d175 Andy McCall ↶ 73
- 2026-09-27 03:55 iteration 8: hooks. 102 tests.
ebef3715 Andy McCall ↶ 74
- 2026-09-27 03:25 iteration 7: .oxyattributes. 101 tests.
4eb6dd3d Andy McCall ↶ 75
- 2026-09-27 02:55 iteration 6: multiple workspaces. 97 tests.
af5bd4e6 Andy McCall ↶ 76
- 2026-09-27 02:25 iteration 5: tags. 96 tests.
717ef88f Andy McCall ↶ 77
- 2026-09-27 01:50 iteration 4: shallow clone and deepen. 95 tests.
d1d1e9e8 Andy McCall ↶ 78
- 2026-09-27 01:05 iteration 3: sparse checkout. 94 tests.
13eddcc4 Andy McCall ↶ 79
- 2026-09-27 00:45 iteration 2: lazy blobs and objects endpoint. 93 tests.
7a716eb9 Andy McCall ↶ 80
- 2026-09-27 00:05 iteration 1: checkpoints, pack merging, CLI presentation, shell prompt. 91 tests.
e02bf0cd Andy McCall ↶ 81
82
## Phase 4 (2026-09-27): what Oxygenhub needs (oxygenhub/docs/spec.md section 18)
83
84
- [x] Bare repositories: `Repo::init_bare`, `is_bare`, `oxy init --bare`; `working_copy` refuses; `log`, `log_for_path` work from the bookmark state; `log_state`, `blame_in`, `log_for_path_in` for any state.
85
- [x] Embeddable server: `oxy_server::{Backend, Server, Endpoint, Access, Failure, DirBackend, bearer_token}`; `router` is `DirBackend` on `/{repo}/oxy/v1/{endpoint}`; `after_push` hook.
05e71ed6 Andy McCall ↶ 86
- [x] Anonymous (zero-author) changes are adopted by the pusher's seal (spec §14.4), found when pushing oxygen's own pre-identity history to the hub.
75167672 Andy McCall 87
- [x] Unbound hosted repositories (`Repo::init_bare_unbound`, `bind_id`, `Oxy-Repo-Id` header): an existing repository pushed to an empty hosted one keeps its id (protocol.md §2).
7aa2dc93 Andy McCall ↶ 88
- [x] `oxy identity sign-challenge` and `identity::{sign_challenge, verify_challenge, challenge_target}` (hub 0.2).
e02bf0cd Andy McCall ↶ 89
- [ ] `oxy seal <hash>..` and `--proposal URL` (hub 0.3).
90
- [ ] `oxy proposal show|land URL` (hub 0.3).
84b9e9ac Andy McCall ↶ 91
- [x] Delegate-signed changes from an edit set without a workspace: `Repo::build_change(heads, edits, message, author)`, `host_ref_update(bookmark, heads, signer)`, `apply_hosted(change, identities, update)` through `receive_push_report`; delegates of a rule's identities may sign reference attestations (spec §14.5, Draft 0.8g).
eae59556 Andy McCall ↶ 92
- [x] `Repo::log_window` and the shared `oxy_core::lanes::layout`; `State` is `Send + Sync` (closure cache behind a mutex) so hosts can cache loaded states.
7aa2dc93 Andy McCall ↶ 93
- [x] `receive_push_report` with `PushReport { results, updated: [RefUpdated { name, old_state, new_state, heads }], imported }`; `Backend::after_push` receives it.
119bbf50 Andy McCall ↶ 94
95
## Roadmap
96
97
The cross-product order of work is in `docs/roadmap.md`.