OXYGENHUB
◆ push-notes oxygen / docs / plan.md
15.5 KiB · Markdown · last changed Plan: tick oxy seal and oxy proposal, which exist 1 month ago renderedsource blame history raw

Overnight work plan

Each loop iteration: pick the first unchecked item, implement it with tests, run the gated checks, commit with oxy, tick the box here with a one-line note, and commit again. Update docs/spec.md (and its changelog) whenever behaviour the spec describes changes. If an item cannot be finished in two iterations, note why under it and move on.

Rules

  • Never run git init in this directory or push this project anywhere. Oxygen tracks itself with target/release/oxy.
  • Never modify ~/development/tetriblox; use only mirror clones in the scratch directory.
  • Tests must stay hermetic (hermetic() / OXY_HOME); never touch ~/.config/oxy.
  • Gate every commit on cargo test --workspace exit status and a clean cargo clippy --workspace --all-targets.
  • No new external services, no publishing, no artifacts.

Items

  • Checkpoints (spec §11) — done 2026-09-27 00:05. At 5,002 changes: status 0.10 s → 0.03 s, commit 0.20 s → 0.05 s; checkpoint write 0.10 s, payload 1.7 MB; states now hold change metadata only. serialize a State to a checkpoint object, load the nearest checkpoint in load_state and replay only the difference, create one automatically every core.checkpoint-interval changes (default 256) and on demand (oxy checkpoint). Benchmark before/after on a 5,000-change synthetic repository and record the numbers here.
  • Pack merging (spec §13.4) — done 2026-09-27 00:05; folds packs into one above 16, also after auto-pack. merge packs when more than 16 exist; oxy gc reports it.
  • Modern CLI presentation (user request, 2026-09-26 23:20) — done 2026-09-27 00:05 via src/cli/ui.rs (console + indicatif): colour, spinners on long commands, --plain, NO_COLOR and non-TTY detection; status/log/show/commit/checkout/verify/sync/gc restyled. colour and bold for ids, paths, and statuses; a live spinner or updating status line while a command works (snapshot, fetch, push, import, gc); indented, grouped output like Claude Code's. Use console/indicatif or equivalent; detect a TTY and honour NO_COLOR and --no-color/--plain so pipes and scripts get plain text; keep --json where it exists. Touch every command's output for consistency.
  • Shell prompt integration (user request, 2026-09-26 23:30) — done 2026-09-27 00:05: oxy prompt [--format], manifest-based dirty check (<10 ms), ahead/behind from tracking, docs/prompt.md with bash/oh-my-bash/zsh snippets. oxy prompt prints a compact one-line summary for shell prompts in a few milliseconds and with no side effects (no snapshot operation recorded, no locks): bookmark, working-copy ChangeId prefix, head count, a dirty marker (compare the file system against the working copy cheaply: sizes and mtimes first, content only when they differ), conflict marker, and ahead/behind counts against remotes/origin/<bookmark> when known. Provide docs/prompt.md with an oh-my-bash/bash snippet (oxy_prompt_info function, silent outside an oxy repository, colours via --plain off) and a zsh variant. Format flags: --format with placeholders, --plain.
  • (Pending, for the morning) Ask Andy to complete his cut-off request ("Can we also look at ..."). (Answered 2026-09-27: it was a stray copy of the prompt request. Done: oxy prompt --glyphs --color, new placeholders %s %U %O %i, and an oh-my-bash powerline segment in contrib/oh-my-bash/oxy.sh, installed to ~/.oh-my-bash/custom/oxy.sh; ~/.local/bin/oxy symlinks the release binary. GPG question settled: Andy said to leave GPG out.)
  • Lazy blobs and the objects endpoint — done 2026-09-27 00:45: clone --max-blob, filter.max-blob, .oxy/lazy/<ref> registry, fetch on first read (checkout, export), server objects route; status/commit need no blobs. (spec §13.5, §16.1, §17): clone --max-blob SIZE, absent blobs recorded with their remote, fetched on first read; server endpoint; wire encoding in spec §16.3.
  • Sparse checkout (spec §17) — done 2026-09-27 01:05: filter.paths, clone --sparse (sets max-blob 0), oxy sparse PATH.., snapshot/checkout/prompt honour scope; out-of-scope blobs never fetched. filter.paths in config, clone --sparse PATH.., status/commit/checkout honour it.
  • Shallow clone (spec §17) — done 2026-09-27 01:50: clone --depth N takes the remote's checkpoint plus N levels of history, .oxy/shallow boundary, negotiation and imports tolerate it, oxy deepen; log works from checkpoint metadata. clone --depth N, shallow boundary with a checkpoint, oxy deepen.
  • Tags (spec §10.1, §14.4) — done 2026-09-27 02:25: oxy tag create|list, checkout by tag, tag attestations, tags in views/refs/push, refused when existing with another state, policy rule tags/<name>, Git export as tags. oxy tag create|list, tag attestations, tags in views, wire, and export.
  • Multiple workspaces (spec §12.2) — done 2026-09-27 02:55: oxy workspace add|list|forget, secondary workspaces via a .oxy file, per-workspace manifest and snapshot lock with stale takeover. oxy workspace add|list|forget, per-workspace directory metadata, snapshot lock.
  • .oxyattributes (spec §19) — done 2026-09-27 03:25: gitattributes-style rules; binary/text, eol=lf|crlf (LF stored, CRLF on checkout), nocompress chunks packed raw, lockable read-only until oxy lock; attributes taken from the files being committed. binary, eol=lf|crlf, nocompress, lockable (advisory locally).
  • Hooks (spec §20) — done 2026-09-27 03:55: executables in .oxy/hooks/<event>, JSON on stdin, pre-* veto via exit status with stderr as the message, semantic-deps adds explicit deps when hooks.semantic-deps = true. pre-commit, post-commit, pre-push, post-fetch, post-checkout, and the semantic-deps hook with its JSON interface.
  • Amend on non-head changes; oxy split — done 2026-09-27 04:45: oxy amend --into CHANGE appends the working copy's edits to an older change when they reference only its closure (identity correspondence for dependents, refused otherwise with a clear message); oxy split CHANGE PATH.. rebuilds a head as two changes. Found and fixed a graph bug: a delete/insert conflict now requires the deleter and inserter to commute (both directions).
  • Reify orphaned references on rewrite as conflicts instead of refusing (spec §8.4); define the conflict shape in the spec. (Done: re-anchoring to the nearest surviving line, else the chain's anchors, so order conflicts arise naturally; vanished nodes drop dependent edits; notes reported by oxy abandon.)
  • User request (2026-09-27, mid-loop): make oxy log and all output look modern like git log with lines, spacing, and glyphs when the terminal supports them. (Done: rail-and-bullet log with -n/-v, relative times, authors, signature marks; status/show/op log/tags/workspaces/remotes/fetch/push/policy/identity restyled; ASCII fallback when piped or with --plain/NO_COLOR/non-UTF-8 locale.)
  • Identity lifecycle: oxy identity rotate|revoke, expiry, delegates (oxy identity delegate add|remove), multi-signer policy updates. (Done: rotation as supersession with chain-aware policy authorization, renew, delegates, revocation attestations and local markers, expiry checks, root add/remove/threshold, proposal files and oxy policy sign; spec 0.6n.)
  • Push and serve the chain of policies since the receiver's serial, so several offline rotations or updates publish in one push or fetch. (Done in iteration 12: .oxy/policies/ index, refs history, bundle policies.)
  • Rename detection on Git import (similarity ≥ 50%); oxy git fetch/push against a Git remote via a mirror. (Done: detection lives in the change builder, so working-copy renames are tracked too; fetch/push through .oxy/git/<key>/mirror.git with fast-forward chaining; spec 0.6p.)
  • Hardening: property tests for the CBOR decoder and pack reader against random and mutated inputs; a concurrent-push test against the server; a 1 GB single-file benchmark for chunking and blobs. (Done: random/mutated CBOR never panics and accepted input is canonical; 300 rounds of pack/index corruption fail cleanly, with new index bounds checks; 8 concurrent HTTP pushes serialise with pull-and-retry; bench_one_gibibyte_file is an ignored release-mode test, run with OXY_BENCH_DIR on a real disk.)
  • Performance follow-up from the 1 GiB benchmark: commit takes 5.6 s on tmpfs but 63 s on the SSD, and clone 49 s, because every object is written with its own fsync (13,715 chunks). Write committed and fetched objects straight into a pack with one fsync, or batch loose writes. (Done: store write batches; small batches stay loose, large ones spill to one pack. SSD: commit 63 s → 8.8 s, clone 49 s → 15.9 s; spec 0.6r.)
  • Performance follow-up: oxy status re-hashes a clean 1 GiB working copy (2.4 s). Record size and mtime in the manifest and skip hashing files whose stat is unchanged (spec allows a stat cache). (Done: manifest records blob refs for opaque files; unchanged files are not read; two-second racy window; status on a clean 1 GiB copy 2.4 s → 4 ms; spec 0.6s.)
  • Documentation: docs/guide.md (user guide by task) and docs/protocol.md (for other implementers, derived from spec §16 and the wire encodings). (Done; every command and flag in the guide was checked against --help.)
  • Spec review pass: resolve or re-word every "MAY ... until" allowance introduced during implementation, and close what can be closed in §27. (Done as Draft 0.7: no "MAY ... until" wording remained; oxy status --json implemented to meet §9.5; §12.4, §16.1 and §24 aligned with the implementation; new §26.1 status table; §27 records workspace conflicts and server locks as open.)

Phase 2 (2026-09-27, Andy's request: the everyday tools Git has)

Rules as above. Spec Draft 0.8 defines each item; implement in this order, each with tests, a gate, and an oxy commit.

  • Multiple bookmarks (spec §10.1): oxy bookmark create|set|delete|list|switch, per-workspace current bookmark, oxy merge BOOKMARK, oxy checkout NAME, bookmark names resolved before change prefixes; push/pull/log act on the current bookmark; oxy push -b.
  • Subrepositories (spec §7.8, §23): node kind subrepo, pin encoding, .oxymodules, snapshot stops at nested repositories and reads their pin, checkout moves or leaves nested repositories, oxy submodule add|update|status|remove; Git import/export of gitlinks and .gitmodules (test with a mirror of tetriblox, never the live repository).
  • Partial commits (spec §12.3): oxy commit PATH.. and oxy commit --interactive (hunk = one edit or tree operation, dependent hunks pulled in), sharing the split machinery.
  • Blame (spec §27): oxy blame PATH [--json] from atom keys, with author and date; renames followed.
  • Bisect (spec §27): oxy bisect start|good|bad|run|reset over dependency-closed prefixes; state under .oxy/bisect/.
  • Squash (spec §8.5): oxy squash CHANGE --into DEST generalising amend --into, with rewrite propagation.
  • Rebase (spec §8.5): oxy rebase CHANGE --onto HEADS.. re-deriving edits against a new base; refused on conflicted files.
  • Guide, README and §26.1 updated for each; spec changelog per item.

Phase 3 (2026-09-27): machine interface for the VS Code extension (oxygen-vscode/docs/spec.md section 4)

  • oxy root, oxy cat PATH [-r CHANGE], oxy version --json, oxy discard PATH.. | --hunks N,M.
  • JSON outputs: log --json [--path P] [--bookmark B], show --json, diff --json [CHANGE], hunks --json, commit --hunks, bookmark list --json, op log --json, resolve --json, submodule status --json, bisect status --json, verify --json, policy show --json, remote list --json, fetch|pull|push --json.
  • An integration test that runs every JSON command against a scratch repository and parses each output.
  • Spec §24 and the guide mention the machine interface; docs/protocol.md gets a pointer.

Log

  • 2026-09-26 23:01 plan created.
  • 2026-09-27 12:15 morning: glyph/colour prompt modes and the oh-my-bash powerline segment. 116 tests.
  • 2026-09-27 10:00 iteration 18: spec review pass (Draft 0.7) and oxy status --json. 113 tests.
  • 2026-09-27 09:35 iteration 17: user guide and protocol document. 112 tests.
  • 2026-09-27 09:10 iteration 16: manifest stat cache (status 2.4 s → 4 ms on 1 GiB). 112 tests.
  • 2026-09-27 08:40 iteration 15: batched object writes (commit 8.8 s, clone 15.9 s for 1 GiB on SSD). 111 tests.
  • 2026-09-27 08:05 iteration 14: hardening (CBOR and pack fuzz-style tests, concurrent pushes, 1 GiB benchmark: write 0.7 s, commit 63 s on SSD / 5.6 s on tmpfs, status 2.4 s, clone 49 s, read back 1.0 s, store 1025.6 MiB, 13,715 objects). 110 tests, 1 ignored benchmark.
  • 2026-09-27 07:20 iteration 13: rename detection (snapshots and Git import) and oxy git fetch/push via a mirror (spec 0.6p). 108 tests.
  • 2026-09-27 06:30 iteration 12: identity lifecycle and multi-signer policy updates; policy chains on fetch and push (spec 0.6n). 106 tests.
  • 2026-09-27 05:45 iteration 11: modern CLI output across all commands (user request). 105 tests.
  • 2026-09-27 05:15 iteration 10: orphaned references reified on rewrite (spec 0.6m); gc fixed to tolerate old views whose working copies were pruned under a zero op-retention during development (this repo's oldest 20 operations). 104 tests.
  • 2026-09-27 04:45 iteration 9: amend --into and split; graph delete/insert fix. 104 tests.
  • 2026-09-27 03:55 iteration 8: hooks. 102 tests.
  • 2026-09-27 03:25 iteration 7: .oxyattributes. 101 tests.
  • 2026-09-27 02:55 iteration 6: multiple workspaces. 97 tests.
  • 2026-09-27 02:25 iteration 5: tags. 96 tests.
  • 2026-09-27 01:50 iteration 4: shallow clone and deepen. 95 tests.
  • 2026-09-27 01:05 iteration 3: sparse checkout. 94 tests.
  • 2026-09-27 00:45 iteration 2: lazy blobs and objects endpoint. 93 tests.
  • 2026-09-27 00:05 iteration 1: checkpoints, pack merging, CLI presentation, shell prompt. 91 tests.

Phase 4 (2026-09-27): what OxygenHub needs (oxygenhub/docs/spec.md section 18)

  • Bare repositories: Repo::init_bare, is_bare, oxy init --bare; working_copy refuses; log, log_for_path work from the bookmark state; log_state, blame_in, log_for_path_in for any state.
  • Embeddable server: oxy_server::{Backend, Server, Endpoint, Access, Failure, DirBackend, bearer_token}; router is DirBackend on /{repo}/oxy/v1/{endpoint}; after_push hook.
  • Anonymous (zero-author) changes are adopted by the pusher's seal (spec §14.4), found when pushing oxygen's own pre-identity history to the hub.
  • Unbound hosted repositories (Repo::init_bare_unbound, bind_id, Oxy-Repo-Id header): an existing repository pushed to an empty hosted one keeps its id (protocol.md §2).
  • oxy identity sign-challenge and identity::{sign_challenge, verify_challenge, challenge_target} (hub 0.2).
  • oxy seal <hash>.. and --proposal URL (hub 0.3).
  • oxy proposal show|land URL (hub 0.3).
  • Delegate-signed changes from an edit set without a workspace: Repo::build_change(heads, edits, message, author), host_ref_update(bookmark, heads, signer), apply_hosted(change, identities, update) through receive_push_report; delegates of a rule's identities may sign reference attestations (spec §14.5, Draft 0.8g).
  • Repo::log_window and the shared oxy_core::lanes::layout; State is Send + Sync (closure cache behind a mutex) so hosts can cache loaded states.
  • receive_push_report with PushReport { results, updated: [RefUpdated { name, old_state, new_state, heads }], imported }; Backend::after_push receives it.

Roadmap

The cross-product order of work is in docs/roadmap.md.