Drops are backed up and restored as drops (refs #280) 9041db4e4d92
An owner's backup runs now take its drops with its repositories. A drop's manifest and records are keyed drops~ID so a run stays one level deep; restore brings the store back and the row as it was (kind, secrecy, title, description). Admin > Backups counts drops with the repositories.
- Author
- Andy McCall @andymccall <[email protected]>
- via
- agent
agent:claude@andymccall <[email protected]> · keyless, under your grant for changes,refs,seal · key 0b52438ba53a
- When
- · 2026-10-21 00:20 UTC
- Signature
- ◈ sealed by agent:claude@andymccall
- Hash
9041db4e4d925a26a14f3fcafe1a3c4c771bfd49051ebcbd7c1e0cf258547f2d- Depends on
- 166f6e9bf4f1 1e8a6e8bb91b 35fea4237cc9 3b3a51afde59 5ef6cf72cd19 6dbd999ad794 a7e6e8e6094c b5668108487f d3b048b27277 d7d182467382
- Depended on by
- 5588bb3631d2
- In bookmarks
- api-unknown-fields diffusion-pin-bookmark draft-ready main proposal-list-fast
- In tags
- v0.7.0 v0.8.0 v0.9.0 v0.9.1 v0.9.11 v0.9.15 v0.9.16 v0.9.19 v0.9.2 v0.9.20 v0.9.21 v0.9.22 v0.9.23 v0.9.24 v0.9.3 v0.9.4 v0.9.5 v0.9.6 v0.9.7 v0.9.8 v0.9.9
- Edits
- 13 atoms, 0 tree operations, 4 files
Browse the tree at this change
modified CHANGELOG.md +1 −0
| … | 3 unchanged lines |
||
| 4 | 4 | ||
| 5 | 5 | ## Unreleased |
|
| 6 | 6 | ||
| 7 | + | - **Drops are backed up (#280):** an owner's backups now include its drops, each copied like a repository and restored as a drop (secret or public, with its title and description); Admin › Backups counts them with the repositories they cover. |
|
| 7 | 8 | - **A skip marker quoted in code does not skip:** a change message that mentions `[skip ci]` between backticks or in a fenced block (documenting the marker, say) now starts its run as usual; only a marker in the text itself skips it. |
|
| 8 | 9 | - **Administration has a sidebar, one page per section (#288):** the admin area was one long page plus separate pages for runners, images, Diffusion, backups and storage. Every page now has the same sidebar, grouped as Overview, People (accounts, registration and invitations, organisations), Sign-in (providers, security contact), Mail, Flow (runners, images, runner settings, insights, variables, secrets), Integrations (AI assistants, Diffusion), Data (storage and maintenance, backups) and Audit, each at its own address (`/admin/mail`, `/admin/audit`, …). `/admin` is an overview of the site's numbers. Forms keep their addresses and answer with their own page, and a kept `/admin#mail` link opens the Mail page. |
|
| 9 | 10 | - **Help: AI identities (#287):** a section on letting an AI agent work for you: keyless agents (a grant with `oh agent grant`, an agent home with `oxy auth grant`, ten-minute certified keys, nothing secret on disk), how its work shows (`via agent:NAME@YOU`), admitting it to a repository's trust policy (`oxy policy grant-subject`), agents with keys of their own (delegation), agent-marked tokens, approval gates, and revoking. "Letting the hub act for you" now says to export your identity again after delegating and upload that. |
|
| … | 272 unchanged lines |
modified crates/hub/src/backups.rs +15 −8
| … | 177 unchanged lines |
||
| 178 | 178 | pub async fn overview(pool: &Pool) -> sqlx::Result<(Vec<Overview>, Vec<(String, i64)>)> { |
|
| 179 | 179 | let settings: Vec<Setting> = sqlx::query_as("SELECT * FROM backup_settings ORDER BY owner").fetch_all(pool).await?; |
|
| 180 | 180 | let counts: Vec<(String, i64)> = |
|
| 181 | - | sqlx::query_as("SELECT owner, COUNT(*) FROM repositories WHERE kind = 'repo' GROUP BY owner ORDER BY owner") |
|
| 182 | - | .fetch_all(pool) |
|
| 183 | - | .await?; |
|
| 181 | + | sqlx::query_as("SELECT owner, COUNT(*) FROM repositories GROUP BY owner ORDER BY owner").fetch_all(pool).await?; |
|
| 184 | 182 | let day_ago = db::now_ms() - 86_400_000; |
|
| 185 | 183 | let mut out = Vec::new(); |
|
| 186 | 184 | for s in settings { |
|
| … | 4 unchanged lines |
||
| 191 | 189 | .bind(day_ago) |
|
| 192 | 190 | .fetch_one(pool) |
|
| 193 | 191 | .await?; |
|
| 194 | - | let repos = db::repos::list(pool).await?.iter().filter(|r| r.owner == s.owner && included(&s, r, false)).count() as i64; |
|
| 192 | + | let repos = db::repos::list_with_drops(pool).await?.iter().filter(|r| r.owner == s.owner && included(&s, r, false)).count() as i64; |
|
| 195 | 193 | out.push(Overview { setting: s, last, failed_day, repos }); |
|
| 196 | 194 | } |
|
| 197 | 195 | let uncovered = counts.into_iter().filter(|(owner, _)| !out.iter().any(|o| o.setting.owner == *owner && o.repos > 0)).collect(); |
|
| … | 678 unchanged lines |
||
| 876 | 874 | /// `repos` or `wikis`. |
|
| 877 | 875 | pub kind: String, |
|
| 878 | 876 | pub files: Vec<File>, |
|
| 877 | + | } |
|
| 878 | + | ||
| 879 | + | /// A repository's name as one key segment: a drop's `drops/ID` becomes |
|
| 880 | + | /// `drops~ID` (#280), so a run's manifests and records stay one level deep; |
|
| 881 | + | /// the files inside keep the real name, which restore uses. |
|
| 882 | + | fn key_name(name: &str) -> String { |
|
| 883 | + | name.replace('/', "~") |
|
| 879 | 884 | } |
|
| 880 | 885 | ||
| 881 | 886 | fn blob_key(hash: &str) -> String { |
|
| … | 617 unchanged lines |
||
| 1499 | 1504 | let d = dest.clone(); |
|
| 1500 | 1505 | tokio::task::spawn_blocking(move || last_manifests(d.as_ref())).await? |
|
| 1501 | 1506 | }; |
|
| 1502 | - | // Its repositories, but those left out (§5). |
|
| 1507 | + | // Its repositories and drops (#280), but those left out (§5). |
|
| 1503 | 1508 | let mine: Vec<db::repos::RepoRow> = |
|
| 1504 | - | db::repos::list(&app.db).await?.into_iter().filter(|r| r.owner == s.owner && included(s, r, by_hand)).collect(); |
|
| 1509 | + | db::repos::list_with_drops(&app.db).await?.into_iter().filter(|r| r.owner == s.owner && included(s, r, by_hand)).collect(); |
|
| 1505 | 1510 | let mut problems = Vec::new(); |
|
| 1506 | 1511 | let mut sent = Sent::default(); |
|
| 1507 | 1512 | let mut repos = 0i64; |
|
| … | 36 unchanged lines |
||
| 1544 | 1549 | sent.sent += s1.sent + s2.sent; |
|
| 1545 | 1550 | sent.bytes += s1.bytes + s2.bytes; |
|
| 1546 | 1551 | let m = Manifest { owner: row.owner.clone(), name: row.name.clone(), kind: kind.into(), files }; |
|
| 1547 | - | let key = format!("runs/{at}/repos/{kind}-{}.json", row.name); |
|
| 1552 | + | let key = format!("runs/{at}/repos/{kind}-{}.json", key_name(&row.name)); |
|
| 1548 | 1553 | tokio::task::block_in_place(|| dest.put(&key, &serde_json::to_vec(&m)?))?; |
|
| 1549 | 1554 | if kind == "repos" { |
|
| 1550 | 1555 | repos += 1; |
|
| … | 17 unchanged lines |
||
| 1568 | 1573 | }; |
|
| 1569 | 1574 | match rec.await { |
|
| 1570 | 1575 | Ok(r) => { |
|
| 1571 | - | tokio::task::block_in_place(|| dest.put(&format!("runs/{at}/records/{}.json", row.name), &serde_json::to_vec(&r)?))?; |
|
| 1576 | + | tokio::task::block_in_place(|| { |
|
| 1577 | + | dest.put(&format!("runs/{at}/records/{}.json", key_name(&row.name)), &serde_json::to_vec(&r)?) |
|
| 1578 | + | })?; |
|
| 1572 | 1579 | // Release assets live beside the stores, named by their |
|
| 1573 | 1580 | // BLAKE3: each sent once, as the stores' files are. |
|
| 1574 | 1581 | for hash in r["release_assets"].as_array().into_iter().flatten().filter_map(|a| a["hash"].as_str()) { |
|
| … | 387 unchanged lines |
modified crates/hub/tests/backups.rs +38 −0
| … | 154 unchanged lines |
||
| 155 | 155 | assert!(refused.detail.contains("not a backup directory this hub allows"), "{}", refused.detail); |
|
| 156 | 156 | } |
|
| 157 | 157 | ||
| 158 | + | /// oxygenhub#280: an owner's drops are in its backups, and come back as |
|
| 159 | + | /// drops: their kind, secrecy and title with their store. |
|
| 158 | 160 | #[tokio::test(flavor = "multi_thread")] |
|
| 161 | + | async fn drops_are_backed_up_and_restored_as_drops() { |
|
| 162 | + | let store = tempfile::tempdir().unwrap(); |
|
| 163 | + | let h = hub(store.path()).await; |
|
| 164 | + | db::accounts::create_account(&h.app.db, "andy", "[email protected]", "x", "salt", false).await.unwrap(); |
|
| 165 | + | create(&h.app, "andy", "demo", hub_api::Visibility::Public).await; |
|
| 166 | + | let id = "0123456789abcdef01234567"; |
|
| 167 | + | let name = RepoName::drop("andy", id).unwrap(); |
|
| 168 | + | h.app.repos.create(&name, None).unwrap(); |
|
| 169 | + | db::repos::create_drop(&h.app.db, "andy", id, "Notes", "a secret drop", hub_api::Visibility::Private).await.unwrap(); |
|
| 170 | + | ||
| 171 | + | let mut s = Setting::off("andy", "user"); |
|
| 172 | + | s.every = "hourly".into(); |
|
| 173 | + | s.target = store.path().display().to_string(); |
|
| 174 | + | backups::save(&h.app.db, &s).await.unwrap(); |
|
| 175 | + | let run = backups::run(&h.app, "andy", "now").await.unwrap(); |
|
| 176 | + | assert_eq!(run.status, "ok", "{}", run.detail); |
|
| 177 | + | let dest = Directory { root: store.path().join("andy") }; |
|
| 178 | + | let repos = dest.list(&format!("runs/{}/repos", run.at)).unwrap(); |
|
| 179 | + | assert!(repos.contains(&format!("repos-drops~{id}.json")), "the drop's manifest, one level deep: {repos:?}"); |
|
| 180 | + | assert!(dest.list(&format!("runs/{}/records", run.at)).unwrap().contains(&format!("drops~{id}.json"))); |
|
| 181 | + | let (overview, _) = backups::overview(&h.app.db).await.unwrap(); |
|
| 182 | + | assert_eq!(overview[0].repos, 2, "the repository and the drop"); |
|
| 183 | + | ||
| 184 | + | let empty = tempfile::tempdir().unwrap(); |
|
| 185 | + | let fresh = hub(empty.path()).await; |
|
| 186 | + | db::accounts::create_account(&fresh.app.db, "andy", "[email protected]", "x", "salt", false).await.unwrap(); |
|
| 187 | + | let dest: Arc<dyn Destination> = Arc::new(Directory { root: store.path().join("andy") }); |
|
| 188 | + | let done = backups::restore_into_hub(&fresh.app, dest, None).await.unwrap(); |
|
| 189 | + | assert!(done.contains(&format!("andy/drops/{id} (repos)")), "{done:?}"); |
|
| 190 | + | assert!(fresh.app.repos.exists(&name), "the drop's store"); |
|
| 191 | + | let back = db::repos::drop_by_id(&fresh.app.db, id).await.unwrap().expect("the drop's row"); |
|
| 192 | + | assert!(back.is_drop() && back.visibility == "private" && back.title == "Notes", "{back:?}"); |
|
| 193 | + | assert!(db::repos::list(&fresh.app.db).await.unwrap().iter().all(|r| !r.is_drop()), "still no drop among repositories"); |
|
| 194 | + | } |
|
| 195 | + | ||
| 196 | + | #[tokio::test(flavor = "multi_thread")] |
|
| 159 | 197 | async fn backups_fall_due_on_their_schedule() { |
|
| 160 | 198 | let store = tempfile::tempdir().unwrap(); |
|
| 161 | 199 | let h = hub(store.path()).await; |
|
| … | 733 unchanged lines |
modified docs/backups.md +1 −1
| … | 21 unchanged lines |
||
| 22 | 22 | ||
| 23 | 23 | ## 2. What a backup holds |
|
| 24 | 24 | ||
| 25 | - | For each repository the owner has (a user's own, or an organisation's): |
|
| 25 | + | For each repository the owner has (a user's own, or an organisation's), and each of its drops (#280: a drop is a small repository of its own, kept as `repos-drops~ID` and `records/drops~ID` so its name stays one key deep, and restored as a drop, secret or public, with its title): |
|
| 26 | 26 | ||
| 27 | 27 | - **Its objects:** the store's content-addressed objects (changes, |
|
| 28 | 28 | blobs, chunks, states, identities, signatures, seals, policies). An |
|
| … | 234 unchanged lines |