OXYGENHUB

Drops are backed up and restored as drops (refs #280) 9041db4e4d92

An owner's backup runs now take its drops with its repositories. A drop's
manifest and records are keyed drops~ID so a run stays one level deep;
restore brings the store back and the row as it was (kind, secrecy, title,
description). Admin > Backups counts drops with the repositories.
Author
Andy McCall @andymccall <[email protected]>
via
agent agent:claude@andymccall <[email protected]> · keyless, under your grant for changes,refs,seal · key 0b52438ba53a
When
· 2026-10-21 00:20 UTC
Signature
◈ sealed by agent:claude@andymccall
Hash
9041db4e4d925a26a14f3fcafe1a3c4c771bfd49051ebcbd7c1e0cf258547f2d
Depends on
166f6e9bf4f1 1e8a6e8bb91b 35fea4237cc9 3b3a51afde59 5ef6cf72cd19 6dbd999ad794 a7e6e8e6094c b5668108487f d3b048b27277 d7d182467382
Depended on by
5588bb3631d2
In bookmarks
api-unknown-fields diffusion-pin-bookmark draft-ready main proposal-list-fast
In tags
v0.7.0 v0.8.0 v0.9.0 v0.9.1 v0.9.11 v0.9.15 v0.9.16 v0.9.19 v0.9.2 v0.9.20 v0.9.21 v0.9.22 v0.9.23 v0.9.24 v0.9.3 v0.9.4 v0.9.5 v0.9.6 v0.9.7 v0.9.8 v0.9.9
Edits
13 atoms, 0 tree operations, 4 files

Browse the tree at this change

modified CHANGELOG.md +1 −0
…
3 unchanged lines
4 4
5 5
## Unreleased
6 6
7 +
- **Drops are backed up (#280):** an owner's backups now include its drops, each copied like a repository and restored as a drop (secret or public, with its title and description); Admin › Backups counts them with the repositories they cover.
7 8
- **A skip marker quoted in code does not skip:** a change message that mentions `[skip ci]` between backticks or in a fenced block (documenting the marker, say) now starts its run as usual; only a marker in the text itself skips it.
8 9
- **Administration has a sidebar, one page per section (#288):** the admin area was one long page plus separate pages for runners, images, Diffusion, backups and storage. Every page now has the same sidebar, grouped as Overview, People (accounts, registration and invitations, organisations), Sign-in (providers, security contact), Mail, Flow (runners, images, runner settings, insights, variables, secrets), Integrations (AI assistants, Diffusion), Data (storage and maintenance, backups) and Audit, each at its own address (`/admin/mail`, `/admin/audit`, …). `/admin` is an overview of the site's numbers. Forms keep their addresses and answer with their own page, and a kept `/admin#mail` link opens the Mail page.
9 10
- **Help: AI identities (#287):** a section on letting an AI agent work for you: keyless agents (a grant with `oh agent grant`, an agent home with `oxy auth grant`, ten-minute certified keys, nothing secret on disk), how its work shows (`via agent:NAME@YOU`), admitting it to a repository's trust policy (`oxy policy grant-subject`), agents with keys of their own (delegation), agent-marked tokens, approval gates, and revoking. "Letting the hub act for you" now says to export your identity again after delegating and upload that.
…
272 unchanged lines
modified crates/hub/src/backups.rs +15 −8
…
177 unchanged lines
178 178
pub async fn overview(pool: &Pool) -> sqlx::Result<(Vec<Overview>, Vec<(String, i64)>)> {
179 179
    let settings: Vec<Setting> = sqlx::query_as("SELECT * FROM backup_settings ORDER BY owner").fetch_all(pool).await?;
180 180
    let counts: Vec<(String, i64)> =
181 -
        sqlx::query_as("SELECT owner, COUNT(*) FROM repositories WHERE kind = 'repo' GROUP BY owner ORDER BY owner")
182 -
            .fetch_all(pool)
183 -
            .await?;
181 +
        sqlx::query_as("SELECT owner, COUNT(*) FROM repositories GROUP BY owner ORDER BY owner").fetch_all(pool).await?;
184 182
    let day_ago = db::now_ms() - 86_400_000;
185 183
    let mut out = Vec::new();
186 184
    for s in settings {
…
4 unchanged lines
191 189
                .bind(day_ago)
192 190
                .fetch_one(pool)
193 191
                .await?;
194 -
        let repos = db::repos::list(pool).await?.iter().filter(|r| r.owner == s.owner && included(&s, r, false)).count() as i64;
192 +
        let repos = db::repos::list_with_drops(pool).await?.iter().filter(|r| r.owner == s.owner && included(&s, r, false)).count() as i64;
195 193
        out.push(Overview { setting: s, last, failed_day, repos });
196 194
    }
197 195
    let uncovered = counts.into_iter().filter(|(owner, _)| !out.iter().any(|o| o.setting.owner == *owner && o.repos > 0)).collect();
…
678 unchanged lines
876 874
    /// `repos` or `wikis`.
877 875
    pub kind: String,
878 876
    pub files: Vec<File>,
877 +
}
878 +
879 +
/// A repository's name as one key segment: a drop's `drops/ID` becomes
880 +
/// `drops~ID` (#280), so a run's manifests and records stay one level deep;
881 +
/// the files inside keep the real name, which restore uses.
882 +
fn key_name(name: &str) -> String {
883 +
    name.replace('/', "~")
879 884
}
880 885
881 886
fn blob_key(hash: &str) -> String {
…
617 unchanged lines
1499 1504
        let d = dest.clone();
1500 1505
        tokio::task::spawn_blocking(move || last_manifests(d.as_ref())).await?
1501 1506
    };
1502 -
    // Its repositories, but those left out (§5).
1507 +
    // Its repositories and drops (#280), but those left out (§5).
1503 1508
    let mine: Vec<db::repos::RepoRow> =
1504 -
        db::repos::list(&app.db).await?.into_iter().filter(|r| r.owner == s.owner && included(s, r, by_hand)).collect();
1509 +
        db::repos::list_with_drops(&app.db).await?.into_iter().filter(|r| r.owner == s.owner && included(s, r, by_hand)).collect();
1505 1510
    let mut problems = Vec::new();
1506 1511
    let mut sent = Sent::default();
1507 1512
    let mut repos = 0i64;
…
36 unchanged lines
1544 1549
                    sent.sent += s1.sent + s2.sent;
1545 1550
                    sent.bytes += s1.bytes + s2.bytes;
1546 1551
                    let m = Manifest { owner: row.owner.clone(), name: row.name.clone(), kind: kind.into(), files };
1547 -
                    let key = format!("runs/{at}/repos/{kind}-{}.json", row.name);
1552 +
                    let key = format!("runs/{at}/repos/{kind}-{}.json", key_name(&row.name));
1548 1553
                    tokio::task::block_in_place(|| dest.put(&key, &serde_json::to_vec(&m)?))?;
1549 1554
                    if kind == "repos" {
1550 1555
                        repos += 1;
…
17 unchanged lines
1568 1573
        };
1569 1574
        match rec.await {
1570 1575
            Ok(r) => {
1571 -
                tokio::task::block_in_place(|| dest.put(&format!("runs/{at}/records/{}.json", row.name), &serde_json::to_vec(&r)?))?;
1576 +
                tokio::task::block_in_place(|| {
1577 +
                    dest.put(&format!("runs/{at}/records/{}.json", key_name(&row.name)), &serde_json::to_vec(&r)?)
1578 +
                })?;
1572 1579
                // Release assets live beside the stores, named by their
1573 1580
                // BLAKE3: each sent once, as the stores' files are.
1574 1581
                for hash in r["release_assets"].as_array().into_iter().flatten().filter_map(|a| a["hash"].as_str()) {
…
387 unchanged lines
modified crates/hub/tests/backups.rs +38 −0
…
154 unchanged lines
155 155
    assert!(refused.detail.contains("not a backup directory this hub allows"), "{}", refused.detail);
156 156
}
157 157
158 +
/// oxygenhub#280: an owner's drops are in its backups, and come back as
159 +
/// drops: their kind, secrecy and title with their store.
158 160
#[tokio::test(flavor = "multi_thread")]
161 +
async fn drops_are_backed_up_and_restored_as_drops() {
162 +
    let store = tempfile::tempdir().unwrap();
163 +
    let h = hub(store.path()).await;
164 +
    db::accounts::create_account(&h.app.db, "andy", "[email protected]", "x", "salt", false).await.unwrap();
165 +
    create(&h.app, "andy", "demo", hub_api::Visibility::Public).await;
166 +
    let id = "0123456789abcdef01234567";
167 +
    let name = RepoName::drop("andy", id).unwrap();
168 +
    h.app.repos.create(&name, None).unwrap();
169 +
    db::repos::create_drop(&h.app.db, "andy", id, "Notes", "a secret drop", hub_api::Visibility::Private).await.unwrap();
170 +
171 +
    let mut s = Setting::off("andy", "user");
172 +
    s.every = "hourly".into();
173 +
    s.target = store.path().display().to_string();
174 +
    backups::save(&h.app.db, &s).await.unwrap();
175 +
    let run = backups::run(&h.app, "andy", "now").await.unwrap();
176 +
    assert_eq!(run.status, "ok", "{}", run.detail);
177 +
    let dest = Directory { root: store.path().join("andy") };
178 +
    let repos = dest.list(&format!("runs/{}/repos", run.at)).unwrap();
179 +
    assert!(repos.contains(&format!("repos-drops~{id}.json")), "the drop's manifest, one level deep: {repos:?}");
180 +
    assert!(dest.list(&format!("runs/{}/records", run.at)).unwrap().contains(&format!("drops~{id}.json")));
181 +
    let (overview, _) = backups::overview(&h.app.db).await.unwrap();
182 +
    assert_eq!(overview[0].repos, 2, "the repository and the drop");
183 +
184 +
    let empty = tempfile::tempdir().unwrap();
185 +
    let fresh = hub(empty.path()).await;
186 +
    db::accounts::create_account(&fresh.app.db, "andy", "[email protected]", "x", "salt", false).await.unwrap();
187 +
    let dest: Arc<dyn Destination> = Arc::new(Directory { root: store.path().join("andy") });
188 +
    let done = backups::restore_into_hub(&fresh.app, dest, None).await.unwrap();
189 +
    assert!(done.contains(&format!("andy/drops/{id} (repos)")), "{done:?}");
190 +
    assert!(fresh.app.repos.exists(&name), "the drop's store");
191 +
    let back = db::repos::drop_by_id(&fresh.app.db, id).await.unwrap().expect("the drop's row");
192 +
    assert!(back.is_drop() && back.visibility == "private" && back.title == "Notes", "{back:?}");
193 +
    assert!(db::repos::list(&fresh.app.db).await.unwrap().iter().all(|r| !r.is_drop()), "still no drop among repositories");
194 +
}
195 +
196 +
#[tokio::test(flavor = "multi_thread")]
159 197
async fn backups_fall_due_on_their_schedule() {
160 198
    let store = tempfile::tempdir().unwrap();
161 199
    let h = hub(store.path()).await;
…
733 unchanged lines
modified docs/backups.md +1 −1
…
21 unchanged lines
22 22
23 23
## 2. What a backup holds
24 24
25 -
For each repository the owner has (a user's own, or an organisation's):
25 +
For each repository the owner has (a user's own, or an organisation's), and each of its drops (#280: a drop is a small repository of its own, kept as `repos-drops~ID` and `records/drops~ID` so its name stays one key deep, and restored as a drop, secret or public, with its title):
26 26
27 27
- **Its objects:** the store's content-addressed objects (changes,
28 28
  blobs, chunks, states, identities, signatures, seals, policies). An
…
234 unchanged lines