Admin › Backups: whether each owner's backup ran, and who has none 1e8a6e8bb91b
docs/backups.md §5. A table of every owner's backup: how often, where (the directory or gs://bucket/prefix), how long runs are kept, the repositories it covers, the last run with its outcome and what it sent, and the runs that failed in the last day; then the owners of repositories that no backup covers, with how many each has. Administrators only; an administrator sees that a backup ran, never what it wrote. The site's own backup runs on its host, outside the hub, and the page says so. closes #176
- Author
- Andy McCall @andymccall <[email protected]>
- When
- · 2026-10-14 03:10 UTC
- Signature
- ✔ signed by Andy McCall
- Hash
1e8a6e8bb91b2505c412628644d3f0ed64538798f61cc9852987c92c71c641bb- Depends on
- 18e381ea88c2 19bb31ff1f5c 5ef6cf72cd19 6dbd999ad794 757195ad7a5d d23464c0ac60 d7d182467382 e8e7158bbc73
- Depended on by
- 9041db4e4d92 737de0fac722 166f6e9bf4f1 5d90e0e045e2 2e9188ca4f98 7b9f4570daf5 d3b048b27277
- In bookmarks
- api-unknown-fields cancel-landed-runs diffusion-pin-bookmark draft-ready main oh-comment-body-file proposal-list-fast
- In tags
- v0.5.0 v0.5.1 v0.5.2 v0.6.0 v0.6.1 v0.7.0 v0.8.0 v0.9.0 v0.9.1 v0.9.11 v0.9.15 v0.9.16 v0.9.19 v0.9.2 v0.9.20 v0.9.21 v0.9.22 v0.9.23 v0.9.24 v0.9.3 v0.9.4 v0.9.5 v0.9.6 v0.9.7 v0.9.8 v0.9.9
- Edits
- 11 atoms, 1 tree operation, 7 files
Browse the tree at this change
added templates/admin_backups.html +35 −0
| 1 | + | {% extends "base.html" %} |
|
| 2 | + | {% block title %}Backups · Administration{% endblock %} |
|
| 3 | + | {% block mainclass %} class="wide"{% endblock %} |
|
| 4 | + | {% block content %} |
|
| 5 | + | <div class="head"><h1>Backups <small>{{ owners.len()|plural("owner", "owners") }} · <a href="/admin">Administration</a></small></h1></div> |
|
| 6 | + | <p class="small mute">Each user and organisation backs up its own repositories, to a place it controls (docs/backups.md). Here is whether each one ran and how it went; what a run wrote stays with its owner. This site's own backup runs on its host, outside the hub.</p> |
|
| 7 | + | <section class="card-box"> |
|
| 8 | + | <div class="card-head"><h2>Owners' backups</h2><span class="count-chip">{{ owners.len() }}</span></div> |
|
| 9 | + | {% if owners.is_empty() %}<p class="empty small">Nobody has set up a backup yet.</p> |
|
| 10 | + | {% else %} |
|
| 11 | + | <div class="table"><table class="backup-runs"> |
|
| 12 | + | <tr><th>Owner</th><th>How often</th><th>Where</th><th>Kept</th><th>Repositories</th><th>Last run</th><th>Failed today</th></tr> |
|
| 13 | + | {% for o in owners %} |
|
| 14 | + | <tr> |
|
| 15 | + | <td><a href="/{{ o.setting.owner }}">{{ o.setting.owner }}</a> <span class="mute small">{% if o.setting.kind == "org" %}organisation{% else %}user{% endif %}</span></td> |
|
| 16 | + | <td>{% if o.setting.every == "off" %}<span class="mute">off</span>{% else %}{{ o.setting.every }}{% endif %}</td> |
|
| 17 | + | <td><code>{{ o.place() }}</code></td> |
|
| 18 | + | <td>{{ o.setting.keep_days }} days</td> |
|
| 19 | + | <td>{{ o.repos }}</td> |
|
| 20 | + | <td>{% if let Some(r) = o.last %}<span class="backup-status st-{{ r.status }}">{{ r.status }}</span> <time title="{{ r.started_ms|when_i }}">{{ r.started_ms|ago_i }}</time> <span class="mute small">{{ r.repos }} repos, {{ (r.bytes / 1024) }} KiB sent</span>{% if !r.detail.is_empty() %}<div class="small mute">{{ r.detail }}</div>{% endif %}{% else %}<span class="mute">never</span>{% endif %}</td> |
|
| 21 | + | <td>{% if o.failed_day > 0 %}<span class="backup-status st-failed">{{ o.failed_day }}</span>{% else %}<span class="mute">0</span>{% endif %}</td> |
|
| 22 | + | </tr> |
|
| 23 | + | {% endfor %} |
|
| 24 | + | </table></div> |
|
| 25 | + | {% endif %} |
|
| 26 | + | </section> |
|
| 27 | + | <section class="card-box"> |
|
| 28 | + | <div class="card-head"><h2>Not backed up</h2><span class="count-chip">{{ uncovered.len() }}</span></div> |
|
| 29 | + | {% if uncovered.is_empty() %}<p class="empty small">Every owner of a repository backs it up.</p> |
|
| 30 | + | {% else %} |
|
| 31 | + | <p class="small mute">These own repositories that no backup covers: their owners set one up under Settings › Backups (an organisation's under its settings).</p> |
|
| 32 | + | <ul class="plain">{% for (owner, n) in uncovered %}<li><a href="/{{ owner }}">{{ owner }}</a> <span class="mute small">{{ n|plural("repository", "repositories") }}</span></li>{% endfor %}</ul> |
|
| 33 | + | {% endif %} |
|
| 34 | + | </section> |
|
| 35 | + | {% endblock %} |
modified crates/hub/src/backups.rs +48 −0
| … | 110 unchanged lines |
||
| 111 | 111 | sqlx::query_as("SELECT * FROM backup_runs WHERE owner = ? ORDER BY id DESC LIMIT ?").bind(owner).bind(limit).fetch_all(pool).await |
|
| 112 | 112 | } |
|
| 113 | 113 | ||
| 114 | + | /// One owner's backup as Admin › Backups shows it (§5): its setting, its |
|
| 115 | + | /// newest run, and how many runs failed in the last day. An administrator |
|
| 116 | + | /// sees that it ran, never what it wrote. |
|
| 117 | + | #[derive(Clone, Debug, Serialize)] |
|
| 118 | + | pub struct Overview { |
|
| 119 | + | pub setting: Setting, |
|
| 120 | + | pub last: Option<Run>, |
|
| 121 | + | pub failed_day: i64, |
|
| 122 | + | /// The repositories it covers now. |
|
| 123 | + | pub repos: i64, |
|
| 124 | + | } |
|
| 125 | + | ||
| 126 | + | impl Overview { |
|
| 127 | + | /// Where its runs go, as a person reads it. |
|
| 128 | + | pub fn place(&self) -> String { |
|
| 129 | + | let s = &self.setting; |
|
| 130 | + | match s.destination.as_str() { |
|
| 131 | + | "gcs" if s.prefix.is_empty() => format!("gs://{}", s.target), |
|
| 132 | + | "gcs" => format!("gs://{}/{}", s.target, s.prefix), |
|
| 133 | + | _ => format!("{}/{}", s.target, s.owner), |
|
| 134 | + | } |
|
| 135 | + | } |
|
| 136 | + | } |
|
| 137 | + | ||
| 138 | + | /// Every owner's backup (Admin › Backups), and the owners of repositories |
|
| 139 | + | /// that none covers, with how many each has. |
|
| 140 | + | pub async fn overview(pool: &Pool) -> sqlx::Result<(Vec<Overview>, Vec<(String, i64)>)> { |
|
| 141 | + | let settings: Vec<Setting> = sqlx::query_as("SELECT * FROM backup_settings ORDER BY owner").fetch_all(pool).await?; |
|
| 142 | + | let counts: Vec<(String, i64)> = |
|
| 143 | + | sqlx::query_as("SELECT owner, COUNT(*) FROM repositories GROUP BY owner ORDER BY owner").fetch_all(pool).await?; |
|
| 144 | + | let day_ago = db::now_ms() - 86_400_000; |
|
| 145 | + | let mut out = Vec::new(); |
|
| 146 | + | for s in settings { |
|
| 147 | + | let last = runs(pool, &s.owner, 1).await?.into_iter().next(); |
|
| 148 | + | let failed_day: i64 = |
|
| 149 | + | sqlx::query_scalar("SELECT COUNT(*) FROM backup_runs WHERE owner = ? AND status = 'failed' AND started_ms >= ?") |
|
| 150 | + | .bind(&s.owner) |
|
| 151 | + | .bind(day_ago) |
|
| 152 | + | .fetch_one(pool) |
|
| 153 | + | .await?; |
|
| 154 | + | let repos = counts.iter().find(|(o, _)| *o == s.owner).map_or(0, |c| c.1); |
|
| 155 | + | out.push(Overview { setting: s, last, failed_day, repos }); |
|
| 156 | + | } |
|
| 157 | + | let uncovered = |
|
| 158 | + | counts.into_iter().filter(|(owner, _)| !out.iter().any(|o| o.setting.owner == *owner && o.setting.every != "off")).collect(); |
|
| 159 | + | Ok((out, uncovered)) |
|
| 160 | + | } |
|
| 161 | + | ||
| 114 | 162 | // ---------------------------------------------------------------- where |
|
| 115 | 163 | ||
| 116 | 164 | /// Where backups are written: keys are `/`-separated paths. |
|
| … | 1184 unchanged lines |
modified crates/hub/src/web/backups.rs +21 −1
| … | 5 unchanged lines |
||
| 6 | 6 | use super::settings::{csrf, require}; |
|
| 7 | 7 | use super::{filters, Error, Page, Shell}; |
|
| 8 | 8 | use crate::auth::Viewer; |
|
| 9 | - | use crate::backups::{self, Run, Setting}; |
|
| 9 | + | use crate::backups::{self, Overview, Run, Setting}; |
|
| 10 | 10 | use crate::Shared; |
|
| 11 | 11 | use askama::Template; |
|
| 12 | 12 | use axum::extract::{Path, State}; |
|
| … | 228 unchanged lines |
||
| 241 | 241 | csrf(&viewer, &f.csrf_token, &shell)?; |
|
| 242 | 242 | test(&app, shell, "org", &org.name).await |
|
| 243 | 243 | } |
|
| 244 | + | ||
| 245 | + | #[derive(Template)] |
|
| 246 | + | #[template(path = "admin_backups.html")] |
|
| 247 | + | pub struct AdminPage { |
|
| 248 | + | shell: Shell, |
|
| 249 | + | owners: Vec<Overview>, |
|
| 250 | + | /// Owners of repositories no backup covers, with how many each has. |
|
| 251 | + | uncovered: Vec<(String, i64)>, |
|
| 252 | + | } |
|
| 253 | + | ||
| 254 | + | /// Admin › Backups (docs/backups.md §5): every owner's backup, whether it |
|
| 255 | + | /// ran and how it went, and who has none. Administrators only. |
|
| 256 | + | pub async fn admin(State(app): State<Shared>, viewer: Viewer) -> Result<Page<AdminPage>, Error> { |
|
| 257 | + | let shell = Shell::new(&app, "/admin/backups", &viewer); |
|
| 258 | + | if !viewer.is_admin() { |
|
| 259 | + | return Err(Error::not_found(shell)); |
|
| 260 | + | } |
|
| 261 | + | let (owners, uncovered) = backups::overview(&app.db).await.map_err(|e| Error::from_db(shell.clone(), e))?; |
|
| 262 | + | Ok(Page(AdminPage { shell, owners, uncovered })) |
|
| 263 | + | } |
modified crates/hub/src/web/mod.rs +1 −0
| … | 125 unchanged lines |
||
| 126 | 126 | .route("/admin/runners/settings", get(runners::settings).post(runners::save_settings)) |
|
| 127 | 127 | .route("/admin/runners/images", get(runners::images)) |
|
| 128 | 128 | .route("/admin/diffusion", get(diffusion::admin)) |
|
| 129 | + | .route("/admin/backups", get(backups::admin)) |
|
| 129 | 130 | .route("/{owner}/{name}/diffusion", get(diffusion::page)) |
|
| 130 | 131 | .route("/admin/runners/token", post(runners::new_token)) |
|
| 131 | 132 | .route("/admin/runners/{id}", get(runners::show)) |
|
| … | 621 unchanged lines |
modified crates/hub/tests/issues.rs +13 −0
| … | 1006 unchanged lines |
||
| 1007 | 1007 | let club = get(app, "/orgs/club/settings/backups", Some(&andy)).await; |
|
| 1008 | 1008 | assert!(club.body.contains("club</a>'s repositories"), "{}", club.body); |
|
| 1009 | 1009 | assert!(get(app, "/orgs/club/settings", Some(&andy)).await.body.contains("/orgs/club/settings/backups")); |
|
| 1010 | + | ||
| 1011 | + | // Admin › Backups (docs/backups.md §5): whether each ran, and who has |
|
| 1012 | + | // none; administrators only. |
|
| 1013 | + | sqlx::query("UPDATE accounts SET is_admin = (username = 'andy')").execute(&app.db).await.unwrap(); |
|
| 1014 | + | app.repos.create(&oxygenhub::repos::RepoName::parse("bob/game").unwrap(), None).unwrap(); |
|
| 1015 | + | db::repos::create(&app.db, "bob", "game", "", hub_api::Visibility::Public).await.unwrap(); |
|
| 1016 | + | assert_eq!(get(app, "/admin/backups", Some(&bob)).await.status, StatusCode::NOT_FOUND); |
|
| 1017 | + | let admin = get(app, "/admin/backups", Some(&andy)).await; |
|
| 1018 | + | assert_eq!(admin.status, StatusCode::OK); |
|
| 1019 | + | assert!(admin.body.contains("gs://andy-backup") && admin.body.contains("hourly") && admin.body.contains("never"), "{}", admin.body); |
|
| 1020 | + | let missing = &admin.body[admin.body.find("Not backed up").unwrap()..]; |
|
| 1021 | + | assert!(missing.contains("href=\"/bob\"") && missing.contains("1 repository"), "{missing}"); |
|
| 1022 | + | assert!(get(app, "/admin", Some(&andy)).await.body.contains("/admin/backups")); |
|
| 1010 | 1023 | } |
modified docs/backups.md +8 −4
| … | 5 unchanged lines |
||
| 6 | 6 | and wiki's store byte for byte, the records as JSON, retention, failure |
|
| 7 | 7 | notices, and restore (`oxygenhub admin backup run|restore`). Since |
|
| 8 | 8 | 2026-10-14 (oxygenhub#175) a restore brings back the records too (§7) and |
|
| 9 | - | release assets travel with the stores. Next: Admin › Backups, |
|
| 9 | + | release assets travel with the stores; Admin › Backups (#176). Next: |
|
| 10 | 10 | per-repository settings, S3. |
|
| 11 | 11 | ||
| 12 | 12 | ## 1. Why |
|
| … | 89 unchanged lines |
||
| 102 | 102 | ||
| 103 | 103 | - **A user:** Settings › Backups, for the repositories they own. |
|
| 104 | 104 | - **An organisation's owners:** the organisation's Settings › Backups. |
|
| 105 | - | - **The site administrator:** Admin › Backups, a table of every owner's |
|
| 106 | - | backup (destination, schedule, last run, outcome) and the site's own |
|
| 107 | - | backup. They may not read an owner's bucket, only see that it ran. |
|
| 105 | + | - **The site administrator:** Admin › Backups (built 2026-10-14, |
|
| 106 | + | oxygenhub#176), a table of every owner's backup (how often, where, how |
|
| 107 | + | long runs are kept, the repositories it covers, the last run with its |
|
| 108 | + | outcome, and the runs that failed in the last day), and the owners of |
|
| 109 | + | repositories no backup covers. They may not read an owner's bucket, only |
|
| 110 | + | see that it ran. The site's own backup runs on its host, outside the |
|
| 111 | + | hub, and the page says so. |
|
| 108 | 112 | - **A repository** (later): opt a repository out of its owner's backup, |
|
| 109 | 113 | or give it its own schedule. |
|
| 110 | 114 | ||
| … | 87 unchanged lines |
modified templates/admin.html +1 −0
| … | 18 unchanged lines |
||
| 19 | 19 | <a href="/admin/runners">Runners ↗</a> |
|
| 20 | 20 | <a href="/admin/runners/images">Images ↗</a> |
|
| 21 | 21 | <a href="/admin/diffusion">Diffusion ↗</a> |
|
| 22 | + | <a href="/admin/backups">Backups ↗</a> |
|
| 22 | 23 | <a href="/admin/storage">Storage and maintenance ↗</a> |
|
| 23 | 24 | </nav> |
|
| 24 | 25 | <div class="sections"> |
|
| … | 74 unchanged lines |