OXYGENHUB
# 50d93fa443dc oxygenhub / docs / reports.md
7.0 KiB · Markdown · last changed docs/reports.md: the Reports design 26 days ago renderedsource blame history raw
50d93fa4 Andy McCall ↶ 1 
# Reports
2
3 
*Design, 2026-10-11. Supersedes the separate Coverage and Tests tabs
4 
(spec §13.6, §13.7), which become two kinds of report.*
5
6 
A project produces more than pass and fail: coverage, test results,
7 
frame timings, sizes, benchmark curves. Today the hub reads two of them,
8 
LCOV coverage and JUnit tests, each with a tab of its own. This design
9 
makes reports one thing: a project declares what a report is, the hub
10 
keeps every run's copy, and readers move through its history.
11
12 
## 1. What a report is
13
14 
A **report** is a directory a Flow job writes, or anyone uploads:
15
16 
```
17 
report/
18 
  report.toml        the declaration (required)
19 
  index.html         a page to show (custom and performance kinds)
20 
  data/*.csv|json    datasets the page draws from
21 
  assets/…           images, scripts, styles: everything the page needs
22 
```
23
24 
`report.toml` says what it is:
25
26 
```toml
27 
kind    = "performance"        # coverage | tests | performance | custom
28 
title   = "Frame timing"
29 
summary = "Median frame 16.4 ms, p99 18.1 ms over 12 scenes"
30
31 
[metrics]                      # the numbers the hub tracks across runs
32 
median_frame_ms = { value = 16.4, unit = "ms", better = "lower" }
33 
p99_frame_ms    = { value = 18.1, unit = "ms", better = "lower", fail_above = 20 }
34 
scenes          = { value = 12 }
35 
```
36
37 
- **Built-in kinds** name their data file and need no page:
38 
  `kind = "coverage"` with `file = "lcov.info"` (LCOV or Cobertura), and
39 
  `kind = "tests"` with `file = "junit.xml"`. The hub reads them as it
40 
  does today, and works out their metrics itself (line coverage; passed,
41 
  failed, skipped).
42 
- **`performance`** is a page with metrics, as above.
43 
- **`custom`** is a page, metrics optional.
44 
- **A metric** is a number with an optional `unit`, `better` (`higher` or
45 
  `lower`, for colouring changes) and `fail_above` / `fail_below`, which
46 
  set a check on the change (§5).
47
48 
**Implicit declarations:** a job artifact named `coverage` or `junit`, the
49 
names used today, is read as a report of that kind with no `report.toml`,
50 
so every existing pipeline keeps working unchanged.
51
52 
## 2. How a report arrives
53
54 
**From Flow**, a job declares it in its outputs:
55
56 
```kdl
57 
job "perf" {
58 
    exec "sh" "-eu" ".flow/jobs/perf.sh"
59 
    output {
60 
        report "frame-timing" {
61 
            path "/src/out/report"
62 
            when "always"
63 
        }
64 
    }
65 
}
66 
```
67
68 
The runner uploads the directory as one bundle. `when "always"` keeps a
69 
failing job's report, as for artifacts.
70
71 
**From anywhere else:**
72 
`oh report upload frame-timing ./report --bookmark develop` attaches it to
73 
the bookmark's current state, or `--change ID` to a change's. The API is
74 
`POST /repos/{o}/{n}/reports/{name}` with the bundle as a tar.
75
76 
## 3. How the hub keeps it
77
78 
Three layers.
79
80 
1. **The bundle, per job, content-addressed.** Each file is a blob in the
81 
   repository's own store, chunked and deduplicated as artifacts are, and
82 
   the bundle is a small **tree**: a manifest of path → blob. A file that
83 
   did not change since the last run (the page, its scripts, a dataset that
84 
   did not move) is stored once however many runs name it; one that
85 
   changed a little costs only its changed chunks. Keeping every run's
86 
   report is cheap.
87 
2. **An index in the database:**
88 
   - `reports`: one row per report per job or upload: repository, name,
89 
     kind, title, summary, run and job (or uploader), bookmark, the
90 
     measured state and heads, time, and the tree's blob.
91 
   - `report_metrics`: one row per metric per report: key, value, unit,
92 
     better, thresholds.
93
94 
   History, trends, the scrubber and checks read only this. Coverage's
95 
   and Tests' existing tables become their kinds' details, keyed by
96 
   report.
97 
3. **Retention, by policy:**
98 
   - metrics: forever (they are a few bytes a run);
99 
   - bundles: every run on the default bookmark; the last 50 on any other
100 
     bookmark; a proposal's until it lands or closes plus 30 days; a tagged
101 
     release's forever. Set per repository in Settings › Reports.
102
103 
   An expired bundle's row keeps its metrics and loses its tree; the blobs
104 
   go at the next garbage collection.
105
106 
**Garbage collection** keeps every tree and file blob a kept report names,
107 
as it keeps artifacts (oxygenhub#126): the database is the only thing that
108 
names them.
109
110 
## 4. What readers see
111
112 
- **A Reports tab** (in place of Coverage and Tests), shown once a
113 
  repository has a report: each report's latest on the chosen bookmark,
114 
  with its kind, summary, metrics and their change against the run
115 
  before. Coverage and Tests are entries in it; their old addresses
116 
  redirect.
117 
- **A report's page:**
118 
  - **built-in kinds** show their own views (the coverage tree and file
119 
    view, the test cases and their history), as today;
120 
  - **performance and custom** show `index.html` in a **sandboxed frame
121 
    served from a separate origin**, with no network access and no access
122 
    to the hub's cookies or session, so a page may run any scripts and
123 
    charts it likes and still cannot act as the viewer;
124 
  - **metrics** as a strip of numbers with sparklines across runs.
125 
- **The timeline:** along the top, one stop per run on the bookmark,
126 
  oldest to newest, each labelled with the change it measured.
127 
  - **Scrub** to any run, or step with the arrow keys.
128 
  - **Play** steps through the runs at a chosen speed, so a chart can be
129 
    watched changing.
130 
  - **Compare** two runs side by side, with every metric's delta; on a
131 
    proposal the default is its base against its head.
132 
- **A frame interface** for pages that animate rather than reload: inside
133 
  the frame, `report.run()` gives the run on screen, and
134 
  `report.history("dataset")` that dataset across the runs, so a page can
135 
  redraw as the timeline moves or chart its own trends.
136 
- **Downloads:** the whole bundle as a zip, and each dataset alone.
137
138 
## 5. Checks
139
140 
A metric with `fail_above` or `fail_below` sets a check named
141 
`report/NAME/metric` on the change, failing when crossed, which a proposal
142 
may require (Settings › Protection). Coverage's existing check becomes
143 
this.
144
145 
## 6. The API and `oh`
146
147 
- `GET /repos/{o}/{n}/reports` (`?bookmark=`): the latest of each.
148 
- `GET /repos/{o}/{n}/reports/{name}` (`?bookmark=&limit=&before=`): its
149 
  history, metrics included.
150 
- `GET /repos/{o}/{n}/reports/{name}/{id}`: one report, and
151 
  `…/{id}/files/{path}` its files.
152 
- `POST /repos/{o}/{n}/reports/{name}` (repo:write): an upload.
153 
- `oh report list | view NAME | upload NAME DIR`.
154
155 
## 7. For a project adopting it
156
157 
1. Make the test or benchmark job write `out/report/` with a `report.toml`
158 
   and, for a page, `index.html` with its libraries bundled (no CDN: the
159 
   frame has no network) and its data in `data/`.
160 
2. Declare `report "NAME" { path "/src/out/report" when "always" }` in the
161 
   job's output.
162 
3. Push. The report appears on the Reports tab after the run.
163
164 
## 8. Order of work
165
166 
1. Storage and index: the `report` output kind in OxygenFlow, the runner's
167 
   bundle upload, the hub's tree, tables and gc roots; coverage and junit
168 
   read through it.
169 
2. The Reports tab and kinds; the sandboxed frame; metrics and checks.
170 
3. The timeline: scrub, play, compare, and the frame interface.
171 
4. `oh report` and the API; this document's §7 as the guide.