docs/reports.md: the Reports design 50d93fa443dc

# Reports

*Design, 2026-10-11. Supersedes the separate Coverage and Tests tabs

(spec §13.6, §13.7), which become two kinds of report.*

A project produces more than pass and fail: coverage, test results,

frame timings, sizes, benchmark curves. Today the hub reads two of them,

LCOV coverage and JUnit tests, each with a tab of its own. This design

makes reports one thing: a project declares what a report is, the hub

keeps every run's copy, and readers move through its history.

## 1. What a report is

A **report** is a directory a Flow job writes, or anyone uploads:

```

report/

  report.toml        the declaration (required)

  index.html         a page to show (custom and performance kinds)

  data/*.csv|json    datasets the page draws from

  assets/…           images, scripts, styles: everything the page needs

```

`report.toml` says what it is:

```toml

kind    = "performance"        # coverage | tests | performance | custom

title   = "Frame timing"

summary = "Median frame 16.4 ms, p99 18.1 ms over 12 scenes"

[metrics]                      # the numbers the hub tracks across runs

median_frame_ms = { value = 16.4, unit = "ms", better = "lower" }

p99_frame_ms    = { value = 18.1, unit = "ms", better = "lower", fail_above = 20 }

scenes          = { value = 12 }

```

- **Built-in kinds** name their data file and need no page:

  `kind = "coverage"` with `file = "lcov.info"` (LCOV or Cobertura), and

  `kind = "tests"` with `file = "junit.xml"`. The hub reads them as it

  does today, and works out their metrics itself (line coverage; passed,

  failed, skipped).

- **`performance`** is a page with metrics, as above.

- **`custom`** is a page, metrics optional.

- **A metric** is a number with an optional `unit`, `better` (`higher` or

  `lower`, for colouring changes) and `fail_above` / `fail_below`, which

  set a check on the change (§5).

**Implicit declarations:** a job artifact named `coverage` or `junit`, the

names used today, is read as a report of that kind with no `report.toml`,

so every existing pipeline keeps working unchanged.

## 2. How a report arrives

**From Flow**, a job declares it in its outputs:

```kdl

job "perf" {

    exec "sh" "-eu" ".flow/jobs/perf.sh"

    output {

        report "frame-timing" {

            path "/src/out/report"

            when "always"

        }

    }

}

```

The runner uploads the directory as one bundle. `when "always"` keeps a

failing job's report, as for artifacts.

**From anywhere else:**

`oh report upload frame-timing ./report --bookmark develop` attaches it to

the bookmark's current state, or `--change ID` to a change's. The API is

`POST /repos/{o}/{n}/reports/{name}` with the bundle as a tar.

## 3. How the hub keeps it

Three layers.

1. **The bundle, per job, content-addressed.** Each file is a blob in the

   repository's own store, chunked and deduplicated as artifacts are, and

   the bundle is a small **tree**: a manifest of path → blob. A file that

   did not change since the last run (the page, its scripts, a dataset that

   did not move) is stored once however many runs name it; one that

   changed a little costs only its changed chunks. Keeping every run's

   report is cheap.

2. **An index in the database:**

   - `reports`: one row per report per job or upload: repository, name,

     kind, title, summary, run and job (or uploader), bookmark, the

     measured state and heads, time, and the tree's blob.

   - `report_metrics`: one row per metric per report: key, value, unit,

     better, thresholds.

   History, trends, the scrubber and checks read only this. Coverage's

   and Tests' existing tables become their kinds' details, keyed by

   report.

3. **Retention, by policy:**

   - metrics: forever (they are a few bytes a run);

   - bundles: every run on the default bookmark; the last 50 on any other

     bookmark; a proposal's until it lands or closes plus 30 days; a tagged

     release's forever. Set per repository in Settings › Reports.

   An expired bundle's row keeps its metrics and loses its tree; the blobs

   go at the next garbage collection.

**Garbage collection** keeps every tree and file blob a kept report names,

as it keeps artifacts (oxygenhub#126): the database is the only thing that

names them.

## 4. What readers see

- **A Reports tab** (in place of Coverage and Tests), shown once a

  repository has a report: each report's latest on the chosen bookmark,

  with its kind, summary, metrics and their change against the run

  before. Coverage and Tests are entries in it; their old addresses

  redirect.

- **A report's page:**

  - **built-in kinds** show their own views (the coverage tree and file

    view, the test cases and their history), as today;

  - **performance and custom** show `index.html` in a **sandboxed frame

    served from a separate origin**, with no network access and no access

    to the hub's cookies or session, so a page may run any scripts and

    charts it likes and still cannot act as the viewer;

  - **metrics** as a strip of numbers with sparklines across runs.

- **The timeline:** along the top, one stop per run on the bookmark,

  oldest to newest, each labelled with the change it measured.

  - **Scrub** to any run, or step with the arrow keys.

  - **Play** steps through the runs at a chosen speed, so a chart can be

    watched changing.

  - **Compare** two runs side by side, with every metric's delta; on a

    proposal the default is its base against its head.

- **A frame interface** for pages that animate rather than reload: inside

  the frame, `report.run()` gives the run on screen, and

  `report.history("dataset")` that dataset across the runs, so a page can

  redraw as the timeline moves or chart its own trends.

- **Downloads:** the whole bundle as a zip, and each dataset alone.

## 5. Checks

A metric with `fail_above` or `fail_below` sets a check named

`report/NAME/metric` on the change, failing when crossed, which a proposal

may require (Settings › Protection). Coverage's existing check becomes

this.

## 6. The API and `oh`

- `GET /repos/{o}/{n}/reports` (`?bookmark=`): the latest of each.

- `GET /repos/{o}/{n}/reports/{name}` (`?bookmark=&limit=&before=`): its

  history, metrics included.

- `GET /repos/{o}/{n}/reports/{name}/{id}`: one report, and

  `…/{id}/files/{path}` its files.

- `POST /repos/{o}/{n}/reports/{name}` (repo:write): an upload.

- `oh report list | view NAME | upload NAME DIR`.

## 7. For a project adopting it

1. Make the test or benchmark job write `out/report/` with a `report.toml`

   and, for a page, `index.html` with its libraries bundled (no CDN: the

   frame has no network) and its data in `data/`.

2. Declare `report "NAME" { path "/src/out/report" when "always" }` in the

   job's output.

3. Push. The report appears on the Reports tab after the run.

## 8. Order of work

1. Storage and index: the `report` output kind in OxygenFlow, the runner's

   bundle upload, the hub's tree, tables and gc roots; coverage and junit

   read through it.

2. The Reports tab and kinds; the sandboxed frame; metrics and checks.

3. The timeline: scrub, play, compare, and the frame interface.

4. `oh report` and the API; this document's §7 as the guide.