OXYGENHUB
# e8fa53694f68 oxygenhub / docs / reports.md
10.0 KiB · Markdown · last changed Reports retention: files beyond the newest 50 runs off the default bookmark expire 24 days ago renderedsource blame history raw

Reports

Design, 2026-10-11. Supersedes the separate Coverage and Tests tabs (spec §13.6, §13.7), which become two kinds of report.

A project produces more than pass and fail: coverage, test results, frame timings, sizes, benchmark curves. Today the hub reads two of them, LCOV coverage and JUnit tests, each with a tab of its own. This design makes reports one thing: a project declares what a report is, the hub keeps every run's copy, and readers move through its history.

1. What a report is

A report is a directory a Flow job writes, or anyone uploads:

report/
  report.toml        the declaration (required)
  index.html         a page to show (custom and performance kinds)
  data/*.csv|json    datasets the page draws from
  assets/…           images, scripts, styles: everything the page needs

report.toml says what it is:

kind    = "performance"        # coverage | tests | performance | custom
title   = "Frame timing"
summary = "Median frame 16.4 ms, p99 18.1 ms over 12 scenes"

[metrics]                      # the numbers the hub tracks across runs
median_frame_ms = { value = 16.4, unit = "ms", better = "lower" }
p99_frame_ms    = { value = 18.1, unit = "ms", better = "lower", fail_above = 20 }
scenes          = { value = 12 }
  • Built-in kinds name their data file and need no page: kind = "coverage" with file = "lcov.info" (LCOV or Cobertura), and kind = "tests" with file = "junit.xml". The hub reads them as it does today, and works out their metrics itself (line coverage; passed, failed, skipped).
  • performance is a page with metrics, as above.
  • custom is a page, metrics optional.
  • A metric is a number with an optional unit, better (higher or lower, for colouring changes) and fail_above / fail_below, which set a check on the change (§5).

Implicit declarations: a job artifact named coverage or junit, the names used today, is read as a report of that kind with no report.toml, so every existing pipeline keeps working unchanged.

2. How a report arrives

From Flow, a job declares it in its outputs:

job "perf" {
    exec "sh" "-eu" ".flow/jobs/perf.sh"
    output {
        report "frame-timing" {
            path "/src/out/report"
            when "always"
        }
    }
}

The runner uploads the directory as one bundle. when "always" keeps a failing job's report, as for artifacts.

The bundle travels file by file: each to POST /api/v1/flow/jobs/{job}/reports/{name}/files with its path in X-Flow-Report-Path, answered with its blob; then the manifest, {"version":1,"files":[{"path","blob","size"}]}, as the job's artifact under the report's name, which the receipt names. The hub refuses a manifest naming a file it does not hold, reads report.toml and indexes the report; a report.toml it cannot read is kept as a custom report whose summary says why, so the job's log and the tab show what to fix. Paths are /-separated names of letters, digits, ., _, - and +; a report holds at most 2000 files and 512 MiB.

From anywhere else: oh report upload frame-timing ./report --bookmark develop attaches it to the bookmark's current state, or --change ID to a change's. The API is POST /repos/{o}/{n}/reports/{name} with the bundle as a tar.

3. How the hub keeps it

Three layers.

  1. The bundle, per job, content-addressed. Each file is a blob in the repository's own store, chunked and deduplicated as artifacts are, and the bundle is a small tree: a manifest of path → blob. A file that did not change since the last run (the page, its scripts, a dataset that did not move) is stored once however many runs name it; one that changed a little costs only its changed chunks. Keeping every run's report is cheap.

  2. An index in the database:

    • reports: one row per report per job or upload: repository, name, kind, title, summary, run and job (or uploader), bookmark, the measured state and heads, time, and the tree's blob.
    • report_metrics: one row per metric per report: key, value, unit, better, thresholds.
    • report_files: one row per file per report: path, blob, size; what garbage collection keeps and the frame serves.

    History, trends, the scrubber and checks read only this. Coverage's and Tests' existing tables become their kinds' details, keyed by report.

  3. Retention, by policy:

    • metrics: forever (they are a few bytes a run);
    • bundles: every run on the default bookmark; the last 50 on any other bookmark; a proposal's until it lands or closes plus 30 days; a tagged release's forever. Set per repository in Settings › Reports.

    An expired bundle's row keeps its metrics and loses its tree; the blobs go at the next garbage collection.

Garbage collection keeps every tree and file blob a kept report names, as it keeps artifacts (oxygenhub#126): the database is the only thing that names them.

4. What readers see

  • A Reports tab (in place of Coverage and Tests), shown once a repository has a report: each report's latest on the chosen bookmark, with its kind, summary, metrics and their change against the run before. Coverage and Tests are entries in it; their old addresses redirect.
  • A report's page:
    • built-in kinds show their own views (the coverage tree and file view, the test cases and their history), as today;
    • performance and custom show index.html in a sandboxed frame served from a separate origin, with no network access and no access to the hub's cookies or session, so a page may run any scripts and charts it likes and still cannot act as the viewer;
    • metrics as a strip of numbers with sparklines across runs.
  • The timeline: along the top, one stop per run on the bookmark, oldest to newest, each labelled with the change it measured.
    • Scrub to any run, or step with the arrow keys.
    • Play steps through the runs at a chosen speed, so a chart can be watched changing.
    • Compare two runs side by side, with every metric's delta; on a proposal the default is its base against its head.
  • A frame interface for pages that animate rather than reload: inside the frame, report.run() gives the run on screen, and report.history("dataset") that dataset across the runs, so a page can redraw as the timeline moves or chart its own trends.
  • Downloads: the whole bundle as a zip, and each dataset alone.

5. Checks

A metric with fail_above or fail_below sets a check named report/NAME/metric on the change, failing when crossed, which a proposal may require (Settings › Protection). Coverage's existing check becomes this.

6. The API and oh

  • GET /repos/{o}/{n}/reports (?bookmark=): the latest of each.
  • GET /repos/{o}/{n}/reports/{name} (?bookmark=&limit=&before=): its history, metrics included.
  • GET /repos/{o}/{n}/reports/{name}/{id}: one report, and …/{id}/files/{path} its files.
  • POST /repos/{o}/{n}/reports/{name} (repo:write): an upload.
  • oh report list | view NAME | upload NAME DIR.

7. For a project adopting it

  1. Make the test or benchmark job write out/report/ with a report.toml and, for a page, index.html with its libraries bundled (no CDN: the frame has no network) and its data in data/.
  2. Declare report "NAME" { path "/src/out/report" when "always" } in the job's output.
  3. Push. The report appears on the Reports tab after the run.

Fitting the hub

  • Height: the hub adds a small script to a report's HTML pages (/static/report-frame.js) that tells the page around the frame how tall the report is, so the frame grows to fit and the report never scrolls inside it. Lay the page out by its content: nothing sized to the viewport (100vh, height: 100% on html/body), or it grows with the frame.

  • Look: link the hub's stylesheet, and drop any fonts or colours of the report's own:

    <link rel="stylesheet" href="/static/report.css">

    It sets the hub's fonts and colour tokens (--bg, --panel, --soft, --ink, --mute, --line, --acc, --ok, --bad, --warn, --m1..--m5 for series), dark or light as the hub is, switching when the viewer switches. Charts read them with getComputedStyle(document.documentElement).getPropertyValue("--m1") and redraw on a data-theme change if they cache colours.

8. Order of work

Status, 2026-10-11: storage and index (1) are deployed; the Reports tab, each report's page with its metrics, changes and sparklines, its own page in a sandboxed frame, and checks from thresholds (2) are built, Coverage and Tests listed on the tab while their own pages stay. The frame is sandboxed (allow-scripts only) rather than on a second origin: it carries no session, so its files are served under a per-report MAC in the link, with Content-Security-Policy: sandbox allow-scripts and nothing beyond this hub to load. The timeline (3) is built too: one stop per run on the bookmark, arrow keys to step, Play at 1 to 10 s a run (carried in the URL as play), Compare against any other run (compare) with every metric's difference and the two pages side by side, and the frame interface: inside the frame, report.run() and report.history("file") answer from the hub page by postMessage, history reading that file of data/ from each run's capability URL. Retention (§3.3) runs before each garbage collection: every run on the default bookmark keeps its files, and the newest 50 runs of each report on any other bookmark (proposal and tag rules, and a per-repository setting, follow); an expired run keeps its metrics and summary, and its page says its files expired. oh report (4) follows.

  1. Storage and index: the report output kind in OxygenFlow, the runner's bundle upload, the hub's tree, tables and gc roots; coverage and junit read through it.
  2. The Reports tab and kinds; the sandboxed frame; metrics and checks.
  3. The timeline: scrub, play, compare, and the frame interface.
  4. oh report and the API; this document's §7 as the guide.