OXYGENHUB
# 260c33b4b6c4 oxygenhub / docs / reports.md
7.4 KiB · Markdown · last changed docs/reports.md: a report's free-form [context] (a budget, a frame rate) 25 days ago renderedsource blame history raw

Reports

Design, 2026-10-11. Supersedes the separate Coverage and Tests tabs (spec §13.6, §13.7), which become two kinds of report.

A project produces more than pass and fail: coverage, test results, frame timings, sizes, benchmark curves. Today the hub reads two of them, LCOV coverage and JUnit tests, each with a tab of its own. This design makes reports one thing: a project declares what a report is, the hub keeps every run's copy, and readers move through its history.

1. What a report is

A report is a directory a Flow job writes, or anyone uploads:

report/
  report.toml        the declaration (required)
  index.html         a page to show (custom and performance kinds)
  data/*.csv|json    datasets the page draws from
  assets/…           images, scripts, styles: everything the page needs

report.toml says what it is:

kind    = "performance"        # coverage | tests | performance | custom
title   = "Frame timing"
summary = "Median frame 16.4 ms, p99 18.1 ms over 12 scenes"

[metrics]                      # the numbers the hub tracks across runs
median_frame_ms = { value = 16.4, unit = "ms", better = "lower" }
p99_frame_ms    = { value = 18.1, unit = "ms", better = "lower", fail_above = 20 }
scenes          = { value = 12 }

[context]                      # free-form, shown with the report
target_fps    = 60
budget_cycles = 133333
  • Built-in kinds name their data file and need no page: kind = "coverage" with file = "lcov.info" (LCOV or Cobertura), and kind = "tests" with file = "junit.xml". The hub reads them as it does today, and works out their metrics itself (line coverage; passed, failed, skipped).
  • performance is a page with metrics, as above.
  • custom is a page, metrics optional.
  • [context] is free-form: what a reader needs to read the numbers, shown with the report and kept per run, such as a performance budget (target_fps = 30, budget_cycles = 268800). The hub assumes no frame rate or budget; each report's metrics and thresholds carry their own.
  • A metric is a number with an optional unit, better (higher or lower, for colouring changes) and fail_above / fail_below, which set a check on the change (§5).

Implicit declarations: a job artifact named coverage or junit, the names used today, is read as a report of that kind with no report.toml, so every existing pipeline keeps working unchanged.

2. How a report arrives

From Flow, a job declares it in its outputs:

job "perf" {
    exec "sh" "-eu" ".flow/jobs/perf.sh"
    output {
        report "frame-timing" {
            path "/src/out/report"
            when "always"
        }
    }
}

The runner uploads the directory as one bundle. when "always" keeps a failing job's report, as for artifacts.

From anywhere else: oh report upload frame-timing ./report --bookmark develop attaches it to the bookmark's current state, or --change ID to a change's. The API is POST /repos/{o}/{n}/reports/{name} with the bundle as a tar.

3. How the hub keeps it

Three layers.

  1. The bundle, per job, content-addressed. Each file is a blob in the repository's own store, chunked and deduplicated as artifacts are, and the bundle is a small tree: a manifest of path → blob. A file that did not change since the last run (the page, its scripts, a dataset that did not move) is stored once however many runs name it; one that changed a little costs only its changed chunks. Keeping every run's report is cheap.

  2. An index in the database:

    • reports: one row per report per job or upload: repository, name, kind, title, summary, run and job (or uploader), bookmark, the measured state and heads, time, and the tree's blob.
    • report_metrics: one row per metric per report: key, value, unit, better, thresholds.

    History, trends, the scrubber and checks read only this. Coverage's and Tests' existing tables become their kinds' details, keyed by report.

  3. Retention, by policy:

    • metrics: forever (they are a few bytes a run);
    • bundles: every run on the default bookmark; the last 50 on any other bookmark; a proposal's until it lands or closes plus 30 days; a tagged release's forever. Set per repository in Settings › Reports.

    An expired bundle's row keeps its metrics and loses its tree; the blobs go at the next garbage collection.

Garbage collection keeps every tree and file blob a kept report names, as it keeps artifacts (oxygenhub#126): the database is the only thing that names them.

4. What readers see

  • A Reports tab (in place of Coverage and Tests), shown once a repository has a report: each report's latest on the chosen bookmark, with its kind, summary, metrics and their change against the run before. Coverage and Tests are entries in it; their old addresses redirect.
  • A report's page:
    • built-in kinds show their own views (the coverage tree and file view, the test cases and their history), as today;
    • performance and custom show index.html in a sandboxed frame served from a separate origin, with no network access and no access to the hub's cookies or session, so a page may run any scripts and charts it likes and still cannot act as the viewer;
    • metrics as a strip of numbers with sparklines across runs.
  • The timeline: along the top, one stop per run on the bookmark, oldest to newest, each labelled with the change it measured.
    • Scrub to any run, or step with the arrow keys.
    • Play steps through the runs at a chosen speed, so a chart can be watched changing.
    • Compare two runs side by side, with every metric's delta; on a proposal the default is its base against its head.
  • A frame interface for pages that animate rather than reload: inside the frame, report.run() gives the run on screen, and report.history("dataset") that dataset across the runs, so a page can redraw as the timeline moves or chart its own trends.
  • Downloads: the whole bundle as a zip, and each dataset alone.

5. Checks

A metric with fail_above or fail_below sets a check named report/NAME/metric on the change, failing when crossed, which a proposal may require (Settings › Protection). Coverage's existing check becomes this.

6. The API and oh

  • GET /repos/{o}/{n}/reports (?bookmark=): the latest of each.
  • GET /repos/{o}/{n}/reports/{name} (?bookmark=&limit=&before=): its history, metrics included.
  • GET /repos/{o}/{n}/reports/{name}/{id}: one report, and …/{id}/files/{path} its files.
  • POST /repos/{o}/{n}/reports/{name} (repo:write): an upload.
  • oh report list | view NAME | upload NAME DIR.

7. For a project adopting it

  1. Make the test or benchmark job write out/report/ with a report.toml and, for a page, index.html with its libraries bundled (no CDN: the frame has no network) and its data in data/.
  2. Declare report "NAME" { path "/src/out/report" when "always" } in the job's output.
  3. Push. The report appears on the Reports tab after the run.

8. Order of work

  1. Storage and index: the report output kind in OxygenFlow, the runner's bundle upload, the hub's tree, tables and gc roots; coverage and junit read through it.
  2. The Reports tab and kinds; the sandboxed frame; metrics and checks.
  3. The timeline: scrub, play, compare, and the frame interface.
  4. oh report and the API; this document's §7 as the guide.