Live pages: the design c0e39717895d
docs/live.md: topic events over server-sent events, the page fetching the HTML fragment that changed, no single-page app; at scale a durable log (NATS JetStream) and a stateless live gateway. Steps L1-L7 are #203-#209. Closes #202
- Author
- Andy McCall @andymccall <[email protected]>
- When
- · 2026-10-14 18:28 UTC
- Signature
- ✔ signed by Andy McCall
- Hash
c0e39717895d70a9b135eb6acc634cdb41f335a7c5c980e9e0a8711375217421- Depends on
- 93be9ce4653e
- Depended on by
- 33d98147511b a10446c68043 8a48dd830178 b5a0a69b3622 67be8ac00c24 7e01a7665aa5
- In bookmarks
- api-unknown-fields cancel-landed-runs diffusion-pin-bookmark draft-ready main oh-comment-body-file proposal-list-fast
- In tags
- v0.5.1 v0.5.2 v0.6.0 v0.6.1 v0.7.0 v0.8.0 v0.9.0 v0.9.1 v0.9.11 v0.9.15 v0.9.16 v0.9.19 v0.9.2 v0.9.20 v0.9.21 v0.9.22 v0.9.23 v0.9.24 v0.9.3 v0.9.4 v0.9.5 v0.9.6 v0.9.7 v0.9.8 v0.9.9
- Edits
- 1 atom, 1 tree operation, 1 file
Browse the tree at this change
added docs/live.md +105 −0
| 1 | + | # Live pages |
|
| 2 | + | ||
| 3 | + | Status: design, 2026-10-14 (oxygenhub#202). Nothing here is built yet |
|
| 4 | + | beyond what §2 lists. |
|
| 5 | + | ||
| 6 | + | ## 1. The problem |
|
| 7 | + | ||
| 8 | + | Pages are rendered once. When something happens behind them (an |
|
| 9 | + | assistant answers, a run finishes, a review arrives, a bump is opened) |
|
| 10 | + | the page does not know until it is reloaded. Proposal 38 on |
|
| 11 | + | doomsdayone/bomberman94 showed it: the assistant's answer, or the reason |
|
| 12 | + | there was none, only appeared on reload. |
|
| 13 | + | ||
| 14 | + | ## 2. What exists |
|
| 15 | + | ||
| 16 | + | - **Flow runs** stream their logs and job states over Server-Sent Events |
|
| 17 | + | (`/{owner}/{name}/flow/{id}/live`, read by `EventSource` in |
|
| 18 | + | `assets/hub.js`). |
|
| 19 | + | - **The conflict resolver** keeps in-memory rooms (`crates/hub/src/live.rs`): |
|
| 20 | + | a `tokio::sync::broadcast` channel per conflict, for presence and |
|
| 21 | + | choices. |
|
| 22 | + | - **Notifications and webhooks** already describe most writes as events |
|
| 23 | + | (`notify::deliver`, `hooks::emit`). |
|
| 24 | + | ||
| 25 | + | ## 3. Not a single-page app |
|
| 26 | + | ||
| 27 | + | Moving to Next.js or another single-page app was considered and set |
|
| 28 | + | aside: |
|
| 29 | + | ||
| 30 | + | - every page would be rendered twice (Rust templates and JavaScript), and |
|
| 31 | + | the two drift; |
|
| 32 | + | - public pages would still need server rendering; |
|
| 33 | + | - pages would stop working without JavaScript, and accessibility gets |
|
| 34 | + | harder; |
|
| 35 | + | - permissions would be checked in two places; |
|
| 36 | + | - it rewrites about a hundred templates to gain one thing: pages that |
|
| 37 | + | update themselves. |
|
| 38 | + | ||
| 39 | + | ## 4. The design: notify, then fetch |
|
| 40 | + | ||
| 41 | + | 1. **Events name a topic, not data.** Every write publishes a small |
|
| 42 | + | event: a topic, a version, and a kind. Topics: |
|
| 43 | + | - `repo:<id>` (bookmarks moved, settings); |
|
| 44 | + | - `repo:<id>/proposal:<n>` and `repo:<id>/issue:<n>` (comments, |
|
| 45 | + | reviews, assistant answers, labels, state, checks); |
|
| 46 | + | - `run:<id>` (jobs and logs, as today); |
|
| 47 | + | - `user:<id>` (inbox, dashboard); |
|
| 48 | + | - `diffusion:<repo id>` (bumps, pills). |
|
| 49 | + | 2. **One stream per page.** A page asks for its topics on one |
|
| 50 | + | connection: `GET /live?topics=...`, `text/event-stream`. The browser |
|
| 51 | + | reconnects by itself; `Last-Event-ID` resumes, and the server replays |
|
| 52 | + | what was missed from a short buffer, or says "reload" when it cannot. |
|
| 53 | + | 3. **The page fetches what changed.** An event says which part of the |
|
| 54 | + | page is stale. The page fetches that part as HTML from the normal |
|
| 55 | + | route (`?fragment=comments`, `?fragment=checks` and so on) and |
|
| 56 | + | swaps it in. So: |
|
| 57 | + | - the templates that render the page render the update; |
|
| 58 | + | - permission is checked once, by the route that renders the |
|
| 59 | + | fragment; an event reaching the wrong viewer reveals nothing but a |
|
| 60 | + | topic and a version; |
|
| 61 | + | - fragments can be cached and are tested like pages. |
|
| 62 | + | 4. **Subscribing is checked too.** `/live` only accepts topics the viewer |
|
| 63 | + | may read (a private repository's topics need read access), so even |
|
| 64 | + | the fact that something changed does not leak. |
|
| 65 | + | 5. **The page still works without it.** No JavaScript or no connection: |
|
| 66 | + | the page is what it is today, and a reload shows everything. Pending |
|
| 67 | + | work shows a placeholder ("Claude is reviewing…") that the stream |
|
| 68 | + | replaces. |
|
| 69 | + | 6. **A small client, ours.** About 2 KB in `hub.js`: elements carry |
|
| 70 | + | `data-live="<topic>"` and `data-fragment="<name>"`; an event for the |
|
| 71 | + | topic refetches the fragment and swaps it, keeping focus, scroll and |
|
| 72 | + | anything being typed. No framework. |
|
| 73 | + | ||
| 74 | + | ## 5. At scale, across services |
|
| 75 | + | ||
| 76 | + | When the hub is split into services: |
|
| 77 | + | ||
| 78 | + | - **Services publish events to a durable log.** NATS JetStream is the |
|
| 79 | + | choice: small, self-hosted, clustered, with replay, so a sovereign site |
|
| 80 | + | runs it on its own machines. Postgres `LISTEN/NOTIFY` is an acceptable |
|
| 81 | + | first step for a single database. |
|
| 82 | + | - **A stateless live gateway** holds the browsers' connections. It |
|
| 83 | + | checks a viewer's subscription (asking the owning service, cached |
|
| 84 | + | briefly), relays topic events, and replays from the log for |
|
| 85 | + | `Last-Event-ID`. It never renders and never reads repositories, so it |
|
| 86 | + | scales out on its own; losing an instance means browsers reconnect. |
|
| 87 | + | - **Fragments stay with the service that owns the data** (issues, |
|
| 88 | + | proposals, Flow, Diffusion). Moving a page to a new service does not |
|
| 89 | + | touch the live layer. |
|
| 90 | + | - **The resolver's rooms** become topics on the log, so people on |
|
| 91 | + | different instances see each other. |
|
| 92 | + | - **Later:** WebTransport where streams need to be bidirectional and |
|
| 93 | + | many; SSE stays the default. |
|
| 94 | + | ||
| 95 | + | ## 6. Order |
|
| 96 | + | ||
| 97 | + | | Step | What | Issue | |
|
| 98 | + | |---|---|---| |
|
| 99 | + | | L1 | Event bus inside the hub (broadcast per topic, versions, a short replay buffer); publish from the writes that notify and webhooks already see | #203 | |
|
| 100 | + | | L2 | `GET /live?topics=` with subscription checks, `Last-Event-ID` replay, and the `hub.js` client with `data-live` / `data-fragment` | #204 | |
|
| 101 | + | | L3 | The proposal page live: comments, reviews, assistant answers and pending placeholders, checks, landing state | #205 | |
|
| 102 | + | | L4 | Dashboard and inbox live, with the notification count in the top bar | #206 | |
|
| 103 | + | | L5 | Issue page, Diffusion tab and repository overview live | #207 | |
|
| 104 | + | | L6 | Fold the Flow run stream and the resolver's rooms into the same bus | #208 | |
|
| 105 | + | | L7 | Services: JetStream behind the bus and the live gateway as its own service | #209 | |