OXYGENHUB

Design: images only runners hold, moved by the swarm (refs #94) 3e3cc9475ab3

Author
Andy McCall @andymccall <[email protected]>
via
agent agent:claude@andymccall <[email protected]> · keyless, under your grant for changes,refs,seal · key 13df425e82a3
When
· 2026-10-27 01:58 UTC
Signature
◈ sealed by agent:claude@andymccall
Hash
3e3cc9475ab303e9a21584fa4cec5c59832fb8b0e041cceebbdcf810f48839ae
Depends on
2814f4213e7c b14aeaae683a c90dd8d5fc36
Depended on by
32f54b393396
In bookmarks
main
In tags
v0.5.0 v0.5.1 v0.5.2 v0.5.3
Edits
1 atom, 0 tree operations, 1 file

Browse the tree at this change

modified docs/jetstream.md +32 −0
…
294 unchanged lines
295 295
upload_mib_s = 50
296 296
```
297 297
298 +
#### 8.1.1 Images only runners hold (oxygenflow#94, oxygenhub#342)
299 +
300 +
Until now an image had to be sent to the hub (`flow rootfs push`) before any
301 +
runner but the one that imported it could take its jobs: the hub hands out
302 +
the manifest, and placement offers a job to a runner that fetches only when
303 +
the hub holds its image. On 2026-10-25 the Rust image was re-pinned and not
304 +
pushed, and for a day every Rust job queued on the one runner that had
305 +
imported it while the others sat idle.
306 +
307 +
- **Runners say what they hold.** A claim already lists the images a runner
308 +
  holds. When the hub keeps no manifest for one of them, the claim's answer
309 +
  asks for it (`send_manifests`), and the runner sends the manifest alone
310 +
  (`PUT /flow/images/{hash}/held`, a few MB even for a 14 GB image): no
311 +
  chunks.
312 +
- **The hub keeps it as peer-held.** It checks the manifest is well formed
313 +
  and names the hash it is filed under, and records the image as held by
314 +
  runners, not by itself: it cannot read the image to check it. A pushed
315 +
  image (`flow rootfs push`, verified whole) replaces a peer-held record.
316 +
- **Trust still comes from the hashes.** The pipeline pins the image's hash;
317 +
  a fetcher asks for the manifest by that hash, checks every chunk against
318 +
  its name, and checks the assembled image against the pin before it keeps
319 +
  it (`jet::images::assemble`). A wrong manifest can make a fetch fail; it
320 +
  cannot change what a job runs. Only runner tokens may send one.
321 +
- **Placement counts peers.** A job whose image the hub holds only as
322 +
  peer-held goes to a runner that fetches only when a runner of the same
323 +
  swarm site holds the image, and says so when none does ("held only by
324 +
  earth-1, in another site").
325 +
- **The hub is no seed for it.** The announce answer says whether the hub
326 +
  holds the chunks (`seed`); when it does not, the fetcher asks peers only,
327 +
  and a fetch with no peer left fails with that reason rather than after
328 +
  the hub refused chunk after chunk.
329 +
298 330
### 8.2 Runners update themselves
299 331
300 332
Proposed 2026-10-25 from Andy's experience: keeping a large fleet's
…
241 unchanged lines