Design: images only runners hold, moved by the swarm (refs #94) 3e3cc9475ab3
- Author
- Andy McCall @andymccall <[email protected]>
- via
- agent
agent:claude@andymccall <[email protected]> · keyless, under your grant for changes,refs,seal · key 13df425e82a3
- When
- · 2026-10-27 01:58 UTC
- Signature
- ◈ sealed by agent:claude@andymccall
- Hash
3e3cc9475ab303e9a21584fa4cec5c59832fb8b0e041cceebbdcf810f48839ae- Depends on
- 2814f4213e7c b14aeaae683a c90dd8d5fc36
- Depended on by
- 32f54b393396
- In bookmarks
- main
- In tags
- v0.5.0 v0.5.1 v0.5.2 v0.5.3
- Edits
- 1 atom, 0 tree operations, 1 file
Browse the tree at this change
modified docs/jetstream.md +32 −0
| … | 294 unchanged lines |
||
| 295 | 295 | upload_mib_s = 50 |
|
| 296 | 296 | ``` |
|
| 297 | 297 | ||
| 298 | + | #### 8.1.1 Images only runners hold (oxygenflow#94, oxygenhub#342) |
|
| 299 | + | ||
| 300 | + | Until now an image had to be sent to the hub (`flow rootfs push`) before any |
|
| 301 | + | runner but the one that imported it could take its jobs: the hub hands out |
|
| 302 | + | the manifest, and placement offers a job to a runner that fetches only when |
|
| 303 | + | the hub holds its image. On 2026-10-25 the Rust image was re-pinned and not |
|
| 304 | + | pushed, and for a day every Rust job queued on the one runner that had |
|
| 305 | + | imported it while the others sat idle. |
|
| 306 | + | ||
| 307 | + | - **Runners say what they hold.** A claim already lists the images a runner |
|
| 308 | + | holds. When the hub keeps no manifest for one of them, the claim's answer |
|
| 309 | + | asks for it (`send_manifests`), and the runner sends the manifest alone |
|
| 310 | + | (`PUT /flow/images/{hash}/held`, a few MB even for a 14 GB image): no |
|
| 311 | + | chunks. |
|
| 312 | + | - **The hub keeps it as peer-held.** It checks the manifest is well formed |
|
| 313 | + | and names the hash it is filed under, and records the image as held by |
|
| 314 | + | runners, not by itself: it cannot read the image to check it. A pushed |
|
| 315 | + | image (`flow rootfs push`, verified whole) replaces a peer-held record. |
|
| 316 | + | - **Trust still comes from the hashes.** The pipeline pins the image's hash; |
|
| 317 | + | a fetcher asks for the manifest by that hash, checks every chunk against |
|
| 318 | + | its name, and checks the assembled image against the pin before it keeps |
|
| 319 | + | it (`jet::images::assemble`). A wrong manifest can make a fetch fail; it |
|
| 320 | + | cannot change what a job runs. Only runner tokens may send one. |
|
| 321 | + | - **Placement counts peers.** A job whose image the hub holds only as |
|
| 322 | + | peer-held goes to a runner that fetches only when a runner of the same |
|
| 323 | + | swarm site holds the image, and says so when none does ("held only by |
|
| 324 | + | earth-1, in another site"). |
|
| 325 | + | - **The hub is no seed for it.** The announce answer says whether the hub |
|
| 326 | + | holds the chunks (`seed`); when it does not, the fetcher asks peers only, |
|
| 327 | + | and a fetch with no peer left fails with that reason rather than after |
|
| 328 | + | the hub refused chunk after chunk. |
|
| 329 | + | ||
| 298 | 330 | ### 8.2 Runners update themselves |
|
| 299 | 331 | ||
| 300 | 332 | Proposed 2026-10-25 from Andy's experience: keeping a large fleet's |
|
| … | 241 unchanged lines |