Jetstream J1: images by hash (push to the hub, runners fetch what they lack) issue #39 closed: completed andymccall opened this 24 days ago · 2 comments

First Jetstream deliverable (refs issue #38, docs/jetstream.md §8). On 2026-10-11 images were copied between runners by hand twice, and a job waited hours for an image whose pin came from a different, non-reproducible build.

• flow rootfs push HASH uploads an image from a Flow home to OxygenHub in FastCDC chunks (Oxygen's chunker): the hub answers which chunks it lacks, only those are sent, then the manifest (hash, size, arch, chunks). A rebuilt image moves only its changed chunks; zero runs collapse to one chunk.
• The hub keeps images site-wide (<data>/images), checks each assembled image against its BLAKE3 before accepting it, and records its architecture.
• Placement offers a job to a runner that lacks its image when the hub holds it; the runner fetches the manifest and missing chunks, writes the image sparse, checks the hash, and runs the job (its log says so).
• flow rootfs pull HASH does the same for a local Flow home, so flow run gets pinned images without a hand import.
• The format and chunking live in a new jet crate, the start of Jetstream.