Cylinder: Draft 0.3 of the proposal 82739cfbafe2
Corrects 0.2 after review: a Firecracker microVM gets an artifact as a block device served from the chunk tree (it has no virtio-fs); the file index finds exact files, versions come from an SBOM package index; revocation refuses new pods at once and replaces running ones within a grace period; encryption is the store's, with customer keys, so chunks still deduplicate and verify; leases are scoped and bounded; upstream packages are unpacked before chunking. C0 is recorded as built (oxygenhub #146), and the open questions are open again, each with the position proposed in review. refs #108
- Author
- Andy McCall @andymccall <[email protected]>
- When
- · 2026-10-12 19:32 UTC
- Signature
- ✔ signed by Andy McCall
- Hash
82739cfbafe29cdcabc52aee6ffe481a2a6ae7c7565707e9636452c0a21c11ce- Depends on
- d0016e229b04
- Depended on by
- nothing yet
- In bookmarks
- carried-not-removed doctor-trust identical-sides keyless-tags main merge-remote-bookmark only-help-103 paths-for-resolutions-and-amend push-notes show-stat subrepo-pin-state
- In tags
- v0.10.0 v0.11.0 v0.4.4 v0.4.5 v0.4.6 v0.5.0 v0.5.1 v0.6.0 v0.7.0 v0.8.0 v0.9.0
- Edits
- 15 atoms, 0 tree operations, 1 file
Browse the tree at this change
modified docs/cylinder.md +81 −50
| 1 | 1 | # Cylinder: artifacts, the Oxygen way |
|
| 2 | 2 | ||
| 3 | - | Status: Draft 0.2 (proposal revision 2), 2026-10-12. Incorporates review of |
|
| 3 | + | Status: Draft 0.3 (proposal revision 3), 2026-10-12. Draft 0.2 added the |
|
| 4 | 4 | object model, deployment leases, inverted vulnerability indexing, |
|
| 5 | - | multi-party consensus, agent scopes, and revset queries. Not built. |
|
| 6 | - | "Cylinder" is a working name (the artifact-repository idea parked on |
|
| 7 | - | 2026-10-02); nothing here is decided until Andy says so. |
|
| 5 | + | multi-party consensus, agent scopes and revset queries; 0.3 corrects the |
|
| 6 | + | MicroVM mount path, the reach of the file index, revocation, encryption, |
|
| 7 | + | leases and upstream quarantine, and keeps §7 open. C0 is built; the rest |
|
| 8 | + | is not. "Cylinder" is a working name (the artifact-repository idea parked |
|
| 9 | + | on 2026-10-02); nothing here is decided until Andy says so. |
|
| 8 | 10 | ||
| 9 | 11 | **Cylinder** keeps and moves everything a build produces: runner images, |
|
| 10 | 12 | container images, packages, binaries, SBOMs and reports. It would be a |
|
| … | 100 unchanged lines |
||
| 111 | 113 | after fetching only the files it reads. |
|
| 112 | 114 | - Every artifact has exactly one name for its content: the BLAKE3 of its |
|
| 113 | 115 | canonical CBOR encoding. |
|
| 114 | - | - **MicroVMs and virtio-fs:** Jetstream microVM runners mount an artifact's |
|
| 115 | - | tree directly via virtio-fs over `flow-casfs`, eliminating ext4 block |
|
| 116 | - | images, loop devices, and cold disk allocation. Where a raw block device |
|
| 117 | - | is required by legacy hardware or external hypervisors, an ext4 block |
|
| 118 | - | map is synthesized on the fly from the chunk tree without duplicating |
|
| 119 | - | bytes. |
|
| 116 | + | - **MicroVMs:** Flow's MicroVM tier runs on Firecracker, which has no |
|
| 117 | + | virtio-fs, so a microVM gets the artifact as a block device: an ext4 |
|
| 118 | + | image served on the fly from the chunk tree (a userspace block device, |
|
| 119 | + | ublk, or an NBD server), fetching blocks as the guest reads them and |
|
| 120 | + | duplicating no bytes. That removes copying whole images onto each |
|
| 121 | + | runner's disk without changing hypervisor. The syscall tier and |
|
| 122 | + | containers mount the tree directly over `flow-casfs`; mounting it with |
|
| 123 | + | virtio-fs would need Cloud Hypervisor or QEMU instead of Firecracker. |
|
| 120 | 124 | ||
| 121 | 125 | ### 4.2 Names are signed histories, not pointers |
|
| 122 | 126 | ||
| … | 51 unchanged lines |
||
| 174 | 178 | - **Upstream lineage:** an artifact built from other artifacts (a base |
|
| 175 | 179 | image, a package) names their hashes in its receipt inputs, so the chain |
|
| 176 | 180 | reaches back transitively to their sources. |
|
| 181 | + | - **An SBOM with every artifact:** the job that publishes an artifact also |
|
| 182 | + | records what it contains by package and version (an SBOM, itself an |
|
| 183 | + | artifact named in the receipt's outputs), so "which artifacts hold a |
|
| 184 | + | vulnerable version of this package" is a lookup, not a scan (§4.6). |
|
| 177 | 185 | ||
| 178 | 186 | ### 4.4 Distribution is built in |
|
| 179 | 187 | ||
| … | 13 unchanged lines |
||
| 193 | 201 | ### 4.5 Sovereign storage, and our own transparency |
|
| 194 | 202 | ||
| 195 | 203 | - Bytes live on any Strata store: local disk, S3, GCS or Azure, in the |
|
| 196 | - | customer's own account and region, encrypted with customer-managed KMS. |
|
| 204 | + | customer's own account and region, encrypted at rest by the store with |
|
| 205 | + | customer-managed keys (for example S3 with a KMS key). Chunks stay named |
|
| 206 | + | by the hash of their plain bytes: encrypting them on the client with |
|
| 207 | + | separate keys would stop them deduplicating and stop peers checking them |
|
| 208 | + | by hash. |
|
| 197 | 209 | - Channel moves and signatures are recorded in the hub's append-only |
|
| 198 | 210 | Merkle log. Auditing requires no dependence on public ledgers like |
|
| 199 | 211 | Rekor, though Cylinder can bridge to them when desired. |
|
| … | 8 unchanged lines |
||
| 208 | 220 | and assurance levels. |
|
| 209 | 221 | - `oxy` and `flow` verify what they fetch. |
|
| 210 | 222 | ||
| 211 | - | **The global inverted vulnerability index:** Because artifacts are trees |
|
| 212 | - | of content-addressed files and chunks rather than opaque tarballs, |
|
| 213 | - | Cylinder maintains an inverted index from chunk/file hash to artifacts: |
|
| 223 | + | **The inverted file index:** Because artifacts are trees of |
|
| 224 | + | content-addressed files rather than opaque tarballs, Cylinder keeps an |
|
| 225 | + | index from each file's hash to the artifacts that contain it: |
|
| 214 | 226 | ||
| 215 | 227 | $$\text{FileHash} \longrightarrow [\text{ArtifactHash}]$$ |
|
| 216 | 228 | ||
| 217 | - | When a security advisory or zero-day is disclosed for a binary or |
|
| 218 | - | library (e.g., a vulnerable `libcrypto.so.3` or Go binary hash): |
|
| 219 | - | - Cylinder performs an immediate $O(1)$ lookup to identify every |
|
| 220 | - | container image, runner image, and package across the fleet containing |
|
| 221 | - | that file. |
|
| 222 | - | - There is no need to unpack gigabytes of tarball layers across thousands |
|
| 223 | - | of images to discover exposure. |
|
| 224 | - | - **Continuous revocation:** If an upstream library or base image is |
|
| 225 | - | revoked in policy, point-of-use admission hooks immediately block |
|
| 226 | - | downstream pods from running any artifact containing that chunk, |
|
| 227 | - | eliminating the gap between vulnerability discovery and redeployment. |
|
| 229 | + | - **Exact files, at once.** Given a known-bad file (a compromised binary, |
|
| 230 | + | a leaked key, the exact `libcrypto.so.3` an advisory names), one lookup |
|
| 231 | + | lists every container image, runner image and package across the fleet |
|
| 232 | + | holding it, with no layers to unpack. |
|
| 233 | + | - **Not versions.** A patched or rebuilt library is a different file with |
|
| 234 | + | a different hash, so this index does not answer "which images have a |
|
| 235 | + | vulnerable version of OpenSSL". That comes from package identity: the |
|
| 236 | + | SBOM each artifact carries (§4.3), indexed by package and version. The |
|
| 237 | + | two together answer both questions; the file index alone answers the |
|
| 238 | + | first. |
|
| 239 | + | - **Revocation without an outage.** When an artifact, base image or file |
|
| 240 | + | is revoked in policy, admission hooks refuse new pods that use it at |
|
| 241 | + | once. Pods already running are not touched by admission; a controller |
|
| 242 | + | marks them, then replaces or evicts them within a grace period the |
|
| 243 | + | policy sets, so one policy change cannot take production down. |
|
| 228 | 244 | ||
| 229 | 245 | ### 4.7 Retention by reachability and deployment leases |
|
| 230 | 246 | ||
| … | 15 unchanged lines |
||
| 246 | 262 | ||
| 247 | 263 | Active leases act as garbage collection roots. If a workload terminates |
|
| 248 | 264 | or a cluster is decommissioned, the lease expires naturally, enabling |
|
| 249 | - | safe automated retention without operational risk. |
|
| 265 | + | safe automated retention without operational risk. A lease is signed by |
|
| 266 | + | a key scoped to leases and to its `scope`, and has a maximum length (a |
|
| 267 | + | day, renewed while the workload runs), so a compromised node cannot hold |
|
| 268 | + | storage forever or pin artifacts outside its own scope. |
|
| 250 | 269 | ||
| 251 | 270 | ### 4.8 Compatibility at the edge |
|
| 252 | 271 | ||
| … | 9 unchanged lines |
||
| 262 | 281 | indexes, npm packuments, PyPI simple indexes) dynamically from artifact |
|
| 263 | 282 | manifests. |
|
| 264 | 283 | - **Upstream dependency quarantine:** When Flow builds fetch third-party |
|
| 265 | - | packages from crates.io or npm, Cylinder proxies and chunks them into |
|
| 266 | - | the organization's Strata store, locking the hashes into internal |
|
| 267 | - | mirror channels. Upstream outages or deletions cannot break builds. |
|
| 284 | + | packages from crates.io or npm, Cylinder proxies them into the |
|
| 285 | + | organization's Strata store, locking the hashes into internal mirror |
|
| 286 | + | channels. Upstream outages or deletions cannot break builds. Packages |
|
| 287 | + | arrive compressed (`.crate` and `.tgz` are gzip), which chunks poorly, |
|
| 288 | + | so the original archive is kept for clients that check its hash and its |
|
| 289 | + | unpacked files are chunked as an artifact tree, where they deduplicate |
|
| 290 | + | with everything else. |
|
| 268 | 291 | ||
| 269 | 292 | ### 4.9 AI-native agent integration |
|
| 270 | 293 | ||
| … | 87 unchanged lines |
||
| 358 | 381 | | Consensus | Single-builder attestation | Multi-party consensus receipts (`reproduced(N)`) | |
|
| 359 | 382 | | Lazy start | Bolted-on snapshotter (SOCI, Nydus) | Native: file trees mounted via `flow-casfs` | |
|
| 360 | 383 | | Peer to peer | Separate mesh daemon (Dragonfly) | Built-in Jetstream swarm, TLS-pinned | |
|
| 361 | - | | Vulnerability search | Unpack and scan every layer ($O(N)$) | $O(1)$ inverted index by file and chunk hash | |
|
| 384 | + | | Vulnerability search | Unpack and scan every layer | Exact files by one index lookup; versions by SBOM package index | |
|
| 362 | 385 | | Retention | Tag rules + manual GC | Reachability + active deployment leases | |
|
| 363 | 386 | | AI workflows | Uncontrolled service tokens | Scoped delegation (`artifacts`, `channels`) | |
|
| 364 | 387 | | Querying | Disconnected CI/CD dashboards | Unified revsets (`main ~ artifact(app:stable)`) | |
|
| … | 1 unchanged lines |
||
| 366 | 389 | ||
| 367 | 390 | ## 6. How we could build it |
|
| 368 | 391 | ||
| 369 | - | - **C0. Images in the hub, now.** A Flow › Images page: each image, where |
|
| 370 | - | it is (hub, runners, swarm activity) and what pins it. |
|
| 392 | + | - **C0. Images in the hub (built 2026-10-12, oxygenhub #146).** Admin › |
|
| 393 | + | Runners › Images: every runner image, its size and architecture, where |
|
| 394 | + | it is (the hub, each runner, swarm fetches in progress), the |
|
| 395 | + | repositories whose jobs used it, and an image held nowhere marked. |
|
| 371 | 396 | - **C1. The service boundary and artifact model.** Implement the |
|
| 372 | 397 | `Artifact` CBOR object type and FastCDC chunk engine in `cylinder`; |
|
| 373 | 398 | unify runner images with artifacts. |
|
| … | 2 unchanged lines |
||
| 376 | 401 | revset predicates. |
|
| 377 | 402 | - **C3. The OCI façade and containerd snapshotter.** OCI pull/push |
|
| 378 | 403 | translation; `containerd` snapshotter over `flow-casfs`. |
|
| 379 | - | - **C4. Leases and the inverted index.** Kubernetes deployment lease |
|
| 380 | - | operator for safe GC; global inverted chunk index for instantaneous |
|
| 381 | - | CVE discovery. |
|
| 404 | + | - **C4. Leases and the indexes.** Kubernetes deployment lease operator |
|
| 405 | + | for safe GC; the inverted file index and the SBOM package index; the |
|
| 406 | + | controller that replaces revoked running workloads within their grace |
|
| 407 | + | period. |
|
| 382 | 408 | - **C5. Reach and package façades.** Package façades for Cargo, npm, |
|
| 383 | 409 | and PyPI; upstream dependency quarantine mirroring. |
|
| 384 | 410 | ||
| 385 | 411 | ## 7. Open questions |
|
| 386 | 412 | ||
| 387 | - | - **The name:** Keep **Cylinder**. It fits the elemental progression: |
|
| 388 | - | *Oxygen* (VCS) is transported in *Cylinders* (artifacts) via |
|
| 389 | - | *Jetstream* (fleet transport) driven by *Flow* (pipelines). |
|
| 390 | - | - **Where it lives:** Initially in `oxygenflow` / `crates/oxygen-cylinder` |
|
| 391 | - | alongside `jet` and `flow` to share `flow-casfs` and chunking primitives, |
|
| 392 | - | splitting into an independent repository at C3. |
|
| 393 | - | - **Local builds:** Allowed, but marked with lower assurance level |
|
| 394 | - | (`local`). Production channel rules require higher assurance levels |
|
| 395 | - | (`tier-microvm` + `reproduced(2)`). |
|
| 396 | - | - **A public catalogue / Federation:** Public hubs federate by publishing |
|
| 397 | - | signed Strata bundles of channels and artifacts, verified offline |
|
| 398 | - | without centralized intermediaries. |
|
| 399 | - | - **Place in the queue:** C0/C1 should proceed alongside Jetstream's |
|
| 400 | - | service boundary work, ensuring Jetstream's image store is built |
|
| 401 | - | directly on Cylinder rather than creating a temporary mechanism. |
|
| 413 | + | These are Andy's decisions. Each carries the position proposed in review, |
|
| 414 | + | not an answer. |
|
| 402 | 415 | ||
| 416 | + | - **The name.** Proposed: keep **Cylinder**. It fits the elemental |
|
| 417 | + | progression: *Oxygen* (VCS) is carried in *Cylinders* (artifacts) by |
|
| 418 | + | *Jetstream* (the fleet), driven by *Flow* (pipelines). |
|
| 419 | + | - **Where it lives.** Proposed: first in `oxygenflow` as |
|
| 420 | + | `crates/oxygen-cylinder`, beside `jet` and `flow`, to share `flow-casfs` |
|
| 421 | + | and the chunking code; its own repository from C3. |
|
| 422 | + | - **Local builds.** Proposed: allowed, at the lowest assurance level |
|
| 423 | + | (`local`); production channels require more (`tier-microvm` and |
|
| 424 | + | `reproduced(2)`). |
|
| 425 | + | - **A public catalogue and federation.** Proposed: public hubs federate |
|
| 426 | + | by publishing signed Strata bundles of channels and artifacts, checked |
|
| 427 | + | offline with no central intermediary. |
|
| 428 | + | - **Its place in the queue.** Proposed: C1 alongside Jetstream's service |
|
| 429 | + | boundary, so Jetstream's image store is built on Cylinder from the |
|
| 430 | + | start rather than on a temporary mechanism. Against: the core work of |
|
| 431 | + | 2026-10-03 (Strata's remaining stores, agents, hub polish) waits |
|
| 432 | + | longer. |
|
| 433 | + | ||
| 403 | 434 | ## Sources |
|
| 404 | 435 | ||
| 405 | 436 | - Docker Hub pull limits: [docs.docker.com, pull usage and limits](https://docs.docker.com/docker-hub/usage/pulls/); [heise, "only 10 pulls per hour"](https://heise.de/-10329047) |
|
| … | 9 unchanged lines |