docs/workspaces.md: the runner side of workspaces, step by step 448568561f2a
Andy's decisions (2026-10-17): runner side first, files lazy from the start (casfs over vsock into the VM, an overlay for edits; the guest kernel needs FUSE first), editor traffic over one WebSocket per workspace with yamux. Five steps, each landing working. Refs andymccall/oxygenhub#69
- Author
- Andy McCall @andymccall <[email protected]>
- via
- agent
Claude Code <[email protected]> · delegate for changes, until 2026-11-13 19:00Z · key ea5f78160088
- When
- · 2026-10-17 13:54 UTC
- Signature
- ✔ signed by Claude Code
- Hash
448568561f2a236cf2bb2fa553a069b1078e240e31344b1081245d28d4151056- Depends on
- e0918b887850
- Depended on by
- 0662754815cf
- In bookmarks
- main
- In tags
- v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.5.0 v0.5.1 v0.5.2 v0.5.3
- Edits
- 1 atom, 1 tree operation, 1 file
Browse the tree at this change
added docs/workspaces.md +69 −0
| 1 | + | # Workspaces: VS Code in a microVM, the runner side |
|
| 2 | + | ||
| 3 | + | oxygenhub#69, hub spec §13.3. Written 2026-10-17 with Andy's decisions: |
|
| 4 | + | build it, **runner side first**; files **lazy from the start**; editor |
|
| 5 | + | traffic over a **WebSocket with multiplexing**. This is the plan for the |
|
| 6 | + | runner; the hub's part (the workspaces table, the button, the proxy, |
|
| 7 | + | commits) follows it. |
|
| 8 | + | ||
| 9 | + | ## What is there today |
|
| 10 | + | ||
| 11 | + | - Jobs run to their exit under a hard time limit, with a job token that |
|
| 12 | + | lasts at most two hours (hub `JOB_TOKEN_MAX_AGE_MS`). |
|
| 13 | + | - A microVM has no network device. Its only traffic is guest-initiated: |
|
| 14 | + | the egress proxy over vsock (`flow-init/src/egress.rs`). |
|
| 15 | + | - The runner speaks to the hub in short HTTPS calls; nothing long-lived. |
|
| 16 | + | - flow-casfs mounts a state lazily with FUSE, on the host. Firecracker has |
|
| 17 | + | no virtio-fs, so a FUSE mount on the host cannot be handed to a guest. |
|
| 18 | + | - The pinned guest kernel (6.1.155) has overlayfs and vsock, but no FUSE. |
|
| 19 | + | ||
| 20 | + | ## The shape |
|
| 21 | + | ||
| 22 | + | ``` |
|
| 23 | + | browser ──HTTPS/WS──► hub /{owner}/{name}/workspace/{id}/ ──┐ |
|
| 24 | + | │ one WebSocket per workspace, |
|
| 25 | + | runner ◄────────────────────────────────────────────────────┘ opened by the runner (yamux streams) |
|
| 26 | + | │ vsock (host connects in) |
|
| 27 | + | ▼ |
|
| 28 | + | microVM: flow-init ─► openvscode-server on 127.0.0.1:3000 |
|
| 29 | + | /workspace = overlay(lower: casfs over vsock, upper: scratch disk) |
|
| 30 | + | ``` |
|
| 31 | + | ||
| 32 | + | ## Steps |
|
| 33 | + | ||
| 34 | + | Each is an issue of its own, and each lands working and tested. |
|
| 35 | + | ||
| 36 | + | 1. **A guest kernel with FUSE.** The same 6.1 kernel and Firecracker's |
|
| 37 | + | guest config, with `CONFIG_FUSE_FS=y`; pinned by BLAKE3 in `flow.toml` |
|
| 38 | + | on earth-1, plex-1 and mars-1. Pipeline jobs are unaffected. |
|
| 39 | + | 2. **casfs over vsock.** flow-casfs gains a server half: on the host it |
|
| 40 | + | answers lookups, listings and reads for one state over a vsock port of |
|
| 41 | + | the workspace's VM, from the same graph-key cache it uses today. The |
|
| 42 | + | guest half is a small FUSE client in the workspace rootfs. `/workspace` |
|
| 43 | + | is an overlay: the state read-only below, the VM's scratch disk above, |
|
| 44 | + | so edits are plain files. Opening a workspace costs a listing, not an |
|
| 45 | + | export; the target is under a second on the Linux tree. |
|
| 46 | + | 3. **The workspace rootfs.** openvscode-server (pinned release, its |
|
| 47 | + | checksum checked), `oxy`, the Oxygen VCS extension, a non-root user; |
|
| 48 | + | built and imported like the rust image (`images/workspace`). |
|
| 49 | + | 4. **The `workspace` job kind.** `Claim` says `kind: "workspace"`. The |
|
| 50 | + | runner keeps the VM until it is told to stop, or the hub's idle limit |
|
| 51 | + | passes, instead of waiting for an exit; it reports liveness and open |
|
| 52 | + | connections every 30 s, and the hub renews the job token while the |
|
| 53 | + | workspace lives. A runner advertises whether it takes workspaces. |
|
| 54 | + | 5. **The tunnel.** flow-init accepts host connections on a vsock port and |
|
| 55 | + | forwards each to openvscode-server. The runner opens one WebSocket to |
|
| 56 | + | the hub per workspace (`/api/v1/flow/workspaces/{id}/tunnel`, its |
|
| 57 | + | runner token) and carries each browser connection as a yamux stream to |
|
| 58 | + | that port. Works for runners behind NAT and through Cloudflare. |
|
| 59 | + | ||
| 60 | + | Then the hub (hub spec §13.3): the `workspaces` table and limits, "Open in |
|
| 61 | + | workspace" behind a site setting (off), the owner-only reverse proxy for |
|
| 62 | + | HTTP and WebSocket into the tunnel, and commits pushed to `ws/{account}/{id}` |
|
| 63 | + | and signed by the hub as the account's delegate. |
|
| 64 | + | ||
| 65 | + | ## Downloads this needs |
|
| 66 | + | ||
| 67 | + | Linux 6.1 sources (kernel.org) and Firecracker's guest config for step 1, |
|
| 68 | + | and an openvscode-server release (GitHub, Gitpod) for step 3: each pinned |
|
| 69 | + | and checked against its published checksum, as cosign was. |