OXYGENHUB

docs/workspaces.md: the runner side of workspaces, step by step 448568561f2a

Andy's decisions (2026-10-17): runner side first, files lazy from the
start (casfs over vsock into the VM, an overlay for edits; the guest
kernel needs FUSE first), editor traffic over one WebSocket per workspace
with yamux. Five steps, each landing working.

Refs andymccall/oxygenhub#69
Author
Andy McCall @andymccall <[email protected]>
via
agent Claude Code <[email protected]> · delegate for changes, until 2026-11-13 19:00Z · key ea5f78160088
When
· 2026-10-17 13:54 UTC
Signature
✔ signed by Claude Code
Hash
448568561f2a236cf2bb2fa553a069b1078e240e31344b1081245d28d4151056
Depends on
e0918b887850
Depended on by
0662754815cf
In bookmarks
main
In tags
v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.5.0 v0.5.1 v0.5.2 v0.5.3
Edits
1 atom, 1 tree operation, 1 file

Browse the tree at this change

added docs/workspaces.md +69 −0
1 +
# Workspaces: VS Code in a microVM, the runner side
2 +
3 +
oxygenhub#69, hub spec §13.3. Written 2026-10-17 with Andy's decisions:
4 +
build it, **runner side first**; files **lazy from the start**; editor
5 +
traffic over a **WebSocket with multiplexing**. This is the plan for the
6 +
runner; the hub's part (the workspaces table, the button, the proxy,
7 +
commits) follows it.
8 +
9 +
## What is there today
10 +
11 +
- Jobs run to their exit under a hard time limit, with a job token that
12 +
  lasts at most two hours (hub `JOB_TOKEN_MAX_AGE_MS`).
13 +
- A microVM has no network device. Its only traffic is guest-initiated:
14 +
  the egress proxy over vsock (`flow-init/src/egress.rs`).
15 +
- The runner speaks to the hub in short HTTPS calls; nothing long-lived.
16 +
- flow-casfs mounts a state lazily with FUSE, on the host. Firecracker has
17 +
  no virtio-fs, so a FUSE mount on the host cannot be handed to a guest.
18 +
- The pinned guest kernel (6.1.155) has overlayfs and vsock, but no FUSE.
19 +
20 +
## The shape
21 +
22 +
```
23 +
browser ──HTTPS/WS──► hub /{owner}/{name}/workspace/{id}/ ──┐
24 +
                                                            │ one WebSocket per workspace,
25 +
runner ◄────────────────────────────────────────────────────┘ opened by the runner (yamux streams)
26 +
  │ vsock (host connects in)
27 +
  ▼
28 +
microVM: flow-init ─► openvscode-server on 127.0.0.1:3000
29 +
         /workspace = overlay(lower: casfs over vsock, upper: scratch disk)
30 +
```
31 +
32 +
## Steps
33 +
34 +
Each is an issue of its own, and each lands working and tested.
35 +
36 +
1. **A guest kernel with FUSE.** The same 6.1 kernel and Firecracker's
37 +
   guest config, with `CONFIG_FUSE_FS=y`; pinned by BLAKE3 in `flow.toml`
38 +
   on earth-1, plex-1 and mars-1. Pipeline jobs are unaffected.
39 +
2. **casfs over vsock.** flow-casfs gains a server half: on the host it
40 +
   answers lookups, listings and reads for one state over a vsock port of
41 +
   the workspace's VM, from the same graph-key cache it uses today. The
42 +
   guest half is a small FUSE client in the workspace rootfs. `/workspace`
43 +
   is an overlay: the state read-only below, the VM's scratch disk above,
44 +
   so edits are plain files. Opening a workspace costs a listing, not an
45 +
   export; the target is under a second on the Linux tree.
46 +
3. **The workspace rootfs.** openvscode-server (pinned release, its
47 +
   checksum checked), `oxy`, the Oxygen VCS extension, a non-root user;
48 +
   built and imported like the rust image (`images/workspace`).
49 +
4. **The `workspace` job kind.** `Claim` says `kind: "workspace"`. The
50 +
   runner keeps the VM until it is told to stop, or the hub's idle limit
51 +
   passes, instead of waiting for an exit; it reports liveness and open
52 +
   connections every 30 s, and the hub renews the job token while the
53 +
   workspace lives. A runner advertises whether it takes workspaces.
54 +
5. **The tunnel.** flow-init accepts host connections on a vsock port and
55 +
   forwards each to openvscode-server. The runner opens one WebSocket to
56 +
   the hub per workspace (`/api/v1/flow/workspaces/{id}/tunnel`, its
57 +
   runner token) and carries each browser connection as a yamux stream to
58 +
   that port. Works for runners behind NAT and through Cloudflare.
59 +
60 +
Then the hub (hub spec §13.3): the `workspaces` table and limits, "Open in
61 +
workspace" behind a site setting (off), the owner-only reverse proxy for
62 +
HTTP and WebSocket into the tunnel, and commits pushed to `ws/{account}/{id}`
63 +
and signed by the hub as the account's delegate.
64 +
65 +
## Downloads this needs
66 +
67 +
Linux 6.1 sources (kernel.org) and Firecracker's guest config for step 1,
68 +
and an openvscode-server release (GitHub, Gitpod) for step 3: each pinned
69 +
and checked against its published checksum, as cosign was.