OXYGENHUB
# 448568561f2a oxygenflow / docs / workspaces.md
3.6 KiB · Markdown · last changed docs/workspaces.md: the runner side of workspaces, step by step 18 days ago renderedsource blame history raw
1
# Workspaces: VS Code in a microVM, the runner side
2
3
oxygenhub#69, hub spec §13.3. Written 2026-10-17 with Andy's decisions:
4
build it, **runner side first**; files **lazy from the start**; editor
5
traffic over a **WebSocket with multiplexing**. This is the plan for the
6
runner; the hub's part (the workspaces table, the button, the proxy,
7
commits) follows it.
8
9
## What is there today
10
11
- Jobs run to their exit under a hard time limit, with a job token that
12
  lasts at most two hours (hub `JOB_TOKEN_MAX_AGE_MS`).
13
- A microVM has no network device. Its only traffic is guest-initiated:
14
  the egress proxy over vsock (`flow-init/src/egress.rs`).
15
- The runner speaks to the hub in short HTTPS calls; nothing long-lived.
16
- flow-casfs mounts a state lazily with FUSE, on the host. Firecracker has
17
  no virtio-fs, so a FUSE mount on the host cannot be handed to a guest.
18
- The pinned guest kernel (6.1.155) has overlayfs and vsock, but no FUSE.
19
20
## The shape
21
22
```
23
browser ──HTTPS/WS──► hub /{owner}/{name}/workspace/{id}/ ──┐
24
                                                            │ one WebSocket per workspace,
25
runner ◄────────────────────────────────────────────────────┘ opened by the runner (yamux streams)
26
  │ vsock (host connects in)
27
  ▼
28
microVM: flow-init ─► openvscode-server on 127.0.0.1:3000
29
         /workspace = overlay(lower: casfs over vsock, upper: scratch disk)
30
```
31
32
## Steps
33
34
Each is an issue of its own, and each lands working and tested.
35
36
1. **A guest kernel with FUSE.** The same 6.1 kernel and Firecracker's
37
   guest config, with `CONFIG_FUSE_FS=y`; pinned by BLAKE3 in `flow.toml`
38
   on earth-1, plex-1 and mars-1. Pipeline jobs are unaffected.
39
2. **casfs over vsock.** flow-casfs gains a server half: on the host it
40
   answers lookups, listings and reads for one state over a vsock port of
41
   the workspace's VM, from the same graph-key cache it uses today. The
42
   guest half is a small FUSE client in the workspace rootfs. `/workspace`
43
   is an overlay: the state read-only below, the VM's scratch disk above,
44
   so edits are plain files. Opening a workspace costs a listing, not an
45
   export; the target is under a second on the Linux tree.
46
3. **The workspace rootfs.** openvscode-server (pinned release, its
47
   checksum checked), `oxy`, the Oxygen VCS extension, a non-root user;
48
   built and imported like the rust image (`images/workspace`).
49
4. **The `workspace` job kind.** `Claim` says `kind: "workspace"`. The
50
   runner keeps the VM until it is told to stop, or the hub's idle limit
51
   passes, instead of waiting for an exit; it reports liveness and open
52
   connections every 30 s, and the hub renews the job token while the
53
   workspace lives. A runner advertises whether it takes workspaces.
54
5. **The tunnel.** flow-init accepts host connections on a vsock port and
55
   forwards each to openvscode-server. The runner opens one WebSocket to
56
   the hub per workspace (`/api/v1/flow/workspaces/{id}/tunnel`, its
57
   runner token) and carries each browser connection as a yamux stream to
58
   that port. Works for runners behind NAT and through Cloudflare.
59
60
Then the hub (hub spec §13.3): the `workspaces` table and limits, "Open in
61
workspace" behind a site setting (off), the owner-only reverse proxy for
62
HTTP and WebSocket into the tunnel, and commits pushed to `ws/{account}/{id}`
63
and signed by the hub as the account's delegate.
64
65
## Downloads this needs
66
67
Linux 6.1 sources (kernel.org) and Firecracker's guest config for step 1,
68
and an openvscode-server release (GitHub, Gitpod) for step 3: each pinned
69
and checked against its published checksum, as cosign was.