OXYGENHUB

flow runner drain: drain the running runner by a request file 4dd392d16454

The way to stop a Windows runner without losing its job (no SIGTERM
there); works on Linux too.

refs #83
Author
Andy McCall @andymccall <[email protected]>
When
· 2026-10-26 03:32 UTC
Signature
◈ sealed by Andy McCall
Hash
4dd392d16454f997a30b1e111f470747123aeb813fac7d6bcaacf2c280e14ccc
Depends on
3007d8f011fa 494a1b9a8ac2 62829d15ef09 731413202557 85275e1ef3b7 89171fca079c a083cc802892 c3db750ed96e e0918b887850
Depended on by
f4dfd742b30d
In bookmarks
main
In tags
v0.4.0 v0.5.0 v0.5.1 v0.5.2 v0.5.3
Edits
11 atoms, 0 tree operations, 6 files

Browse the tree at this change

modified CHANGELOG.md +1 −0
…
3 unchanged lines
4 4
5 5
## Unreleased
6 6
7 +
- **`flow runner drain` (#83):** asks the home's running runner to drain, as SIGTERM does on Linux, through a `drain.request` file its loop reads: the way to stop a Windows runner without losing its job. A starting runner clears a stale request.
7 8
- **A Windows runner updates itself (#83, #85):** `flow runner update` on Windows checks a PE build, renames `flow.exe` to `flow.prev.exe` and `flow.new.exe` into its place with `MoveFileExW` (retrying while a virus scanner holds a new file; a running program cannot rename its own file, so a copy of it, `flow __rename`, does), tells the running runner through an `update.request` file in its home (no SIGUSR1 there), and the runner leaves and exits 75 so a Task Scheduler task set to restart on failure starts the new build, which proves itself by reaching the hub. Tried in the Windows VM: install, leave, exit 75, restart, proven. A rollback runs the restored build and ends with it. Spec §11.5.3.
8 9
- **`flow` builds for Windows (#83, step 2 begun):** `cargo build --target x86_64-pc-windows-gnu` gives a `flow.exe` that offers the WASM tier: the syscall tier, the guest helpers and the egress proxy are Linux's only (a Windows build says the syscall tier "runs only on Linux"), cache pruning by reads stays on Linux, and file modes, links and sizes go through a small `portable` module (modes are not kept on Windows). Its claims report free disk (`GetDiskFreeSpaceExW`); load and kernel are not measured there yet, and a Windows runner does not update itself. `dist` builds `flow-windows-x86_64.exe` too. Nothing changes on Linux or Android. Tried in the Windows VM on Earth: `flow.exe tiers` offers wasm, a local `flow run` of a 120-step WASM job passes, and registered as runner `windows-1` (label `windows`) it ran andymccall/windows-runner-demo's job from the hub (run #1). Helpers start from the running binary's path on Windows (`/proc/self/exe` on Linux and Android).
9 10
- **The syscall tier's loopback reaches TCP (#73):** with `network "loopback"`, a job may open IPv4 and IPv6 sockets, which in its own network namespace reach only its `lo`, so a server it starts on 127.0.0.1 answers it (a conformance suite over an emulator's HTTP control port, which skipped every test on narvi-1). With `network "isolated"` sockets other than `AF_UNIX` are still refused.
…
101 unchanged lines
modified README.md +5 −3
…
61 unchanged lines
62 62
`runtime "wasm"` sandbox and `runs-on "windows"`. To keep it running, a
63 63
Task Scheduler task at start-up with that command (restarting on failure)
64 64
does; set it to restart on failure, as a runner that leaves for a new build
65 -
exits 75 (`flow runner update`, spec §11.5.3). What a Windows runner does
66 -
not do yet: report load and kernel, or drain when Windows stops it
67 -
(oxygenflow#83).
65 +
exits 75 (`flow runner update`, spec §11.5.3). To stop it without losing
66 +
its job, `flow.exe --home C:\flow runner drain` (Windows has no SIGTERM;
67 +
ending the task kills it). What a Windows runner does not report yet: load
68 +
and kernel (oxygenflow#83).
68 69
69 70
## Commands
70 71
…
5 unchanged lines
76 77
    flow hash FILE             # BLAKE3, for pinning in flow.toml
77 78
    flow runner                # claim and run jobs for the hub in flow.toml
78 79
    flow runner update --from FILE [--dry-run]   # install a new flow build in place of this one (spec §11.5.3)
80 +
    flow runner drain          # the running runner claims nothing more and stops once its jobs are reported
79 81
80 82
## A Flow home
81 83
…
60 unchanged lines
modified crates/flow/src/main.rs +13 −0
…
43 unchanged lines
44 44
    /// Measure this machine as a runner does: its speed score (100 on the
45 45
    /// reference machine) and class, free disk and load (spec §11.1)
46 46
    Bench,
47 +
    /// Ask this home's running runner to drain: claim nothing more, finish
48 +
    /// and report what it holds, then stop (as SIGTERM does on Linux; on
49 +
    /// Windows, the way to stop a runner without losing its job)
50 +
    Drain,
47 51
    /// Install a new flow build in place of this one (docs/jetstream.md
48 52
    /// §8.2): check it, keep this one as flow.prev, and have the running
49 53
    /// runner leave with notice so its unit starts the new build. A build
…
234 unchanged lines
284 288
            println!("registered as {} (runner {}); start it with `flow runner`", got.name, got.id);
285 289
            Ok(())
286 290
        }
291 +
        Cmd::Runner { cmd: Some(RunnerCmd::Drain) } => {
292 +
            let home = Home::locate(cli.home.as_deref())?;
293 +
            if flow::update::ask_to_drain(&home.root)? {
294 +
                println!("asked the runner to drain: it claims nothing more and stops once its jobs are done and reported");
295 +
            } else {
296 +
                println!("no runner of this home is running; the next one starts afresh");
297 +
            }
298 +
            Ok(())
299 +
        }
287 300
        Cmd::Runner { cmd: Some(RunnerCmd::Update { from, notice, dry_run, allow_older, exe }) } => {
288 301
            let home = Home::locate(cli.home.as_deref())?;
289 302
            let exe = match exe {
…
342 unchanged lines
modified crates/flow/src/runner.rs +4 −0
…
824 unchanged lines
825 825
        let mut update_asked: Option<Instant> = None;
826 826
        let mut update_said = String::new();
827 827
        loop {
828 +
            if !stopping() && crate::update::drain_requested(&home.root) {
829 +
                eprintln!("flow runner {name}: asked to drain (flow runner drain)");
830 +
                stop();
831 +
            }
828 832
            if UPDATE.swap(false, std::sync::atomic::Ordering::SeqCst) || crate::update::requested(&home.root) {
829 833
                let p = crate::update::pending(&home.root);
830 834
                let notice = Duration::from_secs(p.as_ref().map_or(0, |p| p.notice_secs));
…
1427 unchanged lines
modified crates/flow/src/update.rs +27 −0
…
320 unchanged lines
321 321
    false
322 322
}
323 323
324 +
/// In the runner's home: drain (claim nothing more, finish what it holds,
325 +
/// then stop), as SIGTERM asks on Linux: `flow runner drain`, for Windows.
326 +
const DRAIN: &str = "drain.request";
327 +
328 +
/// Ask the runner of `home` to drain; whether one is running to ask.
329 +
pub fn ask_to_drain(home: &Path) -> std::io::Result<bool> {
330 +
    std::fs::write(home.join(DRAIN), "")?;
331 +
    Ok(home.join(PID).is_file())
332 +
}
333 +
334 +
/// Whether the runner has been asked, by file, to drain (once).
335 +
pub fn drain_requested(home: &Path) -> bool {
336 +
    std::fs::remove_file(home.join(DRAIN)).is_ok()
337 +
}
338 +
324 339
/// Whether the runner has been asked, by file, to leave for a new build
325 340
/// (and forget the asking).
326 341
pub fn requested(home: &Path) -> bool {
…
32 unchanged lines
359 374
/// Note this process as the home's runner, for `flow runner update` to tell.
360 375
pub fn note_runner(home: &Path) {
361 376
    let _ = std::fs::remove_file(home.join(REQUEST));
377 +
    let _ = std::fs::remove_file(home.join(DRAIN));
362 378
    let _ = std::fs::write(home.join(PID), std::process::id().to_string());
363 379
}
364 380
…
133 unchanged lines
498 514
            .contains("not a Windows"));
499 515
        assert_eq!(beside(Path::new("C:/flow/flow.exe"), PREV), Path::new("C:/flow/flow.prev.exe"));
500 516
        assert_eq!(beside(Path::new("/usr/local/bin/flow"), NEW), Path::new("/usr/local/bin/flow.new"));
517 +
    }
518 +
519 +
    #[test]
520 +
    fn a_drain_is_asked_by_file_once() {
521 +
        let dir = tempfile::tempdir().unwrap();
522 +
        assert!(!ask_to_drain(dir.path()).unwrap(), "no runner running");
523 +
        assert!(drain_requested(dir.path()) && !drain_requested(dir.path()));
524 +
        note_runner(dir.path());
525 +
        assert!(ask_to_drain(dir.path()).unwrap());
526 +
        note_runner(dir.path());
527 +
        assert!(!drain_requested(dir.path()), "a starting runner clears a stale request");
501 528
    }
502 529
503 530
    #[test]
…
35 unchanged lines
modified docs/spec.md +3 −0
…
805 unchanged lines
806 806
  systemd kills what is left. A new binary is installed by replacing it
807 807
  and restarting the unit (`systemctl restart --no-block`): the old runner
808 808
  finishes its job, and `Restart=always` starts the new one.
809 +
  `flow runner drain` asks the same by a `drain.request` file in the
810 +
  runner's home, which its loop reads: the way on Windows, which has no
811 +
  SIGTERM (oxygenflow#83).
809 812
810 813
### 11.5.2 Handing back (oxygenflow#76)
811 814
…
97 unchanged lines