The login throttle test races the wall clock and fails on a loaded machine issue #354 open andymccall opened this · 2 comments
andymccall set labels bug ·
andymccall
A second test with the same fault, 2026-11-01 00:35 GMT: tests/flow.rs::scheduled_jobs_run_when_due sleeps a fixed 300 ms for a push's run to be recorded (line ~2440), and at load 35 it was not yet (2 runs, expected 3) in the Postgres pass; alone it passes in 4.7 s. Wait for the run (poll with a deadline) instead of a fixed sleep.
andymccall
A third, in CI run 532 (2026-11-01 03:45): assistants::tests::a_slow_steady_stream_is_read_and_a_silent_one_cut serves parts 200 ms apart against a 400 ms silence limit; on a busy runner a gap passed 400 ms ("no answer within 0 seconds"). Widen the margin (or drive the clock) as for the others.
andymccall referenced by change f87a74fffc87 on main ·
tests/accounts.rs::guessing_passwords_is_slowed_per_account_and_per_addressdepends on the wall clock: the throttle's first wait is 2 s (FIRST_WAIT_MS), and the test makes six wrong guesses then expects the right password to be held back. Each guess hashes a password; on a loaded machine (load 25 on 8 cores, 2026-11-01) six hashes take longer than 2 s, the wait expires and the right password signs in (303, "not signed in" assertion). It failed twice in a row locally while CI on Earth passed.Fix: let the throttle take its clock from the app (as
wait(keys, now_ms)already takesnow_ms) and drive it from the test, or freeze time in the test, so it never races real time.