Diffusion: a landing does not refresh the consumer's pin index in production, so it shows behind with no bump issue #210 closed: completed andymccall opened this · 1 comment
andymccall
Also add a reconciler (Andy, 2026-10-15): today the pin index is only written on push/landing, at start (backfill) and on turning following on (catch_up); nothing re-checks it. Every few minutes, for each followed bookmark compare its current state hash with the state the index last recorded (new column) and re-read pins only where they differ (cheap, fine for GCS-backed repositories); in the same pass, run catch_up for a consumer whose pin is behind its upstream's newest green state with no open bump. Log and count each repair (feeds roadmap A6); repairs should trend to zero once the event path is fixed.
andymccall referenced by change 5d54cf746e00 on main ·
closed by change 0e04dfc20211 ·
doomsdayone/metalgear follows rstar-api (green). rstar-api main is ded953a5 (landed issue #105, run issue #10 passed); metalgear's bump issue #308 pinning it was opened 17:06 UTC and landed 17:39 UTC on 2026-10-14 (issue #307 at 16:30 likewise). Yet diffusion_pins for metalgear/develop was last written 16:02 UTC, by the backfill at the 17:01 BST restart, so the hub still holds the older pin: the overview says behind, and no bump is proposed because the newest upstream state is already in metalgear. land_as calls diffusion::after_push -> record, which deletes and re-inserts the rows in one transaction but returns silently when begin() fails and commits only if every statement succeeded; nothing was logged. The flow.rs Diffusion test (the index followed the landing) passes, so this is production-only (contention on SQLite under load is the first suspect; pins_in on a large state the second). Fix: log every record() failure, retry it, refresh the index on any move of a followed bookmark (not only in the landing path), and have consumers() fall back to reading the pin from the consumer's current state when the index is older than its bookmark. Workaround: a hub restart re-indexes every repository.