A durable job queue (SKIP LOCKED on Postgres, polled on SQLite) issue #276 open andymccall opened this · 1 comment
andymccall set labels feature ·
andymccall referenced by change 6b195ed3b637 on main ·
referenced by 2 changes on 21 Oct 2026
First slice built and live (hub 021dd6df, in 0.7.0): work_queue (migration 0091) and db::queue: enqueue with a dedupe key, a leased claim (UPDATE … WHERE id IN (SELECT … FOR UPDATE SKIP LOCKED) on PostgreSQL), backoff 30 s → 1 h, dead after 8 attempts; queue.worker in every worker process; oxyhub_queue_items{state} in /metrics. Tenants: a notification email that failed, and the Git mirror export after a push that failed (both still try at once).
Checked the rest of the list: the apt index is built on request and cached (no rebuild job to queue); Diffusion's bumps have diffusion.reconciler mending what a push missed; webhook deliveries already have their own durable table, retries and worker (hooks.worker). So what is left is optional: moving deliveries onto work_queue to have one queue, and a nudge on the event bus so a worker takes an item at once instead of within 2 s. Leaving open for Andy to decide whether that is worth doing.
A jobs table (kind, payload, run_after, attempts, claimed_by, claimed_until), claimed with SELECT … FOR UPDATE SKIP LOCKED. First tenants: webhook deliveries, mail, Git mirror export, apt rebuild, Diffusion bumps, gc. Workers scale out by taking jobs; the event bus only nudges. Design: docs/roles.md (2026-10-19).