Skip to content

docs(rfc): consumer hold — postpone a delivery as a partition barrier - #486

Merged
behinddwalls merged 1 commit into
mainfrom
preetam/consumer-backoff
Aug 4, 2026
Merged

docs(rfc): consumer hold — postpone a delivery as a partition barrier#486
behinddwalls merged 1 commit into
mainfrom
preetam/consumer-backoff

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

Queue controllers have no way to say "this message is fine, but it must wait." The outcome model is ack/nack/reject, so the stovepipe process and buildsignal stages fake waiting by acking and republishing fresh copies of their own messages with delayed visibility. That workaround makes consumers publishers to their own topic, requires per-stage message-id minting to dodge the publish dedup (one scheme already shipped a stalled-loop bug), resets the delivery attempt count every cycle so the DLQ backstop never fires, hangs the loop's only liveness on a publish succeeding, and churns a row per wait tick. The process stage RFC weighed the alternatives, chose neither, and deferred to a future consumer-owned primitive — this is that primitive.

What?

Design doc for a hold/postpone primitive. A controller records a hold on its delivery view and returns success; the framework postpones the delivery instead of acking. The postponed message becomes a barrier: its partition is not consumed past it until the delay elapses, then it redelivers first, in order — a backoff loop that costs no goroutine, no lease renewal, and no new rows between wake-ups. A postponed redelivery resets failure accounting (a postponed mark on the delivery state exempts the next delivery from the attempt increment), so holds never burn the dead-letter budget, while real failures still do. Failed postpone writes lapse into normal visibility-timeout redelivery, so wait-loop liveness is framework-owned. Rejected alternatives (park-and-extend, non-blocking postpone, sentinel errors, gate write path, framework hold caps) and stovepipe/orchestrator migrations as named follow-ups.

Test Plan

Issues

Stack

  1. @ docs(rfc): consumer hold — postpone a delivery as a partition barrier #486
  2. feat(platform): hold/postpone — a fourth delivery outcome for backing off #487
  3. feat(stovepipe): migrate process and buildsignal waits to the hold primitive #488
  4. feat(orchestrator): migrate the buildsignal poll loop to the hold primitive #489
  5. feat(messagequeue)!: remove PublishAfter and the visible_after column #490
  6. feat(messagequeue)!: drop the requeue delay parameter from Nack #491
  7. feat(consumergate)!: rebuild the gate on the postpone primitive #492

## Summary

### Why?

Queue controllers have no way to say "this message is fine, but it must wait." The outcome model is ack/nack/reject, so the stovepipe process and buildsignal stages fake waiting by acking and republishing fresh copies of their own messages with delayed visibility. That workaround makes consumers publishers to their own topic, requires per-stage message-id minting to dodge the publish dedup (one scheme already shipped a stalled-loop bug), resets the delivery attempt count every cycle so the DLQ backstop never fires, hangs the loop's only liveness on a publish succeeding, and churns a row per wait tick. The process stage RFC weighed the alternatives, chose neither, and deferred to a future consumer-owned primitive — this is that primitive.

### What?

Design doc for a hold/postpone primitive. A controller records a hold on its delivery view and returns success; the framework postpones the delivery instead of acking. The postponed message becomes a barrier: its partition is not consumed past it until the delay elapses, then it redelivers first, in order — a backoff loop that costs no goroutine, no lease renewal, and no new rows between wake-ups. A postponed redelivery resets failure accounting (a postponed mark on the delivery state exempts the next delivery from the attempt increment), so holds never burn the dead-letter budget, while real failures still do. Failed postpone writes lapse into normal visibility-timeout redelivery, so wait-loop liveness is framework-owned. Rejected alternatives (park-and-extend, non-blocking postpone, sentinel errors, gate write path, framework hold caps) and stovepipe/orchestrator migrations as named follow-ups.
@behinddwalls
behinddwalls added this pull request to the merge queue Aug 4, 2026
Merged via the queue into main with commit 568e9d5 Aug 4, 2026
27 checks passed
@behinddwalls
behinddwalls deleted the preetam/consumer-backoff branch August 4, 2026 23:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants