At-least-once delivery, ack timing, and idempotency keys — designing queue consumers that neither lose nor duplicate

Definition

Most queues deliver a message at least once. A queue does not delete a message until it receives an ack (acknowledgement) from the worker saying processing succeeded. If processing fails or no response comes back, the queue delivers the message again. That means the same message can arrive more than once.

Idempotent means that running the same request several times gives the same result as running it once. You build this by attaching the same number (the idempotency key) to the same request and skipping any number that has already been processed.

Why it matters

Two kinds of incidents happen, and they come from opposite directions.

1. Loss: A worker fails, but the code catches the exception and returns normally (= ack). The queue thinks it succeeded and deletes the message. The user has already been told "received," so the data disappears and nobody knows.
2. Duplication: Processing succeeds, but the network drops while the ack is on its way. The queue treats it as a failure and delivers again. Processing runs twice, so the same row gets stored twice or the same notification goes out twice.

The fix is a pair. Ack only on success to prevent loss, and make execution safe to repeat so you can withstand redelivery.

How it works

The idempotency key flows like this.

1. At the moment a single user action is created (such as a form submission), generate a unique number (a UUID).
2. Put the number in the message and enqueue it. (If the queue supports a deduplication option, pass the same number there too.)
3. At the final storage step, check "does this number already exist?" If it does, don't write, and respond with success.

In practice

async function handle(message: { id: string; values: Values }) {
  const result = await store.appendOnce(message.id, message.values); // no-op if the same id already exists
  if (result.kind === "retryable") throw new Error(result.code); // throwing triggers redelivery
  if (result.kind === "fatal") {
    report({ id: message.id, code: result.code }); // report together with the identifier
    return; // the only place where we ack on purpose
  }
}

• Don't ack the exception path. Code that swallows an error with try/catch and finishes is the most common cause of loss.
• Set a maximum delivery count. When it is exhausted, report to your error tracking tool with the identifier, then ack or move the message to a dead-letter queue (a queue that isolates failed messages).
• Check for duplicates at the final store. Even if you filter only in front of the queue, redelivery happens behind the queue.
• Keep three cases as tests: processing the same message twice, redelivery after a failure, and reporting on exhaustion.

Trade-offs

When not to use it

• For log-style events where duplicates are not actually harmful, a complex idempotency store is overkill.
• "Ack first, process later" should only be used for data where loss is acceptable.
• If the server generates a new idempotency key on every request, deduplication doesn't work. Generate it only once per user action.

Common mistakes

• Swallowing every exception with catch and returning normally. This is the most common cause of silent loss.
• Trusting that the redelivery interval behaves as configured. Check the actual delivery times in the logs. SDK options can be ignored depending on the environment.
• Leaving a message that fails forever (a poison message) alone. It blocks the queue, so you need an isolation path.
• Acking without reporting exhaustion. Nothing signals a person to notice.

Related concepts

• queue-based-load-leveling — the queue and worker structure that needs this ack rule
• retry-backoff-with-jitter — redelivery intervals and how to classify retries
• optimistic-outbox-rebase — a client-side solution to a similar problem: stacking up pending work and reapplying it in order