Retries, Exponential Backoff, and Jitter — Keeping Failed Callers from Coming Back at the Same Moment

Definition

When a call fails for a transient reason (the other side is briefly busy, the network wobbles), trying again is a retry. But simply setting "try again after 1 second" can make the problem bigger, so two things are added.

• Exponential backoff: Each time a call fails, the wait time doubles, such as 1 second, 2 seconds, 4 seconds.
• Jitter: A random value is mixed into the wait time so that each caller waits a slightly different amount.

Why it matters

Suppose ten callers fail at the same moment. If they all use "retry in exactly 1 second", ten of them come back at the same time one second later and get rejected again for the same reason. This repeating phenomenon is called a thundering herd. Even when the other side is truly down, hitting it at fixed intervals keeps slowing its recovery.

Backoff gives the other side time to rest, and jitter spreads out the callers' arrival times so they do not all pile up at once.

How it works

The simplest way to produce jitter is to pick a random value from a range.

const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

async function withRetry<T>(
  fn: () => Promise<T>,
  { attempts = 3, minMs = 1000, maxMs = 4000 } = {},
): Promise<T> {
  for (let attempt = 1; ; attempt++) {
    try {
      return await fn();
    } catch (error) {
      if (!isRetryable(error) || attempt >= attempts) throw error;
      await sleep(minMs + Math.random() * (maxMs - minMs));
    }
  }
}

A two-layer structure is also common. Inside a single call, retry a few times over a short range (for example 1 to 4 seconds) to resolve momentary contention. If that still fails, throw the error and let the queue redeliver at a longer interval (several seconds to several minutes) (see queue-based-load-leveling).

In practice

• Classify what to retry. Retry only what "might succeed if tried again": lock waits, transient 5xx responses, timeouts, and HTML error pages. Bad input (4xx) or authentication failures give the same result no matter how many times you retry.
• Set limits. Without a cap on the number of attempts and on the interval, a request stays alive forever or the interval keeps growing.
• Decide what happens after giving up. When the limit is reached, make it clear in code where the request goes (queue redelivery, error report, a message to the user).
• Multiply the total attempts. With 3 attempts inside a call times 8 queue redeliveries, a single request can reach the other side up to 24 times.

Trade-offs

When not to use it

• For operations that are not safe to repeat (such as one that charges twice if it runs twice), do not retry without an idempotency key (idempotent-consumer-ack-timing).
• On a path where the user is waiting in front of the screen, do not use a long backoff. Keep it short and hand the rest off asynchronously.
• If you retry a permanent failure (a bad request), you repeat the same failure and just keep hammering the other side.

Common mistakes

• Using backoff without jitter. Callers that started at the same moment keep overlapping in timing.
• Retrying every error. Retrying even 4xx responses only grows the logs.
• Not capping the number and interval of retries.
• Not multiplying retries across layers. Each layer is reasonable on its own, but put together they put too much load on the other side.

Related concepts

• queue-based-load-leveling — the structure for deciding which layer does the retrying
• idempotent-consumer-ack-timing — why retries can execute the same request twice, and how to handle it
• playwright-flaky-vs-failed-triage — a way to tell failures that pass on retry apart from defects that keep failing