In the previous article we saw how Retry helps improve application resilience by making new attempts in the face of transient failures. However, we also learned that a naive implementation can produce exactly the opposite effect, increasing the pressure on already degraded services and causing a Retry Storm.
In this article we will see how Exponential Backoff works, why it reduces overload in distributed systems, and which limitations still remain.
Recalling the problem
In the previous article we used as an example an environment running approximately 500 application instances. At some point, an external API starts returning HTTP 503 (Service Unavailable). All instances detect the failure and immediately try again. Within a few milliseconds, the service that was already unavailable receives another 500 requests. Since it remains unavailable, new retries are performed, quickly driving the request volume to thousands per second. Instead of helping the service recover, clients start amplifying its unavailability.
The problem is not trying again. The problem is trying again immediately, without giving the dependency enough time to recover. The Google Cloud documentation itself classifies immediate retries without a waiting strategy as an anti-pattern, precisely because of their potential to amplify outages.
Suppose our server finds itself without enough resources to handle the requests, due to lack of memory or CPU caused by a sudden spike in request volume, and starts returning responses that indicate capacity limits, such as HTTP 429 (Too Many Requests) or HTTP 503 (Service Unavailable). This behavior of repeated retries increases the pressure on the dependency even more, potentially degrading internal processes and prolonging recovery even further.
What is Exponential Backoff?
Exponential Backoff is a Retry strategy where the interval between attempts increases progressively after each failure. This reduces the pressure placed on degraded services, lowering the chances of a Retry Storm.
Unlike fixed-interval or linear-increment strategies, Exponential Backoff increases the wait time exponentially between attempts.
The graph gives you an idea of how it works:
Notice that the interval between attempts is no longer constant. With each new failure, the waiting time increases exponentially, gradually reducing the frequency of new attempts. The longer the failure streak, the longer the wait until the next attempt. In this way, the dependency gets more time to recover its capacity before receiving new requests.
In most implementations it is recommended to define a maximum limit for Backoff growth (Max Backoff). Otherwise, a dependency that has already recovered may remain without receiving new attempts for an excessively long time. Max Backoff defines an upper bound for the interval between attempts, avoiding excessively long waits once the dependency has started responding normally again.
This formula represents only one possible implementation. Some libraries use small variations in the calculation, but all follow the same principle: progressively increase the interval between attempts.
Although base 2 is the most common implementation, different algorithms may use different growth functions as long as they preserve the exponential increase characteristic.
With that, the longer the dependency stays unavailable, the lower the frequency of new attempts should be. In this way, we reduce the load on a degraded service while increasing its chances of recovery.
Comparing Retry strategies
There are different strategies for defining the interval between Retry attempts. The main difference between them lies in how the waiting time evolves after each failure. While Fixed Retry keeps the same interval every time, Incremental Retry increases that interval linearly. Exponential Backoff, on the other hand, increases the waiting time exponentially, progressively reducing the pressure on degraded services.
The images below illustrate how each strategy distributes attempts over time.
Fixed Interval
Incremental Interval
Exponential Backoff
We can see side by side how each wait behaves.
| Attempt | Fixed Retry | Incremental Retry | Exponential Backoff |
|---|---|---|---|
| 1 | 100ms | 100ms | 100ms |
| 2 | 100ms | 200ms | 200ms |
| 3 | 100ms | 300ms | 400ms |
| 4 | 100ms | 400ms | 800ms |
| 5 | 100ms | 500ms | 1600ms |
Notice that Exponential Backoff does not reduce the number of attempts made. It only spreads those attempts over time. This simple change significantly reduces the pressure placed on degraded services.
Implementation
The implementation of Exponential Backoff is extremely simple. In practice, the only difference compared to a traditional Retry lies in how the waiting time between attempts is calculated.
In a fixed-interval Retry strategy, all attempts wait exactly the same period before being executed again.
delay := baseDelayIn an incremental Retry strategy, the waiting time grows linearly.
delay := baseDelay * time.Duration(retryCount)When using Exponential Backoff, the waiting time grows exponentially after each new failure, respecting a maximum limit defined by Max Backoff.
backoff := baseDelay * time.Duration(1<<retryCount)
if backoff > maxBackoff {
backoff = maxBackoff
}
delay := backoffNotice that only a few lines of code are enough to completely change the behavior of the application. Instead of executing new attempts at constant intervals, the system starts to gradually reduce the frequency of requests, lowering pressure on degraded dependencies and increasing their chances of recovery.
Limitations of Exponential Backoff
Despite its benefits, Exponential Backoff does not solve every problem related to Retry. Although it significantly reduces pressure on the service, Exponential Backoff still has an important limitation: different clients continue calculating exactly the same intervals between attempts. As a result, thousands of applications may start making new requests at practically the same moment, creating new traffic spikes.
Exponential Backoff + Jitter
Although Exponential Backoff increases the interval between attempts, different clients still use exactly the same calculation. This causes thousands of applications to retry at practically the same time.
Jitter adds an element of randomness to the value generated by the exponential backoff calculation, so the produced wait times are not all within the same millisecond. By introducing a small random component on top of the calculated time, retries stop happening simultaneously and the load is naturally distributed over time.
The image below illustrates how attempts become distributed over time after applying Full Jitter.
Full Jitter implementation
Just like Exponential Backoff requires only a few lines of code, adding Full Jitter is also quite simple. The difference is that, instead of using the exact backoff value, we pick a random time between 0 and the calculated maximum value.
backoff := baseDelay * time.Duration(1<<retryCount)
if backoff > maxBackoff {
backoff = maxBackoff
}
delay := time.Duration(rand.Int63n(int64(backoff)))In this way, different clients start retrying at different instants, naturally distributing the load across the dependency.
The different types of Jitter
There are different strategies for adding randomness to Backoff calculations. The best known are:
- Full Jitter: picks a random value between 0 and the calculated Backoff. It is the most used strategy because it significantly reduces synchronization between clients.
- Equal Jitter: keeps part of the calculated waiting time and applies randomness only to the other half, avoiding very small delays.
- Decorrelated Jitter: uses the previous delay to calculate the next interval, producing more unpredictable timings and avoiding synchronization between clients over long periods.
In this article we use Full Jitter because it is the simplest approach and also the one recommended in several documents, such as AWS.
Conclusion
Retry remains one of the most important mechanisms for dealing with transient failures. However, as we saw in this article, the strategy used to define the interval between attempts makes all the difference. Exponential Backoff significantly reduces pressure on degraded systems and is now widely used in modern distributed architectures. Even so, when thousands of clients run the same algorithm simultaneously, new traffic spikes can still emerge. That is exactly the scenario that motivates Jitter.