Speed Limits and Resilient Error Management

Gains:

Can interpret speed limits (RPM/ITPM/OTPM) and 429 errors
Implements exponential backoff and retry with retry-after
Correctly classifies and handles common HTTP error codes (400/401/429/500/529)

In a production environment, no API responds perfectly all the time. Sometimes you send requests too quickly and hit the limit; sometimes the server is temporarily busy; Sometimes your request is wrong from the beginning. What distinguishes a solid integration from an amateur attempt is that it handles these situations predictively and automatically. In this unit you will learn about rate limits (RPM/ITPM/OTPM), 429 error, retry with exponential backoff, and proper classification of common HTTP error codes. The goal: to build a flow that is so robust that a user will never notice it.

What are Speed Limits?

The provider limits how much work a switch can do in a given period of time. This protection; It protects both the infrastructure and you from sudden cost explosions. There are three common types of limits:

RPM (Requests Per Minute): Number of requests per minute.
ITPM (Input Tokens Per Minute): Input token that can be processed per minute.
OTPM (Output Tokens Per Minute): Output token that can be produced per minute.

If you exceed any of these limits, the provider rejects the request and returns a 429 error code. Limits generally vary depending on your account level (tier) and may be increased over time.

Tip: You can watch when you are approaching the limit from the response headers. Most providers report your remaining quota with headers like x-ratelimit-remaining-*. Monitoring these values and throttle the traffic in front is the most mature way to prevent the problem without getting a 429.

429 and Exponential Retracement

429 (rate limit) is a temporary and retryable error. The correct response is to wait for the request for a while and try again. But a constant wait is not enough; If everyone tries again at the same time, the limit will be reached again. The solution is exponential backoff: increasing the waiting time exponentially with each failed attempt.

# Exponential backoff logic trial 1 → 429 → wait 1 sec trial 2 → 429 → wait 2 sec trial 3 → 429 → wait 4 sec trial 4 → 429 → wait 8 sec (+ small random "jitter")... give up and report after at most N trials

Adding a little randomness (jitter) to this prevents requests colliding when trying to retry at the same time. Additionally, the 429 response often carries a `retry-after` header: "try again in this many seconds". Respecting this title is more accurate than blindly waiting.

Caution: When you get a 429, "forcing it by sending more requests" will make the situation worse; The limit continues to be filled and no requests go through. The correct response is retreat, not acceleration. Good news: most official SDKs automatically retry 429 and server errors with a backoff — use this behavior of the SDK before installing it manually.

Classifying HTTP Error Codes

Not every mistake is the same. Critical distinction: can it be retried or is it a request/identity issue?

Code

Meaning

Can it be tried again?

correct response

400

Invalid request (format/parameter error)

Correct the request; do not send the same again

401

Authentication error (key invalid/missing)

Fix key/title

403

No authorization (no access to model/feature)

Check permissions/scope

404

Not Found (incorrect model ID/endpoint)

Correct model ID/address

429

Speed limit exceeded

Yes

Retreat + retry-after

500

Server error

Yes

Try again with retreat

529

Server overloaded

Yes

Try again with retreat

Golden rule: 429, 500 and 529 are temporary; It is tried again with withdrawal. 400, 401, 403, 404 are request/identity issues; Trying again won't solve it, and it wastes effort. Your code must distinguish between these two groups.

Step by Step: Durable Call

Submit the request. If successful, continue.
Classify the error code. Can it be tried again?
If tryable: follow retry-after, apply exponential backoff + jitter, try a limited number of times (e.g. 5 max).
If not tried: Fix (format/key) and stop; Do not repeat the same erroneous request in the loop.
Consider giving up. If still unsuccessful after n attempts, show a polite message to the user and log the event (tracking unit 11).

# Robust call pseudo-codedene = 0repeat: response = request_at() if response.success: return response if response.code in [429, 500, 529] and try < 5: wait = retry_after ?? (2^try sec + jitter) sleep(wait); try += 1; git again if response.code in [400, 401, 403, 404]: save_error(response); return "request must be fixed" return "permanent error, try later"

# Polite feedback to the user (when retries are exhausted) "I'm busy right now, I couldn't process your request. Try again soon, or I've saved your request, I'll get back to you when it's ready."

Weak prompt / Strong prompt (here: error message design)

# WEAK (displays raw error to user)"Error 429: rate_limit_error"

# STRONG (user-friendly, reassuring, action-suggesting) "There was a temporary congestion in the system. We have received your request safely and it is being tried again automatically. If a result does not appear within a few seconds, you can refresh the page."

Revealing the raw technical error to the end user both undermines trust and can be a security vulnerability. Categorize errors internally and give the user a calm, action-oriented message; just write the technical detail for the record.

Three Mini Cases

Case 1 — Boat crashed in traffic explosion. A customer service bot received 429 in surge traffic on campaign day; There was no retry in the code, every error was reflected directly to the user as an "error". They added exponential retracement + retry-after; with the same traffic, requests passed with a delay of several seconds, the user did not see any errors.

Case 2 — Trying 400 in the loop. An integration was getting a 404 due to an invalid model ID, but was treating all errors as "transient" and trying again in an infinite loop; The log became swollen and unnecessary load was created. They added error classification: 404 is considered permanent, the loop is stopped and the model ID is corrected. Lesson: don't try every mistake again.

Case 3 — Managing the limit from the front. A data enrichment job was constantly running at the 429 limit. They followed the x-ratelimit-remaining header and throttled the traffic according to the quota. So they kept a steady pace just below the limit, without taking any 429s; The job was done more predictably and faster.

Common mistakes

Increasing speed in 429: Makes the situation worse; Switch to retreat.
Retrying each error: 400/401/404 is permanent; Trying again is a waste.
Using fixed wait: Creates a collision; Use exponential + jitter.
Ignoring 'retry-after': It is most accurate to comply with the time specified by the provider.
Revealing the raw error to the user: Shakes trust, creates vulnerabilities; Classify inside.
Unlimited retries: Set an upper limit (e.g. 5 retries); then give up gracefully.

Deeper: Queuing, Concurrency, and Circuit Breakers

The endurance of a single desire is the first step; The real maturity is to manage a large number of requests without hitting the limits. Three concepts come into play here.

Queue: You put requests in a queue to send them at a controlled pace rather than immediately. Queuing smoothes out sudden bursts of traffic: Even if 1,000 requests arrive at once, the queue will release them at a rate below the limit. This way you prevent 429, then you don't have to worry about fixing it.

Concurrency limit: You limit how many requests are "in the air" at the same time. Unlimited parallel requests quickly fill RPM and TPM limits. A reasonable concurrency ceiling (e.g. no more than 10 concurrent requests) both maintains limits and makes the system predictable.

Circuit breaker: If the provider keeps returning 500/529, instead of doggedly trying every request, you "break the circuit" for a while and quickly fail the request without ever sending it. After a wait, you turn the circuit back on and try. This pattern prevents your system from crashing in the event of a temporary provider failure.

Together, these three establish system-level resiliency beyond the retry logic of a single call. On a small scale, the SDK's automatic retry is sufficient; As scale grows, queuing, concurrency, and circuit breaker become indispensable. They all have the same common goal: to reflect a temporary problem to the user not as a crash, but as an invisible delay of a few seconds.

In summary

429 returns when speed limits (RPM/ITPM/OTPM) are exceeded; This is a temporary error and will be retried using retry-after and exponential backoff + jitter. 500 and 529 are also provisional; 400/401/403/404 is a request/identity issue and cannot be resolved by trying again. A robust flow separates errors into these two groups, tries a limited number of times, monitors the limit from the front and shows calm messages to the user.

Application task

Consider your integration. (1) List the error codes you may encounter and separate them into "retryable / permanent". (2) Write down your exponential pullback plan (initial hold, coefficient, cap, jitter). (3) Specify how to use the retry-after header. (4) Write the polite message to be displayed to the user when retries are exhausted.

checklist

[ ] I can explain RPM/ITPM/OTPM limits and 429.
[ ] I can apply the logic of exponential retreat + jitter + retry-after.
[ ] I can classify error codes as retryable/permanent.
[ ] I know that we should not try every mistake.
[ ] Instead of a raw error, I can show the user a calm, action-oriented message.

What are Speed ​​Limits?