01
Inspect the client’s effective retry configuration
Configuration can come from environment variables, shared settings, and a client Config object. Inspect the client involved in the failure instead of assuming every application uses the same SDK defaults. Record its SDK version and configured mode.
Boto3 documents an important counting distinction: max_attempts in a Config object counts retries after the first attempt, while the similarly named shared-file or environment setting counts total attempts. Using total_max_attempts in Config makes that local choice explicit.
print(client.meta.config.retries)
# Share configuration selectively; do not enable broad credential-bearing logs.02
Set a bounded total-attempt budget
The following example chooses standard mode and three total attempts, including the initial request. Choose values from the operation’s latency budget and error behavior, not because this small demonstration recovered on attempt three.
Connect and read timeouts bound individual network operations; the whole call can still take longer because of retries and backoff. A caller deadline, cancellation policy, and safe operation semantics need to be considered separately.
import boto3
from botocore.config import Config
config = Config(
retries={"mode": "standard", "total_max_attempts": 3},
connect_timeout=2,
read_timeout=2,
)
client = boto3.Session(profile_name="work").client(
"dynamodb", region_name="eu-west-1", config=config
)03
Verify behavior with a controlled response sequence
Our local HTTP fixture returned ProvisionedThroughputExceededException-shaped errors for the first two requests, then a successful GetItem response. It exercised the real SDK retry loop rather than manually calling a mock method several times.
A total budget of two ended with the throttling error. A budget of three reached success, and ResponseMetadata.RetryAttempts was 2. These counts verify this client and fixture, not the capacity or recovery time of an AWS table.
# After a successful Boto3 operation:
print(response["ResponseMetadata"]["RetryAttempts"])04
Handle exhausted retries deliberately
Classify the final error and decide whether the caller should receive a retryable failure, a queued retry, or another application-specific outcome. Avoid stacking multiple independent retry loops that multiply load and exceed the intended deadline.
For writes, a timeout can leave uncertainty about whether a request took effect. Use the service’s appropriate idempotency or conditional-write mechanism instead of assuming another attempt is always harmless. A retry budget controls effort; it does not establish exactly-once execution.
05
Investigate persistent throttling at its source
If errors continue, inspect the actual operation, resource, access distribution, and service evidence. A hot key, burst, concurrency setting, or resource limit can require a different repair from waiting for a transient fault. Do not automatically buy more capacity or grant broader permissions.
SDK retry behavior evolves and can include version-dependent options. Preserve the tested version, mode, attempt count, and final outcome in the record. A deterministic local fixture is useful for guarding the application’s budget, while live service diagnosis remains a separate operational check.
Know the limits
Tested with Boto3/Botocore 1.42.97 in standard mode against a local HTTP fixture. It returned two DynamoDB throttling-shaped responses, then success. No AWS requests, real quota exhaustion, production throttling, or new retry-feature opt-in behavior was exercised.
Sources & further reading
A technical publication by CoreTrailKit.
Browse all articles