Inside the SDK, REST, streaming with SSE, timeouts and retries

A gateway cuts off at 30 seconds and returns a 504, but the worker behind it keeps waiting for the response for several more minutes. Nobody configured anything: the problem lies in the SDK defaults, which nobody read. Integrating Claude well means knowing what the SDK does on top of the REST API and when you need to change it.

What the SDK does for you

The Messages API is REST: a POST with JSON. The client SDK (Python, TypeScript and others) wraps it and adds:

• authentication with the key and the API version header;
• automatic retries with backoff for connection errors, 429s and 5xx errors: two by default, configurable with max_retries;
• a per-request timeout: ten minutes by default, configurable on the client or per request (in Python, with with_options);
• typed exceptions per error code and the ID of each request for debugging;
• an async client for many calls in parallel.

When there is a shorter limit in front (a gateway, a user waiting), the SDK timeout has to sit below it, with fewer retries, and the application returns a fallback response. Otherwise the worker stays stuck long after the gateway gave up.

To parallelize, use bounded concurrency: the async client with a cap on in-flight requests, for example a semaphore. Launching everything at once with an unbounded gather ends in 429s.

Streaming: SSE, not websockets

With stream: true, the API sends the response as server-sent events (SSE): message_start, then the content_block_delta events carrying each text delta, and finally message_delta and message_stop.

What gets asked the most:

• It is not a websocket. If the frontend already uses websockets with the browser, the backend consumes the stream with the SDK and relays the deltas over that websocket. The key never reaches the browser.
• An error can arrive inside the stream, as an error event (for example overloaded_error), after a 200. Without message_stop, the response is incomplete: discard it and retry.
• A tool’s input arrives in pieces. input_json_delta events carry JSON fragments that aren’t valid on their own: accumulate them and parse when the block closes, or use the SDK helper that assembles the final message.

Long requests

With a very large max_tokens and no streaming, the connection sits idle and some networks drop it. The SDK even refuses a non-streaming request it expects to take longer than about ten minutes. The way out is to use the SDK’s streaming helper and ask it for the complete final message: it works just like a non-streaming call, even if you show nothing on screen. If the work can wait, the other option is the Message Batches API.

The usual traps

• Opening a websocket from the browser to the API, or putting the key in the frontend.
• Keeping the defaults when there is a 30-second limit in front.
• Adding your own retry loop on top of the SDK’s.
• Saving a response cut short by an error event as complete.
• Parsing each input_json_delta fragment separately.
• Believing streaming is only useful for showing progress.

On the exam

Look for the limit that rules the scenario: a gateway, a user watching the screen, a request that takes minutes. Then pick the option that tunes the SDK to that limit (timeout, retries, streaming with the final message) rather than the one that switches models or rewrites the integration from scratch.