Request Timeouts: Prevent Cascades & Resource Exhaustion

Request Timeouts

Why Engineers Care About This

Timeouts prevent requests from hanging indefinitely. When a dependency is slow or down, timeouts fail fast instead of waiting forever. This prevents resource exhaustion (connections held open, memory used) and enables graceful degradation (fail fast, use fallbacks). But timeouts require careful configuration—too short causes premature failures, too long causes resource exhaustion.

When requests hang indefinitely, or resources are exhausted by slow requests, or cascading timeouts cause system-wide failures, you're hitting timeout problems. These problems compound. Without timeouts, slow dependencies cause requests to hang, consuming resources. With poorly configured timeouts, requests fail prematurely or hang too long. Good timeouts prevent these problems by failing fast when dependencies are slow.

In interviews, when someone asks "How would you handle slow dependencies?", they're really asking: "Do you understand timeouts? Do you know how to configure timeout durations? Do you understand cascading timeouts?" Most engineers don't. They use default timeouts (often too long) or don't set timeouts at all.

Core Intuitions You Must Build

• Timeouts should be set at multiple layers. Client timeouts (how long client waits for response), server timeouts (how long server processes request), database timeouts (how long database query runs). Each layer needs timeouts—client timeout should be longer than server timeout (allow server to process), server timeout should be longer than database timeout (allow query to complete). Don't set timeouts at only one layer—set them at all layers.

• Timeout duration depends on operation type. Fast operations (cache lookups, simple queries) need short timeouts (100-500ms). Slow operations (complex queries, external API calls) need longer timeouts (1-5 seconds). Very slow operations (batch processing, reports) need very long timeouts (30-60 seconds). Configure timeouts based on expected operation time—don't use one timeout for everything.

• Cascading timeouts cause system-wide failures. When service A calls service B with a 5-second timeout, and service B calls service C with a 5-second timeout, service A must wait up to 10 seconds (sum of timeouts). If service C is slow, service B times out, then service A times out. This cascades through the system. Design timeouts to prevent cascading—use shorter timeouts for dependencies, or implement circuit breakers to fail fast.

• Timeout errors should be handled gracefully. When timeouts occur, don't just return errors—handle them gracefully. Use retries for transient timeouts (network hiccups), fail fast for persistent timeouts (dependency is down), or use fallbacks (cached data, default values). Also, return clear timeout error messages—tell clients what timed out and when to retry.

• Resource exhaustion is prevented by timeouts. Without timeouts, slow requests hold resources (connections, memory, threads) indefinitely. This exhausts resources, preventing new requests from being processed. Timeouts free resources when requests are slow, preventing exhaustion. Set timeouts to prevent resource exhaustion—don't let requests hang indefinitely.

• Timeout configuration should be tunable. Timeout durations should be configurable (environment variables, configuration files), not hardcoded. This enables tuning based on actual performance (increase timeouts if too many premature failures, decrease if too many resource issues). Also, monitor timeout rates—high timeout rates indicate timeouts are too short or dependencies are slow.

Subtopics (Taught Through Real Scenarios)

Multi-Layer Timeouts

What people usually get wrong:

Engineers often set timeouts at only one layer (e.g., client timeout). But timeouts are needed at multiple layers—client, server, database. Each layer needs appropriate timeouts—client timeout should be longer than server timeout (allow server to process), server timeout should be longer than database timeout (allow query to complete). Don't set timeouts at only one layer—set them at all layers.

How this breaks systems in the real world:

A service set only client timeouts (5 seconds). When database queries were slow (3 seconds), requests completed but were slow. When database queries were very slow (10 seconds), client timeouts occurred, but server continued processing (wasting resources). The fix? Set timeouts at all layers—client (5 seconds), server (3 seconds), database (2 seconds). Now timeouts are coordinated, and resources are freed when operations are slow. But the real lesson is: timeouts are needed at multiple layers. Set them at all layers.

What interviewers are really listening for:

They want to hear you talk about multi-layer timeouts, timeout coordination, and layer-specific timeouts. Junior engineers say "just set client timeout." Senior engineers say "set timeouts at multiple layers (client, server, database)—client timeout should be longer than server timeout, server timeout should be longer than database timeout, to allow operations to complete." They're testing whether you understand that timeouts are about coordination, not just "setting a value."

Cascading Timeouts

What people usually get wrong:

Engineers often don't consider cascading timeouts. When service A calls service B with timeout T1, and service B calls service C with timeout T2, service A must wait up to T1 + T2. If service C is slow, timeouts cascade (service B times out, then service A times out). Design timeouts to prevent cascading—use shorter timeouts for dependencies, or implement circuit breakers to fail fast.

How this breaks systems in the real world:

A service called another service with a 5-second timeout. That service called a database with a 5-second timeout. When the database was slow (4 seconds), the second service timed out (5 seconds exceeded), then the first service timed out (5 seconds exceeded). Timeouts cascaded, causing system-wide failures. The fix? Use shorter timeouts for dependencies (2 seconds for database, 3 seconds for service) or implement circuit breakers to fail fast. Now timeouts don't cascade. But the real lesson is: cascading timeouts cause system-wide failures. Design timeouts to prevent cascading.

What interviewers are really listening for:

They want to hear you talk about cascading timeouts, timeout coordination, and prevention strategies. Junior engineers say "just set timeouts." Senior engineers say "cascading timeouts occur when service timeouts sum (A calls B with T1, B calls C with T2, A waits T1+T2)—prevent by using shorter timeouts for dependencies or circuit breakers to fail fast." They're testing whether you understand that timeouts can cascade and must be designed to prevent it.

Timeout Configuration

What people usually get wrong:

Engineers often hardcode timeout values. But timeout durations should be configurable (environment variables, configuration files) to enable tuning based on actual performance. Also, different operations need different timeouts—fast operations need short timeouts, slow operations need long timeouts. Don't use one timeout for everything—configure timeouts based on operation types.

How this breaks systems in the real world:

A service used hardcoded timeouts (5 seconds for all operations). Fast operations (cache lookups, 10ms) had unnecessarily long timeouts. Slow operations (external API calls, 8 seconds) had too short timeouts, causing premature failures. The fix? Make timeouts configurable and set different timeouts for different operations—short timeouts (100ms) for fast operations, long timeouts (10 seconds) for slow operations. Now timeouts are appropriate for each operation. But the real lesson is: timeout configuration should be flexible. Different operations need different timeouts.

What interviewers are really listening for:

They want to hear you talk about timeout configuration, operation-specific timeouts, and tunability. Junior engineers say "just use 5 seconds for everything." Senior engineers say "make timeouts configurable and set different timeouts for different operations—fast operations need short timeouts, slow operations need long timeouts—to optimize for each operation type." They're testing whether you understand that timeout configuration is about optimization, not just "setting a value."

---

• Timeouts should be set at multiple layers—client, server, database, each with appropriate durations
• Timeout duration depends on operation type—fast operations need short timeouts, slow need long
• Cascading timeouts cause system-wide failures—design timeouts to prevent cascading
• Timeout errors should be handled gracefully—retry, fail fast, or use fallbacks
• Resource exhaustion is prevented by timeouts—free resources when requests are slow
• Timeout configuration should be tunable—make timeouts configurable for optimization
• Good timeouts prevent resource exhaustion and enable graceful degradation

---

• Error Handling & Logging - Handling timeout errors
• Circuit Breakers - Using circuit breakers with timeouts
• System Design - Designing systems with proper timeouts