Three-Way Handshake (TCP)

Three-Way Handshake (TCP)

Why This Matters

Think of the three-way handshake like a phone call. Before you can talk, you need to establish that both parties are ready. You say "Hello?" (SYN), the other person says "Hello, I can hear you" (SYN-ACK), and you say "Great, I can hear you too" (ACK). Only then do you start talking. TCP does the same thing—it establishes that both client and server are ready before sending data.

This matters because every TCP connection requires this handshake. For short-lived connections, the handshake overhead can be significant (one round trip = 50-100ms). If you're creating many connections, this overhead adds up. Understanding the handshake helps you optimize connection usage (connection pooling, HTTP keep-alive).

In interviews, when someone asks "How would you optimize a service that makes many network calls?", they're testing whether you understand the TCP handshake overhead. Do you know why connection pooling helps? Do you understand keep-alive? Most engineers don't. They create new connections for each request and wonder why it's slow.

What Engineers Usually Get Wrong

Most engineers think "TCP connections are instant." But TCP requires a three-way handshake, which takes one round trip (50-100ms on typical networks). For short-lived connections, this overhead is significant. Also, each connection consumes resources (memory, file descriptors). Creating too many connections can exhaust system resources.

Engineers also don't understand that the handshake establishes sequence numbers. These numbers are used to ensure reliable, ordered delivery. If the handshake fails, no connection is established, and no data can be sent. This is why connection failures can be silent—the handshake fails, but your application might not know why.

How This Breaks Systems in the Real World

A microservice was making HTTP requests to another service. Each request opened a new TCP connection, sent data, then closed the connection. Under normal load, this worked fine. But during a traffic spike, the service tried to open 10,000 connections per second. Each connection required a three-way handshake (one round trip = 50ms). The handshake queue filled up. New connections timed out. The service became unresponsive.

The fix? Use HTTP keep-alive and connection pooling. Reuse TCP connections instead of creating new ones for each request. This reduced connection overhead from 10,000 handshakes/second to just maintaining a pool of connections. But the real lesson is: TCP connections have overhead. Reuse them.

Another story: A service was creating TCP connections without timeouts. When the server was slow to respond to SYN packets, connections hung in SYN_SENT state. Over time, thousands of connections accumulated in this state, consuming resources. The service ran out of file descriptors. The fix? Always set connection timeouts. If the handshake doesn't complete within the timeout, fail fast and retry.

The Process

Step 1: SYN (Synchronize)

Client → Server

• Client sends SYN packet with initial sequence number (ISN)
• Client enters SYN_SENT state

Step 2: SYN-ACK (Synchronize-Acknowledge)

Server → Client

• Server receives SYN, sends SYN-ACK
• Server sends its own ISN and acknowledges client's ISN + 1
• Server enters SYN_RECEIVED state

Step 3: ACK (Acknowledge)

Client → Server

• Client receives SYN-ACK, sends ACK
• Client acknowledges server's ISN + 1
• Both enter ESTABLISHED state
• Connection is now ready for data transfer

Sequence Diagram

Client                    Server
  |                         |
  |-------- SYN (seq=x) ---->|
  |                         | (SYN_RECEIVED)
  |<-- SYN-ACK (seq=y,      |
  |     ack=x+1) -----------|
  | (SYN_SENT)              |
  |-------- ACK (ack=y+1) -->|
  | (ESTABLISHED)           | (ESTABLISHED)
  |                         |
  |<===== Data Transfer ===>|

Examples

TCP Connection in Code

1// Client side
2import net from 'net';
3
4const socket = new net.Socket();
5socket.connect(80, 'example.com', () => {
6  // Three-way handshake completed
7  console.log('Connected!');
8  socket.write('GET / HTTP/1.1\r\n\r\n');
9});

Wireshark Capture

1. Client → Server: SYN, Seq=1000
2. Server → Client: SYN-ACK, Seq=5000, Ack=1001
3. Client → Server: ACK, Ack=5001

Common Pitfalls

• SYN flood attacks: Too many SYN packets without completing handshake. Fix: SYN cookies, rate limiting
• Not handling timeouts: Client waits forever if server doesn't respond. Fix: Set connection timeout
• Sequence number confusion: Not understanding ISN selection. Fix: Random ISN for security
• State machine errors: Not tracking connection state correctly. Fix: Proper state management

Interview Questions

Beginner

Q: What is the TCP three-way handshake and why is it needed?

A: The three-way handshake is the process TCP uses to establish a connection:

1. Client sends SYN: "I want to connect, my sequence number is X"
2. Server sends SYN-ACK: "I acknowledge your SYN (X+1), my sequence number is Y"
3. Client sends ACK: "I acknowledge your SYN (Y+1)"

Why needed:

• Synchronize sequence numbers: Both sides agree on starting sequence numbers
• Establish connection: Both sides confirm they're ready to communicate
• Reliability: Ensures both sides can send and receive data

Intermediate

Q: What happens if the ACK packet in the three-way handshake is lost?

A:

Scenario: Client sends ACK, but it's lost in transit.

What happens:

• Server doesn't receive ACK, remains in SYN_RECEIVED state
• Server will retransmit SYN-ACK (with exponential backoff)
• Client receives duplicate SYN-ACK, sends another ACK
• Connection eventually establishes

Timeout handling:

• Server has timeout (typically 30-120 seconds)
• If no ACK received, server closes connection
• Client can retry connection

Key point: TCP is reliable - lost packets are retransmitted until acknowledged.

Senior

Q: Design a high-performance TCP server that handles millions of connections. How do you optimize the three-way handshake? Handle SYN flood attacks?

A:

Optimizations:

1class HighPerformanceTCPServer {
2  // 1. SYN Cookies (prevent SYN flood)
3  async handleSYN(synPacket: SYN): Promise<void> {
4    // Instead of storing connection state, use SYN cookie
5    const cookie = this.generateSYNCookie(synPacket);
6    
7    // Send SYN-ACK with cookie (no state stored)
8    await this.sendSYNACK(synPacket, cookie);
9  }
10
11  generateSYNCookie(syn: SYN): number {
12    // Cryptographic hash of client info + secret
13    return hash(syn.sourceIP + syn.sourcePort  secret

SYN Flood Protection:

1. SYN Cookies: Don't store state until ACK received
2. Rate limiting: Limit SYNs per IP
3. Connection limits: Max connections per IP
4. Firewall rules: Block suspicious IPs
5. DDoS protection: Use CDN/proxy for filtering

Performance:

• Epoll/kqueue: Efficient I/O multiplexing
• Zero-copy: Minimize data copying
• Connection reuse: Keep-alive, connection pooling
• Worker processes: Distribute load across CPUs

Failure Stories You'll Recognize

The Connection Storm: A microservice was making HTTP requests to another service. Each request opened a new TCP connection, sent data, then closed the connection. Under normal load, this worked fine. But during a traffic spike, the service tried to open 10,000 connections per second. Each connection required a three-way handshake (one round trip = 50ms). The handshake queue filled up. New connections timed out. The service became unresponsive. The fix? Use HTTP keep-alive and connection pooling. Reuse TCP connections instead of creating new ones for each request. This reduced connection overhead from 10,000 handshakes/second to just maintaining a pool of connections.

The Hanging Connections: A service was creating TCP connections without timeouts. When the server was slow to respond to SYN packets, connections hung in SYN_SENT state. Over time, thousands of connections accumulated in this state, consuming resources. The service ran out of file descriptors. The fix? Always set connection timeouts. If the handshake doesn't complete within the timeout, fail fast and retry.

The SYN Flood: A service was hit by a SYN flood attack. Attackers sent many SYN packets but never completed the handshake. The server allocated resources for each SYN packet, exhausting its resources. The service became unavailable. The fix? Use SYN cookies or rate limiting. SYN cookies allow the server to handle SYN packets without allocating resources until the handshake completes.

What Interviewers Are Really Testing

They want to hear you talk about TCP handshake overhead and optimization strategies. Junior engineers say "TCP establishes connections." Senior engineers say "TCP handshake takes one round trip (50-100ms). For short-lived connections, this overhead is significant. Use connection pooling and keep-alive to reuse connections. Always set timeouts to prevent hanging connections."

When they ask "How would you optimize a service that makes many network calls?", they're testing:

• Do you understand TCP handshake overhead?

• Do you know why connection pooling helps?

• Can you optimize connection usage?

• TCP vs UDP - Understanding TCP's connection-oriented nature explains why the three-way handshake is necessary

• TCP Connection Termination - Learn how TCP connections are closed, complementing the connection establishment process

• Connection Pooling - Optimize TCP connection reuse to minimize the overhead of repeated three-way handshakes

• HTTP/1 vs HTTP/2 vs HTTP/3 - HTTP connections use TCP, understanding the handshake helps optimize HTTP performance

• OSI Model (7 Layers) - The three-way handshake occurs at Layer 4 (Transport), understanding the OSI model provides context

• Three-way handshake establishes TCP connection: SYN → SYN-ACK → ACK

• Synchronizes sequence numbers: Both sides agree on starting sequence numbers

• Reliable: Lost packets are retransmitted until acknowledged

• State tracking: Both sides track connection state (SYN_SENT, SYN_RECEIVED, ESTABLISHED)

• Security: Random ISN prevents sequence number prediction attacks

• Optimization: SYN cookies, TCP Fast Open, connection pooling for performance

• SYN flood protection: Rate limiting, SYN cookies, connection limits

How InterviewCrafted Will Teach This

We'll teach this through production failures, not protocol specifications. Instead of memorizing "SYN, SYN-ACK, ACK," you'll learn through scenarios like "why did our service become unresponsive when we tried to create 10,000 connections per second?"

You'll see how TCP handshake overhead affects performance and system design. When an interviewer asks "how would you optimize a service that makes many network calls?", you'll think about connection pooling, keep-alive, and handshake overhead—not just "TCP establishes connections."