Networking Stack in OS (Socket APIs)

Networking Stack in OS (Socket APIs)

Why This Matters

Think of the networking stack like a postal system. You write a letter (application data), put it in an envelope with an address (TCP/IP headers), and drop it in a mailbox (socket). The postal system (OS networking stack) handles routing, delivery, and reliability. Socket APIs are the interface to this system—they let applications send and receive data over networks.

This matters because network programming is fundamental to modern applications. Web servers, APIs, microservices—they all use sockets to communicate. Understanding the networking stack helps you write efficient network code, debug connectivity issues, and design distributed systems.

In interviews, when someone asks "How does a web server handle requests?", they're testing whether you understand sockets and the networking stack. Do you know how sockets work? Do you understand TCP/IP? Most engineers don't. They just use HTTP libraries and assume they work.

What Engineers Usually Get Wrong

Most engineers think "sockets are just for network programming." But sockets are the interface between applications and the OS networking stack. When you create a socket, bind it to an address, and listen for connections, you're using the OS networking stack. Understanding this helps you understand how network applications work.

Engineers also don't understand that the networking stack has layers (application, transport, network, link, physical). Your application uses sockets (application layer), which use TCP/IP (transport/network layers), which use the network interface (link/physical layers). Understanding these layers helps you debug network issues and optimize performance.

How This Breaks Systems in the Real World

A service was creating many sockets but not closing them. Each socket consumes resources (file descriptors, memory). Over time, sockets accumulated, exhausting resources. The service ran out of file descriptors and couldn't accept new connections. The fix? Always close sockets when done. Use connection pooling to reuse sockets. Monitor socket usage and set limits.

Another story: A service was using blocking sockets. Each socket operation blocked the thread until completion. With many concurrent connections, threads were blocked, and the system became unresponsive. The fix? Use non-blocking sockets or async I/O. Don't block threads—use event loops or async/await. This allows the system to handle many connections efficiently.

---

Networking Stack Layers

OSI Model (7 layers):

1. Application: Sockets API (user programs)
2. Transport: TCP/UDP (reliability, flow control)
3. Network: IP (routing, addressing)
4. Link: Ethernet (frame delivery)
5. Physical: Hardware (cables, signals)

TCP/IP Model (4 layers):

1. Application: HTTP, FTP, SSH (sockets)
2. Transport: TCP, UDP
3. Internet: IP
4. Link: Ethernet, Wi-Fi

---

Socket APIs

Socket Lifecycle:

1. socket(): Create socket
2. bind(): Bind to address/port
3. listen(): Listen for connections (server)
4. accept(): Accept connection (server)
5. connect(): Connect to server (client)
6. send()/recv(): Send/receive data
7. close(): Close socket

Example (TCP Server):

// Create socket
int sock = socket(AF_INET, SOCK_STREAM, 0);

// Bind to address
struct sockaddr_in addr;
addr.sin_family = AF_INET;
addr.sin_port = htons(8080);
addr.sin_addr.s_addr = INADDR_ANY;
bind(sock, (struct sockaddr*)&addr, sizeof(addr));

// Listen for connections
listen(sock, 10);

// Accept connection
int client = accept(sock, NULL, NULL);

// Send/receive data
send(client, "Hello", 5, 0);
recv(client, buffer, 1024, 0);

// Close
close(client);
close(sock);

---

Examples

Example 1: Blocking vs Non-blocking Sockets

Blocking socket (default):

// Blocks until data arrives
recv(sock, buffer, 1024, 0);
// Thread blocked, can't handle other connections

Non-blocking socket:

// Set non-blocking
fcntl(sock, F_SETFL, O_NONBLOCK);

// Returns immediately (EAGAIN if no data)
int n = recv(sock, buffer, 1024, 0);
if (n < 0 && errno == EAGAIN) {
    // No data available, continue other work
}

Benefit: Non-blocking allows one thread to handle many connections

Example 2: Socket Resource Exhaustion

Problem: Creating sockets but not closing them

while (1) {
    int sock = socket(...);  // Creates socket
    connect(sock, ...);
    // Forgot to close!
    // File descriptors exhausted
}

Solution: Always close sockets

int sock = socket(...);
connect(sock, ...);
// ... use socket ...
close(sock);  // Always close!

Example 3: Connection Pooling

Without pooling (creates new socket per request):

Request 1 → socket() → connect() → send() → close()
Request 2 → socket() → connect() → send() → close()
// Slow: socket creation/connection overhead

With pooling (reuses sockets):

Request 1 → get from pool → send() → return to pool
Request 2 → get from pool → send() → return to pool
// Fast: reuse existing connections

---

Common Pitfalls

Pitfall 1: Not closing sockets

• Problem: Sockets consume file descriptors, exhausting system resources
• Solution: Always close sockets when done, use connection pooling
• Example: Creating sockets without closing exhausts file descriptor limit (typically 1024)

Pitfall 2: Using blocking sockets for high concurrency

• Problem: Blocking sockets block threads, limiting concurrency
• Solution: Use non-blocking sockets or async I/O (epoll, kqueue, async/await)
• Example: Blocking sockets limit server to ~1000 connections (one thread per connection)

Pitfall 3: Not handling socket errors

• Problem: Socket operations can fail (network errors, timeouts)
• Solution: Always check return values, handle errors appropriately
• Example: Assuming send() always succeeds can cause silent failures

Pitfall 4: Not setting socket options

• Problem: Default socket options may not be optimal
• Solution: Set appropriate options (SO_REUSEADDR, SO_KEEPALIVE, TCP_NODELAY)
• Example: Not setting SO_REUSEADDR causes "Address already in use" errors

Pitfall 5: Ignoring network stack layers

• Problem: Not understanding how layers interact causes debugging difficulties
• Solution: Understand TCP/IP stack, use appropriate tools (tcpdump, netstat)
• Example: Not understanding TCP flow control causes performance issues

---

Interview Questions

Beginner

Q: What are sockets and how do they work?

A: Sockets are the interface between applications and the OS networking stack. They provide a way for programs to send and receive data over networks. The socket lifecycle involves: creating a socket (socket()), binding to an address (bind()), listening for connections (listen() for servers) or connecting (connect() for clients), sending/receiving data (send()/recv()), and closing (close()). Sockets abstract the networking stack layers (TCP/IP) and provide a simple API for network programming.

---

Intermediate

Q: What is the difference between blocking and non-blocking sockets, and when would you use each?

A: Blocking sockets cause the calling thread to wait until the operation completes. For example, recv() blocks until data arrives. Non-blocking sockets return immediately, returning an error (EAGAIN) if the operation can't complete immediately.

Blocking sockets:

• Use for: Simple applications, low concurrency, sequential processing
• Advantage: Simple to program
• Disadvantage: One thread per connection, poor scalability

Non-blocking sockets:

• Use for: High-concurrency servers, event-driven architectures
• Advantage: One thread can handle many connections
• Disadvantage: More complex (need event loops, state management)

Example: A web server handling 10,000 connections needs non-blocking sockets (or async I/O) because creating 10,000 threads is impractical.

---

Senior

Q: How would you design a high-performance network server that handles 100,000 concurrent connections efficiently?

A: I would use an event-driven architecture:

1. Non-blocking I/O:

   • Use non-blocking sockets for all connections
   • Use epoll (Linux) or kqueue (BSD) for efficient event notification
   • One thread can handle thousands of connections

2. Event loop:

   • Single event loop thread (or worker threads)
   • Register socket events (read, write, error)
   • Process events as they occur

3. Connection management:

   • Use connection pools to reuse sockets
   • Implement connection limits and timeouts
   • Handle connection lifecycle (connect, idle, close)

4. Memory management:

   • Pre-allocate buffers for socket I/O
   • Use buffer pools to reduce allocation overhead
   • Implement zero-copy where possible (sendfile)

5. Load balancing:

   • Use multiple worker threads/processes
   • Distribute connections across workers
   • Use lock-free data structures for shared state

6. Performance optimization:

   • Use TCP_NODELAY to reduce latency
   • Implement read-ahead and write-batching
   • Use scatter-gather I/O (readv/writev) for efficiency

7. Monitoring:

   • Track connection count, throughput, latency
   • Monitor socket resource usage
   • Detect and handle connection issues

This design can handle 100,000+ concurrent connections efficiently on a single server.

---

• Sockets: Interface between applications and OS networking stack

• Socket APIs: socket, bind, listen, accept, connect, send, recv, close

• Networking stack layers: Application (sockets), Transport (TCP/UDP), Network (IP), Link (Ethernet), Physical

• Blocking vs non-blocking: Blocking (simple but blocks threads), non-blocking (enables concurrency)

• Socket lifecycle: Create → bind → listen/connect → send/receive → close

• Best practices: Close sockets when done, use connection pooling, handle errors and timeouts

• System Calls - How socket operations are implemented through system calls

• I/O Management - How network I/O is managed by the OS

• Interrupts and Traps - How network interrupts trigger packet processing

• Process vs Thread - How blocking sockets affect processes and threads

• Context Switching - How network I/O blocking triggers context switches