Networking for Developers: WebSockets Explained (When REST API Isn’t Enough)
How I watched 300,000 empty HTTP requests destroy our database — and what bidirectional connections actually fix
Networking for Developers: WebSockets Explained (When REST API Isn’t Enough)
How I watched 300,000 empty HTTP requests destroy our database — and what bidirectional connections actually fix
The difference between polling hell and persistent connections isn’t philosophical — it’s measurable in database queries per second and your AWS bill.
Our chat feature launched on a Tuesday. By Thursday, our primary database was on fire.
Not metaphorically. The CPU graphs looked like someone had drawn vertical lines in red Sharpie. I pulled the logs at night because our on-call got paged for the fifth time. Ten thousand concurrent users. Each one hitting our /messages/new endpoint every two seconds. That's 300,000 requests per minute—roughly 5,000 requests per second hitting your database. Ninety-five percent returned empty JSON arrays because the query had to check for new messages even when none existed.
The database didn’t care that most queries returned nothing. It still had to execute them.
I stared at the tcpdump output on my second monitor. TCP handshakes everywhere. Connection opened, request sent, response returned, connection closed. Two seconds later: do it again. Each request involved multiple packets for TCP handshake (SYN, SYN-ACK, ACK), HTTP headers, and connection teardown — all that overhead before you even transfer data. On slow or mobile networks, this multiplies painfully.
Our pool of 100 database connections maxed out continuously. Every poll acquired a connection, ran a query, released it. Under load, clients started timing out just waiting for an available connection.
Why Polling Is Fake Real-Time
Here’s what I assumed going in: HTTP is simple. Clients ask, servers answer. Want updates? Ask again.
Wrong. Or at least, wrong at scale.
The math is brutal. Ten thousand users polling every two seconds means your server handles five thousand requests per second doing absolutely nothing useful. We were burning CPU cycles on handshakes. Establishing a TCP connection takes those 3 packets before you even send your HTTP request, meaning each polling request incurs heavy network overhead just to ask “anything new?”
And the latency. Two-second polling means two-second worst-case lag. Send a message, wait up to two seconds for the recipient to poll and see it. Not exactly instant messaging.
I spent Tuesday night reading through our Nginx logs. Lines and lines of GET /messages/new. Status 200. Empty response body. I watched money burn in real-time. Speaking of burning money, our AWS bill had tripled in a week. The database instance kept auto-scaling because we couldn't keep up.
Actually, that’s when I realized we’d designed ourselves into a corner. Polling works fine with fifty users in testing. At ten thousand? The pattern breaks down completely.
The HTTP Upgrade Handshake
WebSockets don’t replace HTTP. They upgrade it.
The client sends what looks like a normal HTTP request, but with specific headers:
GET /chat HTTP/1.1
Host: api.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13
Clients may include Sec-WebSocket-Protocol to negotiate subprotocols like specific chat formats, but it's optional. The server responds:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
That 101 status code means "we're not doing HTTP anymore." The TCP connection that carried that HTTP request? It stays open. That's your WebSocket. No more request-response. No more opening and closing connections.
Bidirectional. The server can push data without the client asking.
This changed everything for us. Instead of 300,000 requests per minute, we had 10,000 open connections. One per user. When a message arrived, we pushed it immediately. No polling. No wasted queries. No fake real-time with multi-second lag.
What Actually Happens on the Wire
I wasn’t convinced until I saw it in Wireshark.
Our old polling system generated packet storms. I filtered to one client IP and watched: TCP handshake, HTTP request, response, FIN/ACK teardown. Two seconds of silence. Then do it all again.
With WebSockets, the initial handshake looked similar — TCP SYN, SYN-ACK, ACK, then the HTTP upgrade request. But after the 101 Switching Protocols response? The connection just sat there. Quiet.
Then a message came through. Not an HTTP request — a WebSocket frame. Each frame begins with flags and length info, followed by payload data. No HTTP headers or request lines. The frame format minimizes overhead: FIN bit (is this the final frame?), opcode (text or binary?), payload length, then the actual data. Thirty bytes total for a short message.
The client never had to ask “anything new?” The server just sent it.
Different protocols solve different problems. WebSockets aren’t always the answer — sometimes you just need events flowing one way, and SSE does that with half the complexity.
When Connections Stay Open
Each WebSocket connection holds an open file descriptor. On Linux, that’s a socket in the kernel. On typical servers, each open socket might use less than 100KB of RAM total, but tens of thousands strain file descriptor and memory limits.
Ten thousand connections meant ten thousand file descriptors. Our Node.js process held them all. I checked with lsof | grep node | wc -l. The number matched our connected user count.
This is different from HTTP request handling. With traditional request-response, you handle the request, send the response, close the socket, free the resources. With WebSockets, the resources stay allocated. The connection lives as long as the user stays connected.
We hit our first scaling wall at around 30,000 concurrent connections. AWS ALB has documented limits near 25–30k concurrent connections, and we hit it hard. Connections got distributed at connection time, not per-request. Once a user connected to one backend server, they stayed there.
User distribution became uneven. Server 1 had 15,000 connections. Server 2 had 8,000. Actually, this is where sticky sessions became critical — but also where our real problems started.
The Multi-Server Problem
Here’s where it got interesting.
User A connects to Server 1. User B connects to Server 2. User A sends a message to User B. Server 1 receives it… but User B’s WebSocket lives on Server 2. How does the message get there?
Our first attempt was naive: shared database. Server 1 writes the message to the database. Server 2 polls the database for new messages. We’d just reinvented polling at the server level. Terrible.
The answer was Redis pub/sub. Each server subscribed to a Redis channel. When Server 1 received a message, it saved to database (persistence), then published to Redis (distribution). All servers listening received it and pushed to their local connections.
const redis = require('redis');
const subscriber = redis.createClient();
const publisher = redis.createClient();
// Each server listens for messages
subscriber.subscribe('chat_messages');
subscriber.on('message', (channel, data) => {
const msg = JSON.parse(data);
const socket = connections.get(msg.recipientId); // Check local connections
if (socket && socket.readyState === WebSocket.OPEN) {
socket.send(JSON.stringify(msg)); // Push if user connected here
}
});
function sendMessage(msg) {
saveToDatabase(msg); // Persist first
publisher.publish('chat_messages', JSON.stringify(msg)); // Then broadcast
}
Redis pub/sub is fast but non-durable — messages can be lost if Redis restarts. For higher reliability, consider augmenting with persistent queues. We accepted the risk because our database had the source of truth anyway.
This let us scale horizontally. We went from three servers to fifteen without rewriting anything. Redis became our message bus.
Keeping Connections Alive
WebSocket connections don’t last forever. Networks fail. NAT gateways time out idle connections. Proxies close sockets they think are dead. Heartbeats detect silent drops caused by NAT timeouts, proxy disconnects, or mobile network transitions — basically anywhere the network kills your connection without telling you.
We implemented heartbeats. Every thirty seconds, the server sent a ping frame. The client responded with a pong. If we didn’t get a pong within ten seconds, we assumed the connection was dead.
const PING_INTERVAL = 30000; // 30 seconds between pings
const PONG_TIMEOUT = 10000; // 10 seconds to wait for pong
function setupHeartbeat(ws) {
let pongReceived = true;
const interval = setInterval(() => {
if (!pongReceived) {
ws.terminate(); // Connection dead, clean up
clearInterval(interval);
return;
}
pongReceived = false; // Assume dead until proven alive
ws.ping(); // Send ping frame
}, PING_INTERVAL);
ws.on('pong', () => pongReceived = true); // Got response, connection alive
}
The ping/pong frames are built into the WebSocket protocol. The browser’s WebSocket API handles pongs automatically — you never see them in JavaScript. On the server, you have to listen for them.
This solved our zombie connection problem. Before heartbeats, our connection counts slowly drifted upward as mobile clients disconnected without closing cleanly.
When WebSockets Are Wrong
I got excited and tried to use WebSockets for everything. That was a mistake.
Our API had a /profile endpoint. GET request, returns user data. Some product manager suggested making it "real-time" with WebSockets so profile updates would push instantly. I implemented it. It was stupid.
Profiles don’t update often. Maybe once per session. Maybe never. Each idle WebSocket connection consumes precious memory and CPU cycles. For infrequent updates, polling or SSE may be far more resource-friendly. Holding open a WebSocket just to push rare profile updates was wasteful. Each connection costs memory. Each connection uses a file descriptor. For what? So we could save one HTTP request?
I reverted it. Went back to REST for profile data.
WebSockets make sense when updates are frequent (messages every few seconds), latency matters (gaming, collaborative editing), or bidirectional communication is actually needed. They don’t make sense when updates are rare, the flow is purely one-way (use Server-Sent Events instead), or you’re just trying to look fancy.
The Gotcha That Cost Me a Week
Our WebSocket connections kept dying after exactly sixty seconds.
I checked our heartbeat logic. Fine. I checked AWS load balancer settings. Fine. I checked nginx. Fine. I checked our application code for timeouts. Nothing.
Then I looked at our database connection pooling. We acquired a database connection when a WebSocket connected — to load initial state. We never released it. The pool had a sixty-second idle timeout. This 60-second cutoff matched our database pool timeout, not WebSocket or network defaults. When the connection died, it triggered an error that crashed the WebSocket.
The fix was embarrassing:
// WRONG - holds DB connection forever
wss.on('connection', async (ws) => {
const dbConn = await pool.acquire(); // Get connection
const initialData = await dbConn.query('SELECT * FROM messages WHERE...');
ws.send(JSON.stringify(initialData));
// Never released! Connection held until timeout
});
// RIGHT - release immediately after use
wss.on('connection', async (ws) => {
const dbConn = await pool.acquire();
const initialData = await dbConn.query('SELECT * FROM messages WHERE...');
dbConn.release(); // THIS LINE - release after query
ws.send(JSON.stringify(initialData));
});
Seven days debugging that. The logs said “database connection timeout” but I kept looking at WebSocket configuration. Wrong layer entirely. I assumed the connection would live for the WebSocket lifetime. It shouldn’t have. Database connections are for queries, not storage.
Start Here
If you’re building something new and you think you need WebSockets, start with REST and polling. Measure your polling rate and empty response percentage. When empty responses exceed 80% or your RPS becomes painful, switch.
Use Socket.IO — it handles fallbacks and makes the client code simple. Plan for horizontal scaling from day one with Redis pub/sub. Implement heartbeats immediately.
The debugging tools that saved me:
# Watch packets in real-time
tcpdump -i eth0 port 443
# Count active connections
netstat -an | grep ESTABLISHED | wc -l
# Monitor Redis pub/sub traffic
redis-cli MONITOR
Networks fail. Connections drop. Clients disappear without saying goodbye. Design for that reality and you’ll be fine.
Tomorrow I’ll write about the WebSocket scaling crisis we hit at 100K concurrent connections, and why we had to rethink everything about our infrastructure again.
Enjoyed the read? Let’s stay connected!
- 🚀 Follow The Speed Engineer for more Rust, Go and high-performance engineering stories.
- 💡 Like this article? Follow for daily speed-engineering benchmarks and tactics.
- ⚡ Stay ahead in Rust and Go — follow for a fresh article every morning & night.
Your support means the world and helps me create more content you’ll love. ❤️