Real-Time Distributed Chat System

Role: Personal project · Year: 2024 - 2025 · Status: shipped

tldr: A chat system built to study what breaks when WebSocket traffic has to live across several servers. Goroutines handle connections, Redis Pub/Sub fans messages out between nodes, and a small custom load balancer sits in front. The interesting part isn’t the chat. It’s what stops working when you go from one node to two.

Problem

A single-node chat server keeps every connection in memory, and that works. Add a second node and state fragments: a user on server A can’t reach a user on server B, because A has no handle on B’s sockets. Connection management has to be separated from message distribution.

Constraints

• Sockets can’t move. A WebSocket is a long-lived TCP connection pinned to one process. Moving it means a disconnect.
• Memory per node is tight. No heavy framework layer between the socket and the chat logic.
• Order and tracking. Messages have to arrive in order, and the system has to keep track of who is connected where as nodes join and leave.

Decisions

• Go for the servers. Goroutines keep idle connections cheap and avoid callback-style event loops.
• Redis Pub/Sub as the bus. Nodes don’t synchronize client lists. A message that lands on server A is published to a channel, and server B receives it and writes it to its own local clients.
• Docker for the topology, so going from one node to several is a repeatable local setup.

Outcome

In horizontal scaling tests each node held 500+ concurrent connections, with idle goroutines under 10 KB each. The main trade-off is delivery. Redis Pub/Sub is at-most-once: a node that is restarting when a message is published never sees it. For a study of fan-out I accepted that and chose eventual consistency over distributed locking. Durable delivery would mean moving to Redis Streams or a log with consumer offsets.