Scaling Real-Time Systems

A single server can handle thousands of WebSocket connections, but eventually you'll need multiple servers for reliability and capacity. This creates a coordination problem: how does a message from a user on Server 1 reach users connected to Server 2?

The Scaling Challenge

Each server maintains its own set of WebSocket connections. When User A (connected to Server 1) sends a message to a chat room, Server 1 can broadcast to its local connections. But User B might be connected to Server 2 — Server 1 doesn't know about that connection.

User A → Server 1 → [broadcasts locally]
                    User B on Server 2 never receives the message

You need a way for servers to communicate with each other.

The Pub/Sub Solution

Publish/subscribe (pub/sub) messaging solves this elegantly. All servers subscribe to shared channels. When any server receives a message, it publishes to the channel. All subscribed servers receive it and broadcast to their local connections:

User A → Server 1 → Redis Pub/Sub → Server 1 → Local users
                         ↓
                    Server 2 → Local users (including User B)
                         ↓
                    Server 3 → Local users

Redis Pub/Sub Implementation

Redis provides simple, fast pub/sub:

import aioredis

class ScalableConnectionManager:
    def __init__(self):
        self.redis = aioredis.from_url("redis://localhost")
        self.local_connections = {}
    
    async def subscribe_to_room(self, room: str):
        pubsub = self.redis.pubsub()
        await pubsub.subscribe(f"room:{room}")
        
        # Listen for messages from other servers
        async for message in pubsub.listen():
            if message["type"] == "message":
                await self.broadcast_locally(
                    room, 
                    message["data"]
                )
    
    async def send_to_room(self, room: str, message: str):
        # Publish to all servers
        await self.redis.publish(f"room:{room}", message)
    
    async def broadcast_locally(self, room: str, message: str):
        for conn in self.local_connections.get(room, []):
            await conn.send_text(message)

Each server publishes messages to Redis and listens for messages from other servers. Redis handles the fan-out.

Alternative Approaches

Socket.IO with Redis adapter handles this automatically. It's a popular choice that abstracts away the complexity while providing fallbacks for environments where WebSockets don't work.

Managed services like Pusher, Ably, or AWS IoT Core handle scaling entirely. You publish messages to their API, and they deliver to connected clients. This trades control for simplicity.

Cloud pub/sub services like Google Cloud Pub/Sub or AWS SNS can coordinate servers, though they add latency compared to Redis.

Considerations

Pub/sub adds latency — messages travel through the broker before reaching recipients. For most chat applications, this is imperceptible. For real-time games, you might need more sophisticated solutions.

Also consider message ordering. Pub/sub doesn't guarantee order across servers. If strict ordering matters, you'll need additional coordination.

See More

• WebSocket Fundamentals
• Building Chat Applications
• Message Queues Explained
• Horizontal and Vertical Scaling

Further Reading

• Redis Pub/Sub Documentation