Real-Time Applications

Use WebSocket when you need full-duplex communication: both client and server send messages continuously (chat, collaborative editing, multiplayer games). Use SSE when data flows primarily from server to client: live feeds, AI streaming responses, notification pushes. SSE is simpler to implement, works over standard HTTP, and handles reconnection automatically; WebSocket offers lower overhead at high frequency but requires more infrastructure to scale horizontally.

WebSocket connections are stateful: a client connected to instance A can't receive messages published to instance B without a shared message bus. We configure Redis Pub/Sub or NATS as the inter-instance messaging layer: when a message targets a connection on instance B, instance A publishes it to Redis/NATS and instance B delivers it to the client. This allows true stateless horizontal scaling behind a load balancer with sticky sessions.

We use the OpenAI or Anthropic streaming API endpoints which emit token-by-token Server-Sent Events. On the Next.js side, we use a Route Handler that reads the streaming response and pipes it to the client. The React UI updates the visible text incrementally using state management that appends tokens as they arrive. We include abort controls (stop generating), error recovery (retry on network interruption), and proper loading/done states.

We implement exponential backoff reconnection logic: the client attempts to reconnect after 1s, 2s, 4s, 8s (capped at ~30s) on connection loss. Missed messages during the gap are replayed from a short-lived queue (Redis Streams or a database cursor position). For collaborative features, we use CRDTs or version vectors so reconnecting clients can merge diverged state without conflict.

A single Node.js WebSocket server can handle 10,000–100,000 concurrent connections depending on message frequency and payload size. For higher scale, we use horizontal scaling with Redis Pub/Sub. We validate your target concurrency with a load test (Artillery or k6) during the delivery sprint, so you have empirical data before going live.