Scalable WebSockets Chat Application

A highly scalable real-time chat application built with modern technologies and a distributed architecture approach.

🚀 Overview

This project demonstrates a scalable approach to building real-time applications using WebSockets. It addresses the common challenge of scaling WebSocket connections across multiple server instances while maintaining seamless real-time communication.

🏗️ Architecture

The application uses a Redis Pub/Sub model to enable horizontal scaling of WebSocket servers:

This diagram shows how multiple server instances can handle user connections while Redis pub/sub ensures message distribution across all servers.

The architecture allows:

• Multiple Socket.IO servers to handle WebSocket connections
• Redis Pub/Sub to route messages between server instances
• Kafka for durable message processing and database persistence
• PostgreSQL for permanent storage of message history

💻 Tech Stack

Backend

• Node.js with TypeScript: For type-safe server-side code
• Socket.IO: Handling WebSocket connections
• Redis: For Pub/Sub messaging between server instances
• Kafka: For durable message processing and queueing
• Prisma ORM: For database operations
• PostgreSQL: For persistent message storage

Frontend

• Next.js: React framework for the web client
• Socket.IO Client: For WebSocket communication
• React Context: For state management

Infrastructure

• Turborepo: Monorepo management for shared code and configurations
• TypeScript: Type safety across the entire application

🔧 Scalability Features

Horizontal Scaling

• Deploy multiple Socket.IO server instances behind a load balancer
• Add more servers as user load increases
• Redis Pub/Sub ensures all instances receive all messages

Fault Tolerance

• Kafka ensures messages are not lost during service disruptions
• Automatic retries for database operations
• Message persistence across system restarts

High Performance

• Low latency with Redis for real-time communication
• High throughput with Kafka for message processing
• Asynchronous database operations

📦 Project Structure

Scaleable-WebSockets/
├── apps/
│   ├── server/         # WebSocket & messaging backend
│   ├── web/            # Next.js frontend
│   └── docs/           # Documentation site
└── packages/
    ├── eslint-config/  # Shared ESLint configs
    ├── typescript-config/ # Shared TS configs
    └── ui/             # Shared UI components

🏁 Getting Started

Prerequisites

• Node.js >= 18
• Yarn package manager
• Redis server
• Kafka broker
• PostgreSQL database

Installation

1. Clone the repository:

git clone https://github.com/rishav-RG/Scaleable-RealTime-Chat.git
cd Scaleable-WebSockets

2. Install dependencies:

yarn install

3. Set up environment variables in each app directory.

4. Start the development servers:

yarn dev

📝 How It Works

1. Users connect to any available Socket.IO server
2. When a user sends a message:
   • Their server receives it via Socket.IO
   • The server publishes it to Redis
   • All servers receive the message from Redis
   • All servers broadcast to their connected clients
   • The message is sent to Kafka for persistence
   • A Kafka consumer stores the message in PostgreSQL

This architecture ensures that all users receive all messages in real-time, regardless of which server instance they're connected to.

🛠️ Development

To build all apps and packages:

yarn build

To develop all apps and packages:

yarn dev

📄 License

MIT