Realtime Event Push Platform
Designed and implemented Realtime Event Push Platform, a low-latency, cost-efficient platform for real-time backend-to-frontend communication.
Realtime Event Push Platform Infrastructure
High-level system architecture overview
1. System Overview
Realtime Event Push Platform is a high-performance, real-time backend-to-frontend push infrastructure designed to eliminate polling and deliver instant notifications to users. Built with NATS and WebSockets, it processes events for 50K+ concurrent connections with sub-300ms latency.
Core Capabilities
- • Multi-tenant Architecture: Complete tenant isolation with custom delivery modes
- • Real-time Event Push: Persistent WebSocket connections for instant delivery
- • Smart Delivery Modes: BLIND, WITHOUT_ACK, WITH_ACK based on criticality
- • Network-aware Prioritization: Intelligent routing based on network conditions
- • Graceful Reconnection: Session recovery with message replay capabilities
Impact Metrics
- • 90% reduction in UPI transaction polling
- • 85% reduction in balance API polling
- • $15K-20K/month projected network cost savings
- • 300ms p95 latency for real-time event delivery
- • 99.9% uptime with fault-tolerant design
2. Actors & Usage Patterns
Human Actors
End Users
Receive real-time notifications within mobile and web applications
System Administrators
Monitor system health, manage configurations, and handle operational tasks
Operations Team
Handle incident response and system maintenance procedures
Tenant Administrators
Manage tenant-specific configurations and user permissions
System Actors
Upstream Services
Generate notification events and publish to Kafka topics
Client Applications
iOS, Android, and web applications maintaining persistent connections
Authentication Service
Validates JWT tokens for secure connection establishment
Load Balancers
Distribute traffic across DCM instances for optimal performance
Usage Patterns & Scale Requirements
Notification Types
- • System Alerts: Maintenance notifications, service disruptions
- • Service Events: Order updates, payment confirmations, status changes
- • Promotional: Marketing messages with targeting capabilities
- • Priority Levels: HIGH, MEDIUM, LOW with smart routing
Quality Requirements
- • Latency: <300ms P95 end-to-end delivery
- • Availability: 99.9% uptime with fault tolerance
- • Consistency: At-least-once delivery guarantee
- • Scale: 200% year-over-year growth projection
3. Architecture Design
High-Level System Architecture
Distributed real-time messaging infrastructure with NATS-based connection management
Design Goals
System Boundaries
Input Boundary
Kafka topics receiving notification events from upstream services
Processing Boundary
Internal workers, connection managers, and databases
Output Boundary
Client applications via persistent WebSocket/NATS connections
Management Boundary
Retool interfaces for operational tasks and monitoring
Core Components
Distributed Connection Manager (DCM)
- • Maintains persistent WebSocket connections
- • Routes notifications to appropriate clients
- • Handles JWT authentication and authorization
- • Tracks connection health with 30s ping intervals
- • Provides graceful shutdown with connection migration
Priority-Based Notification Worker
- • Consumes notifications from Kafka with priority ordering
- • Validates tenant permissions and configurations
- • Routes notifications to appropriate DCM instances
- • Implements exponential backoff retry logic
- • Tracks delivery metrics and status reporting
NATS Core Cluster
- • High-performance message routing backbone
- • 3-node cluster for high availability
- • Gossip protocol for cluster membership
- • Sub-millisecond message routing latency
- • Built-in clustering and load balancing
Storage Layer
- • PostgreSQL for persistent notification storage
- • Redis cluster for distributed connection state
- • AES-256 encryption for sensitive data at rest
- • Automated backup and disaster recovery
- • 90-day retention with automated archival
Data Flow & Message Routing
4. Technology Decisions & Trade-offs
Technology Stack Evaluation
| Technology | Pros | Cons | Decision |
|---|---|---|---|
| NATS Core | • High performance (M+ msg/sec) • Built-in clustering • Low latency (<1ms) • Simple deployment | • At-most-once delivery • No persistence by default • Limited ecosystem | |
| Socket.IO | • Real-time bidirectional • Fallback mechanisms • Rich ecosystem • Browser compatibility | • No horizontal scaling • Memory overhead • Sticky sessions required • Complex load balancing | |
| gRPC Streaming | • HTTP/2 based • Strong typing • Load balancing • Protobuf efficiency | • No native pub/sub • Custom connection mgmt • Higher protocol overhead • Complex implementation | |
| AWS IoT Core | • Fully managed • MQTT support • Auto-scaling • Security features | • Vendor lock-in • High costs at scale • Limited customization • Regional constraints |
Key Technology Decisions
NATS over RabbitMQ
Better clustering with gossip protocol, lower resource usage, and simpler operational model
WebSocket over HTTP/2 Streaming
Lower protocol overhead, better browser support, and simpler connection management
Redis for Distributed State
Enables stateless DCM design, provides persistence during restarts and failover
Kafka for Event Processing
Durable event processing, replay capabilities, and integration with existing systems
Implementation Trade-offs
Latency vs Durability
Choice: NATS for routing, Kafka for persistence
Trade-off: Kafka provides better durability, NATS provides sub-ms latency
Memory vs Network
Choice: Stateless design with Redis state
Trade-off: Network latency for state calls vs horizontal scalability
Complexity vs Performance
Choice: Connection pooling and load balancing
Trade-off: Increased operational complexity for better resource utilization
Protocol Comparison Matrix
| Protocol | Latency | Throughput | Scalability | Reliability | Complexity | Cost |
|---|---|---|---|---|---|---|
| NATS + WebSocket | Excellent | High | Excellent | Good | Medium | Low |
| Socket.IO | Good | Medium | Poor | Good | Low | Medium |
| AWS API Gateway | Good | High | Excellent | Excellent | Low | High |
5. Cost Analysis & Comparison
Usage Metrics (400K DAU Realistic Load)
Cost Comparison Table
| Solution | Monthly Cost | Annual Cost | vs NATS | Notes |
|---|---|---|---|---|
| Self-hosted NATS | $340 | $4,080 | Baseline | Cheapest; needs DevOps expertise |
| AWS API Gateway WebSocket | $555 | $6,660 | +63% | Fully managed; AWS-native |
| Ably Real-time Platform | $727-1,200+ | $8,724-14,400+ | +114-253% | Feature-rich but expensive at scale |
| Synadia NATS Cloud | $2,000+ | $24,000+ | +488%+ | Enterprise managed NATS |
Self-hosted NATS Breakdown
Break-even Analysis
vs AWS API Gateway
Save $215/month (39% cheaper)
Annual savings: $2,580
vs Ably Pro Plan
Save $387-860/month (53-72% cheaper)
Annual savings: $4,644-10,320
Cost Advantage Grows
Higher traffic makes self-hosted even more economical
Linear vs exponential cost scaling
Cost Sensitivity Analysis
2x Message Volume Impact (1.02B messages/month)
2x Connection Scale Impact (60k peak users)
Final Recommendation: Self-hosted NATS
Why NATS Wins
- • 63% cheaper than AWS API Gateway
- • 114% cheaper than Ably Pro Plan
- • Full control over infrastructure
- • No vendor lock-in or dependency risks
- • Scales economically with growth
- • Existing expertise in NATS operations
Cost Optimization Strategy
- • Reserved Instances: 30-40% savings on predictable workloads
- • Spot Instances: 60-70% savings for non-critical workers
- • Regional optimization: ap-south-1 already cost-effective
- • Auto-scaling: Pay only for actual usage
- • Connection pooling: Optimize resource utilization
6. Performance & Scale
Performance Targets & Achievements
Latency Breakdown
Throughput Metrics
Scale & Growth Projections
| Metric | Current | 1 Year | 2 Year | Infrastructure Impact |
|---|---|---|---|---|
| Concurrent Connections | 50,000 | 100,000 | 150,000 | Scale DCM instances to 6-9 nodes |
| Message Volume/Month | 510M | 1.02B | 1.53B | Increase worker pool capacity |
| Tenant Count | 1,000 | 3,000 | 10,000 | Database sharding consideration |
| Data Storage Growth | 5GB | 25GB | 50GB | Implement automated archival |
Load Testing Results
Peak Load Test
Successfully handled 75K concurrent connections
150% of target capacity validated
Burst Traffic
Handled 5x message spike with <500ms latency
Auto-scaling response time: 30 seconds
Failure Recovery
Node failure recovered in <10 seconds
Zero message loss during failover
Optimization Opportunities
7. Security & PII Handling
Security Architecture Overview
Authentication Layer
- • JWT-based authentication with configurable expiration
- • Token validation on every connection establishment
- • Multi-tenant isolation through JWT claims
- • Automatic token refresh mechanisms
Transport Security
- • TLS 1.3 for all client connections
- • Certificate pinning for mobile applications
- • HSTS headers for web clients
- • Perfect Forward Secrecy (PFS) enabled
Data Protection
- • AES-256 encryption for sensitive data at rest
- • Field-level encryption for PII data
- • Encrypted backup storage with key rotation
- • Secure key management with HSM integration
Access Control
- • Role-Based Access Control (RBAC) implementation
- • Tenant-based data isolation and permissions
- • API rate limiting and DDoS protection
- • Administrative access logging and monitoring
PII Data Handling (Category 2)
Sensitive Data Types
- • User identifiers and session tokens
- • Notification content with personal information
- • Connection metadata and usage patterns
- • Device identifiers and network information
Protection Measures
- • End-to-end encryption for sensitive payloads
- • PII anonymization in logs and metrics
- • Data masking for development environments
- • Regular security audits and compliance checks
Compliance & Retention
- • GDPR-compliant data handling procedures
- • 90-day retention with automated purging
- • Right to erasure implementation
- • Data portability and export capabilities
8. Deployment Strategy
Infrastructure Topology
Load Balancer Layer
- • HAProxy/NGINX for connection distribution
- • Health check configuration for DCM instances
- • SSL termination and certificate management
- • Geographic routing for latency optimization
Application Layer
- • 3-node NATS cluster for high availability
- • Auto-scaling Kubernetes pods for workers
- • Service mesh for inter-service communication
- • Rolling deployment with zero downtime
Data Layer
- • PostgreSQL cluster with read replicas
- • Redis cluster with sentinel configuration
- • Automated backup and disaster recovery
- • Data encryption at rest and in transit
Monitoring Layer
- • Prometheus/Grafana stack for observability
- • ELK stack for centralized logging
- • Distributed tracing with Jaeger
- • Alert manager for incident notifications
Operational Excellence
Deployment Strategy
- • Blue-green deployment for zero downtime
- • Canary releases for gradual rollout
- • Feature flags for controlled feature activation
- • Automated rollback on health check failures
Monitoring & Alerting
- • Real-time dashboard for system health
- • SLA-based alerting (latency, availability)
- • Capacity planning and scaling alerts
- • Security incident detection and response
Disaster Recovery
- • Multi-AZ deployment for high availability
- • Automated backup with point-in-time recovery
- • Cross-region replication for DR scenarios
- • RTO: 4 hours, RPO: 1 hour targets
Cost Optimization
- • Reserved instances for predictable workloads
- • Spot instances for batch processing
- • Auto-scaling based on demand patterns
- • Regular cost analysis and optimization