Realtime Event Push Platform

Designed and implemented Realtime Event Push Platform, a low-latency, cost-efficient platform for real-time backend-to-frontend communication.

Realtime Event Push Platform Infrastructure

High-level system architecture overview

Realtime Event Push Platform System Architecture
Scroll to zoom • Drag to pan • Click fullscreen for more options

1. System Overview

Realtime Event Push Platform is a high-performance, real-time backend-to-frontend push infrastructure designed to eliminate polling and deliver instant notifications to users. Built with NATS and WebSockets, it processes events for 50K+ concurrent connections with sub-300ms latency.

Core Capabilities

  • • Multi-tenant Architecture: Complete tenant isolation with custom delivery modes
  • • Real-time Event Push: Persistent WebSocket connections for instant delivery
  • • Smart Delivery Modes: BLIND, WITHOUT_ACK, WITH_ACK based on criticality
  • • Network-aware Prioritization: Intelligent routing based on network conditions
  • • Graceful Reconnection: Session recovery with message replay capabilities

Impact Metrics

  • • 90% reduction in UPI transaction polling
  • • 85% reduction in balance API polling
  • • $15K-20K/month projected network cost savings
  • • 300ms p95 latency for real-time event delivery
  • • 99.9% uptime with fault-tolerant design

2. Actors & Usage Patterns

Human Actors

End Users

Receive real-time notifications within mobile and web applications

System Administrators

Monitor system health, manage configurations, and handle operational tasks

Operations Team

Handle incident response and system maintenance procedures

Tenant Administrators

Manage tenant-specific configurations and user permissions

System Actors

Upstream Services

Generate notification events and publish to Kafka topics

Client Applications

iOS, Android, and web applications maintaining persistent connections

Authentication Service

Validates JWT tokens for secure connection establishment

Load Balancers

Distribute traffic across DCM instances for optimal performance

Usage Patterns & Scale Requirements

50K
Peak Concurrent Users
10K/sec
Message Ingestion Rate
1,000+
Multi-tenant Support

Notification Types

  • • System Alerts: Maintenance notifications, service disruptions
  • • Service Events: Order updates, payment confirmations, status changes
  • • Promotional: Marketing messages with targeting capabilities
  • • Priority Levels: HIGH, MEDIUM, LOW with smart routing

Quality Requirements

  • • Latency: <300ms P95 end-to-end delivery
  • • Availability: 99.9% uptime with fault tolerance
  • • Consistency: At-least-once delivery guarantee
  • • Scale: 200% year-over-year growth projection

3. Architecture Design

Realtime Event Push Platform System Architecture
Scroll to zoom • Drag to pan • Click fullscreen for more options

High-Level System Architecture

Distributed real-time messaging infrastructure with NATS-based connection management

Design Goals

High Availability: System remains operational during node failures with 99.9% uptime
Horizontal Scalability: Add capacity by scaling out DCM instances seamlessly
Multi-tenancy: Complete isolation between tenant data and configurations
Low Latency: Sub-300ms delivery for 95% of notifications
Fault Tolerance: Graceful degradation and automatic recovery mechanisms

System Boundaries

Input Boundary

Kafka topics receiving notification events from upstream services

Processing Boundary

Internal workers, connection managers, and databases

Output Boundary

Client applications via persistent WebSocket/NATS connections

Management Boundary

Retool interfaces for operational tasks and monitoring

Core Components

Distributed Connection Manager (DCM)

  • • Maintains persistent WebSocket connections
  • • Routes notifications to appropriate clients
  • • Handles JWT authentication and authorization
  • • Tracks connection health with 30s ping intervals
  • • Provides graceful shutdown with connection migration

Priority-Based Notification Worker

  • • Consumes notifications from Kafka with priority ordering
  • • Validates tenant permissions and configurations
  • • Routes notifications to appropriate DCM instances
  • • Implements exponential backoff retry logic
  • • Tracks delivery metrics and status reporting

NATS Core Cluster

  • • High-performance message routing backbone
  • • 3-node cluster for high availability
  • • Gossip protocol for cluster membership
  • • Sub-millisecond message routing latency
  • • Built-in clustering and load balancing

Storage Layer

  • • PostgreSQL for persistent notification storage
  • • Redis cluster for distributed connection state
  • • AES-256 encryption for sensitive data at rest
  • • Automated backup and disaster recovery
  • • 90-day retention with automated archival

Data Flow & Message Routing

1
Event Ingestion: Upstream services publish notification events to Kafka topics with priority classification
2
Worker Processing: Priority-based workers consume events, validate tenants, and check client online status
3
DCM Routing: Workers route notifications to appropriate DCM instances via NATS subjects
4
Client Delivery: DCM delivers notifications to clients via persistent WebSocket connections
5
Acknowledgment: Clients acknowledge receipt, DCM updates delivery status and metrics

4. Technology Decisions & Trade-offs

Technology Stack Evaluation

TechnologyProsConsDecision
NATS Core• High performance (M+ msg/sec)
• Built-in clustering
• Low latency (<1ms)
• Simple deployment
• At-most-once delivery
• No persistence by default
• Limited ecosystem
Socket.IO• Real-time bidirectional
• Fallback mechanisms
• Rich ecosystem
• Browser compatibility
• No horizontal scaling
• Memory overhead
• Sticky sessions required
• Complex load balancing
gRPC Streaming• HTTP/2 based
• Strong typing
• Load balancing
• Protobuf efficiency
• No native pub/sub
• Custom connection mgmt
• Higher protocol overhead
• Complex implementation
AWS IoT Core• Fully managed
• MQTT support
• Auto-scaling
• Security features
• Vendor lock-in
• High costs at scale
• Limited customization
• Regional constraints

Key Technology Decisions

NATS over RabbitMQ

Better clustering with gossip protocol, lower resource usage, and simpler operational model

WebSocket over HTTP/2 Streaming

Lower protocol overhead, better browser support, and simpler connection management

Redis for Distributed State

Enables stateless DCM design, provides persistence during restarts and failover

Kafka for Event Processing

Durable event processing, replay capabilities, and integration with existing systems

Implementation Trade-offs

Latency vs Durability

Choice: NATS for routing, Kafka for persistence

Trade-off: Kafka provides better durability, NATS provides sub-ms latency

Memory vs Network

Choice: Stateless design with Redis state

Trade-off: Network latency for state calls vs horizontal scalability

Complexity vs Performance

Choice: Connection pooling and load balancing

Trade-off: Increased operational complexity for better resource utilization

Protocol Comparison Matrix

ProtocolLatencyThroughputScalabilityReliabilityComplexityCost
NATS + WebSocketExcellentHighExcellentGoodMediumLow
Socket.IOGoodMediumPoorGoodLowMedium
AWS API GatewayGoodHighExcellentExcellentLowHigh

5. Cost Analysis & Comparison

Usage Metrics (400K DAU Realistic Load)

30K
Peak Concurrent Users
510M
Messages/Month
180M
Connection Minutes/Month
1TB
Data Transfer/Month

Cost Comparison Table

SolutionMonthly CostAnnual Costvs NATSNotes
Self-hosted NATS$340$4,080BaselineCheapest; needs DevOps expertise
AWS API Gateway WebSocket$555$6,660+63%Fully managed; AWS-native
Ably Real-time Platform$727-1,200+$8,724-14,400++114-253%Feature-rich but expensive at scale
Synadia NATS Cloud$2,000+$24,000++488%+Enterprise managed NATS

Self-hosted NATS Breakdown

NATS Core Cluster (3x m5.large)$210/month
Network Load Balancer$20/month
Data Transfer Out (1TB)$90/month
Monitoring/Logging/Backup$20/month
Total Self-hosted NATS$340/month

Break-even Analysis

vs AWS API Gateway

Save $215/month (39% cheaper)

Annual savings: $2,580

vs Ably Pro Plan

Save $387-860/month (53-72% cheaper)

Annual savings: $4,644-10,320

Cost Advantage Grows

Higher traffic makes self-hosted even more economical

Linear vs exponential cost scaling

Cost Sensitivity Analysis

2x Message Volume Impact (1.02B messages/month)

AWS API Gateway+$510/month
Ably+$0/month
Self-hosted NATS+$30/month

2x Connection Scale Impact (60k peak users)

AWS API Gateway+$45/month
Ably+$328/month
Self-hosted NATS+$140/month

Final Recommendation: Self-hosted NATS

Why NATS Wins

  • • 63% cheaper than AWS API Gateway
  • • 114% cheaper than Ably Pro Plan
  • • Full control over infrastructure
  • • No vendor lock-in or dependency risks
  • • Scales economically with growth
  • • Existing expertise in NATS operations

Cost Optimization Strategy

  • • Reserved Instances: 30-40% savings on predictable workloads
  • • Spot Instances: 60-70% savings for non-critical workers
  • • Regional optimization: ap-south-1 already cost-effective
  • • Auto-scaling: Pay only for actual usage
  • • Connection pooling: Optimize resource utilization

6. Performance & Scale

Performance Targets & Achievements

<300ms
P95 End-to-End Latency
✓ Target Met
50K/sec
Notification Delivery
✓ Target Achieved
99.9%
System Uptime
✓ SLA Met
10K/sec
Message Ingestion
✓ Capacity Available

Latency Breakdown

Connection Establishment<100ms P95
Health Ping Response<50ms P95
Database Write Operations<10ms P95
NATS Message Routing<1ms Average

Throughput Metrics

Kafka Message Consumption10K msg/sec
WebSocket Delivery Rate50K msg/sec
Health Ping Processing1,667 pings/sec
Acknowledgment Processing40K acks/sec

Scale & Growth Projections

MetricCurrent1 Year2 YearInfrastructure Impact
Concurrent Connections50,000100,000150,000Scale DCM instances to 6-9 nodes
Message Volume/Month510M1.02B1.53BIncrease worker pool capacity
Tenant Count1,0003,00010,000Database sharding consideration
Data Storage Growth5GB25GB50GBImplement automated archival

Load Testing Results

Peak Load Test

Successfully handled 75K concurrent connections

150% of target capacity validated

Burst Traffic

Handled 5x message spike with <500ms latency

Auto-scaling response time: 30 seconds

Failure Recovery

Node failure recovered in <10 seconds

Zero message loss during failover

Optimization Opportunities

Connection Pooling:Reduce memory usage by 30%
Message Batching:Improve throughput by 40%
Compression:Reduce bandwidth by 60%
Edge Deployment:Reduce latency by 50%

7. Security & PII Handling

Security Architecture Overview

Authentication Layer

  • • JWT-based authentication with configurable expiration
  • • Token validation on every connection establishment
  • • Multi-tenant isolation through JWT claims
  • • Automatic token refresh mechanisms

Transport Security

  • • TLS 1.3 for all client connections
  • • Certificate pinning for mobile applications
  • • HSTS headers for web clients
  • • Perfect Forward Secrecy (PFS) enabled

Data Protection

  • • AES-256 encryption for sensitive data at rest
  • • Field-level encryption for PII data
  • • Encrypted backup storage with key rotation
  • • Secure key management with HSM integration

Access Control

  • • Role-Based Access Control (RBAC) implementation
  • • Tenant-based data isolation and permissions
  • • API rate limiting and DDoS protection
  • • Administrative access logging and monitoring

PII Data Handling (Category 2)

Sensitive Data Types

  • • User identifiers and session tokens
  • • Notification content with personal information
  • • Connection metadata and usage patterns
  • • Device identifiers and network information

Protection Measures

  • • End-to-end encryption for sensitive payloads
  • • PII anonymization in logs and metrics
  • • Data masking for development environments
  • • Regular security audits and compliance checks

Compliance & Retention

  • • GDPR-compliant data handling procedures
  • • 90-day retention with automated purging
  • • Right to erasure implementation
  • • Data portability and export capabilities

8. Deployment Strategy

Infrastructure Topology

Load Balancer Layer

  • • HAProxy/NGINX for connection distribution
  • • Health check configuration for DCM instances
  • • SSL termination and certificate management
  • • Geographic routing for latency optimization

Application Layer

  • • 3-node NATS cluster for high availability
  • • Auto-scaling Kubernetes pods for workers
  • • Service mesh for inter-service communication
  • • Rolling deployment with zero downtime

Data Layer

  • • PostgreSQL cluster with read replicas
  • • Redis cluster with sentinel configuration
  • • Automated backup and disaster recovery
  • • Data encryption at rest and in transit

Monitoring Layer

  • • Prometheus/Grafana stack for observability
  • • ELK stack for centralized logging
  • • Distributed tracing with Jaeger
  • • Alert manager for incident notifications

Operational Excellence

Deployment Strategy

  • • Blue-green deployment for zero downtime
  • • Canary releases for gradual rollout
  • • Feature flags for controlled feature activation
  • • Automated rollback on health check failures

Monitoring & Alerting

  • • Real-time dashboard for system health
  • • SLA-based alerting (latency, availability)
  • • Capacity planning and scaling alerts
  • • Security incident detection and response

Disaster Recovery

  • • Multi-AZ deployment for high availability
  • • Automated backup with point-in-time recovery
  • • Cross-region replication for DR scenarios
  • • RTO: 4 hours, RPO: 1 hour targets

Cost Optimization

  • • Reserved instances for predictable workloads
  • • Spot instances for batch processing
  • • Auto-scaling based on demand patterns
  • • Regular cost analysis and optimization