Feat: add prometheus compatible endpoint, seed fallback store and add env.example

This commit is contained in:
Ayobami
2025-08-13 22:41:37 +01:00
parent 3f8f456eef
commit da78487047
10 changed files with 987 additions and 268 deletions
+43 -1
View File
@@ -1,6 +1,7 @@
# Ticket Scaling Microservice - Design Document
## Table of Contents
1. [Architecture Overview](#architecture-overview)
2. [System Components](#system-components)
3. [Scalability Strategies](#scalability-strategies)
@@ -15,6 +16,7 @@
## Architecture Overview
### High-Level Architecture
```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Load Balancer │ │ Prometheus │ │ Grafana │
@@ -38,6 +40,7 @@
```
### Design Principles
1. **High Availability**: Fallback mechanisms ensure service continuity
2. **Atomic Operations**: Redis Lua scripts prevent race conditions
3. **Horizontal Scalability**: Stateless design enables easy scaling
@@ -47,6 +50,7 @@
## System Components
### 1. Core Application (server.js)
- **Technology**: Node.js with Express framework
- **Responsibilities**:
- HTTP request handling
@@ -55,6 +59,7 @@
- PDF generation coordination
### 2. Redis Client (redis-client.js)
- **Technology**: Redis with Lua scripting
- **Responsibilities**:
- Atomic ticket operations
@@ -63,6 +68,7 @@
- Script execution
### 3. Fallback Store (fallback-store.js)
- **Technology**: In-memory JavaScript Map
- **Responsibilities**:
- Emergency ticket storage
@@ -70,6 +76,7 @@
- Graceful degradation
### 4. PDF Generator (pdf-generator.js)
- **Technology**: PDFKit library
- **Responsibilities**:
- Professional ticket generation
@@ -77,6 +84,7 @@
- Cleanup operations
### 5. Logging System (logger.js)
- **Technology**: Winston logging framework
- **Responsibilities**:
- Structured logging
@@ -87,6 +95,7 @@
## Scalability Strategies
### Horizontal Scaling
```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Instance 1 │ │ Instance 2 │ │ Instance N │
@@ -102,17 +111,20 @@
```
**Key Features**:
- Stateless application design
- Shared Redis backend
- Load balancer distribution
- Independent scaling
### Vertical Scaling
- **CPU**: Multi-core utilization through Node.js cluster mode
- **Memory**: Configurable heap sizes for high-throughput
- **I/O**: Async operations prevent blocking
### Database Scaling
```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Redis Master │ │ Redis Replica │ │ Redis Replica │
@@ -121,6 +133,7 @@
```
**Strategies**:
- Redis clustering for horizontal scaling
- Read replicas for metrics/stats queries
- Sharding by event ID for massive scale
@@ -128,6 +141,7 @@
## Atomic Operations
### Lua Script Design
Our core purchase operation uses a Redis Lua script to ensure atomicity:
```lua
@@ -145,12 +159,14 @@ local globalKey = KEYS[3] -- global:stats
```
**Benefits**:
- **Race Condition Prevention**: All operations execute atomically
- **Consistency**: No partial state updates
- **Performance**: Single round-trip to Redis
- **Reliability**: All-or-nothing execution
### Concurrency Handling
- **Optimistic Locking**: Lua scripts handle concurrent access
- **Queue Management**: Redis lists provide FIFO ticket distribution
- **Connection Pooling**: Efficient Redis connection reuse
@@ -158,11 +174,13 @@ local globalKey = KEYS[3] -- global:stats
## Fallback Mechanisms
### Activation Triggers
1. **Redis Connection Failure**: Network issues or Redis downtime
2. **Script Execution Errors**: Lua script failures
3. **Timeout Scenarios**: Slow Redis responses
### Fallback Architecture
```
┌─────────────────┐
│ Request Comes │
@@ -181,8 +199,16 @@ local globalKey = KEYS[3] -- global:stats
└─────────────────┘ └─────────────────┘
```
### Fallback Store Improvements
- **Automatic Seeding**: Fallback store is seeded during server startup and when activated
- **Data Synchronization**: Automatic attempt to sync with Redis data when activated
- **Manual Seeding**: Admin endpoint to manually populate fallback store from Redis
- **Resilient Operation**: Continues functioning even when Redis is completely unavailable
### Fallback Limitations
- **Non-Persistent**: Data lost on restart
- **Non-Persistent**: Data lost on restart (mitigated by automatic reseeding)
- **Single Instance**: No cross-instance synchronization
- **Capacity Limited**: Memory constraints
- **Warning Logs**: Clear indication of degraded mode
@@ -190,18 +216,21 @@ local globalKey = KEYS[3] -- global:stats
## Performance Optimizations
### Application Level
1. **Async Operations**: Non-blocking I/O throughout
2. **Connection Pooling**: Reuse Redis connections
3. **Batch Operations**: Bulk ticket seeding
4. **Caching**: Event metadata caching
### Redis Optimizations
1. **Lua Scripts**: Reduced network round-trips
2. **Pipeline Operations**: Batch commands
3. **Memory Management**: Efficient data structures
4. **Persistence**: AOF for durability
### PDF Generation
1. **Async Generation**: Non-blocking PDF creation
2. **Stream Processing**: Memory-efficient file handling
3. **Cleanup Jobs**: Automatic old file removal
@@ -210,6 +239,7 @@ local globalKey = KEYS[3] -- global:stats
## Monitoring & Observability
### Metrics Collection
```json
{
"global": {
@@ -238,12 +268,14 @@ local globalKey = KEYS[3] -- global:stats
```
### Logging Strategy
- **Structured Logging**: JSON format for parsing
- **Request Tracking**: Unique IDs for tracing
- **Performance Metrics**: Response times and throughput
- **Error Categorization**: Different log levels
### Health Checks
- **Application Health**: `/health` endpoint
- **Redis Connectivity**: Connection status
- **Fallback Status**: Degraded mode indication
@@ -252,16 +284,19 @@ local globalKey = KEYS[3] -- global:stats
## Security Considerations
### Input Validation
- **Event ID Validation**: Numeric constraints
- **Request Rate Limiting**: DDoS protection
- **Parameter Sanitization**: Injection prevention
### Container Security
- **Non-Root User**: Principle of least privilege
- **Minimal Base Image**: Alpine Linux for smaller attack surface
- **Health Checks**: Container monitoring
### Data Protection
- **No Sensitive Data**: Tickets are identifiers only
- **Audit Logging**: Purchase tracking
- **Secure Defaults**: Production-ready configuration
@@ -269,6 +304,7 @@ local globalKey = KEYS[3] -- global:stats
## Deployment Strategy
### Development Environment
```bash
# Local development
npm install
@@ -278,6 +314,7 @@ npm run dev # Start with nodemon
```
### Production Environment
```bash
# Docker deployment
docker-compose up -d # Core services
@@ -285,6 +322,7 @@ docker-compose --profile monitoring up # With monitoring
```
### Container Orchestration
- **Docker Compose**: Local and small deployments
- **Kubernetes**: Large-scale deployments
- **Health Checks**: Automatic restart on failure
@@ -293,24 +331,28 @@ docker-compose --profile monitoring up # With monitoring
## Future Enhancements
### Performance Improvements
1. **Redis Clustering**: Horizontal database scaling
2. **CDN Integration**: PDF delivery optimization
3. **Caching Layer**: Application-level caching
4. **Connection Optimization**: Advanced pooling
### Feature Additions
1. **QR Code Generation**: Enhanced ticket security
2. **Email Integration**: Automatic ticket delivery
3. **Payment Processing**: Complete purchase flow
4. **Event Management**: Dynamic event creation
### Monitoring Enhancements
1. **Distributed Tracing**: Request flow tracking
2. **Custom Dashboards**: Business metrics visualization
3. **Alerting**: Proactive issue detection
4. **Performance Profiling**: Bottleneck identification
### Security Hardening
1. **Authentication**: API key management
2. **Rate Limiting**: Advanced throttling
3. **Encryption**: Data in transit protection