telecom-platform/docs/ARCHITECTURE.md

# TaaS Platform Architecture

## System Overview

The TaaS Platform is a multi-layer telecom infrastructure that provides cellular connectivity as a service. It implements 3GPP standards while exposing a modern, developer-friendly API.

## Architecture Layers

```
┌─────────────────────────────────────────────────────────────┐
│                     Developer Interface                      │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │  Web UI      │  │  TypeScript  │  │  REST API    │      │
│  │  Dashboard   │  │  SDK         │  │  Webhooks    │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
└─────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────┐
│                   Business Support System (BSS)              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │  API Server  │  │  Carrier     │  │  Subscriber  │      │
│  │  (Go 1.26)   │  │  Connector   │  │  Management  │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
└─────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────┐
│              Operations Support System (OSS)                 │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │  Charging    │  │  Usage       │  │  eSIM        │      │
│  │  Engine      │  │  Tracking    │  │  Provisioning│      │
│  │  (Rust)      │  │  (Redis)     │  │  (ES2+)      │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
└─────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────┐
│                  Telecom Core Network                        │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │  5G Core     │  │  Packet      │  │  Data Plane  │      │
│  │  (free5GC)   │  │  Gateway     │  │  (eBPF)      │      │
│  │  (Go)        │  │  (Rust)      │  │              │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
└─────────────────────────────────────────────────────────────┘
```

## Component Details

### 1. API Server (Go 1.26)

**Purpose**: Developer-facing RESTful API and business logic

**Technology**:
- Language: Go 1.26 (Green Tea GC, new(expr) syntax)
- Framework: Gin
- Database: MongoDB (subscriber data)
- Cache: Redis (session management)

**Responsibilities**:
- eSIM creation and management
- Authentication and authorization
- Rate limiting and quotas
- Webhook delivery
- API documentation

**Endpoints**:
- `POST /v1/esims` - Create eSIM
- `GET /v1/esims/:id` - Get eSIM details
- `GET /v1/esims/:id/usage` - Get usage stats
- `DELETE /v1/esims/:id` - Terminate eSIM

### 2. Carrier Connector (Go 1.26)

**Purpose**: Integration with GSMA SM-DP+ servers for eSIM provisioning

**Technology**:
- Language: Go 1.26
- Protocol: HTTPS (ES2+ API)
- Vendor: Thales, G+D, or Idemia

**Responsibilities**:
- Profile ordering via ES2+ protocol
- Activation code generation (LPA strings)
- Profile status tracking
- Vendor-specific adaptations

**Flow**:
1. Receive eSIM order from API Server
2. Call SM-DP+ vendor API with IMSI, K, OPc keys
3. Receive activation code (LPA:1$...)
4. Return to API Server for user display

### 3. Charging Engine (Rust 1.94)

**Purpose**: Real-time credit control and billing

**Technology**:
- Language: Rust 1.94 (array_windows, async Tokio)
- Framework: Axum
- Database: Redis (in-memory credit balances)

**Responsibilities**:
- Credit balance management
- Real-time authorization checks
- Usage deduction (atomic operations)
- Credit top-up processing

**API**:
- `POST /v1/credit/:ip/check` - Check if user has credit
- `POST /v1/credit/:ip/deduct` - Deduct bytes used
- `POST /v1/credit/:ip/add` - Add credit
- `GET /v1/credit/:ip/balance` - Get balance

**Performance**:
- Sub-millisecond response times
- Atomic Redis operations (INCR, DECR)
- Horizontal scaling via Redis Cluster

### 4. Packet Gateway (Rust 1.94 + eBPF)

**Purpose**: High-performance packet processing and traffic control

**Technology**:
- Language: Rust 1.94
- Framework: Aya (pure Rust eBPF)
- Attach Point: XDP (eXpress Data Path)
- Protocol: GTP-U decapsulation (UDP port 2152)

**Responsibilities**:
- Parse GTP-U tunneled packets
- Extract user IP addresses
- Count bytes per subscriber
- Block traffic for users without credit
- Sync stats to Redis

**Architecture**:
```
┌─────────────────────────────────────────────┐
│           Network Interface (eth0)          │
└─────────────────────────────────────────────┘
                    │
          ┌─────────▼─────────┐
          │   XDP Hook        │ ← eBPF program runs here
          │   (Kernel)        │
          └─────────┬─────────┘
                    │
          ┌─────────▼─────────┐
          │   eBPF Maps       │
          │  - Packet Stats   │
          │  - Credit Status  │
          └─────────┬─────────┘
                    │
          ┌─────────▼─────────┐
          │  Userspace Daemon │ ← Rust application
          │  (Reads maps,     │
          │   syncs to Redis) │
          └───────────────────┘
```

**Packet Processing**:
1. Packet arrives at NIC
2. XDP hook inspects Ethernet → IP → UDP → GTP-U headers
3. Extract inner IP packet (actual user data)
4. Look up credit status in eBPF map
5. If credit > 0: `XDP_PASS`, else: `XDP_DROP`
6. Update packet statistics in eBPF map
7. Userspace daemon reads stats, syncs to Redis

### 5. 5G Core Network (free5GC)

**Purpose**: 3GPP-compliant 5G standalone core

**Technology**:
- Project: free5GC v4.2.1
- Language: Go
- Database: MongoDB
- Deployment: Kubernetes or bare metal

**Network Functions**:
- **AMF** (Access and Mobility Management)
- **SMF** (Session Management)
- **UPF** (User Plane Function)
- **PCF** (Policy Control)
- **UDM** (Unified Data Management)
- **UDR** (Unified Data Repository)
- **AUSF** (Authentication Server)
- **NRF** (NF Repository)

**Integration Points**:
- API Server writes subscriber data to MongoDB
- UPF forwards GTP-U traffic to Packet Gateway
- AMF handles UE authentication using stored keys (K, OPc)

### 6. Web Dashboard (Next.js 15)

**Purpose**: Developer portal and management interface

**Technology**:
- Framework: Next.js 15 (App Router)
- Language: TypeScript
- Styling: Tailwind CSS
- Data Fetching: TanStack Query

**Features**:
- eSIM creation and management
- Real-time usage charts (Recharts)
- API key management
- Webhook configuration
- Billing and invoices

## Data Flow

### eSIM Creation Flow

```
1. Developer → API Server
   POST /v1/esims { data_plan: "1GB" }

2. API Server → IMSI Allocator
   Generate unique 15-digit IMSI

3. API Server → Key Generator
   Generate K (subscriber key)
   Generate OPc (operator key)

4. API Server → MongoDB (free5GC)
   Insert subscriber data:
   - subscriptionData.authenticationData (K, OPc)
   - subscriptionData.provisionedData (IMSI, AMBR)

5. API Server → Carrier Connector
   Order eSIM profile via ES2+

6. Carrier Connector → SM-DP+ Vendor
   HTTPS POST with IMSI, K, OPc

7. SM-DP+ → Carrier Connector
   Return activation code (LPA string)

8. API Server → Developer
   Return eSIM details + activation code

9. Developer → End User
   Display QR code with LPA string

10. End User's Device → SM-DP+
    Scan QR, download eSIM profile

11. Device → free5GC AMF
    Registration procedure (5G AKA)

12. free5GC → Packet Gateway
    Establish GTP-U tunnel for data
```

### Real-Time Usage Flow

```
1. User Device sends data
   ↓
2. free5GC UPF encapsulates in GTP-U
   ↓
3. Packet Gateway (eBPF XDP)
   - Decapsulate GTP-U
   - Extract user IP
   - Check credit status in eBPF map
   ↓
4. If credit > 0:
   - Pass packet (XDP_PASS)
   - Increment byte counter in eBPF map
   ↓
5. Userspace daemon (Rust)
   - Reads eBPF maps every 1 second
   - Writes stats to Redis
   ↓
6. Charging Engine
   - Reads usage from Redis
   - Deducts from credit balance
   - Updates credit status
   ↓
7. Packet Gateway
   - Reads new credit status from Redis
   - Updates eBPF map
   ↓
8. If credit reaches 0:
   - eBPF drops packets (XDP_DROP)
   - User's internet stops
```

## Scaling Architecture

### Horizontal Scaling

**Stateless Components** (can be replicated):
- API Server: 3+ replicas behind load balancer
- Carrier Connector: 2+ replicas
- Charging Engine: 2+ replicas
- Web Dashboard: CDN + multiple origins

**Stateful Components** (require special handling):
- MongoDB: Replica set (1 primary, 2 secondaries)
- Redis: Cluster mode or Sentinel
- Packet Gateway: One instance per physical server (eBPF)
- free5GC: Kubernetes deployment with dedicated UPF nodes

### Performance Metrics

**Throughput**:
- API Server: 10,000 requests/second
- Charging Engine: 50,000 credit checks/second
- Packet Gateway: 10 Gbps+ per server (eBPF)
- free5GC UPF: 5 Gbps per instance

**Latency**:
- API Server: <50ms p99
- Charging Engine: <1ms p99 (Redis)
- Packet Gateway: <1µs (eBPF in kernel)
- eSIM Provisioning: <2 seconds end-to-end

## Security

### Authentication & Authorization
- API keys with HMAC signatures
- JWT tokens for web dashboard
- OAuth 2.0 for third-party integrations
- Rate limiting per API key

### Data Encryption
- TLS 1.3 for all API endpoints
- MongoDB encryption at rest
- Redis TLS connections
- eBPF packet inspection (no decryption)

### Secrets Management
- Kubernetes Secrets for production
- HashiCorp Vault for sensitive keys
- Environment variables for local dev
- No hardcoded credentials

## Deployment

### Development
```bash
make all           # Build all components
make db-setup      # Set up MongoDB + Redis
make dev           # Start all services
```

### Docker Compose
```bash
docker-compose up -d
```

### Kubernetes
```bash
kubectl apply -f deployments/kubernetes/
```

### Production Checklist
- [ ] MongoDB replica set configured
- [ ] Redis Cluster or Sentinel enabled
- [ ] TLS certificates installed
- [ ] Monitoring (Prometheus, Grafana)
- [ ] Logging (ELK Stack or Loki)
- [ ] Backup strategy implemented
- [ ] Disaster recovery plan documented
- [ ] Load testing completed (10x expected traffic)

## Monitoring

### Metrics (Prometheus)
- `api_requests_total` - Total API requests
- `esim_creations_total` - eSIMs created
- `data_usage_bytes_total` - Total data usage
- `charging_checks_duration_seconds` - Credit check latency
- `packet_gateway_packets_total` - Packets processed

### Alerts
- High error rate (>1%)
- Low credit balances (<10% of users)
- MongoDB replica lag (>10 seconds)
- Redis memory usage (>90%)
- Packet gateway packet loss (>0.1%)

## Future Enhancements

1. **Multi-Region Deployment**
   - Deploy in multiple AWS regions
   - Global load balancing with Route 53
   - Data residency compliance

2. **Advanced Analytics**
   - Real-time dashboards with WebSockets
   - Predictive usage forecasting
   - Anomaly detection (unusual data patterns)

3. **Additional Carriers**
   - Support for multiple SM-DP+ vendors
   - Dynamic carrier selection based on region
   - Fallback mechanisms

4. **IoT Optimization**
   - SGP.32 support (IoT eSIM standard)
   - LPWAN protocol support (NB-IoT, LTE-M)
   - Power-saving features

5. **Developer Tools**
   - CLI tool for API interaction
   - Terraform provider
   - GitHub Actions integration