docs: Restructure documentation with comprehensive architecture guide and new API reference

- Remove docs/API.md (237 lines) with TaaS-specific REST endpoints and SDK examples
- Expand docs/ARCHITECTURE.md with detailed system overview, component architecture, data flow diagrams, scaling strategies, and RBAC security model
- Add Table of Contents with 6 main sections: System Overview, Component Details, Data Flow, Scaling, Security & RBAC, Future Enhancements
- Add detailed component descriptions
This commit is contained in:
nutcas3
2026-04-19 04:35:53 +03:00
parent 8e30626252
commit 4e313a4298
6 changed files with 923 additions and 1099 deletions
+199 -287
View File
@@ -1,10 +1,21 @@
# TaaS Platform Architecture
# Architecture
Comprehensive overview of the TaaS Platform system design, components, data flow, security, and RBAC.
## Table of Contents
1. [System Overview](#system-overview)
2. [Component Details](#component-details)
3. [Data Flow](#data-flow)
4. [Scaling](#scaling)
5. [Security & RBAC](#security--rbac)
6. [Future Enhancements](#future-enhancements)
## System Overview
The TaaS Platform is a multi-layer telecom infrastructure that provides cellular connectivity as a service. It implements 3GPP standards while exposing a modern, developer-friendly API.
The TaaS Platform is a multi-layer telecom infrastructure that provides cellular connectivity as a service, implementing 3GPP standards while exposing a modern, developer-friendly API.
## Architecture Layers
### Architecture Layers
```
┌─────────────────────────────────────────────────────────────┐
@@ -46,363 +57,264 @@ The TaaS Platform is a multi-layer telecom infrastructure that provides cellular
### 1. API Server (Go 1.26)
**Purpose**: Developer-facing RESTful API and business logic
**Purpose**: Developer-facing RESTful API and business logic.
**Technology**:
- Language: Go 1.26 (Green Tea GC, new(expr) syntax)
- Framework: Gin
- Database: MongoDB (subscriber data)
- Cache: Redis (session management)
**Stack**:
- Gin framework
- GORM with PostgreSQL (primary) / MongoDB (free5GC subscriber data)
- Redis (session/cache)
- Casbin for RBAC
- WebSocket for real-time updates
**Responsibilities**:
- eSIM creation and management
- Authentication and authorization
- Rate limiting and quotas
- eSIM lifecycle management
- Authentication (JWT) and authorization (Casbin)
- Rate limiting, quotas, and performance monitoring
- Webhook delivery
- API documentation
- Health checks and alerting
**Endpoints**:
- `POST /v1/esims` - Create eSIM
- `GET /v1/esims/:id` - Get eSIM details
- `GET /v1/esims/:id/usage` - Get usage stats
- `DELETE /v1/esims/:id` - Terminate eSIM
**Key Endpoints**:
- `/v1/auth/*` — authentication
- `/v1/esims/*` — eSIM management
- `/v1/services/*` — service operations
- `/v1/monitoring/*` — metrics & alerts
- `/v1/billing/*` — invoicing
- `/ws` — WebSocket stream
- `/health`, `/ready`, `/live` — probes
### 2. Carrier Connector (Go 1.26)
**Purpose**: Integration with GSMA SM-DP+ servers for eSIM provisioning
**Technology**:
- Language: Go 1.26
- Protocol: HTTPS (ES2+ API)
- Vendor: Thales, G+D, or Idemia
**Responsibilities**:
- Profile ordering via ES2+ protocol
- Activation code generation (LPA strings)
- Profile status tracking
- Vendor-specific adaptations
**Purpose**: Integration with GSMA SM-DP+ servers for eSIM provisioning.
**Flow**:
1. Receive eSIM order from API Server
2. Call SM-DP+ vendor API with IMSI, K, OPc keys
2. Call SM-DP+ vendor API with IMSI, K, OPc keys (ES2+ over HTTPS)
3. Receive activation code (LPA:1$...)
4. Return to API Server for user display
4. Return to API Server
Supports multiple vendors: Thales, G+D, Idemia.
### 3. Charging Engine (Rust 1.94)
**Purpose**: Real-time credit control and billing
**Purpose**: Real-time credit control and billing.
**Technology**:
- Language: Rust 1.94 (array_windows, async Tokio)
- Framework: Axum
- Database: Redis (in-memory credit balances)
**Stack**:
- Axum (async web framework)
- Redis for in-memory credit balances
- Atomic INCR/DECR operations
**Responsibilities**:
- Credit balance management
- Real-time authorization checks
- Usage deduction (atomic operations)
- Credit top-up processing
**Endpoints**:
- `POST /v1/credit/:ip/check` — authorization
- `POST /v1/credit/:ip/deduct` — usage deduction
- `POST /v1/credit/:ip/add` — top-up
- `GET /v1/credit/:ip/balance` — balance query
**API**:
- `POST /v1/credit/:ip/check` - Check if user has credit
- `POST /v1/credit/:ip/deduct` - Deduct bytes used
- `POST /v1/credit/:ip/add` - Add credit
- `GET /v1/credit/:ip/balance` - Get balance
**Performance**:
- Sub-millisecond response times
- Atomic Redis operations (INCR, DECR)
- Horizontal scaling via Redis Cluster
**Performance**: Sub-millisecond p99, horizontally scalable via Redis Cluster.
### 4. Packet Gateway (Rust 1.94 + eBPF)
**Purpose**: High-performance packet processing and traffic control
**Purpose**: High-performance packet processing at line rate.
**Technology**:
- Language: Rust 1.94
- Framework: Aya (pure Rust eBPF)
- Attach Point: XDP (eXpress Data Path)
- Protocol: GTP-U decapsulation (UDP port 2152)
**Responsibilities**:
- Parse GTP-U tunneled packets
- Extract user IP addresses
- Count bytes per subscriber
- Block traffic for users without credit
- Sync stats to Redis
**Stack**:
- Aya (pure Rust eBPF)
- XDP (eXpress Data Path) hook
- GTP-U decapsulation (UDP 2152)
**Architecture**:
```
┌─────────────────────────────────────────────┐
│ Network Interface (eth0) │
└─────────────────────────────────────────────┘
│
┌─────────▼─────────┐
│ XDP Hook │ ← eBPF program runs here
│ (Kernel) │
└─────────┬─────────┘
│
┌─────────▼─────────┐
│ eBPF Maps │
│ - Packet Stats │
│ - Credit Status │
└─────────┬─────────┘
│
┌─────────▼─────────┐
│ Userspace Daemon │ ← Rust application
│ (Reads maps, │
│ syncs to Redis) │
└───────────────────┘
NIC → XDP Hook (kernel eBPF) → eBPF Maps → Userspace Daemon → Redis
```
**Packet Processing**:
**Packet flow**:
1. Packet arrives at NIC
2. XDP hook inspects Ethernet → IP → UDP → GTP-U headers
3. Extract inner IP packet (actual user data)
4. Look up credit status in eBPF map
5. If credit > 0: `XDP_PASS`, else: `XDP_DROP`
6. Update packet statistics in eBPF map
7. Userspace daemon reads stats, syncs to Redis
2. XDP inspects Ethernet → IP → UDP → GTP-U headers
3. Extract inner IP (user data), look up credit in eBPF map
4. If credit > 0: `XDP_PASS`; else: `XDP_DROP`
5. Userspace daemon syncs byte counters to Redis every 1s
### 5. 5G Core Network (free5GC)
### 5. 5G Core Network (free5GC v4.2.1)
**Purpose**: 3GPP-compliant 5G standalone core
3GPP-compliant 5G standalone core with all network functions (AMF, SMF, UPF, PCF, UDM, UDR, AUSF, NRF).
**Technology**:
- Project: free5GC v4.2.1
- Language: Go
- Database: MongoDB
- Deployment: Kubernetes or bare metal
**Network Functions**:
- **AMF** (Access and Mobility Management)
- **SMF** (Session Management)
- **UPF** (User Plane Function)
- **PCF** (Policy Control)
- **UDM** (Unified Data Management)
- **UDR** (Unified Data Repository)
- **AUSF** (Authentication Server)
- **NRF** (NF Repository)
**Integration Points**:
**Integration**:
- API Server writes subscriber data to MongoDB
- UPF forwards GTP-U traffic to Packet Gateway
- AMF handles UE authentication using stored keys (K, OPc)
- UPF forwards GTP-U to Packet Gateway
- AMF authenticates using stored K/OPc keys
### 6. Web Dashboard (Next.js 15)
**Purpose**: Developer portal and management interface
- App Router, TypeScript, Tailwind CSS
- TanStack Query for data fetching
- Real-time updates via WebSocket
- Error boundaries and loading states
**Technology**:
- Framework: Next.js 15 (App Router)
- Language: TypeScript
- Styling: Tailwind CSS
- Data Fetching: TanStack Query
**Features**:
- eSIM creation and management
- Real-time usage charts (Recharts)
- API key management
- Webhook configuration
- Billing and invoices
**Pages**: eSIM management, services, monitoring, billing, config, chaos engineering, plugins, automation.
## Data Flow
### eSIM Creation Flow
### eSIM Creation
```
1. Developer → API Server
POST /v1/esims { data_plan: "1GB" }
2. API Server → IMSI Allocator
Generate unique 15-digit IMSI
3. API Server → Key Generator
Generate K (subscriber key)
Generate OPc (operator key)
4. API Server → MongoDB (free5GC)
Insert subscriber data:
- subscriptionData.authenticationData (K, OPc)
- subscriptionData.provisionedData (IMSI, AMBR)
5. API Server → Carrier Connector
Order eSIM profile via ES2+
6. Carrier Connector → SM-DP+ Vendor
HTTPS POST with IMSI, K, OPc
7. SM-DP+ → Carrier Connector
Return activation code (LPA string)
8. API Server → Developer
Return eSIM details + activation code
9. Developer → End User
Display QR code with LPA string
10. End User's Device → SM-DP+
Scan QR, download eSIM profile
11. Device → free5GC AMF
Registration procedure (5G AKA)
12. free5GC → Packet Gateway
Establish GTP-U tunnel for data
1. Developer → API Server: POST /v1/esims { data_plan: "1GB" }
2. API Server → IMSI Allocator: generate 15-digit IMSI
3. API Server → Key Generator: K + OPc
4. API Server → MongoDB: insert subscriptionData
5. API Server → Carrier Connector: order via ES2+
6. Carrier Connector → SM-DP+: HTTPS POST (IMSI, K, OPc)
7. SM-DP+ → Carrier Connector: activation code (LPA:1$...)
8. API Server → Developer: eSIM + activation code
9. End User scans QR → SM-DP+: profile download
10. Device → free5GC AMF: 5G AKA registration
11. free5GC → Packet Gateway: GTP-U tunnel established
```
### Real-Time Usage Flow
### Real-Time Usage
```
1. User Device sends data
↓
1. User device sends data
2. free5GC UPF encapsulates in GTP-U
↓
3. Packet Gateway (eBPF XDP)
- Decapsulate GTP-U
- Extract user IP
- Check credit status in eBPF map
↓
4. If credit > 0:
- Pass packet (XDP_PASS)
- Increment byte counter in eBPF map
↓
5. Userspace daemon (Rust)
- Reads eBPF maps every 1 second
- Writes stats to Redis
↓
6. Charging Engine
- Reads usage from Redis
- Deducts from credit balance
- Updates credit status
↓
7. Packet Gateway
- Reads new credit status from Redis
- Updates eBPF map
↓
8. If credit reaches 0:
- eBPF drops packets (XDP_DROP)
- User's internet stops
3. Packet Gateway (eBPF XDP):
- Decapsulate, extract user IP
- Check credit in eBPF map
4. If credit > 0: XDP_PASS + increment byte counter
5. Userspace daemon: read eBPF maps → Redis (1s interval)
6. Charging Engine: deduct from credit balance
7. Packet Gateway: reload credit status from Redis → eBPF map
8. If credit = 0: XDP_DROP (user traffic stops)
```
## Scaling Architecture
## Scaling
### Horizontal Scaling
### Horizontal (Stateless)
**Stateless Components** (can be replicated):
- API Server: 3+ replicas behind load balancer
- Carrier Connector: 2+ replicas
- Charging Engine: 2+ replicas
- Web Dashboard: CDN + multiple origins
**Stateful Components** (require special handling):
- MongoDB: Replica set (1 primary, 2 secondaries)
- Redis: Cluster mode or Sentinel
- Packet Gateway: One instance per physical server (eBPF)
- free5GC: Kubernetes deployment with dedicated UPF nodes
### Stateful
### Performance Metrics
- MongoDB: replica set (1 primary, 2 secondaries)
- PostgreSQL: primary + streaming replicas
- Redis: Cluster or Sentinel
- Packet Gateway: one per physical server (eBPF kernel binding)
- free5GC: K8s deployment with dedicated UPF nodes
**Throughput**:
- API Server: 10,000 requests/second
- Charging Engine: 50,000 credit checks/second
- Packet Gateway: 10 Gbps+ per server (eBPF)
- free5GC UPF: 5 Gbps per instance
### Performance Targets
**Latency**:
- API Server: <50ms p99
- Charging Engine: <1ms p99 (Redis)
- Packet Gateway: <1µs (eBPF in kernel)
- eSIM Provisioning: <2 seconds end-to-end
| Component | Throughput | Latency (p99) |
|-----------|-----------|----------------|
| API Server | 10,000 req/s | <100ms |
| Charging Engine | 50,000 checks/s | <1ms |
| Packet Gateway | 10+ Gbps/server | <1µs (kernel) |
| free5GC UPF | 5 Gbps/instance | — |
| eSIM provisioning | — | <2s end-to-end |
## Security
## Security & RBAC
### Authentication & Authorization
- API keys with HMAC signatures
- JWT tokens for web dashboard
- OAuth 2.0 for third-party integrations
- Rate limiting per API key
### Authentication
- **JWT** tokens for web dashboard and API
- **API keys** with HMAC signatures for programmatic access
- **OAuth 2.0** for third-party integrations
- **Refresh tokens** for long-lived sessions
JWT middleware (`@/Users/nutcase/Downloads/telecom-platform-starter/apps/api-server/internal/middleware/auth.go`) validates tokens on all protected routes.
### RBAC with Casbin
Sophisticated policy-based access control using Casbin with a GORM adapter for persistent policies.
**Request flow**:
```
Request → AuthMiddleware (JWT) → CasbinRBAC (policy check) → Handler
```
**Model** (in `@/Users/nutcase/Downloads/telecom-platform-starter/apps/api-server/internal/rbac/casbin.go`):
```
[request_definition]
r = sub, obj, act
[policy_definition]
p = sub, obj, act
[role_definition]
g = _, _
[policy_effect]
e = some(where (p.eft == allow))
[matchers]
m = g(r.sub, p.sub) && keyMatch(r.obj, p.obj) && (r.act == p.act || p.act == "*")
```
**Default policies**:
| Role | Permissions |
|------|-------------|
| `admin` | All resources, all actions |
| `operator` | Services, monitoring, deployments (GET/POST) |
| `viewer` | Read-only on all resources |
**Usage in routes**:
```go
users := protected.Group("/users")
if casbinSvc != nil {
users.Use(middleware.RequireCasbinPermission(casbinSvc, "/v1/users", "GET"))
} else {
users.Use(middleware.RequireRole("admin")) // fallback
}
```
**Policy management**:
```go
casbinSvc.AddPolicy("operator", "/v1/services/*", "GET")
casbinSvc.AddRoleForUser("user123", "operator")
allowed, err := casbinSvc.CheckPermission(userID, "/v1/services", "GET")
```
### Data Encryption
- TLS 1.3 for all API endpoints
- MongoDB encryption at rest
- Redis TLS connections
- eBPF packet inspection (no decryption)
### Secrets Management
- Kubernetes Secrets for production
- HashiCorp Vault for sensitive keys
- Environment variables for local dev
- TLS 1.3 for all API endpoints
- MongoDB/PostgreSQL encryption at rest
- Redis TLS connections
- Secrets via Kubernetes Secrets or HashiCorp Vault
- No hardcoded credentials
## Deployment
### Security Headers
### Development
```bash
make all # Build all components
make db-setup # Set up MongoDB + Redis
make dev # Start all services
Applied globally via middleware:
```
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000; includeSubDomains
```
### Docker Compose
```bash
docker-compose up -d
```
Plus request size limit (10MB), request timeout (30s), and CORS configuration.
### Kubernetes
```bash
kubectl apply -f deployments/kubernetes/
```
### Rate Limiting
### Production Checklist
- [ ] MongoDB replica set configured
- [ ] Redis Cluster or Sentinel enabled
- [ ] TLS certificates installed
- [ ] Monitoring (Prometheus, Grafana)
- [ ] Logging (ELK Stack or Loki)
- [ ] Backup strategy implemented
- [ ] Disaster recovery plan documented
- [ ] Load testing completed (10x expected traffic)
Tiered limits protect against abuse (see `@/Users/nutcase/Downloads/telecom-platform-starter/apps/api-server/internal/middleware/ratelimit.go`):
## Monitoring
### Metrics (Prometheus)
- `api_requests_total` - Total API requests
- `esim_creations_total` - eSIMs created
- `data_usage_bytes_total` - Total data usage
- `charging_checks_duration_seconds` - Credit check latency
- `packet_gateway_packets_total` - Packets processed
### Alerts
- High error rate (>1%)
- Low credit balances (<10% of users)
- MongoDB replica lag (>10 seconds)
- Redis memory usage (>90%)
- Packet gateway packet loss (>0.1%)
| Tier | Limit/min | Burst |
|------|-----------|-------|
| Auth | 10 | 5 |
| Read | 200 | 50 |
| Admin | 50 | 10 |
| Default | 100 | 20 |
## Future Enhancements
1. **Multi-Region Deployment**
- Deploy in multiple AWS regions
- Global load balancing with Route 53
- Data residency compliance
1. **Multi-region deployment** — AWS multi-region with Route 53 global LB and data residency compliance.
2. **Advanced analytics** — predictive usage forecasting, anomaly detection.
3. **Additional carriers** — multi-vendor SM-DP+ with fallback.
4. **IoT optimization** — SGP.32 (IoT eSIM), NB-IoT/LTE-M, power-saving features.
5. **Developer tools** — expanded CLI, Terraform provider, GitHub Actions integration.
2. **Advanced Analytics**
- Real-time dashboards with WebSockets
- Predictive usage forecasting
- Anomaly detection (unusual data patterns)
## Related Docs
3. **Additional Carriers**
- Support for multiple SM-DP+ vendors
- Dynamic carrier selection based on region
- Fallback mechanisms
4. **IoT Optimization**
- SGP.32 support (IoT eSIM standard)
- LPWAN protocol support (NB-IoT, LTE-M)
- Power-saving features
5. **Developer Tools**
- CLI tool for API interaction
- Terraform provider
- GitHub Actions integration
- [`GETTING_STARTED.md`](./GETTING_STARTED.md) — Setup, API reference, SDKs
- [`OPERATIONS.md`](./OPERATIONS.md) — Deployment, monitoring, performance tuning