Commit Graph
304 Commits
Author SHA1 Message Date
NutcaseandGitHub d61aa93bd5 Multi carriersm dp+ (#61)
* refactor: Rename handler package to handlers in carrier-connector

- Rename internal/handler directory to internal/handlers
- Update handler import path in main.go and routes.go

* feat: Add SM-DP+ handler with multi-carrier management and profile operations

- Add SMDPHandler struct with manager field and NewSMDPHandler constructor
- Add AddCarrier endpoint with JSON binding and carrier configuration support
- Add RemoveCarrier endpoint with carrier ID parameter validation
- Add DownloadProfile endpoint with 30-second timeout and context cancellation
- Add GetCarrierStatus endpoint returning all carriers with timestamp
- Add GetProfileStatus endpoint with EID/ICCID validation

* feat: Add SM-DP+ integration layer with multi-carrier system initialization and route registration

- Add SMDPIntegration struct with service, handler, repository, and logger fields
- Add NewSMDPIntegration constructor with default carrier initialization
- Add InitializeSystem method with carrier health logging and status tracking
- Add RegisterRoutes method with profile, carrier, health, and metrics endpoints
- Add healthHandler with carrier health status aggregation and degraded state detection
- Add metricsHandler

* feat: Add SM-DP+ service layer with multi-carrier management and profile download operations

- Add SMDPService struct with manager, repository, and logger fields
- Add NewSMDPService constructor with default config and background health checking
- Add InitializeDefaultCarriers with AT&T, Verizon, and T-Mobile DE configurations
- Add DownloadProfile method with context support and request logging
- Add GetOptimalCarrier wrapping SelectCarrier for best carrier selection
- Add GetCarrierHealth returning

* feat: Add SM-DP+ manager with multi-carrier load balancing, health checking, and profile operations

- Add DefaultManagerConfig with 30s health checks, 3 retries, and circuit breaker configuration
- Add CarrierConfig struct with ES2 settings, MCC/MNC, priority, and feature support
- Add ToCarrier method converting CarrierConfig to Carrier with metrics and capabilities
- Add HealthChecker with periodic carrier health monitoring and status evaluation
- Add LoadBalancer with weighted round-robin, least
2026-05-01 20:24:34 +03:00
nutcas3 1cbe66a0c0 refactor: Add context parameter to MessageQueue Publish method with timeout wrapping
- Add context.Context parameter to Publish method signature
- Replace context.Background() with provided context wrapped in timeout
- Rename timeout context variable from ctx to timeoutCtx for clarity
2026-05-01 09:06:43 +03:00
nutcas3 783d5c147b feat: Add XSS protection and HATEOAS links to subscriber and auth handlers with cursor pagination
- Add bluemonday import to auth.go and subscriber.go
- Add UGCPolicy sanitization for username, email, firstName, lastName, and role in Register
- Add conditional XSS sanitization for firstName, lastName, email, and MSISDN in CreateSubscriber and UpdateSubscriber
- Replace Page and PageSize with Cursor and Limit in ListSubscribers query parameters
- Add HATEOAS links to CreateSubscriber, GetSubscriber
2026-05-01 09:06:32 +03:00
nutcas3 36748d534e refactor: Replace offset pagination with cursor-based pagination for subscriber listing
- Replace Page and PageSize with Cursor and Limit in ListSubscribersRequest
- Replace Total, Page, and PageSize with NextCursor and HasMore in ListSubscribersResponse
- Add default limit of 20 with max cap of 100 in ListSubscribers
- Replace ListSubscribers call with ListSubscribersCursor in subscriber_crud.go
- Remove unused database import from subscriber_crud.go
- Replace context.Background() with context.Without
2026-05-01 09:06:13 +03:00
nutcas3 444d767ceb feat: Add CSRF protection middleware to carrier-connector with secret validation
- Add gin-csrf middleware import to main.go
- Add CSRF_SECRET environment variable support with default fallback warning
- Add CSRF middleware to Gin router with configurable secret
- Add default CSRF secret with production warning for missing environment variable
2026-05-01 09:05:38 +03:00
nutcas3 49dc77d241 feat: Add CSP, Referrer-Policy, and Permissions-Policy headers to SecurityHeadersMiddleware
- Add Content-Security-Policy header with self-only defaults and unsafe-inline styles
- Add Referrer-Policy header with strict-origin-when-cross-origin
- Add Permissions-Policy header blocking geolocation, microphone, and camera access
2026-05-01 09:05:02 +03:00
nutcas3 5b6cee5618 feat: Add cursor-based pagination for subscribers and parameterize test password
- Add ListSubscribersCursor method to Database with ID-based cursor pagination
- Add cursor, hasMore, and nextCursor support for efficient pagination
- Add status, organizationID, and search filters to cursor-based query
- Add limit+1 fetch strategy to detect additional results
- Add fmt import to database/crud.go
- Add TEST_PASSWORD environment variable support in integration tests with default fallback
- Replace hard
2026-05-01 09:04:41 +03:00
nutcas3 4b04048c9b feat: Add SSRF protection to charging engine client with URL validation and domain whitelisting
- Add validateURL method to ChargingEngineClient with private IP blocking and domain whitelisting
- Add SSRF validation to CheckCredit, DeductCredit, AddCredit, GetBalance, and HealthCheck endpoints
- Add errors, net, url, and slices imports to charging_client.go
- Update HealthCheck endpoint from /health to /v1/health for consistency
- Add allowed domains list including localhost, 127.0.0.1, charging
2026-05-01 09:03:25 +03:00
nutcas3 b2376e2133 feat: Add HATEOAS link support for subscriber resources with navigation and action links
- Add Link struct with href, rel, and optional type fields
- Add Links map type for HATEOAS link collections
- Add SubscriberWithLinks and SubscriberListWithLinks wrapper types
- Add NewSubscriberLinks function generating self, update, delete, billing, and sessions links
- Add NewSubscriberListLinks function with self, create, and conditional next pagination links
2026-05-01 09:02:12 +03:00
nutcas3 4e4b13f3e8 refactor: Simplify validation parsing using let-chains in sync.rs and wrap unsafe env::set_var in tests
- Replace nested if-let statements with let-chain pattern for VmRSS memory parsing in /proc/self/status
- Replace nested if-let statements with let-chain pattern for Redis used_memory parsing
- Wrap std::env::set_var calls in unsafe blocks in validation_tests.rs for MAX_BYTES_LIMIT and MAX_AMOUNT_LIMIT
2026-05-01 00:43:54 +03:00
nutcas3 11065b827b refactor: Simplify Redis RPS extraction using let-chains in monitoring_impl.rs
- Replace nested if-let statements with let-chain pattern for Redis RPS parsing
- Update module reference from monitoring to monitoring_impl in monitoring.rs
2026-05-01 00:43:41 +03:00
nutcas3 220b7e0380 refactor: Rename monitoring.rs to monitoring_impl.rs in charging-engine 2026-05-01 00:43:22 +03:00
nutcas3 dd346a7484 refactor: Update Rust version to 1.95, add SSRF protection to ES2 client, and fix CLI API endpoint paths
- Change rust-version from "1.94" to "1.95" in workspace Cargo.toml
- Add validateURL method to ES2Client with private IP blocking and domain whitelisting
- Add SSRF validation to DownloadProfile and GetProfileStatus endpoints
- Add slices, errors, net, and url imports to es2/client.go
- Update CLI charging API endpoints from /v1 to /api/v1 prefix
- Fix DeductCredit response handling to use nil
2026-05-01 00:41:57 +03:00
nutcas3 b1f18e1a4e feat: Add comprehensive Swagger API documentation for subscriber management endpoints
- Add GET /api/v1/subscribers endpoint with pagination and filtering parameters
- Add POST /api/v1/subscribers endpoint for subscriber creation
- Add GET /api/v1/subscribers/{id} and GET /api/v1/subscribers/imsi/{imsi} endpoints
- Add PUT /api/v1/subscribers/{id} endpoint for subscriber updates
- Add POST /api/v1/subscribers/{id}/suspend and /api/v1/subscribers/{id}/terminate endpoints
- Add Swagger definitions
2026-04-30 22:59:46 +03:00
nutcas3 2e813c3d90 refactor: Add CSRF protection and session management dependencies to carrier-connector
- Add github.com/utrack/gin-csrf v0.0.0 for CSRF protection
- Add github.com/gin-contrib/sessions and related session management dependencies
- Add github.com/gorilla/context, gorilla/securecookie, and gorilla/sessions for session handling
- Add github.com/dchest/uniuri for unique identifier generation
- Add indirect dependencies for various session store implementations (redistore, memcache, mongostore)
2026-04-30 22:55:19 +03:00
nutcas3 cebed0a904 feat: Add CSRF protection middleware and Swagger annotations, remove GraphQL handler setup
- Add github.com/utrack/gin-csrf import for CSRF protection
- Remove internal/graphql import and resolver initialization
- Add Swagger API documentation annotations (@title, @version, @description, @host, @BasePath, @securityDefinitions)
- Add CSRF middleware with configurable secret from CSRF_SECRET environment variable
- Add warning log when using default CSRF secret
- Add return statement after database
2026-04-30 22:53:09 +03:00
nutcas3 51c1ad6573 feat: Add Swagger documentation endpoint and update API route prefix from /v1 to /api/v1
- Add swaggo/files and swaggo/gin-swagger imports for Swagger documentation
- Change API route group prefix from "/v1" to "/api/v1"
- Add GET /api/v1/swagger/*any endpoint for Swagger UI documentation
- Update RBAC path from "/v1/users" to "/api/v1/users" for consistency
2026-04-30 22:49:51 +03:00
nutcas3 afe4084ae2 refactor: Replace log.Printf with structured logging using logrus in api-server metrics collection
- Remove standard log import from server_metrics.go
- Add logrus import for structured logging
- Replace log.Printf with logrus.WithError for system metrics storage errors
- Replace log.Printf with logrus.WithError for subscriber count errors (total, active, suspended)
- Replace log.Printf with logrus.WithError for active sessions count errors
- Replace log.Printf with logrus.WithError for /proc/stat, /proc/meminfo, and /proc/net/dev read errors
2026-04-30 22:43:37 +03:00
nutcas3 1bb9b64322 refactor: Replace GraphQL dependencies with CSRF protection and HTML sanitization in api-server
- Remove github.com/99designs/gqlgen and github.com/vektah/gqlparser/v2 GraphQL dependencies
- Add github.com/utrack/gin-csrf v0.0.0 for CSRF protection
- Add github.com/microcosm-cc/bluemonday v1.0.27 for HTML sanitization
- Add github.com/gin-contrib/sessions and related session management dependencies
- Remove GraphQL-related indirect dependencies (agnivade/levenshtein, hashicorp/golang-lru/v2, so
2026-04-30 22:41:21 +03:00
nutcas3 f8504de4ef refactor: Remove generated GraphQL code and resolver files from api-server
- Delete generated.go containing GraphQL type definitions, resolvers, and schema
- Delete resolvers.go containing root Resolver struct and helper functions
- Remove GraphQL enum constants (SubscriberStatus, ProfileStatus, UsageType, etc.)
- Remove GraphQL input types (SubscriberFilter, CreateSubscriberInput, etc.)
- Remove GraphQL output types (PageInfo, SubscriberConnection, UsageStats, etc.)
- Remove resolver interfaces
2026-04-30 04:57:58 +03:00
nutcas3 9b64366e25 refactor: Fix Rust edition, remove duplicate lint rule, and improve error handling in charging-engine and packet-gateway
- Change Rust edition from "2026" to "2024" in workspace Cargo.toml
- Remove duplicate expect_used lint rule from workspace lints
- Prefix unused account variable with underscore in calculate_usage_cost
- Split CARGO_MANIFEST_DIR assignment into separate variable in run_migrations
- Clone plan_id before moving plan in add_rating_plan handler
- Replace expect() with unwrap() in governor
2026-04-30 03:17:46 +03:00
nutcas3 cd392894df refactor: Replace manual map copy with maps.Copy in carrier-connector message queue
- Add maps import to mq.go
- Replace manual for loop copying payload map with maps.Copy in PublishProfileEvent
- Simplify map merging logic while maintaining same functionality
2026-04-29 13:23:41 +03:00
nutcas3 ccb2ab7ee8 refactor: Add comprehensive documentation with examples to charging-engine modules
- Add module-level documentation to lib.rs with usage example
- Document Claims and AuthConfig structs with examples in auth.rs
- Add ChargingEngine struct documentation with initialization example
- Document ChargingRule, SubscriberAccount, AccountStatus, UsageEvent, UsageType, and RatingPlan with examples in types.rs
- Add CircuitState, CircuitBreakerError, and CircuitBreaker documentation with examples in circuit_breaker.rs
2026-04-29 04:36:40 +03:00
nutcas3 8fddb29716 refactor: Replace fmt.Printf with structured logging using logrus in api-server subscriber and invoice services
- Add logrus import to invoice.go, subscriber_crud.go, subscriber_esim.go, and subscriber_lifecycle.go
- Replace fmt.Printf with logrus.WithError and WithField for structured error logging
- Update invoice email notification placeholder to use logrus.Info with invoice_id field
- Replace eSIM provisioning error print with logrus.Warn including subscriber_id field
- Add comment explaining derived
2026-04-29 04:36:02 +03:00
nutcas3 b4e42d7717 refactor: Improve Redis key consistency and add existence check in credit management
- Change Redis key prefix from "block:" to "blocked:" in unblock_user for consistency
- Replace is_some() check with if let pattern in block_user to avoid unused variable
- Add existence check before attempting to delete blocked user key in unblock_user
- Use if let pattern for blocked status check in unblock_user
- Return Ok(()) silently if user is not blocked instead of attempting deletion
2026-04-29 04:24:49 +03:00
nutcas3 4008a19f4a refactor: Pre-allocate Vec capacity in rating_billing.rs to reduce reallocations
- Set applied_rules capacity to 6 (maximum number of charging rules that can be applied)
- Set keys capacity to 100 in get_usage_history and get_all_accounts (matches SCAN batch size)
- Add comments explaining capacity choices for each Vec initialization
2026-04-29 04:24:22 +03:00
nutcas3 5ec01e591e refactor: Replace fallback defaults with required environment variable validation in charging-engine auth
- Change AuthConfig::from_env() return type from Self to Result<Self, String>
- Remove default JWT_SECRET fallback, require environment variable to be set
- Add validation requiring JWT_SECRET to be at least 32 characters for security
- Replace expect() with ok_or_else in create_token timestamp calculation
- Replace expect() with map_err in DATABASE_URL environment variable loading
2026-04-29 04:23:58 +03:00
nutcas3 8bb332964b refactor: Replace unwrap() with unwrap_or_else fallback in rate limiter configuration
- Add fallback to default GovernorConfig (10 req/s, burst 20) if custom config fails
- Replace unwrap() with unwrap_or_else providing sensible defaults
- Add comment explaining fallback behavior for rate limiter initialization
2026-04-29 04:23:11 +03:00
nutcas3 5340469a1a refactor: Replace expect() with unwrap_or_else fallbacks and improve error handling in charging-engine
- Remove unused _account variable in calculate_usage_cost
- Replace HashMap::with_capacity with HashMap::new and use iterator chain in get_all_plans
- Add std::process::Command import and set current_dir to repo root in run_migrations
- Add comment explaining directory change to locate migrations folder
- Replace expect() with unwrap_or_else providing fallback JSON in SubscriberAccount and UsageEvent
2026-04-29 04:22:52 +03:00
nutcas3 1deecf32fe refactor: Replace manual Display and Error implementations with thiserror derive macro in ChargingError
- Add thiserror::Error derive to ChargingError enum
- Replace manual fmt::Display implementation with #[error] attributes on each variant
- Remove manual std::error::Error implementation (provided by thiserror)
- Remove std::fmt import as it's no longer needed
- Maintain same error message formats and functionality
2026-04-29 04:22:02 +03:00
nutcas3 4cc0b80b1c refactor: Remove unnecessary clone() calls and improve error handling in packet-gateway
- Remove clone() from rating plan parameter in add_rating_plan call
- Remove clone() calls from ChargingEngineClient and EbpfManager initialization
- Remove clone() from Redis client Arc wrapping
- Remove clone() calls from PacketGatewayConfig field assignments
- Replace expect() with proper error handling using map_err and anyhow in health check server
- Remove clone() from IP string in session data collection
2026-04-29 04:21:52 +03:00
nutcas3 b3ff568bd4 feat: Add workspace-level linting rules and cargo-audit configuration
- Add cargo-audit metadata section with placeholder for ignored advisories
- Forbid unsafe_code at workspace level
- Enable all and pedantic clippy warnings
- Deny unwrap_used, expect_used, and panic to enforce error handling
- Add thiserror dependency to charging-engine for improved error handling
2026-04-29 04:21:33 +03:00
nutcas3 609bdb16fa feat: Set repository root as working directory for Goose migrations in carrier-connector
- Add filepath import for path manipulation
- Calculate repository root path relative to carrier-connector location (../../)
- Set cmd.Dir to repository root before executing Goose command
- Add comment explaining directory change to locate migrations folder
- Ensure migrations can be found from carrier-connector's nested location
2026-04-29 04:21:19 +03:00
nutcas3 898e58201f refactor: Extract CRUD operations from database.go into separate crud.go file
- Move all subscriber, payment method, transaction, and notification CRUD operations to crud.go
- Remove models import from database.go as it's no longer needed
- Keep database initialization, connection management, and IMSI allocation in database.go
- Maintain ListSubscribersRequest struct definition in crud.go alongside related functions
2026-04-29 03:36:45 +03:00
nutcas3 393685846c fix: Convert Status field to string in ListSubscribers database request
- Cast req.Status to string when passing to database.ListSubscribersRequest
- Ensure type compatibility between service and database layer status fields
2026-04-29 03:31:29 +03:00
nutcas3 71353fc237 feat: Replace GORM AutoMigrate with Goose CLI migrations in api-server database initialization
- Add os/exec import for running external commands
- Replace AutoMigrate function with runMigrations function using goose postgres command
- Call runMigrations in NewDatabase before initializing IMSI allocation
- Remove PoolStats function and GORM-based schema migration code
- Reorganize database methods with section comments (Subscriber CRUD, Payment methods, Transactions, Additional CRUD, IMSI allocation)
-
2026-04-29 03:31:00 +03:00
nutcas3 69241334aa feat: Add automatic Goose migration execution on startup in charging-engine RatingPlansRepo
- Add std::process::Command import for running external commands
- Implement run_migrations function to execute goose postgres migrations
- Call run_migrations in RatingPlansRepo::connect before returning repository instance
- Update function comments to reflect automatic migration execution on startup
- Add error handling and output capture for migration failures
- Add tracing log for successful migration
2026-04-29 03:00:28 +03:00
nutcas3 76d7661443 feat: Add automatic Goose migration execution on startup in carrier-connector PostgresProfileStore
- Add os/exec import for running external commands
- Implement runMigrations function to execute goose postgres migrations
- Call runMigrations in NewPostgresProfileStore before returning store instance
- Update function comment to reflect automatic migration execution on startup
- Add error handling and output capture for migration failures
2026-04-29 02:57:41 +03:00
nutcas3 a8381e6b8f refactor: Remove inline schema creation from PostgresProfileStore in favor of centralized migration system
- Remove AutoMigrate call for profileRow from NewPostgresProfileStore
- Update function comment to reference migration system (make db-migrate) instead of inline schema creation
- Add note that esim_profiles table is created by migration system
- Remove error handling for AutoMigrate execution
2026-04-29 02:55:13 +03:00
nutcas3 85b97c0950 deps: Update lib/pq from v1.10.2 to v1.12.3 in api-server 2026-04-29 02:55:05 +03:00
nutcas3 eb5b9092c9 refactor: Remove inline schema creation from RatingPlansRepo in favor of centralized migration system
- Remove CREATE TABLE IF NOT EXISTS query for rating_plans table from connect() method
- Update comment to reference migration system (make db-migrate) instead of inline schema creation
- Add note that rating_plans table is created by 000001_init_rating_plans.up.sql migration
- Remove sqlx::query execution and error mapping for schema creation
- Keep database connection pool and circuit breaker initialization
2026-04-29 02:54:41 +03:00
nutcas3 4e5ad491ef refactor: Remove unused CircuitBreaker and ChargingError imports from charging-engine
- Remove unused CircuitBreaker import from credit_management.rs
- Remove unused ChargingError import from engine.rs
- Keep CircuitBreakerError import in credit_management.rs
- Keep CircuitBreaker import in engine.rs
2026-04-26 02:32:23 +03:00
nutcas3 db4d8bf56a refactor: Re-enable rate limiter with environment variable configuration in charging-engine routes
- Add tower_governor imports (GovernorConfigBuilder and GovernorLayer)
- Remove commented-out rate_limit module import
- Add environment variable parsing for RATE_LIMIT_RPS (default: 10) and RATE_LIMIT_BURST (default: 20)
- Create GovernorConfigBuilder with per_second and burst_size configuration
- Replace commented-out rate_limiter layer with GovernorLayer::new(governor_conf)
2026-04-26 02:32:14 +03:00
nutcas3 2094a3b952 refactor: Simplify rate limiter to use default configuration in charging-engine
- Remove unused imports: governor clock/state/Quota, std::num::NonZeroU32, std::sync::Arc, and SmartIpKeyExtractor
- Change create_rate_limiter return type from explicit generic parameters to simple GovernorLayer
- Replace custom rate limit configuration (RATE_LIMIT_RPS, RATE_LIMIT_BURST environment variables) with GovernorConfig::default()
- Remove quota builder logic and pass config directly to GovernorLayer::new()
2026-04-26 02:31:59 +03:00
nutcas3 186bba04a8 refactor: Remove duplicate imports and comment out rate limiter implementation in charging-engine
- Remove duplicate CreditManager and RatingPlansRepo imports from engine.rs
- Add circuit_breaker module to main.rs imports
- Comment out rate_limit module in lib.rs
- Update rate_limit.rs imports to use GovernorConfig and GovernorLayer directly
- Change GovernorConfigBuilder to GovernorConfig::builder() and update config initialization
- Update GovernorLayer::new() to take config reference
2026-04-26 02:17:26 +03:00
nutcas3 a4aec94314 refactor: Re-enable circuit breaker implementation in charging-engine credit management and rating plans
- Uncomment CircuitBreaker and CircuitBreakerError imports in credit_management.rs and rating_plans_repo.rs
- Re-enable redis_circuit_breaker execute calls for all Redis operations (get_balance, add_credit, deduct_credit, block_user, unblock_user, is_user_blocked, get_subscriber_account, update_subscriber_account, record_usage_event)
- Re-enable postgres circuit_breaker execute calls for all database operations
2026-04-26 02:13:07 +03:00
nutcas3 1d489a0a8a refactor: Make eBPF dependencies optional and add conditional compilation support to packet-gateway
- Add ebpf feature flag to Cargo.toml with default disabled
- Make aya and aya-log dependencies optional and tied to ebpf feature
- Add #[cfg(feature = "ebpf")] conditional compilation directives for EbpfManager imports and usage
- Add feature-gated readiness check logic (ebpf_ok only checked when feature enabled)
- Add feature-gated metrics_handler implementations (full version with eBPF, stub version without)
- Change
2026-04-26 02:00:27 +03:00
nutcas3 95def1bcab refactor: Re-enable circuit breaker implementation and add conditional eBPF compilation support
- Uncomment CircuitBreaker and CircuitBreakerError imports in charging-engine
- Re-enable redis_circuit_breaker and postgres_circuit_breaker fields in ChargingEngine struct
- Add circuit breaker initialization with environment variable configuration (REDIS_FAILURE_THRESHOLD, REDIS_TIMEOUT, POSTGRES_FAILURE_THRESHOLD, POSTGRES_TIMEOUT)
- Add #[cfg(feature = "ebpf")] conditional compilation directives throughout
2026-04-26 01:50:29 +03:00
nutcas3 10ac6bb861 refactor: Make eBPF build optional in packet-gateway build script
- Change eBPF build from required to optional with graceful fallback
- Add match statement to handle build success, failure, and missing cargo-build-bpf cases
- Add cargo warnings when eBPF build fails or cargo-build-bpf is not installed
- Remove panic on build failure to allow compilation without eBPF toolchain
2026-04-26 01:24:17 +03:00
nutcas3 d5ebb3bfd8 refactor: Comment out circuit breaker implementation and remove thiserror dependency
- Remove thiserror 2.0.18 dependency from Cargo.toml
- Comment out CircuitBreakerError import and all circuit breaker execute calls in credit_management.rs
- Comment out redis_circuit_breaker and postgres_circuit_breaker fields in ChargingEngine struct
- Comment out circuit breaker initialization code in engine.rs
- Comment out circuit_breaker field and initialization in RatingPlansRepo
- Remove circuit breaker
2026-04-26 01:15:58 +03:00