mirror of
https://github.com/andwati/zitadel.git
synced 2026-10-18 13:52:51 +00:00
d10be4c09ad94b725da170e1284d59bfbf2c7f0e
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  
|
f04d873270 |
feat(domain): Organization APIs with relational tables (#10704)
# Which Problems Are Solved This PR implements the endpoints listed in #10445 using relational tables. - [x] UpdateOrganization - [x] ListOrganizations - [x] DeleteOrganization - [x] DeactivateOrganization - [x] ActivateOrganization # How the Problems Are Solved - **UpdateOrganization:** Implemented logic for `Validate()`, `Execute()` and `Events()`. On CQRS side, updating an organization emits events related to the organization and its domains. Separate commanders have been made for the domain events so that their logic is run accordingly and we keep separation of concerns. Implementation of the domains is left to its own ticket. Because some domain-related data is changed during the organization update, its original values are saved as pointers inside [UpdateOrgCommand](https://github.com/zitadel/zitadel/blob/4f87cd8d0cfd75f290aa13c342e4939cecebef73/backend/v3/domain/org_update.go#L20). These variables will be then used as input for the domain commanders (that will run after the update organization one). - **ListOrganizations:** Implemented logic for `Validate()`, `Execute()`. Utility methods have been made to parse the input queries. This commander is used both by the v2 and v2beta endpoint, so a translation layer has been put in place to translate all v2beta requests into v2. The commander strictly handles v2 request only. The translation layer between v2beta and v2 APIs was made so to make it simple to remove it once the v2beta APIs will be removed (a simple deletion should suffice). A `converter` package [is shipped](https://github.com/zitadel/zitadel/blob/4f87cd8d0cfd75f290aa13c342e4939cecebef73/backend/v3/api/org/v2/convert/README.md#L5) as well with instruction for removal. TODOs have been put in place all over the code to indicate that the related piece of code needs to be removed once v2beta -> v2 transition is complete. - **DeleteOrganization:** Implemented logic for `Validate()`, `Execute()` and `Events()`. The `Events()` method is mostly incomplete as it requires a lot of data that is currently not retrievable due to the lack of their relational tables. - **DeactivateOrganization** and **ActivateOrganization**: Implemented logic for `Validate()`, `Execute()` and `Events()`. No notable remarks. 👉 All endpoints return matching errors to the CQRS counterpart: this way allows re-using the integration tests and 👉 All endpoints have been unit-tested through DB mocking 👉 A bunch of TODOs have been put in place for future reworks that are needed (e.g. `Update()` method on repository should also return a timestamp) # Additional Changes - A test utility method to allow mocking query options have been made. This was necessary because `database.QueryOption` is a function and the only comparison possible with functions is `nil` check. So, to mock this, `database.QueryOption` is converted first to `database.QueryOpts`. See 2276742adaef1998ea4844e1f0dca2079e45ccfb - A `BaseCommand` has been created with the idea to collect methods that are going to be needed by other commands. For the moment, it only offers a method to convert query text operations from gRPC to `domain` model. See bb85456dd101282fa76d2e2ad165eed4c8ef61b9 - SQL operations have been reworked to allow for queries using `ContainsIgnoreCase`, `EndsWithIgnoreCase` and other `IgnoreCase` operations. This change is pending approval and might be reworked. See https://github.com/zitadel/zitadel/pull/10704#discussion_r2392886177 # Additional Context `UpdateOrganization` is partially working because the logic for handling domain updates is not done (there is only a draft of the commanders to showcase the call chain). See https://github.com/zitadel/zitadel/pull/10704#discussion_r2356143178 Permissions are not implemented as they are missing. See https://github.com/zitadel/zitadel/pull/10771 - Closes: #10445 --------- Co-authored-by: Silvan <[email protected]> |
||
|
SilvanandGitHub
|
6686c3b16b |
feat(relational): org metadata (#10761)
This pull request introduces a new feature that allows adding, updating, and querying metadata for organizations. The changes are primarily in the backend and include new database tables, repositories, and domain logic to support organization metadata. ## Changes * New `org_metadata` table: A new table `zitadel.org_metadata` is introduced to store metadata for organizations. It includes columns for `instance_id`, `org_id`, `key`, and `value`. The `value` is stored as a `BYTEA` type to allow for flexible data storage. * New `OrganizationMetadataRepository`: A new repository `OrganizationMetadataRepository` is created to handle all database operations for organization metadata. It provides methods to `Get`, `List`, `Set`, and `Remove` metadata. * New `org_metadata_relational_projection`: A new projection `org_metadata_relational_projection` is added to update the `zitadel.org_metadata` table based on events. It handles `MetadataSet`, `MetadataRemoved`, and `MetadataRemovedAll` events. * Updated `OrganizationRepository`: The `OrganizationRepository` is updated to support loading organization metadata. A new method `LoadMetadata` is added to enable joining the `org_metadata` table when querying for organizations. * Updated Organization domain: The Organization domain model is updated to include a new field `Metadata` of type `[]*OrganizationMetadata`. ## Additional Info * Extensible Design: The new metadata feature is designed to be extensible, allowing for future enhancements such as indexing on specific JSON fields within the `value` column. * closes https://github.com/zitadel/zitadel/issues/10206 * closes https://github.com/zitadel/zitadel/issues/10214 |
||
 Tim MöhlmannandGitHub
|
fcb05fd561 |
feat(relational): command permission check (#10771)
# Which Problems Are Solved There is no implementation of the permission framework in the new command pattern. We risk forgetting permission options as we are already migrating the service layer to the new pattern. # How the Problems Are Solved Use an permission checker interface, which will be a sub-interface of the Permission Repository. The permission checker is passed in the command executor and invoker as part of the command options. From here the implementation can call the correct permission method. This PR proposes a check method for each [level](https://github.com/zitadel/zitadel/wiki/Internal-AuthZ#levels). # Additional Changes - none # Additional Context - Draft design: https://github.com/zitadel/zitadel/wiki/Internal-AuthZ#enforcement - Related for queries: https://github.com/zitadel/zitadel/pull/10756 |
||
|
Tim MöhlmannandGitHub
|
a45908b364 |
feat(rt): project repository (#10789)
# Which Problems Are Solved Add projects to the relational tables # How the Problems Are Solved - Define table migrations - Define and implement Project and Project Role repositories. - Provide projection handlers to populate the relational tables. # Additional Changes - Statement Builder now has a constructor which allows setting of a base query with arguments. - Certain operations, like Get, Update and Delete require the Primary Key to be set as conditions. However, this requires knowledge of the implementation and table definition. This PR proposes an additional condition for repositories: `PrimaryKeyCondition`. This gives clarity on the required IDs for these operations. - Added couple of helpers to the repository package, to allow more DRY code. - getOne / getMany: generic functions for query execution and scanning. - checkRestrictingColumns, checkPkCondition: simplify condition checking, instead of using ladders of conditionals. - Added a couple of helpers to the repository test package: - Transaction, savepoint and rollback helpers. - Create instance and organization helpers for objects that depend on them (like projects). # Additional Context - after https://github.com/zitadel/zitadel/pull/10809 - closes #10765 |
||
 IraqandGitHub
|
08f41e034e | feat(idp_table_relational): adding inital idp tables for relational repository (#10334) | ||
|
SilvanandGitHub
|
cccfc816f6 |
refactor: database interaction and error handling (#10762)
This pull request introduces a significant refactoring of the database interaction layer, focusing on improving explicitness, transactional control, and error handling. The core change is the removal of the stateful `QueryExecutor` from repository instances. Instead, it is now passed as an argument to each method that interacts with the database. This change makes transaction management more explicit and flexible, as the same repository instance can be used with a database pool or a specific transaction without needing to be re-instantiated. ### Key Changes - **Explicit `QueryExecutor` Passing:** - All repository methods (`Get`, `List`, `Create`, `Update`, `Delete`, etc.) in `InstanceRepository`, `OrganizationRepository`, `UserRepository`, and their sub-repositories now require a `database.QueryExecutor` (e.g., a `*pgxpool.Pool` or `pgx.Tx`) as the first argument. - Repository constructors no longer accept a `QueryExecutor`. For example, `repository.InstanceRepository(pool)` is now `repository.InstanceRepository()`. - **Enhanced Error Handling:** - A new `database.MissingConditionError` is introduced to enforce required query conditions, such as ensuring an `instance_id` is always present in `UPDATE` and `DELETE` operations. - The database error wrapper in the `postgres` package now correctly identifies and wraps `pgx.ErrTooManyRows` and similar errors from the `scany` library into a `database.MultipleRowsFoundError`. - **Improved Database Conditions:** - The `database.Condition` interface now includes a `ContainsColumn(Column) bool` method. This allows for runtime checks to ensure that critical filters (like `instance_id`) are included in a query, preventing accidental cross-tenant data modification. - A new `database.Exists()` condition has been added to support `EXISTS` subqueries, enabling more complex filtering logic, such as finding an organization that has a specific domain. - **Repository and Interface Refactoring:** - The method for loading related entities (e.g., domains for an organization) has been changed from a boolean flag (`Domains(true)`) to a more explicit, chainable method (`LoadDomains()`). This returns a new repository instance configured to load the sub-resource, promoting immutability. - The custom `OrgIdentifierCondition` has been removed in favor of using the standard `database.Condition` interface, simplifying the API. - **Code Cleanup and Test Updates:** - Unnecessary struct embeddings and metadata have been removed. - All integration and repository tests have been updated to reflect the new method signatures, passing the database pool or transaction object explicitly. - New tests have been added to cover the new `ExistsDomain` functionality and other enhancements. These changes make the data access layer more robust, predictable, and easier to work with, especially in the context of database transactions. |
||
|
SilvanandGitHub
|
22ef817d5c |
fix(eventstore): Make Eventstore Compatible with Relational Table Package (#10687)
Improves compatibility of eventstore and related database components with the new relational table package. ## Which problems are solved 1. **Incompatible Database Interfaces**: The existing eventstore was tightly coupled to the database package, which is incompatible with the new, more abstract relational table package in v3. This prevented the new command-side logic from pushing events to the legacy eventstore. 2. **Missing Health Checks**: The database interfaces in the new package lacked a Ping method, making it impossible to perform health checks on database connections. 3. **Event Publishing Logic**: The command handling logic in domain needed a way to collect and push events to the legacy eventstore after a command was successfully executed. ## How the problems are solved 1. **`LegacyEventstore` Interface**: * A new `LegacyEventstore` interface is introduced in the new `database/eventstore` . This interface exposes a `PushWithNewClient` method that accepts the new `database.QueryExecutor` interface, decoupling the v3 domain from the legacy implementation. * The `internal/eventstore.Eventstore` now implements this interface. A wrapper, PushWithClient, is added to convert the old database client types (`*sql.DB`, `*sql.Tx`) into the new `QueryExecutor` types before calling `PushWithNewClient`. 2. **Database Interface Updates**: * The `database.Pool` and `database.Client` interfaces in `storage/eventstore` have been updated to include a Ping method, allowing for consistent health checks across different database dialects. * The `postgres` and `sql` dialect implementations have been updated to support this new method. 3. **Command and Invoker Refactoring**: * The `Commander` interface in domain now includes an `Events() []legacy_es.Command` method. This allows commands to declare which events they will generate. * The `eventCollector` in the invoker logic has been redesigned. It now ensures a database transaction is started before executing a command. After successful execution, it calls the `Events()` method on the command to collect the generated events and appends them to a list. * The `eventStoreInvoker` then pushes all collected events to the legacy eventstore using the new `LegacyEventstore` interface, ensuring that events are only pushed if the entire command (and any sub-commands) executes successfully within the transaction. 4. **Testing**: * New unit tests have been added for the invoker to verify that events are correctly collected from single commands, batched commands, and nested commands. These changes create a clean bridge between the new v3 command-side logic and the existing v1 eventstore, allowing for incremental adoption of the new architecture while maintaining full functionality. ## Additional Information closes https://github.com/zitadel/zitadel/issues/10442 |
||
 
|
61cab8878e |
feat(backend): state persisted objects (#9870)
This PR initiates the rework of Zitadel's backend to state-persisted objects. This change is a step towards a more scalable and maintainable architecture. ## Changes * **New `/backend/v3` package**: A new package structure has been introduced to house the reworked backend logic. This includes: * `domain`: Contains the core business logic, commands, and repository interfaces. * `storage`: Implements the repository interfaces for database interactions with new transactional tables. * `telemetry`: Provides logging and tracing capabilities. * **Transactional Tables**: New database tables have been defined for `instances`, `instance_domains`, `organizations`, and `org_domains`. * **Projections**: New projections have been created to populate the new relational tables from the existing event store, ensuring data consistency during the migration. * **Repositories**: New repositories provide an abstraction layer for accessing and manipulating the data in the new tables. * **Setup**: A new setup step for `TransactionalTables` has been added to manage the database migrations for the new tables. This PR lays the foundation for future work to fully transition to state-persisted objects for these components, which will improve performance and simplify data access patterns. This PR initiates the rework of ZITADEL's backend to state-persisted objects. This is a foundational step towards a new architecture that will improve performance and maintainability. The following objects are migrated from event-sourced aggregates to state-persisted objects: * Instances * incl. Domains * Orgs * incl. Domains The structure of the new backend implementation follows the software architecture defined in this [wiki page](https://github.com/zitadel/zitadel/wiki/Software-Architecturel). This PR includes: * The initial implementation of the new transactional repositories for the objects listed above. * Projections to populate the new relational tables from the existing event store. * Adjustments to the build and test process to accommodate the new backend structure. This is a work in progress and further changes will be made to complete the migration. --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Iraq Jaber <[email protected]> Co-authored-by: Iraq <[email protected]> Co-authored-by: Tim Möhlmann <[email protected]> |