This pull request introduces a new feature that allows adding, updating,
and querying project grants The changes are primarily in the backend and
include new database tables, repositories, and domain logic to support
project grants.
## Changes
* New `project_grants` table: A new table `zitadel.project_grants` is
introduced to store project grants It includes columns for
`instance_id`, `id`, `state`, `project_id`, `granted_org_id` and
`granting_org_id`.
* New `ProjectGrantRepository`: A new repository
`ProjectGrantRepository` is created to handle all database operations
for project grants. It provides methods to `Get`, `List`, `Create`,
`Update`, and `Delete`.
* New `project_grant_roles` table: A new table
`zitadel.project_grant_roles` is introduced to store project grant roles
It includes columns for `instance_id`, `grant_id`, `key`, `project_id`
and `project_org_id`.
* New `ProjectGrantRoleRepository`: A new repository
`ProjectGrantRoleRepository` is created to handle all database
operations for project grant roles. It provides methods to `Get`,
`List`, `Add`, and `Delete`.
* New `project_grant_relational_projection`: A new projection
`project_grant_relational_projection` is added to update the
`zitadel.project_grants` and `zitadel.project_grant_roles` tables based
on events. It handles all project grant events.
* Updated `ProjectRepository`: The `ProjectRepository` is updated to
list granted projects.
# Additional Context
Closes https://github.com/zitadel/zitadel/issues/10766
---------
Co-authored-by: Stefan Benz <[email protected]>
Co-authored-by: Marco A. <[email protected]>
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
Relational table's primary keys are part of the SQL table definition.
However, by abstraction through the repository interfaces callers
(service layer) have no clear understanding of the primary key definion.
This might lead to confusion during implementation:
- The repository makes a sanity check on required conditions before
query execution and returns an error if the condition was not met.
- When a Get method is executed, an error is returned if not exactly one
row is found. Get replaces the legacy GetXxxByID methods we have on
projections. However, a list of flexible conditions can be passed.
- Operations like UPDATE and DELETE should often only affect a single
row, identified by the primary key. If the primary key conditions are
missing an error is returned.
Because there is no "type safety" enforced here, the errors are returned
during runtime. When all possible combinations of API arguments are not
properly tested such errors are actually bugs in the experience of the
end-user. (The API should do it's own input validation, but a
programmer's error is easily made).
# How the Problems Are Solved
Add `PrimaryKeyColumns` and `PrimaryKeyCondition` to existing repository
intefaces.
# Additional Changes
- Where reducers already used repositories, the PrimaryKeyConditions are
used where possible.
# Additional Context
- Part of
https://github.com/zitadel/zitadel/wiki/Decision-Log#expose-primary-key-definition-in-repository-interfaces
---------
Co-authored-by: Marco A. <[email protected]>
# Which Problems Are Solved
Add projects to the relational tables
# How the Problems Are Solved
- Define table migrations
- Define and implement Project and Project Role repositories.
- Provide projection handlers to populate the relational tables.
# Additional Changes
- Statement Builder now has a constructor which allows setting of a base
query with arguments.
- Certain operations, like Get, Update and Delete require the Primary
Key to be set as conditions. However, this requires knowledge of the
implementation and table definition. This PR proposes an additional
condition for repositories: `PrimaryKeyCondition`. This gives clarity on
the required IDs for these operations.
- Added couple of helpers to the repository package, to allow more DRY
code.
- getOne / getMany: generic functions for query execution and scanning.
- checkRestrictingColumns, checkPkCondition: simplify condition
checking, instead of using ladders of conditionals.
- Added a couple of helpers to the repository test package:
- Transaction, savepoint and rollback helpers.
- Create instance and organization helpers for objects that depend on
them (like projects).
# Additional Context
- after https://github.com/zitadel/zitadel/pull/10809
- closes#10765
This pull request introduces a significant refactoring of the database
interaction layer, focusing on improving explicitness, transactional
control, and error handling. The core change is the removal of the
stateful `QueryExecutor` from repository instances. Instead, it is now
passed as an argument to each method that interacts with the database.
This change makes transaction management more explicit and flexible, as
the same repository instance can be used with a database pool or a
specific transaction without needing to be re-instantiated.
### Key Changes
- **Explicit `QueryExecutor` Passing:**
- All repository methods (`Get`, `List`, `Create`, `Update`, `Delete`,
etc.) in `InstanceRepository`, `OrganizationRepository`,
`UserRepository`, and their sub-repositories now require a
`database.QueryExecutor` (e.g., a `*pgxpool.Pool` or `pgx.Tx`) as the
first argument.
- Repository constructors no longer accept a `QueryExecutor`. For
example, `repository.InstanceRepository(pool)` is now
`repository.InstanceRepository()`.
- **Enhanced Error Handling:**
- A new `database.MissingConditionError` is introduced to enforce
required query conditions, such as ensuring an `instance_id` is always
present in `UPDATE` and `DELETE` operations.
- The database error wrapper in the `postgres` package now correctly
identifies and wraps `pgx.ErrTooManyRows` and similar errors from the
`scany` library into a `database.MultipleRowsFoundError`.
- **Improved Database Conditions:**
- The `database.Condition` interface now includes a
`ContainsColumn(Column) bool` method. This allows for runtime checks to
ensure that critical filters (like `instance_id`) are included in a
query, preventing accidental cross-tenant data modification.
- A new `database.Exists()` condition has been added to support `EXISTS`
subqueries, enabling more complex filtering logic, such as finding an
organization that has a specific domain.
- **Repository and Interface Refactoring:**
- The method for loading related entities (e.g., domains for an
organization) has been changed from a boolean flag (`Domains(true)`) to
a more explicit, chainable method (`LoadDomains()`). This returns a new
repository instance configured to load the sub-resource, promoting
immutability.
- The custom `OrgIdentifierCondition` has been removed in favor of using
the standard `database.Condition` interface, simplifying the API.
- **Code Cleanup and Test Updates:**
- Unnecessary struct embeddings and metadata have been removed.
- All integration and repository tests have been updated to reflect the
new method signatures, passing the database pool or transaction object
explicitly.
- New tests have been added to cover the new `ExistsDomain`
functionality and other enhancements.
These changes make the data access layer more robust, predictable, and
easier to work with, especially in the context of database transactions.
This PR initiates the rework of Zitadel's backend to state-persisted
objects. This change is a step towards a more scalable and maintainable
architecture.
## Changes
* **New `/backend/v3` package**: A new package structure has been
introduced to house the reworked backend logic. This includes:
* `domain`: Contains the core business logic, commands, and repository
interfaces.
* `storage`: Implements the repository interfaces for database
interactions with new transactional tables.
* `telemetry`: Provides logging and tracing capabilities.
* **Transactional Tables**: New database tables have been defined for
`instances`, `instance_domains`, `organizations`, and `org_domains`.
* **Projections**: New projections have been created to populate the new
relational tables from the existing event store, ensuring data
consistency during the migration.
* **Repositories**: New repositories provide an abstraction layer for
accessing and manipulating the data in the new tables.
* **Setup**: A new setup step for `TransactionalTables` has been added
to manage the database migrations for the new tables.
This PR lays the foundation for future work to fully transition to
state-persisted objects for these components, which will improve
performance and simplify data access patterns.
This PR initiates the rework of ZITADEL's backend to state-persisted
objects. This is a foundational step towards a new architecture that
will improve performance and maintainability.
The following objects are migrated from event-sourced aggregates to
state-persisted objects:
* Instances
* incl. Domains
* Orgs
* incl. Domains
The structure of the new backend implementation follows the software
architecture defined in this [wiki
page](https://github.com/zitadel/zitadel/wiki/Software-Architecturel).
This PR includes:
* The initial implementation of the new transactional repositories for
the objects listed above.
* Projections to populate the new relational tables from the existing
event store.
* Adjustments to the build and test process to accommodate the new
backend structure.
This is a work in progress and further changes will be made to complete
the migration.
---------
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Iraq Jaber <[email protected]>
Co-authored-by: Iraq <[email protected]>
Co-authored-by: Tim Möhlmann <[email protected]>
