8 Commits

Include renames

Author	SHA1	Message	Date
3 people	c4221a9c0f	refactor: invoker implementations and clean up code (#10962) ... This pull request refactors the command execution and transaction handling logic within the `domain` package. The primary goal is to simplify the invoker chain, clarify transaction management, and improve the overall structure for better maintainability and readability. ### Key Changes: - **Simplified Invoker Chain**: The previous chain of invokers has been refactored. A new `transactionInvoker` is introduced to explicitly manage database transactions for commands that require them. This responsibility is no longer spread across different parts of the invocation flow. - **`Transactional` Interface**: A new `Transactional` interface has been introduced. Commands that need to run within a database transaction now implement this interface. This provides a clear and explicit contract for transactional operations. - **Consolidated Transaction Logic**: The logic for starting, committing, and rolling back transactions is now centralized within the `transactionInvoker` and the `eventStoreInvoker`. The `EnsureTx` method on `InvokeOpts` has been removed, and transaction handling is now managed automatically by the invocation framework. - **Improved Event Handling**: The `eventStoreInvoker` and its `eventCollector` have been rewritten to be more robust and to correctly handle nested and batched commands, ensuring that all events are collected and published within the same transaction. - **Code Cleanup**: A significant amount of now-unused code, including the old `invoker_test.go`, has been removed. Various comments and variable declarations have also been cleaned up. - **Updated Command Implementations**: All commands in the `domain` package (e.g., `ActivateOrgCommand`, `DeactivateOrgCommand`, `DeleteOrgCommand`, `UpdateOrgCommand`) have been updated to use the new `Transactional` interface where appropriate, removing manual transaction management. - **Enhanced Testing**: Tests have been added and updated to cover the new transaction and event handling logic, ensuring the refactoring is sound. - **Use `PrimaryKeyCondition`**: Use the `repository.PrimaryKeyCondition(...)` method instead of using `database.And(...)` where possible. These changes lead to a cleaner, more decoupled architecture for command handling in ZITADEL. --------- Co-authored-by: Marco A. <[email protected]>	2025-10-27 14:04:06 +00:00
SilvanandGitHub	6686c3b16b	feat(relational): org metadata (#10761) ... This pull request introduces a new feature that allows adding, updating, and querying metadata for organizations. The changes are primarily in the backend and include new database tables, repositories, and domain logic to support organization metadata. ## Changes * New `org_metadata` table: A new table `zitadel.org_metadata` is introduced to store metadata for organizations. It includes columns for `instance_id`, `org_id`, `key`, and `value`. The `value` is stored as a `BYTEA` type to allow for flexible data storage. * New `OrganizationMetadataRepository`: A new repository `OrganizationMetadataRepository` is created to handle all database operations for organization metadata. It provides methods to `Get`, `List`, `Set`, and `Remove` metadata. * New `org_metadata_relational_projection`: A new projection `org_metadata_relational_projection` is added to update the `zitadel.org_metadata` table based on events. It handles `MetadataSet`, `MetadataRemoved`, and `MetadataRemovedAll` events. * Updated `OrganizationRepository`: The `OrganizationRepository` is updated to support loading organization metadata. A new method `LoadMetadata` is added to enable joining the `org_metadata` table when querying for organizations. * Updated Organization domain: The Organization domain model is updated to include a new field `Metadata` of type `[]*OrganizationMetadata`. ## Additional Info * Extensible Design: The new metadata feature is designed to be extensible, allowing for future enhancements such as indexing on specific JSON fields within the `value` column. * closes https://github.com/zitadel/zitadel/issues/10206 * closes https://github.com/zitadel/zitadel/issues/10214	2025-10-14 10:15:16 +01:00
5 people	f69a6ed4f3	chore: rehaul DevX (#10571) ... # Which Problems Are Solved Replaces Turbo by Nx and lays the foundation for the next CI improvements. It enables using Nx Cloud to speed the up the pipelines that affect any node package. It streamlines the dev experience for frontend and backend developers by providing the following commands: | Task | Command | Notes | |------|---------|--------| | **Production** | `nx run PROJECT:prod` | Production server | | **Develop** | `nx run PROJECT:dev` | Hot reloading development server | | **Test** | `nx run PROJECT:test` | Run all tests | | **Lint** | `nx run PROJECT:lint` | Check code style | | **Lint Fix** | `nx run PROJECT:lint-fix` | Auto-fix style issues | The following values can be used for PROJECT: - @zitadel/zitadel (root commands) - @zitadel/api, - @zitadel/login, - @zitadel/console, - @zitadel/docs, - @zitadel/client - @zitadel/proto The project names and folders are streamlined: | Old Folder | New Folder | | --- | --- | | ./e2e | ./tests/functional-ui | | ./load-test | ./benchmark | | ./build/zitadel | ./apps/api | | ./console | ./apps/console (postponed so the PR is reviewable) | Also, all references to the TypeScript repo are removed so we can archive it. # How the Problems Are Solved - Ran `npx nx@latest init` - Replaced all turbo.json by project.json and fixed the target configs - Removed Turbo dependency - All JavaScript related code affected by a PRs changes is quality-checked using the `nx affected` command - We move PR checks that are runnable using Nx into the `check` workflow. For workflows where we don't use Nx, yet, we restore previously built dependency artifacts from Nx. - We only use a single and easy to understand dev container - The CONTRIBUTING.md is streamlined - The setup with a generated client pat is orchestrated with Nx - Everything related to the TypeScript repo is updated or removed. A **Deploy with Vercel** button is added to the docs and the CONTRIBUTING.md. # Additional Changes - NPM package names have a consistent pattern. - Docker bake is removed. The login container is built and released like the core container. - The integration tests build the login container before running, so they don't rely on the login container action anymore. This fixes consistently failing checks on PRs from forks. - The docs build in GitHub actions is removed, as we already build on Vercel. # Additional Context - Internal discussion: https://zitadel.slack.com/archives/C087ADF8LRX/p1756277884928169 - Workflow dispatch test: https://github.com/zitadel/zitadel/actions/runs/17760122959 --------- Co-authored-by: Florian Forster <[email protected]> Co-authored-by: Tim Möhlmann <[email protected]> Co-authored-by: Copilot <[email protected]>	2025-10-08 10:27:02 +02:00
Tim MöhlmannandGitHub	a45908b364	feat(rt): project repository (#10789) ... # Which Problems Are Solved Add projects to the relational tables # How the Problems Are Solved - Define table migrations - Define and implement Project and Project Role repositories. - Provide projection handlers to populate the relational tables. # Additional Changes - Statement Builder now has a constructor which allows setting of a base query with arguments. - Certain operations, like Get, Update and Delete require the Primary Key to be set as conditions. However, this requires knowledge of the implementation and table definition. This PR proposes an additional condition for repositories: `PrimaryKeyCondition`. This gives clarity on the required IDs for these operations. - Added couple of helpers to the repository package, to allow more DRY code. - getOne / getMany: generic functions for query execution and scanning. - checkRestrictingColumns, checkPkCondition: simplify condition checking, instead of using ladders of conditionals. - Added a couple of helpers to the repository test package: - Transaction, savepoint and rollback helpers. - Create instance and organization helpers for objects that depend on them (like projects). # Additional Context - after https://github.com/zitadel/zitadel/pull/10809 - closes #10765	2025-10-01 09:47:04 +00:00
IraqandGitHub	08f41e034e	feat(idp_table_relational): adding inital idp tables for relational repository (#10334)	2025-09-24 13:19:09 +00:00
SilvanandGitHub	cccfc816f6	refactor: database interaction and error handling (#10762) ... This pull request introduces a significant refactoring of the database interaction layer, focusing on improving explicitness, transactional control, and error handling. The core change is the removal of the stateful `QueryExecutor` from repository instances. Instead, it is now passed as an argument to each method that interacts with the database. This change makes transaction management more explicit and flexible, as the same repository instance can be used with a database pool or a specific transaction without needing to be re-instantiated. ### Key Changes - **Explicit `QueryExecutor` Passing:** - All repository methods (`Get`, `List`, `Create`, `Update`, `Delete`, etc.) in `InstanceRepository`, `OrganizationRepository`, `UserRepository`, and their sub-repositories now require a `database.QueryExecutor` (e.g., a `*pgxpool.Pool` or `pgx.Tx`) as the first argument. - Repository constructors no longer accept a `QueryExecutor`. For example, `repository.InstanceRepository(pool)` is now `repository.InstanceRepository()`. - **Enhanced Error Handling:** - A new `database.MissingConditionError` is introduced to enforce required query conditions, such as ensuring an `instance_id` is always present in `UPDATE` and `DELETE` operations. - The database error wrapper in the `postgres` package now correctly identifies and wraps `pgx.ErrTooManyRows` and similar errors from the `scany` library into a `database.MultipleRowsFoundError`. - **Improved Database Conditions:** - The `database.Condition` interface now includes a `ContainsColumn(Column) bool` method. This allows for runtime checks to ensure that critical filters (like `instance_id`) are included in a query, preventing accidental cross-tenant data modification. - A new `database.Exists()` condition has been added to support `EXISTS` subqueries, enabling more complex filtering logic, such as finding an organization that has a specific domain. - **Repository and Interface Refactoring:** - The method for loading related entities (e.g., domains for an organization) has been changed from a boolean flag (`Domains(true)`) to a more explicit, chainable method (`LoadDomains()`). This returns a new repository instance configured to load the sub-resource, promoting immutability. - The custom `OrgIdentifierCondition` has been removed in favor of using the standard `database.Condition` interface, simplifying the API. - **Code Cleanup and Test Updates:** - Unnecessary struct embeddings and metadata have been removed. - All integration and repository tests have been updated to reflect the new method signatures, passing the database pool or transaction object explicitly. - New tests have been added to cover the new `ExistsDomain` functionality and other enhancements. These changes make the data access layer more robust, predictable, and easier to work with, especially in the context of database transactions.	2025-09-24 10:12:31 +00:00
SilvanandGitHub	869282ca49	fix(repo): correct mapping for domains (#10653) ... This pull request fixes an issue where the repository would fail to scan organization or instance structs if the `domains` column was `NULL`. ## Which problems are solved If the `domains` column of `orgs` or `instances` was `NULL`, the repository failed scanning into the structs. This happened because the scanning mechanism did not correctly handle `NULL` JSONB columns. ## How the problems are solved A new generic type `JSONArray[T]` is introduced, which implements the `sql.Scanner` interface. This type can correctly scan JSON arrays from the database, including handling `NULL` values gracefully. The repositories for instances and organizations have been updated to use this new type for the domains field. The SQL queries have also been improved to use `FILTER` with `jsonb_agg` for better readability and performance when aggregating domains. ## Additional changes * An unnecessary cleanup step in the organization domain tests for already removed domains has been removed. * The `pgxscan` library has been replaced with `sqlscan` for scanning `database/sql`.Rows. * Minor cleanups in integration tests.	2025-09-08 09:35:31 +02:00
6 people	61cab8878e	feat(backend): state persisted objects (#9870) ... This PR initiates the rework of Zitadel's backend to state-persisted objects. This change is a step towards a more scalable and maintainable architecture. ## Changes * **New `/backend/v3` package**: A new package structure has been introduced to house the reworked backend logic. This includes: * `domain`: Contains the core business logic, commands, and repository interfaces. * `storage`: Implements the repository interfaces for database interactions with new transactional tables. * `telemetry`: Provides logging and tracing capabilities. * **Transactional Tables**: New database tables have been defined for `instances`, `instance_domains`, `organizations`, and `org_domains`. * **Projections**: New projections have been created to populate the new relational tables from the existing event store, ensuring data consistency during the migration. * **Repositories**: New repositories provide an abstraction layer for accessing and manipulating the data in the new tables. * **Setup**: A new setup step for `TransactionalTables` has been added to manage the database migrations for the new tables. This PR lays the foundation for future work to fully transition to state-persisted objects for these components, which will improve performance and simplify data access patterns. This PR initiates the rework of ZITADEL's backend to state-persisted objects. This is a foundational step towards a new architecture that will improve performance and maintainability. The following objects are migrated from event-sourced aggregates to state-persisted objects: * Instances * incl. Domains * Orgs * incl. Domains The structure of the new backend implementation follows the software architecture defined in this [wiki page](https://github.com/zitadel/zitadel/wiki/Software-Architecturel). This PR includes: * The initial implementation of the new transactional repositories for the objects listed above. * Projections to populate the new relational tables from the existing event store. * Adjustments to the build and test process to accommodate the new backend structure. This is a work in progress and further changes will be made to complete the migration. --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Iraq Jaber <[email protected]> Co-authored-by: Iraq <[email protected]> Co-authored-by: Tim Möhlmann <[email protected]>	2025-09-05 09:54:34 +01:00