Commit Graph
2024 Commits
Author SHA1 Message Date
33c51deb20 Merge commit from fork
* validate IDP linking permissions

* fix: check policy before linking

* apply suggested review comments

---------

Co-authored-by: Max Peintner <[email protected]>
2025-11-12 13:48:40 +01:00
a9846498a9 feat(group): add user groups to token claims (#11009)
# Which Problems Are Solved

If the groups of a user is requested through a scope, details related to
the user's groups are added in the token claims.

# How the Problems Are Solved
By querying the user groups that the user is a part of, and setting them
in the token claims.

This PR introduces two scopes to request user groups: `groups` and
`urn:zitadel:iam:user:groups`
* `groups` returns an array of group names in the token claims with the
same name
* `urn:zitadel:iam:user:groups` returns an array of JSON objects with
information about the user groups in the token claims with the same name

# Additional Changes

N/A

# Additional Context

- Closes #9702 
- Follow-up for PRs:
  * #10455 
  * #10853 
  * #10758 
  * #10940

---------

Co-authored-by: Livio Spring <[email protected]>
2025-11-11 09:06:45 +01:00
Livio SpringandGitHub e4a959c321 fix(authz): ignore unready auth methods for mfa requirement check (#11056)
# Which Problems Are Solved

The recent
[fix](https://github.com/zitadel/zitadel/commit/2a7db648817d95eaf6716b12345f958eecdff15d)
made sure the Zitadel API always requires MFA if a user has set up so
even though not required by the login policy. After the deployment,
multiple users reached out that also users without any MFA set up got
the corresponding `[permission_denied] mfa required (AUTHZ-KI3p0)`error.

# How the Problems Are Solved

- Only check the set up factors with are verified and ready to use.
Ignore all unready auth methods.

# Additional Changes

None

# Additional Context

- relates to
https://github.com/zitadel/zitadel/commit/2a7db648817d95eaf6716b12345f958eecdff15d
- closes https://github.com/zitadel/zitadel/issues/11055
- requires backport to v2.71.x, v3.x and v4.x
2025-11-11 05:55:59 +00:00
d7e9eddb76 fix(user): Updating user info when authenticating with external IDP (#11046)
# Which Problems Are Solved

User profile updates were not propagated when using External OIDC IDP +
Login V2

# How the Problems Are Solved

* `UpdateHumanUserRequest` is added to
`RetrieveIdentityProviderIntentResponse`
* `UpdateHumanUserRequest` is returned in the
`RetrieveIdentityProviderIntentResponse` when the user already exists
during external IDP auth, which is then used in the frontend to update
the user info

# Additional Changes

* Moved integration tests related to user intent to a separate test file
* Fix redirection after external IDP user registration

# Additional Context
- Closes #10838 
- Follow up: https://github.com/zitadel/zitadel/issues/11053

---------

Co-authored-by: Max Peintner <[email protected]>
2025-11-10 09:50:36 +01:00
SilvanandGitHub 345480faeb fix(projection): locking behavior based on configuration (#11014)
Ensure projections await running status if configured, improving
synchronization during event processing.
2025-10-31 13:10:02 +02:00
Livio SpringandGitHub b284f8474e Merge commit from fork 2025-10-29 10:12:50 +01:00
Livio SpringandGitHub b8db8cdf9c Merge commit from fork
* fix: respect lockout policy on password change (with old password)

* add tarpitting

* cleanup
2025-10-29 10:07:35 +01:00
Livio SpringandGitHub 72a5c33e6a Merge commit from fork
* fix: sanitize host headers before use

* add additional test
2025-10-29 10:05:37 +01:00
f4503e07cd feat(api): integrates drupal7 hash verifier from passwap (#10918)
# Which Problems Are Solved

- Integrates the Drupal 7 hash verifier from passwap

# Additional Changes

- The docs inform about the option to use the Drupal 7 hash verifier
- Updates passwap to version v0.10.0

# Additional Context

- Follow-up for PR
[#passwap/pull/70](https://github.com/zitadel/passwap/pull/70)

Co-authored-by: Marco A. <[email protected]>
2025-10-28 16:26:48 +00:00
Marco A.andGitHub 39a29f534a feat(domain): Instance APIs with relational tables (#10953)
# Which Problems Are Solved

This PR implements the endpoints listed in
https://github.com/zitadel/zitadel/issues/10443 .

# How the Problems Are Solved

The implementation follows the same pattern as the Organization one.
There are no peculiarities to this PR. The `Update`, `Delete` are
implementing the `Commander` interface, while `Get` and `List`
instance(s) endpoints are implementing the `Querier` interace.

# Additional Context

- Closes #10443
- Depends on #10445
2025-10-28 14:17:19 +00:00
Gayathri VijayanandGitHub ad8e8bf61f feat(group): manage users in user groups (#10940)
# Which Problems Are Solved

1. Adding users to user groups and removing users from user groups.
2. Searching for users in user groups by group IDs or user IDs

# How the Problems Are Solved

By adding:
1. The API definitions to manage users in users groups
3. The command-layer implementation of adding users/removing users
to/from user groups.
4. The projection table group_users1
5. Query-side implementation to search for users in user groups

# Additional Changes

1. Remove debug statements from unit tests.
2. Fix removal of groups when orgs are removed
3. Add unit tests for groups projection

# Additional Context

* Related to #9702 
* Follow-up for PRs 
  * https://github.com/zitadel/zitadel/pull/10455
  * https://github.com/zitadel/zitadel/pull/10758
  * https://github.com/zitadel/zitadel/pull/10853
2025-10-28 13:23:54 +00:00
Livio SpringandGitHub c2a0b9d187 feat(api): move instance service to v2 (#10919)
# Which Problems Are Solved

As part of our efforts to simplify the structure and versions of our
APIs, were moving all existing v2beta endpoints to v2 and deprecate
them. They will be removed in Zitadel V5.

# How the Problems Are Solved

- This PR moves instance v2beta service and its endpoints to a
corresponding v2 version. The v2beta service and endpoints are
deprecated.
- The docs are moved to the new GA service and its endpoints. The v2beta
is not displayed anymore.
- The comments and have been improved and, where not already done, moved
from swagger annotations to proto.
- All required fields have been marked with (google.api.field_behavior)
= REQUIRED and validation rules have been added where missing
- `Domain` has been renamed to `CustomDomain` to align with naming
conventions
- `..Query` has been renamed to `..Filter` to align with other services
- The `instance_id` parameter can now passed on all endpoints and is
properly used, but requires `system` permissions. It can be omitted to
use the own instance (identified by context as any other service).
  - The following endpoints are affected:
    - GetInstance
    - UpdateInstance
    - ListCustomDomains
    - AddTrustedDomain
    - RemoveTrustedDomain
    - ListTrustedDomains
- InstanceService has been added the InstanceInterceptor's
`explicitInstanceIdServices` to allow passing the id
- If the instance is not found by id, the error is not directly returned
to prevent enumeration.
- Permissions are checked in the API instead of the interceptor for
these calls.
- Setting the same instance name in the update no longer returns an
error, but the previous change date.
 
# Additional Changes

none

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/10772
- requires backport to v4.x
2025-10-28 14:01:14 +01:00
Livio SpringandGitHub 32500e3b0c feat(api): move project service v2beta to GA (and deprecate v2beta) (#10844)
# Which Problems Are Solved

As part of our efforts to simplify the structure and versions of our
APIs, were moving all existing v2beta endpoints to v2 and deprecate
them. They will be removed in Zitadel V5.

# How the Problems Are Solved

- This PR moves project v2beta service and its endpoints to a
corresponding v2 version. The v2beta service and endpoints are
deprecated.
- The comments and have been improved and, where not already done, moved
from swagger annotations to proto.
- All required fields have been marked with `(google.api.field_behavior)
= REQUIRED` and validation rules have been added where missing.
- Name ID of the project always `project_id`
- `UpdateProjectRequest` has been updated to align with the creation and
retrieval of a project:
  - `project_role_check` has been renamed to `authorization_required`
  - `has_project_check` has been renamed to `project_access_required`
- `ListProjectRequest` has been changed:
- `project_grant_resource_owner_filter`,
`project_grant_resource_owner_filter` and
`project_organization_id_filter` have been removed and merged into a
single `organization_id_filter` where a `type` can optionally be
specified to select `owned`, `granted` or both project types within a
specified organization.
- `ListProjectGrantReques` has been changed:
- `project_resource_owner_filter` has been renamed to
`project_organization_id_filter`
- `grant_resource_owner_filter` has been renamed to
`granted_organization_id_filter`

# Additional Changes

Replaced deprecated `intergration.WithAuthorization` with
`integration.WithAuthorizationToken` in integration tests.

# Additional Context

- part of #10772 
- requires backport to v4.x
2025-10-28 12:28:45 +00:00
Livio SpringandGitHub c9ac1ce344 feat(api): move authorization service to v2 (#10914)
# Which Problems Are Solved

As part of our efforts to simplify the structure and versions of our
APIs, were moving all existing v2beta endpoints to v2 and deprecate
them. They will be removed in Zitadel V5.

# How the Problems Are Solved

- This PR moves the authorization v2beta service and its endpoints to a
corresponding v2 version. The v2beta service and endpoints are
deprecated.
- The docs are moved to the new GA service and its endpoints. The v2beta
is not displayed anymore.
- The comments and have been improved and, where not already done, moved
from swagger annotations to proto.
- All required fields have been marked with (google.api.field_behavior)
= REQUIRED and validation rules have been added where missing.
- The `organization_id` to create an authorization is now required to be
always passed. There's no implicit fallback to the project's
organization anymore.
- The `user_id` filter has been removed in favor of the recently added
`in_user_ids` filter.
- The returned `Authorization` object has been reworked to return
`project`, `organization` and `roles` as objects like the granted `user`
already was.
- Additionally the `roles` now not only contain the granted `role_keys`,
but also the `display_name` and `group`. To implement this the query has
been updated internally. Existing APIs are unchanged and still return
just the keys.

# Additional Changes

None

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/10772
- closes #10746 
- requires backport to v4.x
2025-10-28 12:11:12 +00:00
196eaa84d2 fix(user): auth option while listing user metadata (#10968)
# Which Problems Are Solved

A user from `org A` with `ORG_USER_MANAGER` role in `org B` is unable to
list user metadata for a user in `org B`.

# How the Problems Are Solved

The `auth.option` is set to a specific permission (`user.read`) in the
API definition of `ListUserMetadata`, which causes the interceptors to
check for this specific permission. In this case, there is no specific
check for org membership of a user (from org A) in a target organization
(org B), and hence the call fails even though the user has the necessary
permissions.

This has been fixed by setting the `auth.option` to `authenticated`, and
the necessary [permission checks are handled in the
query-layer](https://github.com/zitadel/zitadel/blob/main/internal/query/user_metadata.go#L173).

# Additional Changes
N/A

# Additional Context
- Closes #10925

---------

Co-authored-by: Marco A. <[email protected]>
2025-10-28 11:24:50 +00:00
Gayathri VijayanandGitHub da63abd1ad fix(metrics): incorrect mapping of grpc status codes in the grpc_server_grpc_status_code_total metric (#10989)
# Which Problems Are Solved

Requests without errors were mapped being mapped to GRPC status code
`Unknown`, which were then being mapped as HTTP `500` status code.

# How the Problems Are Solved

By deriving the grpc status codes from the error only when there's an
error.
When the error is `nil`, the grpc status code is set to 0 (`OK`).

# Additional Changes

N/A

# Additional Context
- Closes #10884
2025-10-28 11:36:18 +01:00
812641ee9c chore: change prepareUserAuthMethodTypesQuery log level (#10813)
# Which Problems Are Solved

During normal operations, when listing authentication method types,
activated IDPs are added to the `userAuthMethodTypes` slice but there's
an error in the logs, causing false alarms in our alerting system which
has been configured to trigger on error-level logs:
```
{
    "logDate": "2025-09-29T07:47:49.524794306Z",
    "protocol": 0,
    "requestUrl": "/zitadel.user.v2.UserService/ListAuthenticationMethodTypes",
    "responseStatus": 0,
    "requestHeaders": {
        "authorization": [
            "[REDACTED]"
        ],
        "content-type": [
            "application/grpc+proto"
        ],
        "grpc-accept-encoding": [
            "gzip,br"
        ],
        "host": [
            "zitadel.example.com"
        ],
        "te": [
            "trailers"
        ],
        "user-agent": [
            "connect-es/2.0.4"
        ]
    },
    "responseHeaders": {},
    "instanceId": "329151080840616127",
    "projectId": "329851980840157809",
    "requestedDomain": "zitadel.example.com",
    "requestedHost": "zitadel.example.com"
}

{
    "caller": "/home/runner/work/zitadel/zitadel/internal/query/user_auth_method.go:478",
    "level": "error",
    "msg": "IDP1",
    "time": "2025-09-29T07:47:49Z"
}
```

# How the Problems Are Solved

The log was changed to debug level and the log text was updated to
better describe what is happening.

Another possible solution would be to remove the log line completely.
Reviewers can decide if this is a better solution.

# Additional Changes

None.

# Additional Context

None.

Co-authored-by: Livio Spring <[email protected]>
2025-10-24 12:09:26 +00:00
SilvanandGitHub e7b841a874 fix(assets-api): Add error handling for missing file paths (#10938) 2025-10-22 09:26:10 +00:00
f230cf0fb5 refactor(rt): add primary key columns to repo interfaces (#10843)
# Which Problems Are Solved

Relational table's primary keys are part of the SQL table definition.
However, by abstraction through the repository interfaces callers
(service layer) have no clear understanding of the primary key definion.
This might lead to confusion during implementation:

- The repository makes a sanity check on required conditions before
query execution and returns an error if the condition was not met.
- When a Get method is executed, an error is returned if not exactly one
row is found. Get replaces the legacy GetXxxByID methods we have on
projections. However, a list of flexible conditions can be passed.
- Operations like UPDATE and DELETE should often only affect a single
row, identified by the primary key. If the primary key conditions are
missing an error is returned.

Because there is no "type safety" enforced here, the errors are returned
during runtime. When all possible combinations of API arguments are not
properly tested such errors are actually bugs in the experience of the
end-user. (The API should do it's own input validation, but a
programmer's error is easily made).

# How the Problems Are Solved

Add `PrimaryKeyColumns` and `PrimaryKeyCondition` to existing repository
intefaces.

# Additional Changes

- Where reducers already used repositories, the PrimaryKeyConditions are
used where possible.

# Additional Context

- Part of
https://github.com/zitadel/zitadel/wiki/Decision-Log#expose-primary-key-definition-in-repository-interfaces

---------

Co-authored-by: Marco A. <[email protected]>
2025-10-22 08:40:21 +00:00
f04d873270 feat(domain): Organization APIs with relational tables (#10704)
# Which Problems Are Solved

This PR implements the endpoints listed in #10445 using relational
tables.

- [x] UpdateOrganization
- [x] ListOrganizations
- [x] DeleteOrganization
- [x] DeactivateOrganization
- [x] ActivateOrganization

# How the Problems Are Solved

- **UpdateOrganization:** Implemented logic for `Validate()`,
`Execute()` and `Events()`. On CQRS side, updating an organization emits
events related to the organization and its domains. Separate commanders
have been made for the domain events so that their logic is run
accordingly and we keep separation of concerns. Implementation of the
domains is left to its own ticket.

Because some domain-related data is changed during the organization
update, its original values are saved as pointers inside
[UpdateOrgCommand](https://github.com/zitadel/zitadel/blob/4f87cd8d0cfd75f290aa13c342e4939cecebef73/backend/v3/domain/org_update.go#L20).

These variables will be then used as input for the domain commanders
(that will run after the update organization one).
- **ListOrganizations:** Implemented logic for `Validate()`,
`Execute()`. Utility methods have been made to parse the input queries.
This commander is used both by the v2 and v2beta endpoint, so a
translation layer has been put in place to translate all v2beta requests
into v2. The commander strictly handles v2 request only.

The translation layer between v2beta and v2 APIs was made so to make it
simple to remove it once the v2beta APIs will be removed (a simple
deletion should suffice). A `converter` package [is
shipped](https://github.com/zitadel/zitadel/blob/4f87cd8d0cfd75f290aa13c342e4939cecebef73/backend/v3/api/org/v2/convert/README.md#L5)
as well with instruction for removal. TODOs have been put in place all
over the code to indicate that the related piece of code needs to be
removed once v2beta -> v2 transition is complete.
- **DeleteOrganization:** Implemented logic for `Validate()`,
`Execute()` and `Events()`. The `Events()` method is mostly incomplete
as it requires a lot of data that is currently not retrievable due to
the lack of their relational tables.
- **DeactivateOrganization** and **ActivateOrganization**: Implemented
logic for `Validate()`, `Execute()` and `Events()`. No notable remarks.

👉 All endpoints return matching errors to the CQRS counterpart: this way
allows re-using the integration tests and

👉 All endpoints have been unit-tested through DB mocking

👉 A bunch of TODOs have been put in place for future reworks that are
needed (e.g. `Update()` method on repository should also return a
timestamp)

# Additional Changes

- A test utility method to allow mocking query options have been made.
This was necessary because `database.QueryOption` is a function and the
only comparison possible with functions is `nil` check. So, to mock
this, `database.QueryOption` is converted first to `database.QueryOpts`.
See 2276742adaef1998ea4844e1f0dca2079e45ccfb

- A `BaseCommand` has been created with the idea to collect methods that
are going to be needed by other commands. For the moment, it only offers
a method to convert query text operations from gRPC to `domain` model.
See bb85456dd101282fa76d2e2ad165eed4c8ef61b9

- SQL operations have been reworked to allow for queries using
`ContainsIgnoreCase`, `EndsWithIgnoreCase` and other `IgnoreCase`
operations. This change is pending approval and might be reworked. See
https://github.com/zitadel/zitadel/pull/10704#discussion_r2392886177

# Additional Context

`UpdateOrganization` is partially working because the logic for handling
domain updates is not done (there is only a draft of the commanders to
showcase the call chain). See
https://github.com/zitadel/zitadel/pull/10704#discussion_r2356143178

Permissions are not implemented as they are missing. See
https://github.com/zitadel/zitadel/pull/10771

- Closes: #10445

---------

Co-authored-by: Silvan <[email protected]>
2025-10-21 15:27:38 +02:00
88213d785a fix(oidc): accept localhost redirect URIs without path nor port (#10836)
# Which Problems Are Solved

Some native OIDC applications use localhost without a path as redirect
URI. Currently, setting `http://localhost` as a redirect URI leads to a
compliance warning (`Redirect URIs must begin with your own protocol,
http://127.0.0.1, http://[::1] or http://localhost.`), while
`http://localhost/some/path` and `http://localhost:some-port` are
accepted).

# How the Problems Are Solved

This PR adds additional checks to accept `http://localhost`,
`http://127.0.0.1`, `http://[::1]` and `http://[0:0:0:0:0:0:0:1]` (their
counterpart with port and with path were already accepted).

---------

Co-authored-by: Marco Ardizzone <[email protected]>
2025-10-21 10:10:49 +00:00
Gayathri VijayanandGitHub ebc6c503a3 feat(group): add group permissions (#10853)
# Which Problems Are Solved
Ensuring the user group resource is managed with appropriate
permissions.

# How the Problems Are Solved
By configuring and checking for the relevant permissions needed to
create, read, update, and delete the user groups resource.

# Additional Changes
N/A

# Additional Context
- Related to #9702 
- Follow-up for PRs #10455,  #10758
2025-10-21 11:18:21 +02:00
Livio SpringandGitHub fda19dc85b fix(login v1): handle old sessions in logout (#10926)
# Which Problems Are Solved

Sessions created through login UI (v1) automatically get assigned an ID
after creation. This change was introduced with the OIDC back-channel
logout implementation. Sessions created before that don't have an ID and
are updated on the next (re-)authentication.
A customer now reached out, that a logout from Console was resulting in
an error. This is due to at least one session not having an ID (<null>
in sql) in the same user agent.

# How the Problems Are Solved

Since the sessionID is not used in the specific situation, we just
assign the userID as sessionID. This way all sessions are properly
terminated.

# Additional Changes

None

# Additional Context

- relates to support request
- requires backport to v4.x
2025-10-18 10:18:36 +02:00
Livio SpringandGitHub 8d5f92f80a feat(api): move organization settings endpoints to v2 (#10910)
# Which Problems Are Solved

As part of our efforts to simplify the structure and versions of our
APIs, were moving all existing v2beta endpoints to v2 and deprecate
them. They will be removed in Zitadel V5.

# How the Problems Are Solved

- This PR adds the organization settings endpoints to the settings
service v2. This was split from #10909, since that PR targets v4.x and
the corresponding feature is not yet available in v4.x, but only v5.
- The comments and have been improved and, where not already done, moved
from swagger annotations to proto.

# Additional Changes

None

# Additional Context

- relates to #10909 
- relates to #10772
2025-10-17 11:21:01 +02:00
Livio SpringandGitHub dbf877e028 fix(api): deprecate settings v2beta endpoints (#10909)
# Which Problems Are Solved

As part of our efforts to simplify the structure and versions of our
APIs, were moving all existing v2beta endpoints to v2 and deprecate
them. They will be removed in Zitadel V5.

# How the Problems Are Solved

- This PR deprecates all settings v2beta service and endpoints.
- The comments and have been improved and, where not already done, moved
from swagger annotations to proto.
- `LoginSettings`:
- `AllowUsernamePassword` has been deprecated and a corresponding
`AllowLocalAuthentication` has been introduced
- `SECOND_FACTOR_TYPE_OTP` has been deprecated and a
`SECOND_FACTOR_TYPE_TOTP` has been introduced as enum alias

# Additional Changes

- cleanups of some unused structs

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/10772
- requires backport to v4.x
2025-10-17 09:02:26 +00:00
Livio SpringandGitHub 0281670030 feat(api): move application service v2beta to GA (and deprecate v2beta) (#10846)
# Which Problems Are Solved

As part of our efforts to simplify the structure and versions of our
APIs, were moving all existing v2beta endpoints to v2 and deprecate
them. They will be removed in Zitadel V5.

# How the Problems Are Solved

- This PR moves app v2beta service and its endpoints to a corresponding
to application v2 version. The v2beta service and endpoints are
deprecated.
- The comments and have been improved and, where not already done, moved
from swagger annotations to proto.
- All required fields have been marked with (google.api.field_behavior)
= REQUIRED and validation rules have been added where missing.
- Name ID of the application always `application_id`, previously was
also `id` and `app_id`.
- Get rid of all `app` abbreviations and name it `application` including
the service name, `AppState` -> `ApplicationState` and `AppSorting` ->
`ApplicationSorting`
- Updated `CreateApplicationRequest`:
- renamed `creation_request_type` to `application_type` and all its
options to `XY_configuration` instead of `XY_request`
- `RegenerateClientSecret`
  - renamed method to `GenerateClientSecret`
  - removed `app_type` from request
- `ListApplicationRequest`:
  - removed required `project_id` and provided it as a filter
- Type `ApplicationNameQuery` has been renamed to
`ApplicationNameFilter` as its usage in the request
- Renamed all fields and types from `config` to `configuration`
- Updated `DeleteApplicationKeyRequest`
  - removed `organization_id`
- Updated `GetApplicationKeyRequest`:
  - removed `project_id`, `application_id` and `organization_id``
- Updated `ListApplicationKeysRequest`:
  - removed oneOf `resource_id` and moved the options into filters
- Name ID of the application key always `key_id`.
- removed unnecessary package prefixed (`zitadel.application.v2`)
- formatted using `buf`

# Additional Changes

None

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/10772
- requires backport to v4.x
2025-10-17 10:12:20 +02:00
0f2a349ec1 feat(api): move internal permission service to GA (and deprecate v2beta) (#10898)
# Which Problems Are Solved

As part of our efforts to simplify the structure and versions of our
APIs, were moving all existing v2beta endpoints to v2 and deprecate
them. They will be removed in Zitadel V5.

# How the Problems Are Solved

- This PR moves the internal permission v2beta service and its endpoints
to a corresponding v2 version. The v2beta service and endpoints are
deprecated.
- The docs are moved to the new GA service and its endpoints. The v2beta
is not displayed anymore.
- The comments and have been improved and, where not already done, moved
from swagger annotations to proto.
- All required fields have been marked with (google.api.field_behavior)
= REQUIRED and validation rules have been added where missing.
- Listing administrators of a project grant can now be done with the
`ProjectGrant` (`project_id` and `organization_id`) instead of a
`project_id`, which corresponds to creation of the administrator ship of
such grant.
- formatted using `buf`

# Additional Changes

None

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/10772
- requires backport to v4.x

---------

Co-authored-by: Gayathri Vijayan <[email protected]>
2025-10-17 07:35:35 +02:00
3a91b409c4 fix(zitadel): LDAP binding error with non-ASCII characters in DN (#10578)
<!--
Please inform yourself about the contribution guidelines on submitting a
PR here:
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#submit-a-pull-request-pr.
Take note of how PR/commit titles should be written and replace the
template texts in the sections below. Don't remove any of the sections.
It is important that the commit history clearly shows what is changed
and why.
Important: By submitting a contribution you agree to the terms from our
Licensing Policy as described here:
https://github.com/zitadel/zitadel/blob/main/LICENSING.md#community-contributions.
-->

# Which Problems Are Solved

LDAP binding with non-ASCII characters in DN

# How the Problems Are Solved

The root of the problem is that ParseDN() function messes DN with
non-ASCII character. Instead of using DN object, returned from ParseDN
we use user.DN in binding request. ParseDN stays only for verifying
correctness of DN.

# Additional Context
- Closes #9970

---------

Co-authored-by: Stefan Benz <[email protected]>
2025-10-15 09:40:41 +00:00
a8bbac37d9 chore: fix some eventual consistent integration testing (#10752)
# Which Problems Are Solved

Flakiness in integration tests because of eventual consistentcy.

# How the Problems Are Solved

Split tests related to feature flags and other eventual consistent
resources.

# Additional Changes

None

# Additional Context

None

Co-authored-by: Marco A. <[email protected]>
2025-10-14 18:24:09 +02:00
Tim MöhlmannandGitHub f27ca69749 fix(query): distinct count in user list (#10840)
# Which Problems Are Solved

When listing / searching users, each user got multiplied by the amount
of metadata entries they have, towards the `total_results` count. In
PostgreSQL the `COUNT(*) OVER()` window function does not support
`DISTINCT`. Even tho the query did a distinct select, the count would
still include duplicates.

# How the Problems Are Solved

Wrap the original query in a sub-select, so that the `DISTINCT` gets
handled before the count window function is executed in the outer
function. Filters, permission and solting is applied to the inner query.
Offset, limit and count are applied to the outer query.

# Additional Changes

- none

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/10825
- Backport to 4v
2025-10-14 09:43:59 +00:00
SilvanandGitHub 6686c3b16b feat(relational): org metadata (#10761)
This pull request introduces a new feature that allows adding, updating,
and querying metadata for organizations. The changes are primarily in
the backend and include new database tables, repositories, and domain
logic to support organization metadata.

## Changes

* New `org_metadata` table: A new table `zitadel.org_metadata` is
introduced to store metadata for organizations. It includes columns for
`instance_id`, `org_id`, `key`, and `value`. The `value` is stored as a
`BYTEA` type to allow for flexible data storage.
* New `OrganizationMetadataRepository`: A new repository
`OrganizationMetadataRepository` is created to handle all database
operations for organization metadata. It provides methods to `Get`,
`List`, `Set`, and `Remove` metadata.
* New `org_metadata_relational_projection`: A new projection
`org_metadata_relational_projection` is added to update the
`zitadel.org_metadata` table based on events. It handles `MetadataSet`,
`MetadataRemoved`, and `MetadataRemovedAll` events.
* Updated `OrganizationRepository`: The `OrganizationRepository` is
updated to support loading organization metadata. A new method
`LoadMetadata` is added to enable joining the `org_metadata` table when
querying for organizations.
* Updated Organization domain: The Organization domain model is updated
to include a new field `Metadata` of type `[]*OrganizationMetadata`.

## Additional Info

* Extensible Design: The new metadata feature is designed to be
extensible, allowing for future enhancements such as indexing on
specific JSON fields within the `value` column.
* closes https://github.com/zitadel/zitadel/issues/10206
* closes https://github.com/zitadel/zitadel/issues/10214
2025-10-14 10:15:16 +01:00
Stefan BenzandGitHub 013620baab fix: correct mapping of custom texts (#10725)
# Which Problems Are Solved

Some custom texts are overwritten by incorrect mapped values.

# How the Problems Are Solved

Correct the mapping in the mapping.

# Additional Changes

None

# Additional Context

Closes #10155
2025-10-13 09:44:04 +00:00
Elio BischofandGitHub 7ba6870baf feat: await initial database connection (#10869)
# Which Problems Are Solved

When Postgres was not ready when the API was started, the API failed
immediately.
This made task orchestration hard, especially in a platform agnostic
way:

- The current health check in the Nx target `@zitadel/api:prod` uses the
timeout command, which is not installed on all platforms and behaves
unpredictably
- The current health check in the Nx target `@zitadel/api:prod` requires
the DB to have been started using `@zitadel/zitadel:db`

# How the Problems Are Solved

- Additional configuration option `Database.Postgres.AwaitInitialConn`
is added and defaults to *0m* for backwards compatibility.
- If a duration is configured, the API retries to ping the database
until it succeeds
- The API sleeps for a second between each ping.
- It emits an info-level log with the error on each try.
- When the configured duration times out before the ping is successful,
the error is returned and the command exits with a failure code.
- When the ping succeeds within the configured duration, the API goes on
with the init, setup or start phase.

# Additional Context

- Relates to internally reported problems with the current DB health
check command
[here](https://zitadel.slack.com/archives/C07EUL5H83A/p1759915009839269?thread_ts=1759912259.410789&cid=C07EUL5H83A)
and
[here](https://zitadel.slack.com/archives/C07EUL5H83A/p1759918324246249?thread_ts=1759912259.410789&cid=C07EUL5H83A).
2025-10-09 11:18:34 +00:00
Livio SpringandGitHub b8bff3cdea fix: Revert "feat(oidc): Added new claim in userinfo response to return all requested audience roles (#9861)" (#10874)
# Which Problems Are Solved

#9861 added a `urn:zitadel:iam:org:projects:roles` claims to include all
roles from all requested roles. The intention was to return them on the
userinfo endpoint. But since the claims might also be returned in the id
and access tokens, they can grow big quite fast and break the size
limits for headers.

# How the Problems Are Solved

This PR revert the feature. The information for roles of other projects
is already available as a dedicated claim (for each project):
```json
  "urn:zitadel:iam:org:project:328813096124547391:roles": {
    "r2": {
      "306639557921669515": "zitadel.localhost"
    },
    "r3": {
      "306639557921669515": "zitadel.localhost"
    },
    "role": {
      "306639557921669515": "zitadel.localhost"
    }
  },
  "urn:zitadel:iam:org:project:341406882914631999:roles": {
    "role": {
      "306639557921669515": "zitadel.localhost",
      "328237605990695334": "aa.localhost"
    },
    "test": {
      "306639557921669515": "zitadel.localhost",
      "328237605990695334": "aa.localhost"
    }
  },
  "urn:zitadel:iam:org:project:roles": {
    "r2": {
      "306639557921669515": "zitadel.localhost"
    },
    "r3": {
      "306639557921669515": "zitadel.localhost"
    },
    "role": {
      "306639557921669515": "zitadel.localhost"
    }
  }
 ```

# Additional Changes

None

# Additional Context

- relates to #9861 
- noted issues in production
- requires backport to v4.x
2025-10-09 10:29:49 +00:00
d5cfbc7b00 chore: update crewjam/saml to v0.5.1 (#10652)
# Which Problems Are Solved

No usage of the current version of crewjam/saml.

# How the Problems Are Solved

Update dependency to v0.5.1.

# Additional Changes

None

# Additional Context

Closes #9783

---------

Co-authored-by: Livio Spring <[email protected]>
2025-10-08 11:38:48 +02:00
f69a6ed4f3 chore: rehaul DevX (#10571)
# Which Problems Are Solved

Replaces Turbo by Nx and lays the foundation for the next CI
improvements. It enables using Nx Cloud to speed the up the pipelines
that affect any node package.
It streamlines the dev experience for frontend and backend developers by
providing the following commands:

| Task | Command | Notes |
|------|---------|--------|
| **Production** | `nx run PROJECT:prod` | Production server |
| **Develop** | `nx run PROJECT:dev` | Hot reloading development server
|
| **Test** | `nx run PROJECT:test` | Run all tests |
| **Lint** | `nx run PROJECT:lint` | Check code style |
| **Lint Fix** | `nx run PROJECT:lint-fix` | Auto-fix style issues |

The following values can be used for PROJECT:

- @zitadel/zitadel (root commands)
- @zitadel/api,
- @zitadel/login,
- @zitadel/console,
- @zitadel/docs,
- @zitadel/client
- @zitadel/proto

The project names and folders are streamlined:

| Old Folder | New Folder |
| --- | --- |
| ./e2e | ./tests/functional-ui |
| ./load-test | ./benchmark |
| ./build/zitadel | ./apps/api |
| ./console | ./apps/console (postponed so the PR is reviewable) |  

Also, all references to the TypeScript repo are removed so we can
archive it.

# How the Problems Are Solved

- Ran `npx nx@latest init`
- Replaced all turbo.json by project.json and fixed the target configs
- Removed Turbo dependency
- All JavaScript related code affected by a PRs changes is
quality-checked using the `nx affected` command
- We move PR checks that are runnable using Nx into the `check`
workflow. For workflows where we don't use Nx, yet, we restore
previously built dependency artifacts from Nx.
- We only use a single and easy to understand dev container
- The CONTRIBUTING.md is streamlined
- The setup with a generated client pat is orchestrated with Nx
- Everything related to the TypeScript repo is updated or removed. A
**Deploy with Vercel** button is added to the docs and the
CONTRIBUTING.md.

# Additional Changes

- NPM package names have a consistent pattern.
- Docker bake is removed. The login container is built and released like
the core container.
- The integration tests build the login container before running, so
they don't rely on the login container action anymore. This fixes
consistently failing checks on PRs from forks.
- The docs build in GitHub actions is removed, as we already build on
Vercel.

# Additional Context

- Internal discussion:
https://zitadel.slack.com/archives/C087ADF8LRX/p1756277884928169
- Workflow dispatch test:
https://github.com/zitadel/zitadel/actions/runs/17760122959

---------

Co-authored-by: Florian Forster <[email protected]>
Co-authored-by: Tim Möhlmann <[email protected]>
Co-authored-by: Copilot <[email protected]>
2025-10-08 10:27:02 +02:00
4c078c1474 fix: add CommonName to SAML SP certificate (#10700)
# Which Problems Are Solved

There is no CN (CommonName) defined in the certificates to use an
external SAML IDP.

# How the Problems Are Solved

Add Issuer and CommonName to the certificate information.

# Additional Changes

None

# Additional Context

Closes #9048

Co-authored-by: Gayathri Vijayan <[email protected]>
2025-10-07 14:58:55 +00:00
8e766132b0 feat(group): add user groups query-side and projection implementation (#10758)
# Which Problems Are Solved

This is the second PR related to the backend implementation of
GroupService to manage user groups.
The first [PR](https://github.com/zitadel/zitadel/pull/10455) implements
the Command-side.
This PR implements the query side. 

# How the Problems Are Solved
* Query-side implementation to search/list groups by 
    * a list of Group IDs
    * by the Group name
    * by the Organization ID

# Additional Changes
N/A

# Additional Context
- Follow-up for PR #10455

---------

Co-authored-by: Livio Spring <[email protected]>
2025-10-07 12:53:25 +00:00
Livio SpringandGitHub e25b21a6a4 fix(api): deprecate v2beta endpoints of existing v2 services (#10841)
# Which Problems Are Solved

As part of our efforts to simplify the structure and versions of our
APIs, were moving all existing v2beta endpoints to v2 and deprecate
them. They will be removed in Zitadel V5.

# How the Problems Are Solved

- This PR deprecates all v2beta service and their endpoints, which have
already a corresponding v2 version and should not be used anymore.
- The comments and have been improved and, where not already done, moved
from swagger annotations to proto.
- All required fields have been marked with `(google.api.field_behavior)
= REQUIRED` and validation rules have been added where missing.
- Removed the "required flag" comments on the Action and WebKey service
endpoints, since they were removed in Zitadel v4.
- The `SetSession` endpoint already documented that the token does not
have to be provided anymore and will be ignored if still sent, but it
was actually still checked if provided. The corresponding check has been
removed and the field is now properly deprecated in the proto as well.

# Additional Changes

None

# Additional Context

- part of #10772 
- requires backport to v4.x
2025-10-07 11:17:46 +02:00
a7e1bfb4a3 fix: add email query to github idp if email empty (#10705)
# Which Problems Are Solved

In the integration with Github, private emails are not returned with the
userinfo.

# How the Problems Are Solved

If the scope `user:email` is set in the Github IDP and the email is not
included in the userinfo, a request to Github's API is executed to query
the email of the user.

# Additional Changes

Additional tests.

# Additional Context

Closes #10098

---------

Co-authored-by: Marco A. <[email protected]>
Co-authored-by: Livio Spring <[email protected]>
2025-10-07 06:13:06 +00:00
Stefan BenzandGitHub 6da380628d fix: add InUserIDs query to query of user grants (#10741)
# Which Problems Are Solved

Currently there is only the option to either filter for all usergrants
of an organization or the usergrants of a singluar user.

# How the Problems Are Solved

Add the option to provide a list of userIDs to query user grants.

# Additional Changes

Fixed internal typo for function.

# Additional Context

Closes #9675
2025-10-07 04:56:20 +00:00
Elio BischofandGitHub 4aac34d93b chore: fix code generation (#10835)
# Which Problems Are Solved

Generation of code using `make core_generate_all` failed, produced a
diff and printed scary warnings.

<img width="1548" height="1542" alt="image"
src="https://github.com/user-attachments/assets/31c149b3-5d77-479d-bf04-dfda81c3a34e"
/>

# How the Problems Are Solved

- Removed the [obsolete mock_gen.go file causing the
failure](https://github.com/zitadel/zitadel/pull/10564/files#r2398152163)
- Assuming we are not going to upgrade sass for the login v1, it makes
no sense to migrate the syntax and risk breaking the UI, even though
this [could be done
automatically](https://sass-lang.com/documentation/breaking-changes/import/#automatic-migration).
Therefore, we [silenced the sass deprecation
warnings](https://sass-lang.com/documentation/cli/dart-sass/#silence-deprecation).
- Re-generated code with the [pinned mock-gen binary
version](https://github.com/zitadel/zitadel/pull/10762/files#r2398223267).
2025-10-06 13:08:33 +00:00
b81dedcaea feat(group): group service to create, update, and delete groups (#10455)
# Which Problems Are Solved

This PR adds API definition and backend implementation for GroupService
to manage user groups.

# How the Problems Are Solved
* API definition to create, update, retrieve, and delete groups is added
* Command-side implementation to create, update, and delete user groups
as part of the GroupV2 API is added

# Additional Changes
N/A

# Additional Context
- Related to #10089, #9702 (parent ticket)
- User contribution: https://github.com/zitadel/zitadel/pull/9428/files
- Additional functionalities to list/search user groups, add
permissions, manage users in groups, group scopes will be added in
subsequent PRs.
- Also needs documentation, which will be added once the entire feature
is available

---------

Co-authored-by: Livio Spring <[email protected]>
2025-10-06 11:23:15 +02:00
ee3654fe6f fix: Translation on italian email 2FA templates fix. {{:code}} erroneuously translated as well. (#10805)
# Which Problems Are Solved

There's an error in the italian translation. All templates in the
it.yaml file contain a wrong {{.Code}} anchor which was probably
translated with search/replace without cheking. The resulting
{{.Codice}} will send mails with a missing code.

I opened a bugreport here:
https://github.com/zitadel/zitadel/issues/10806

# How the Problems Are Solved

Fixed the template to include the correct anchor.

# Additional Context

Closes https://github.com/zitadel/zitadel/issues/10806

Co-authored-by: pp <[email protected]>
2025-10-02 08:14:43 +00:00
Tim MöhlmannandGitHub a45908b364 feat(rt): project repository (#10789)
# Which Problems Are Solved

Add projects to the relational tables

# How the Problems Are Solved

- Define table migrations
- Define and implement Project and Project Role repositories.
- Provide projection handlers to populate the relational tables.

# Additional Changes

- Statement Builder now has a constructor which allows setting of a base
query with arguments.
- Certain operations, like Get, Update and Delete require the Primary
Key to be set as conditions. However, this requires knowledge of the
implementation and table definition. This PR proposes an additional
condition for repositories: `PrimaryKeyCondition`. This gives clarity on
the required IDs for these operations.
- Added couple of helpers to the repository package, to allow more DRY
code.
- getOne / getMany: generic functions for query execution and scanning.
- checkRestrictingColumns, checkPkCondition: simplify condition
checking, instead of using ladders of conditionals.
- Added a couple of helpers to the repository test package:
  - Transaction, savepoint and rollback helpers.
- Create instance and organization helpers for objects that depend on
them (like projects).

# Additional Context

- after https://github.com/zitadel/zitadel/pull/10809
- closes #10765
2025-10-01 09:47:04 +00:00
Livio SpringandGitHub 1a0588fef1 fix(idp): SAML signature algorithm (#10795)
# Which Problems Are Solved

https://github.com/zitadel/zitadel/pull/10520 added the possibility to
specify the signature algorithm for SAML auth requests. After releasing,
customer noticed that the Console UI would not correctly display the
selected algorithm and that it was not used in the login V1.

# How the Problems Are Solved

- Correctly map the algorithm in the UI
- Provide the option to the idp when creating a SAML request in login V1

# Additional Changes

None

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/10780
- closes https://github.com/zitadel/zitadel/issues/10792
- requires backport to v4.x
2025-09-26 07:23:24 +00:00
Livio SpringandGitHub f7fbd0cdfd chore: use postgres 17 (#10797)
# Which Problems Are Solved

The current cache interface implementation for postgres is not
compatible with Postgres18, since we rely on partitioned unlogged
tables, which are no longer supported.

# How the Problems Are Solved

Use postgres 17 and update compatibility in the docs.

# Additional Changes

None

# Additional Context

- requires backport to v3.x, v4.x
2025-09-26 06:59:18 +00:00
Livio SpringandGitHub bb9e557760 test: make session integration tests less eventually consistent (#10790)
# Which Problems Are Solved

While reviewing #9954, i noticed eventual consistency issues in the
session integration tests. All creation and change dates as well as
checked_at were tested using a `window` duration, typically one minute
from `time.Now()`. If some precondition took longer, they would all
fail.

# How the Problems Are Solved

Changed the tests to use the information returned by the creation / set
session calls and make sure they're in those timeframes.
Added a clock skew for the factor checks, since there's an inconsistency
in the event payload and event date: #10791


# Additional Changes

None

# Additional Context

- noted in #9954
- requires backport to v4.x
2025-09-25 14:30:10 +02:00
SilvanandGitHub 4900ac477a fix(test): increase retry tick duration in tests (#10787)
Adjust the retry tick duration in various tests to one minute to improve
reliability and reduce the frequency of retries.
2025-09-25 08:47:29 +02:00
IraqandGitHub 08f41e034e feat(idp_table_relational): adding inital idp tables for relational repository (#10334) 2025-09-24 13:19:09 +00:00