Commit Graph
100 Commits
Author SHA1 Message Date
5d2d1d6da6 feat(OIDC): handle logout hint on end_session_endpoint (#10039)
# Which Problems Are Solved

The OIDC session endpoint allows to pass a `id_token_hint` to identify
the session to terminate. In case the application is not able to pass
that, e.g. Console currently allows multiple sessions to be open, but
will only store the id_token of the current session, allowing to pass
the `logout_hint` to identify the user adds some new possibilities.

# How the Problems Are Solved

In case the end_session_endpoint is called with no `id_token_hint`, but
a `logout_hint` and the v2 login UI is configured, the information is
passed to the login UI also as `login_hint` parameter to allow the login
UI to determine the session to be terminated, resp. let the user decide.

# Additional Changes

Also added the `ui_locales` as parameter to handle and pass to the V2
login UI.

# Dependencies ⚠️ 

~These changes depend on https://github.com/zitadel/oidc/pull/774~

# Additional Context

closes #9847

---------

Co-authored-by: Marco Ardizzone <[email protected]>
2025-07-28 13:55:55 +00:00
Livio SpringandGitHub c4e0342c5f chore(tests): fix tests (#10267)
# Which Problems Are Solved

The latest merge on main corrupted some unit tests.

# How the Problems Are Solved

Fix them as intended on the PR.

# Additional Changes

None

# Additional Context

relates to
https://github.com/zitadel/zitadel/commit/4c942f3477b073e3e270079e6424b2b3797765d6
2025-07-15 13:09:22 +00:00
Livio SpringandGitHub 4c942f3477 Merge commit from fork
* fix: require permission to create and update session

* fix: require permission to fail auth requests

* merge main and fix integration tests

* fix merge

* fix integration tests

* fix integration tests

* fix saml permission check
2025-07-15 13:38:00 +02:00
Livio SpringandGitHub 79fcc2f2b6 chore(tests): name integration test packages correctly to let them run (#10242)
# Which Problems Are Solved

After changing some internal logic, which should have failed the
integration test, but didn't, I noticed that some integration tests were
never executed. The make command lists all `integration_test` packages,
but some are named `integration`

# How the Problems Are Solved

Correct wrong integration test package names.

# Additional Changes

None

# Additional Context

- noticed internally
- backport to 3.x and 2.x
2025-07-14 08:01:36 +00:00
Livio SpringandGitHub 1b01fc6c40 fix(api): CORS for connectRPC and grpc-web (#10227)
# Which Problems Are Solved

The CORS handler for the new connectRPC handlers was missing, leading to
unhandled preflight requests and a unusable api for browser based calls,
e.g. cross domain gRPC-web requests.

# How the Problems Are Solved

- Added the http CORS middleware to the connectRPC handlers.
- Added `Grpc-Timeout`, `Connect-Protocol-Version`,`Connect-Timeout-Ms`
to the default allowed headers (this improves also the old grpc-web
handling)
- Added `Grpc-Status`, `Grpc-Message`, `Grpc-Status-Details-Bin` to the
default exposed headers (this improves also the old grpc-web handling)

# Additional Changes

None

# Additional Context

noticed internally while testing other issues
2025-07-11 09:55:01 +00:00
Livio SpringandGitHub 8f61b24532 fix(login v1): correctly auto-link users on organizations with suffixed usernames (#10205) 2025-07-11 05:29:27 -04:00
Livio SpringandGitHub fefeaea56a perf: improve org and org domain creation (#10232)
# Which Problems Are Solved

When an organization domain is verified, e.g. also when creating a new
organization (incl. generated domain), existing usernames are checked if
the domain has been claimed.
The query was not optimized for instances with many users and
organizations.

# How the Problems Are Solved

- Replace the query, which was searching over the users projection with
(computed loginnames) with a dedicated query checking the loginnames
projection directly.
-  All occurrences have been updated to use the new query.

# Additional Changes

None

# Additional Context

- reported through support
- requires backport to v3.x
2025-07-10 15:17:49 +00:00
Livio SpringandGitHub ffe6d41588 fix(login v1): handle password reset when authenticating with email or phone number (#10228)
# Which Problems Are Solved

When authenticating with email or phone number in the login V1, users
were not able to request a password reset and would be given a "User not
found" error.
This was due to a check of the loginname of the auth request, which in
those cases would not match the user's stored loginname.

# How the Problems Are Solved

Switch to a check of the resolved userID in the auth request. (We still
check the user again, since the ID might be a placeholder for an unknown
user and we do not want to disclose any information by omitting a check
and reduce the response time.)

# Additional Changes

None

# Additional Context

- reported through support
- requires backport to v3.x
2025-07-10 09:29:26 +02:00
Livio SpringandGitHub 2821f41c3a fix(login v1): ensure the user's organization is always set into the token context (#10221)
# Which Problems Are Solved

Customers reported, that if the session / access token in Console
expired and they re-authenticated, the user list would be empty.
While reproducing the issue, we discovered that the necessary
organization information, would be missing in the access token, since
this would already be missing in the OIDC session creation when using an
id_token_hint.

# How the Problems Are Solved

- Ensure the user's organization is set in the login v1 auth request.
This is used to create the OIDC and token information.
 
# Additional Changes

None

# Additional Context

- reported by customers
- requires backport to v3.x
2025-07-09 16:51:13 +02:00
Livio SpringandGitHub aa8edee50b chore(docs): prevent readme overwrite (#10170)
# Which Problems Are Solved

To generate the docs, we rely on a protoc plugin to generate an openAPI
definition from connectRPC / proto.
Since the plugin is not available on buf.build, we currently download
the released version. As the tar contains a licence and a readme, this
overwrote existing internal files.

# How the Problems Are Solved

Download and extract the plugin in a separate folder and update
buf.gen.yaml accordingly.

# Additional Changes

None

# Additional Context

relates to #9483
2025-07-07 09:28:38 +02:00
27cd1d8518 docs(api): add new beta services to api reference (#10018)
# Which Problems Are Solved

The unreleased new resource apis have been removed from the docs:
https://github.com/zitadel/zitadel/pull/10015

# How the Problems Are Solved

Add them to the docs sidenav again, since they're now released.

# Additional Changes

none

# Additional Context

none

---------

Co-authored-by: Fabienne <[email protected]>
Co-authored-by: Marco Ardizzone <[email protected]>
2025-07-04 19:00:24 +02:00
9ebf2316c6 feat: exchange gRPC server implementation to connectRPC (#10145)
# Which Problems Are Solved

The current maintained gRPC server in combination with a REST (grpc)
gateway is getting harder and harder to maintain. Additionally, there
have been and still are issues with supporting / displaying `oneOf`s
correctly.
We therefore decided to exchange the server implementation to
connectRPC, which apart from supporting connect as protocol, also also
"standard" gRCP clients as well as HTTP/1.1 / rest like clients, e.g.
curl directly call the server without any additional gateway.

# How the Problems Are Solved

- All v2 services are moved to connectRPC implementation. (v1 services
are still served as pure grpc servers)
- All gRPC server interceptors were migrated / copied to a corresponding
connectRPC interceptor.
- API.ListGrpcServices and API. ListGrpcMethods were changed to include
the connect services and endpoints.
- gRPC server reflection was changed to a `StaticReflector` using the
`ListGrpcServices` list.
- The `grpc.Server` interfaces was split into different combinations to
be able to handle the different cases (grpc server and prefixed gateway,
connect server with grpc gateway, connect server only, ...)
- Docs of services serving connectRPC only with no additional gateway
(instance, webkey, project, app, org v2 beta) are changed to expose that
- since the plugin is not yet available on buf, we download it using
`postinstall` hook of the docs

# Additional Changes

- WebKey service is added as v2 service (in addition to the current
v2beta)

# Additional Context

closes #9483

---------

Co-authored-by: Elio Bischof <[email protected]>
2025-07-04 14:06:20 +00:00
Livio SpringandGitHub 82cd1cee08 fix(service ping): correct endpoint, validate and randomize default interval (#10166)
# Which Problems Are Solved

The production endpoint of the service ping was wrong.
Additionally we discussed in the sprint review, that we could randomize
the default interval to prevent all systems to report data at the very
same time and also require a minimal interval.

# How the Problems Are Solved

- fixed the endpoint
- If the interval is set to @daily (default), we generate a random time
(minute, hour) as a cron format.
- Check if the interval is more than 30min and return an error if not.
- Fixed yaml indent on `ResourceCount`

# Additional Changes

None

# Additional Context

as discussed internally
2025-07-04 13:45:15 +00:00
Livio SpringandGitHub f93a35c7a8 feat: implement service ping (#10080)
This PR is still WIP and needs changes to at least the tests.

# Which Problems Are Solved

To be able to report analytical / telemetry data from deployed Zitadel
systems back to a central endpoint, we designed a "service ping"
functionality. See also https://github.com/zitadel/zitadel/issues/9706.
This PR adds the first implementation to allow collection base data as
well as report amount of resources such as organizations, users per
organization and more.

# How the Problems Are Solved

- Added a worker to handle the different `ReportType` variations. 
- Schedule a periodic job to start a `ServicePingReport`
- Configuration added to allow customization of what data will be
reported
- Setup step to generate and store a `systemID`

# Additional Changes

None

# Additional Context

relates to #9869
2025-07-02 13:57:41 +02:00
Livio SpringandGitHub 71575e8d67 fix(webauthn): allow to use "old" passkeys/u2f credentials on session API (#10150)
# Which Problems Are Solved

To prevent presenting unusable WebAuthN credentials to the user /
browser, we filtered out all credentials, which do not match the
requested RP ID. Since credentials set up through Login V1 and Console
do not have an RP ID stored, they never matched. This was previously
intended, since the Login V2 could be served on a separate domain.
The problem is, that if it is hosted on the same domain, the credentials
would also be filtered out and user would not be able to login.

# How the Problems Are Solved

Change the filtering to return credentials, if no RP ID is stored and
the requested RP ID matches the instance domain.

# Additional Changes

None

# Additional Context

Noted internally when testing the login v2
2025-07-02 11:04:59 +00:00
Livio SpringandGitHub 64a03fba28 fix(api): return typed saml form post data in idp intent (#10136)
<!--
Please inform yourself about the contribution guidelines on submitting a
PR here:
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#submit-a-pull-request-pr.
Take note of how PR/commit titles should be written and replace the
template texts in the sections below. Don't remove any of the sections.
It is important that the commit history clearly shows what is changed
and why.
Important: By submitting a contribution you agree to the terms from our
Licensing Policy as described here:
https://github.com/zitadel/zitadel/blob/main/LICENSING.md#community-contributions.
-->

# Which Problems Are Solved

The current user V2 API returns a `[]byte` containing a whole HTML
document including the form on `StartIdentifyProviderIntent` for intents
based on form post (e.g. SAML POST bindings). This is not usable for
most clients as they cannot handle that and render a whole page inside
their app.
For redirect based intents, the url to which the client needs to
redirect is returned.

# How the Problems Are Solved

- Changed the returned type to a new `FormData` message containing the
url and a `fields` map.
- internal changes:
- Session.GetAuth now returns an `Auth` interfacce and error instead of
(content string, redirect bool)
- Auth interface has two implementations: `RedirectAuth` and `FormAuth`
- All use of the GetAuth function now type switch on the returned auth
object
- A template has been added to the login UI to execute the form post
automatically (as is).

# Additional Changes

- Some intent integration test did not check the redirect url and were
wrongly configured.

# Additional Context

- relates to zitadel/typescript#410
2025-06-30 15:07:33 +00:00
Livio SpringandGitHub 63c92104ba chore: service ping api design (#9984)
# Which Problems Are Solved

Add the possibility to report information and analytical data from
(self-hosted) ZITADEL systems to a central endpoint.
To be able to do so an API has to be designed to receive the different
reports and information.

# How the Problems Are Solved

- Telemetry service definition added, which currently has two endpoints:
- ReportBaseInformation: To gather the zitadel version and instance
information such as id and creation date
- ReportResourceCounts: Dynamically report (based on #9979) different
resources (orgs, users per org, ...)
- To be able to paginate and send multiple pages to the endpoint a
`report_id` is returned on the first page / request from the server,
which needs to be passed by the client on the following pages.
- Base error handling is described in the proto and is based on gRPC
standards and best practices.

# Additional Changes

none

# Additional Context

Public documentation of the behaviour / error handling and what data is
collected, resp. how to configure will be provided in
https://github.com/zitadel/zitadel/issues/9869.

Closes https://github.com/zitadel/zitadel/issues/9872
2025-06-05 10:13:26 +00:00
Livio SpringandGitHub e2a61a6002 docs(api): remove unreleased services from api reference (#10015)
# Which Problems Are Solved

As we migrate resources to the new API, whenever a an implementation got
merged, the API reference was added to the docs sidenav. As these new
services and their implementation are not yet released, it can be
confusing for developers as the corresponding endpoints return 404 or
unimplemented errors.

# How the Problems Are Solved

Currently we just remove it from the sidenav and will add it once
they're released. We're looking into a proper solution for the API
references.

# Additional Changes

None

# Additional Context

None
2025-06-04 08:41:10 +02:00
Livio SpringandGitHub 15902f5bc7 fix(cache): prevent org cache overwrite by other instances (#10012)
# Which Problems Are Solved

A customer reported that randomly certain login flows, such as automatic
redirect to the only configured IdP would not work. During the
investigation it was discovered that they used that same primary domain
on two different instances. As they used the domain for preselecting the
organization, one would always overwrite the other in the cache. Since
The organization and especially it's policies could not be retrieved on
the other instance, it would fallback to the default organization
settings, where the external login and the corresponding IdP were not
configured.

# How the Problems Are Solved

Include the instance id in the cache key for organizations to prevent
overwrites.

# Additional Changes

None

# Additional Context

- found because of a support request
- requires backport to 2.70.x, 2.71.x and 3.x
2025-06-03 14:48:15 +02:00
Livio SpringandGitHub c097887bc5 fix: validate proto header and provide https enforcement (#9975)
# Which Problems Are Solved

ZITADEL uses the notification triggering requests Forwarded or
X-Forwarded-Proto header to build the button link sent in emails for
confirming a password reset with the emailed code. If this header is
overwritten and a user clicks the link to a malicious site in the email,
the secret code can be retrieved and used to reset the users password
and take over his account.

Accounts with MFA or Passwordless enabled can not be taken over by this
attack.

# How the Problems Are Solved

- The `X-Forwarded-Proto` and `proto` of the Forwarded headers are
validated (http / https).
- Additionally, when exposing ZITADEL through https. An overwrite to
http is no longer possible.

# Additional Changes

None

# Additional Context

None
2025-05-28 10:12:27 +02:00
Livio SpringandGitHub 4d66a786c8 feat: JWT IdP intent (#9966)
# Which Problems Are Solved

The login v1 allowed to use JWTs as IdP using the JWT IDP. The login V2
uses idp intents for such cases, which were not yet able to handle JWT
IdPs.

# How the Problems Are Solved

- Added handling of JWT IdPs in `StartIdPIntent` and `RetrieveIdPIntent`
- The redirect returned by the start, uses the existing `authRequestID`
and `userAgentID` parameter names for compatibility reasons.
- Added `/idps/jwt` endpoint to handle the proxied (callback) endpoint ,
which extracts and validates the JWT against the configured endpoint.

# Additional Changes

None

# Additional Context

- closes #9758
2025-05-27 16:26:46 +02:00
Livio SpringandGitHub 833f6279e1 fix: allow invite codes for users with verified mails (#9962)
# Which Problems Are Solved

Users who started the invitation code verification, but haven't set up
any authentication method, need to be able to do so. This might require
a new invitation code, which was currently not possible since creation
was prevented for users with verified emails.

# How the Problems Are Solved

- Allow creation of invitation emails for users with verified emails.
- Merged the creation and resend into a single method, defaulting the
urlTemplate, applicatioName and authRequestID from the previous code (if
one exists). On the user service API, the `ResendInviteCode` endpoint
has been deprecated in favor of the `CreateInviteCode`

# Additional Changes

None

# Additional Context

- Noticed while investigating something internally.
- requires backport to 2.x and 3.x
2025-05-26 13:59:20 +02:00
2cf3ef4de4 feat: federated logout for SAML IdPs (#9931)
# Which Problems Are Solved

Currently if a user signs in using an IdP, once they sign out of
Zitadel, the corresponding IdP session is not terminated. This can be
the desired behavior. In some cases, e.g. when using a shared computer
it results in a potential security risk, since a follower user might be
able to sign in as the previous using the still open IdP session.

# How the Problems Are Solved

- Admins can enabled a federated logout option on SAML IdPs through the
Admin and Management APIs.
- During the termination of a login V1 session using OIDC end_session
endpoint, Zitadel will check if an IdP was used to authenticate that
session.
- In case there was a SAML IdP used with Federated Logout enabled, it
will intercept the logout process, store the information into the shared
cache and redirect to the federated logout endpoint in the V1 login.
- The V1 login federated logout endpoint checks every request on an
existing cache entry. On success it will create a SAML logout request
for the used IdP and either redirect or POST to the configured SLO
endpoint. The cache entry is updated with a `redirected` state.
- A SLO endpoint is added to the `/idp` handlers, which will handle the
SAML logout responses. At the moment it will check again for an existing
federated logout entry (with state `redirected`) in the cache. On
success, the user is redirected to the initially provided
`post_logout_redirect_uri` from the end_session request.

# Additional Changes

None

# Additional Context

- This PR merges the https://github.com/zitadel/zitadel/pull/9841 and
https://github.com/zitadel/zitadel/pull/9854 to main, additionally
updating the docs on Entra ID SAML.
- closes #9228 
- backport to 3.x

---------

Co-authored-by: Silvan <[email protected]>
Co-authored-by: Zach Hirschtritt <[email protected]>
2025-05-23 13:52:25 +02:00
a73acbcfc3 fix(login): render error properly when auto creation fails (#9871)
# Which Problems Are Solved

If an IdP has the `automatic creation` option enabled without the
`account creation allowed (manually)` and does not provide all the
information required (given name, family name, ...) the wrong error
message was presented to the user.

# How the Problems Are Solved

Prevent overwrite of the error when rendering the error in the
`renderExternalNotFoundOption` function.

# Additional Changes

none

# Additional Context

- closes #9766
- requires backport to 2.x and 3.x

Co-authored-by: Stefan Benz <[email protected]>
2025-05-20 17:18:32 +00:00
Livio SpringandGitHub 968d91a3e0 chore: update dependencies (#9784)
# Which Problems Are Solved

Some dependencies are out of date and published new version including
(unaffected) vulnerability fixes.

# How the Problems Are Solved

- Updated at least all direct dependencies apart from i18n, webauthn
(existing issues),
  -  crewjam (https://github.com/zitadel/zitadel/issues/9783) and
- github.com/gorilla/csrf (https://github.com/gorilla/csrf/issues/190,
https://github.com/gorilla/csrf/issues/189,
https://github.com/gorilla/csrf/issues/188,
https://github.com/gorilla/csrf/issues/187,
https://github.com/gorilla/csrf/issues/186)
      -  noteworthy: https://github.com/golang/go/issues/73626
- Some dependencies require Go 1.24, which triggered an update for
zitadel to go 1.24 as well.

# Additional Changes

None

# Additional Context

None
2025-05-19 10:16:49 +00:00
Livio SpringandGitHub 1b2fd23e0b fix: idp user information mapping (#9892)
# Which Problems Are Solved

When retrieving the information of an IdP intent, depending on the IdP
type (e.g. Apple), there was issue when mapping the stored (event)
information back to the specific IdP type, potentially leading to a
panic.

# How the Problems Are Solved

- Correctly initialize the user struct to map the information to.

# Additional Changes

none

# Additional Context

- reported by a support request
- needs backport to 3.x and 2.x
2025-05-19 09:25:17 +00:00
Livio SpringandGitHub 867e9cb15a fix: correctly use single matching user (by loginname) (#9865)
# Which Problems Are Solved

In rare cases there was a possibility that multiple users were found by
a loginname. This prevented the corresponding user to sign in.

# How the Problems Are Solved

Fixed the corresponding query (to correctly respect the org domain
policy).

# Additional Changes

None

# Additional Context

Found during the investigation of a support request
2025-05-08 07:32:41 +00:00
Livio SpringandGitHub d71795c433 fix: remove index es_instance_position (#9862)
# Which Problems Are Solved

#9837 added a new index `es_instance_position` on the events table with
the idea to improve performance for some projections. Unfortunately, it
makes it worse for almost all projections and would only improve the
situation for the events handler of the actions V2 subscriptions.

# How the Problems Are Solved

Remove the index again.

# Additional Changes

None

# Additional Context

relates to #9837
relates to #9863
2025-05-08 08:35:34 +02:00
Livio SpringandGitHub b1e60e7398 Merge commit from fork
* fix: prevent intent token reuse and add expiry

* fix duplicate

* fix expiration
2025-05-02 13:44:24 +02:00
Livio SpringandGitHub 84628671bd chore: only download release relevant artifacts (#9808)
<!--
Please inform yourself about the contribution guidelines on submitting a
PR here:
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#submit-a-pull-request-pr.
Take note of how PR/commit titles should be written and replace the
template texts in the sections below. Don't remove any of the sections.
It is important that the commit history clearly shows what is changed
and why.
Important: By submitting a contribution you agree to the terms from our
Licensing Policy as described here:
https://github.com/zitadel/zitadel/blob/main/LICENSING.md#community-contributions.
-->

# Which Problems Are Solved

https://github.com/zitadel/zitadel/pull/9765 fixed an issue for with
actions cache service. The PR updated the push action, which now also
provides a build summary. The "release" step tries to download all
artifacts, which now fails:
https://github.com/zitadel/zitadel/actions/runs/14660464768/job/41145285454

# How the Problems Are Solved

Only download relevant artifacts, which are published as part of the
release.

# Additional Changes

None

# Additional Context

None
2025-04-28 11:02:33 +02:00
Livio SpringandGitHub 4ffd4ef381 fix(actions): handle empty deny list correctly (#9753)
<!--
Please inform yourself about the contribution guidelines on submitting a
PR here:
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#submit-a-pull-request-pr.
Take note of how PR/commit titles should be written and replace the
template texts in the sections below. Don't remove any of the sections.
It is important that the commit history clearly shows what is changed
and why.
Important: By submitting a contribution you agree to the terms from our
Licensing Policy as described here:
https://github.com/zitadel/zitadel/blob/main/LICENSING.md#community-contributions.
-->

# Which Problems Are Solved

A customer reached out that after an upgrade, actions would always fail
with the error "host is denied" when calling an external API.
This is due to a security fix
(https://github.com/zitadel/zitadel/security/advisories/GHSA-6cf5-w9h3-4rqv),
where a DNS lookup was added to check whether the host name resolves to
a denied IP or subnet.
If the lookup fails due to the internal DNS setup, the action fails as
well. Additionally, the lookup was also performed when the deny list was
empty.

# How the Problems Are Solved

- Prevent DNS lookup when deny list is empty
- Properly initiate deny list and prevent empty entries

# Additional Changes

- Log the reason for blocked address (domain, IP, subnet)

# Additional Context

- reported by a customer
- needs backport to 2.70.x, 2.71.x and 3.0.0 rc
2025-04-25 07:12:42 +00:00
Livio SpringandGitHub 618143931b chore(ci): fix container build (#9765)
<!--
Please inform yourself about the contribution guidelines on submitting a
PR here:
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#submit-a-pull-request-pr.
Take note of how PR/commit titles should be written and replace the
template texts in the sections below. Don't remove any of the sections.
It is important that the commit history clearly shows what is changed
and why.
Important: By submitting a contribution you agree to the terms from our
Licensing Policy as described here:
https://github.com/zitadel/zitadel/blob/main/LICENSING.md#community-contributions.
-->

# Which Problems Are Solved

While creating a new release, the [pipeline
failed](https://github.com/zitadel/zitadel/actions/runs/14509737111/job/40705906723)
as GH sunset the old actions cache service:

https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/#decommissioned-cache-service-brownouts

# How the Problems Are Solved

The `driver-opts` parameter is removed from the buildx actions to use
the latest stable image. ([new cache service is used by BuildKit >=
v0.20.0](https://docs.docker.com/build/ci/github-actions/cache/#cache-backend-api))

# Additional Changes

Updated docker/build-push-action to v6 in a first attempt to solve the
issue, but kept it as it gave some more insights (incl. build summary)

# Additional Context

Since the containers are only built on workflow triggers, here's the
corresponding pipeline run:
https://github.com/zitadel/zitadel/actions/runs/14513926232
2025-04-22 06:22:54 +00:00
a7a5124643 fix(mirror): initialize meter to prevent panic (#9712)
# Which Problems Are Solved

With the change of #9561, the `mirror` command panics as there's no
metrics provider configured.

# How the Problems Are Solved

Correctly initialize the provider (no-op by default) for the mirror
command.

# Additional Changes

None

# Additional Context

relates to #9561 -> needs backports to 2.66.x - 2.71.x and 3.0.0-rc

Co-authored-by: Stefan Benz <[email protected]>
2025-04-07 16:00:15 +00:00
79a5585f91 fix(login): handle requests without auth request correctly (#9713)
# Which Problems Are Solved

We found some paths in the login UI, where requests without any
`AuthRequest` were not handled correctly and could potentially panic.
This also includes providing the `AuthRequest` as part of `ctx` object
in actions V1.

# How the Problems Are Solved

- Check for the existance of an `AuthRequest` were needed and return an
error otherwise.
- Provide correct state of the `AuthRequest` for actions V1

# Additional Changes

None

# Additional Context

- Noticed as part of a support request
- requires backport to at least 2.70.x

Co-authored-by: Stefan Benz <[email protected]>
2025-04-07 17:40:42 +02:00
Livio SpringandGitHub 315503beab Merge commit from fork 2025-03-31 12:45:11 +02:00
Livio SpringandGitHub 14de8ecac2 Merge commit from fork 2025-03-31 12:44:49 +02:00
Livio SpringandGitHub 3424204291 chore(workflow): run jobs on public runners (#9643)
# Which Problems Are Solved

Build and test workflows are currently running on specific GitHub hosted
runners. These is not needed for most worklfows and just costs more.

# How the Problems Are Solved

Moved all the steps apart from integration-tests to public runners.

# Additional Changes

None

# Additional Context

None
2025-03-26 10:06:46 +00:00
Livio SpringandGitHub 57bfecf7f7 fix(actions): correctly handle api.v1.appendUserGrant (#9598)
# Which Problems Are Solved

It was not possible to use the `api.v1.appendUserGrant` function in the
`postCreation` trigger action as documented.

# How the Problems Are Solved

- Correctly initialize the javascript / Goja function
- Added `projectID` and `projectGrantID` (as documented), but kept
`projectId` and `projectGrantId` (for backwards compatibility) when
mapping the object in the `appendUserGrant` function

# Additional Changes

None

# Additional Context

- A customer reached out to support regarding this issue.
- requires back port to 2.70.x
2025-03-25 11:07:54 +00:00
Livio SpringandGitHub e7c7f9b800 fix: correct event type for permission removed (#9626)
# Which Problems Are Solved

When we recently changed some permission for the `SYSTEM_OWNER` role on
QA, we noticed that there we multiple `permission.added` even when we
removed specific permissions.

# How the Problems Are Solved

Fixed the event type when removing permissions.

# Additional Changes

None

# Additional Context

Noticed when rolling out some changes on QA
2025-03-24 17:00:44 +00:00
Livio SpringandGitHub 54e513f312 fix(console): pass verification code for phone verification (#9616)
# Which Problems Are Solved

While testing totally unrelated issues, I noticed that the phone
verification in Console didn't work anymore.

# How the Problems Are Solved

Pass the verification `code` entered by the user to the API.

# Additional Changes

None

# Additional Context

probably relates to https://github.com/zitadel/zitadel/pull/9312
2025-03-24 16:10:47 +01:00
Livio SpringandGitHub c2c02492b9 chore: update dependencies (#9614)
# Which Problems Are Solved

Multiple dependencies used by the Zitadel backend are outdated.

# How the Problems Are Solved

Update all direct dependencies (apart from the following two) to the
latest version:
- github.com/go-webauthn/webauthn: see
https://github.com/zitadel/zitadel/issues/9615
- github.com/nicksnyder/go-i18n/v2 (v2.5.0 / 2.5.1 prevent the use of
reserved keys such as `description`:
https://github.com/nicksnyder/go-i18n/releases/tag/v2.5.0)

# Additional Changes

None

# Additional Context

None
2025-03-24 12:06:35 +02:00
Livio SpringandGitHub f1f500d0e7 chore: use crdb 24.3 (#9558)
# Which Problems Are Solved

E2E tests in pipelines started to fail randomly. While debugging it, i
noticed that we use the `latest` tag of cockroach's docker image. They
tagged 25.1 as latest yesterday.

# How the Problems Are Solved

Since we drop support for CRDB with version 3 as there are anyway
multiple issues with various versions, I pinned the docker image tag to
`latest-v24.3`.

# Additional Changes

None

# Additional Context

relates to https://github.com/zitadel/zitadel/actions/runs/13917603587
and https://github.com/zitadel/zitadel/actions/runs/13904928050
2025-03-18 10:52:38 +01:00
Livio SpringandGitHub 19f022e1cf fix(login): cache scripts and other assets correctly (#9551)
# Which Problems Are Solved

Scripts and other assets for the hosted login UI are served with a
public cache with `max-age` and `s-maxage`. After changing scripts or
assets, old versions might be still used as they might be cached locally
or in a shared cache (CDN, proxy, ...). This can lead to unwanted
behaviour or even errors.

# How the Problems Are Solved

To ensure the correct file is served a query parameter with the build
time is added to the assets filename dynamically. (`?v=2025-03-17...`)

# Additional Changes

None

# Additional Context

- relates to #9485
- requires backport to at least 2.70.x
2025-03-17 11:13:24 +00:00
Livio SpringandGitHub 61c4b1c3fd fix(console): allow management of metadata of users of other organizations again (#9490)
# Which Problems Are Solved

With the recent change in Console to use the User V2 API
(https://github.com/zitadel/zitadel/pull/9312), some functionality still
needs to call the management API, which requires the organization
context. The context was not passed anymore, leading to error in cases
where the calling user (e.g. an IAM_OWNER) was not part of the same
organization.

# How the Problems Are Solved

Added an interceptor to provide the `x-zitadel-orgid` header for the new
management client.

# Additional Changes

None

# Additional Context

- closes #9488
2025-03-14 06:05:45 +00:00
Livio SpringandGitHub d527a1c824 feat(console): specify login UI for SAML apps (#9486)
# Which Problems Are Solved

Users were not yet able to specify (and test) the new login UI or
self-hosted login UI for SAML applications through Console.

# How the Problems Are Solved

Added the configuration for SAML apps (as already available for OIDC) in
Console.

# Additional Changes

None

# Additional Context

- closes #9354
2025-03-12 12:37:43 +00:00
Livio SpringandGitHub ed697bbd69 fix(OIDC): back channel logout work for custom UI (#9487)
# Which Problems Are Solved

When using a custom / new login UI and an OIDC application with
registered BackChannelLogoutUI, no logout requests were sent to the URI
when the user signed out.
Additionally, as described in #9427, an error was logged:
`level=error msg="event of type *session.TerminateEvent doesn't
implement OriginEvent"
caller="/home/runner/work/zitadel/zitadel/internal/notification/handlers/origin.go:24"`

# How the Problems Are Solved

- Properly pass `TriggerOrigin` information to session.TerminateEvent
creation and implement `OriginEvent` interface.
- Implemented `RegisterLogout` in `CreateOIDCSessionFromAuthRequest` and
`CreateOIDCSessionFromDeviceAuth`, both used when interacting with the
OIDC v2 API.
- Both functions now receive the `BackChannelLogoutURI` of the client
from the OIDC layer.

# Additional Changes

None

# Additional Context

- closes #9427
2025-03-11 14:19:09 +00:00
Livio SpringandGitHub e6ce1af003 fix(token exchange): properly return an error if membership is missing (#9468)
# Which Problems Are Solved

When requesting a JWT (`urn:ietf:params:oauth:token-type:jwt`) to be
returned in a Token Exchange request, ZITADEL would panic if the `actor`
was not granted the necessary permission.

# How the Problems Are Solved

Properly check the error and return it.

# Additional Changes

None

# Additional Context

- closes #9436
2025-03-11 11:14:18 +00:00
Livio SpringandGitHub ead9fde16a docs: improve api guidelines (#9478)
# Which Problems Are Solved

The API guidelines were not completely accurate on how we want to check
permissions.

# How the Problems Are Solved

Made the description clearer and added examples.

# Additional Changes

Improved the error code example as initially intended in #9340
and added notes about the `limits` for listing resources.

# Additional Context

popped up in PR review of https://github.com/zitadel/zitadel/pull/9445
2025-03-07 14:19:09 +00:00
Livio SpringandGitHub d9d8339813 Merge commit from fork 2025-03-04 08:49:02 +01:00
Livio SpringandGitHub 4e1868e9bb fix: prevent panic when retrieving session by id in internal calls (#9442)
# Which Problems Are Solved

#9110 introduced more possibilities to search for "own" sessions. Due to
this the permission checks for retrieving a session had to be updated
accordingly. Internal calls, such as retrieving them for sending
notifications do not require a permission, but the code was not properly
adjusted and thus could lead to panics.

# How the Problems Are Solved

- Properly handled (do not require) permission check for internal only
calls when retrieving the session by id.

# Additional Changes

None

# Additional Context

- needs backports to 2.68.x, 2.69.x, 2.70.x
- closes zitadel/devops#117
2025-03-03 11:24:52 +01:00
4df3b6492c chore: API guidelines (#9340)
# Which Problems Are Solved

There were no guideline to how design future APIs and their endpoints.
The V3 documentation was to specific and targeted towards internal
stakeholders.
This PR is intended as base and kept to the minimum. If more details or
additional guideline or rules are needed, they will be added in the
future.

# How the Problems Are Solved

- Removed the V3 description and corresponding examples.
- Provided general guideline for the design of APIs, which includes the
structure, naming, versioning, error handling and more.

# Additional Changes

None

# Additional Context

closes #9184

---------

Co-authored-by: Maximilian <[email protected]>
Co-authored-by: Silvan <[email protected]>
2025-02-27 11:30:39 +00:00
8f88c4cf5b feat: add PKCE option to generic OAuth2 / OIDC identity providers (#9373)
# Which Problems Are Solved

Some OAuth2 and OIDC providers require the use of PKCE for all their
clients. While ZITADEL already recommended the same for its clients, it
did not yet support the option on the IdP configuration.

# How the Problems Are Solved

- A new boolean `use_pkce` is added to the add/update generic OAuth/OIDC
endpoints.
- A new checkbox is added to the generic OAuth and OIDC provider
templates.
- The `rp.WithPKCE` option is added to the provider if the use of PKCE
has been set.
- The `rp.WithCodeChallenge` and `rp.WithCodeVerifier` options are added
to the OIDC/Auth BeginAuth and CodeExchange function.
- Store verifier or any other persistent argument in the intent or auth
request.
- Create corresponding session object before creating the intent, to be
able to store the information.
- (refactored session structs to use a constructor for unified creation
and better overview of actual usage)

Here's a screenshot showing the URI including the PKCE params:


![use_pkce_in_url](https://github.com/zitadel/zitadel/assets/30386061/eaeab123-a5da-4826-b001-2ae9efa35169)

# Additional Changes

None.

# Additional Context

- Closes #6449
- This PR replaces the existing PR (#8228) of @doncicuto. The base he
did was cherry picked. Thank you very much for that!

---------

Co-authored-by: Miguel Cabrerizo <[email protected]>
Co-authored-by: Stefan Benz <[email protected]>
2025-02-26 12:20:47 +00:00
911200aa9b feat(api): allow Device Authorization Grant using custom login UI (#9387)
# Which Problems Are Solved

The OAuth2 Device Authorization Grant could not yet been handled through
the new login UI, resp. using the session API.
This PR adds the ability for the login UI to get the required
information to display the user and handle their decision (approve with
authorization or deny) using the OIDC Service API.

# How the Problems Are Solved

- Added a `GetDeviceAuthorizationRequest` endpoint, which allows getting
the `id`, `client_id`, `scope`, `app_name` and `project_name` of the
device authorization request
- Added a `AuthorizeOrDenyDeviceAuthorization` endpoint, which allows to
approve/authorize with the session information or deny the request. The
identification of the request is done by the `device_authorization_id` /
`id` returned in the previous request.
- To prevent leaking the `device_code` to the UI, but still having an
easy reference, it's encrypted and returned as `id`, resp. decrypted
when used.
- Fixed returned error types for device token responses on token
endpoint:
- Explicitly return `access_denied` (without internal error) when user
denied the request
  - Default to `invalid_grant` instead of `access_denied`
- Explicitly check on initial state when approving the reqeust
- Properly handle done case (also relates to initial check) 
- Documented the flow and handling in custom UIs (according to OIDC /
SAML)

# Additional Changes

- fixed some typos and punctuation in the corresponding OIDC / SAML
guides.
- added some missing translations for auth and saml request

# Additional Context

- closes #6239

---------

Co-authored-by: Tim Möhlmann <[email protected]>
2025-02-25 07:33:13 +01:00
Livio SpringandGitHub e7a73eb6b1 fix(oidc / login v2): always us login v2 if x-zitadel-login-client header is sent (#9336)
# Which Problems Are Solved

As reported in #9311, even when providing a `x-zitadel-login-client`
header, the auth request would be created as hosted login UI / V1
request.
This is due to a change introduced with #9071, where the login UI
version can be specified using the app configuration.
The configuration set to V1 was not considering if the header was sent.

# How the Problems Are Solved

- Check presence of `x-zitadel-login-client` before the configuration.
Use later only if no header is set.

# Additional Changes

None

# Additional Context

- closes #9311 
- needs back ports to 2.67.x, 2.68.x and 2.69.x
2025-02-10 14:46:28 +01:00
Livio SpringandGitHub b63c5fdb17 fix(login): fix migration to allow login by email again (#9315)
# Which Problems Are Solved

The login by email was not possible anymore. This was due to a newly
generated user projection because of #9255 .
Internal logs showed that the computed lower case column for verified
email was missing.

# How the Problems Are Solved

Update name of setup step 25 to rerun the step, since the underlying sql
changed.

# Additional Changes

None

# Additional Context

- relates to #9255
2025-02-06 09:45:22 +00:00
Livio SpringandGitHub 990e1982c7 fix(OTEL): reduce high cardinality in traces and metrics (#9286)
# Which Problems Are Solved

There were multiple issues in the OpenTelemetry (OTEL) implementation
and usage for tracing and metrics, which lead to high cardinality and
potential memory leaks:
- wrongly initiated tracing interceptors
- high cardinality in traces:
  - HTTP/1.1 endpoints containing host names
- HTTP/1.1 endpoints containing object IDs like userID (e.g.
`/management/v1/users/2352839823/`)
- high amount of traces from internal processes (spooler)
- high cardinality in metrics endpoint:
  - GRPC entries containing host names
  - notification metrics containing instanceIDs and error messages

# How the Problems Are Solved

- Properly initialize the interceptors once and update them to use the
grpc stats handler (unary interceptors were deprecated).
- Remove host names from HTTP/1.1 span names and use path as default.
- Set / overwrite the uri for spans on the grpc-gateway with the uri
pattern (`/management/v1/users/{user_id}`). This is used for spans in
traces and metric entries.
- Created a new sampler which will only sample spans in the following
cases:
  - remote was already sampled
- remote was not sampled, root span is of kind `Server` and based on
fraction set in the runtime configuration
- This will prevent having a lot of spans from the spooler back ground
jobs if they were not started by a client call querying an object (e.g.
UserByID).
- Filter out host names and alike from OTEL generated metrics (using a
`view`).
- Removed instance and error messages from notification metrics.

# Additional Changes

Fixed the middleware handling for serving Console. Telemetry and
instance selection are only used for the environment.json, but not on
statically served files.

# Additional Context

- closes #8096 
- relates to #9074
- back ports to at least 2.66.x, 2.67.x and 2.68.x
2025-02-04 09:55:26 +01:00
Livio SpringandGitHub 04b9e9b144 fix(console): add posthog to CSP if configured (#9284)
# Which Problems Are Solved

PostHog scripts are currently blocked by content security policy (CSP).

# How the Problems Are Solved

Add `https://*.i.posthog.com` to the CSP according to
https://posthog.com/docs/advanced/content-security-policy#enabling-the-toolbar
(they suggest  `https://*.posthog.com`)

# Additional Changes

None

# Additional Context

relates to https://github.com/zitadel/zitadel/issues/9076
2025-02-03 08:08:01 +01:00
Livio SpringandGitHub 3fc68e5d60 fix(notifications): cancel on missing channels and Twilio 4xx errors (#9254)
# Which Problems Are Solved

#9185 changed that if a notification channel was not present,
notification workers would no longer retry to send the notification and
would also cancel in case Twilio would return a 4xx error.
However, this would not affect the "legacy" mode.

# How the Problems Are Solved

- Handle `CancelError` in legacy notifier as not failed (event).

# Additional Changes

None

# Additional Context

- relates to #9185 
- requires back port to 2.66.x and 2.67.x
2025-01-28 06:32:09 +00:00
Livio SpringandGitHub c9aa5db2a5 fix(oidc apps): correctly remove last additional origin, redirect uri and post logout redirect uri (#9209)
# Which Problems Are Solved

A customer reached out to support, that the (last) `additional origin`
could not be removed. While testing / implementation it was discovered,
that the same applied to `redirect_uris` and `post_logout_redirect_uris`

# How the Problems Are Solved

- Correctly set the corresponding array to empty in the event so it can
be differentiated to `null` / not set in case of no change.

# Additional Changes

Replaced `reflect.DeepEqual` with `slices.Equal`

# Additional Context

- Reported to support
2025-01-22 07:37:37 +00:00
Livio SpringandGitHub 60857c8d3e fix: cancel notifications on missing channels and configurable (twilio) error codes (#9185)
# Which Problems Are Solved

If a notification channel was not present, notification workers would
retry to the max attempts. This leads to unnecessary load.
Additionally, a client noticed  bad actors trying to abuse SMS MFA. 

# How the Problems Are Solved

- Directly cancel the notification on:
  - a missing channel and stop retries.
  - any `4xx` errors from Twilio Verify

# Additional Changes

None

# Additional Context

reported by customer
2025-01-17 07:42:14 +00:00
Livio SpringandGitHub 40082745f4 fix(login): allow fallback to local auth in case of IdP errors (#9178)
# Which Problems Are Solved

The current login will always prefer external authentication (through an
IdP) over local authentication. So as soon as either the user had
connected to an IdP or even when the login policy was just set up to
have an IdP allowed, users would be redirected to that IdP for
(re)authentication.
This could lead to problems, where the IdP was not available or any
other error occurred in the process (such as secret expired for
EntraID).
Even when local authentication (passkeys or password) was allowed for
the corresponding user, they would always be redirected to the IdP
again, preventing any authentication. If admins were affected, they
might not even be able to update the client secret of the IdP.

# How the Problems Are Solved

Errors during the external IdP flow are handled in an
`externalAuthFailed` function, which will check if the organisation
allows local authentication and if the user has set up such.
If either password or passkeys is set up, the corresponding login page
will be presented to the user. As already with local auth passkeys is
preferred over password authentication.
The user is informed that the external login failed and fail back to
local auth as an error on the corresponding page in a focused mode. Any
interaction or after 5 second the focus mode is disabled.

# Additional Changes

None.

# Additional Context

closes #6466
2025-01-15 10:39:28 +00:00
Livio SpringandGitHub c966446f80 fix: correctly get x-forwarded-for for browser info in events (#9149)
# Which Problems Are Solved

Events like "password check succeeded" store some information about the
caller including their IP.
The `X-Forwarded-For` was not correctly logged, but instead the
RemoteAddress.

# How the Problems Are Solved

- Correctly get the `X-Forwarded-For` in canonical form.

# Additional Changes

None

# Additional Context

closes [#9106](https://github.com/zitadel/zitadel/issues/9106)
2025-01-08 09:30:12 +00:00
Livio SpringandGitHub 8d7a1efd4a fix(idp): correctly get data from cache before parsing (#9134)
# Which Problems Are Solved

IdPs using form callback were not always correctly handled with the
newly introduced cache mechanism
(https://github.com/zitadel/zitadel/pull/9097).

# How the Problems Are Solved

Get the data from cache before parsing it.

# Additional Changes

None

# Additional Context

Relates to https://github.com/zitadel/zitadel/pull/9097
2025-01-06 14:48:32 +01:00
fa5e590aab fix(idp): prevent server errors for idps using form post for callbacks (#9097)
# Which Problems Are Solved

Some IdP callbacks use HTTP form POST to return their data on callbacks.
For handling CSRF in the login after such calls, a 302 Found to the
corresponding non form callback (in ZITADEL) is sent. Depending on the
size of the initial form body, this could lead to ZITADEL terminating
the connection, resulting in the user not getting a response or an
intermediate proxy to return them an HTTP 502.

# How the Problems Are Solved

- the form body is parsed and stored into the ZITADEL cache (using the
configured database by default)
- the redirect (302 Found) is performed with the request id
- the callback retrieves the data from the cache instead of the query
parameters (will fallback to latter to handle open uncached requests)

# Additional Changes

- fixed a typo in the default (cache) configuration: `LastUsage` ->
`LastUseAge`

# Additional Context

- reported by a customer
- needs to be backported to current cloud version (2.66.x)

---------

Co-authored-by: Silvan <[email protected]>
2025-01-06 10:47:46 +01:00
Livio SpringandGitHub bcf416d4cf fix(saml): parse xsd:duration format correctly (#9098)
# Which Problems Are Solved

SAML IdPs exposing an `EntitiesDescriptor` using an `xsd:duration` time
format for the `cacheDuration` property (e.g. `PT5H`) failed parsing.

# How the Problems Are Solved

Handle the unmarshalling for `EntitiesDescriptor` specifically.
[crewjam/saml](https://github.com/crewjam/saml/blob/bbccb7933d5f60512ebc6caec7120c604581983d/metadata.go#L88-L103)
already did this for `EntitiyDescriptor` the same way.

# Additional Changes

None

# Additional Context

- reported by a customer
- needs to be backported to current cloud version (2.66.x)
2024-12-20 16:03:06 +00:00
Livio SpringandGitHub d9a89d9fcf chore: update dependencies (#9087)
Updates most urgent dependencies
2024-12-19 14:01:33 +00:00
Livio SpringandGitHub 50d2b26a28 feat: specify login UI version on instance and apps (#9071)
# Which Problems Are Solved

To be able to migrate or test the new login UI, admins might want to
(temporarily) switch individual apps.
At a later point admin might want to make sure all applications use the
new login UI.

# How the Problems Are Solved

- Added a feature flag `` on instance level to require all apps to use
the new login and provide an optional base url.
- if the flag is enabled, all (OIDC) applications will automatically use
the v2 login.
  - if disabled, applications can decide based on their configuration
- Added an option on OIDC apps to use the new login UI and an optional
base url.
- Removed the requirement to use `x-zitadel-login-client` to be
redirected to the login V2 and retrieve created authrequest and link
them to SSO sessions.
- Added a new "IAM_LOGIN_CLIENT" role to allow management of users,
sessions, grants and more without `x-zitadel-login-client`.

# Additional Changes

None

# Additional Context

closes https://github.com/zitadel/zitadel/issues/8702
2024-12-19 10:37:46 +01:00
Livio SpringandGitHub f20539ef8f fix(login): make sure first email verification is done before MFA check (#9039)
# Which Problems Are Solved

During authentication in the login UI, there is a check if the user's
MFA is already checked or needs to be setup.
In cases where the user was just set up or especially, if the user was
just federated without a verified email address, this can lead to the
problem, where OTP Email cannot be setup as there's no verified email
address.

# How the Problems Are Solved

- Added a check if there's no verified email address on the user and
require a mail verification check before checking for MFA.
Note: that if the user had a verified email address, but changed it and
has not verified it, they will still be prompted with an MFA check
before the email verification. This is make sure, we don't break the
existing behavior and the user's authentication is properly checked.

# Additional Changes

None

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/9035
2024-12-13 11:37:20 +00:00
Livio SpringandGitHub fd70a7de5f fix(api): map REST request body in user invite requests (#9054)
# Which Problems Are Solved

The `CreateInviteCode` and `VerifyInviteCode` methods missed the body
mapping.

# How the Problems Are Solved

Added the mapping.

# Additional Changes

None

# Additional Context

Noticed during internal login UI tests using REST
2024-12-13 10:24:14 +00:00
Livio SpringandGitHub 40fedace3c docs(oidc): add back-channel logout (#9034)
# Which Problems Are Solved

OIDC Back-Channel Logout released with
[V2.65.0](https://github.com/zitadel/zitadel/releases/tag/v2.65.0) were
not yet documented

# How the Problems Are Solved

- Added small guide and description
- Updated claims (added `sid` and `events`)

# Additional Changes

None

# Additional Context

relates to https://github.com/zitadel/zitadel/issues/8467
2024-12-13 09:47:04 +00:00
Livio SpringandGitHub 5c3e917248 chore: remove stable release tag (#8885)
# Which Problems Are Solved

The current "stable" release tag was no longer maintained.

# How the Problems Are Solved

Remove the tag from the docs.

# Additional Changes

Update the docs to reflect that test run with Ubuntu 22.04 instead of
20.04.

# Additional Context

- relates to https://github.com/zitadel/zitadel/issues/8884
2024-12-09 08:29:13 +01:00
Livio SpringandGitHub 7a3ae8f499 fix(notifications): bring back legacy notification handling (#9015)
# Which Problems Are Solved

There are some problems related to the use of CockroachDB with the new
notification handling (#8931).
See #9002 for details.

# How the Problems Are Solved

- Brought back the previous notification handler as legacy mode.
- Added a configuration to choose between legacy mode and new parallel
workers.
  - Enabled legacy mode by default to prevent issues.

# Additional Changes

None

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/9002
- relates to #8931
2024-12-06 10:56:19 +01:00
Livio SpringandGitHub 0017e4daa6 docs: remove autoplay from videos (#9005)
# Which Problems Are Solved

Some videos in the guides start playing automatically. This prevents a
great user / developer experience.

# How the Problems Are Solved

Stop autoplay.

# Additional Changes

None

# Additional Context

Discussed internally
2024-12-05 06:23:59 +00:00
7f0378636b fix(notifications): improve error handling (#8994)
# Which Problems Are Solved

While running the latest RC / main, we noticed some errors including
context timeouts and rollback issues.

# How the Problems Are Solved

- The transaction context is passed and used for any event being written
and for handling savepoints to be able to handle context timeouts.
- The user projection is not triggered anymore. This will reduce
unnecessary load and potential timeouts if lot of workers are running.
In case a user would not be projected yet, the request event will log an
error and then be skipped / retried on the next run.
- Additionally, the context is checked if being closed after each event
process.
- `latestRetries` now correctly only returns the latest retry events to
be processed
- Default values for notifications have been changed to run workers less
often, more retry delay, but less transaction duration.

# Additional Changes

None

# Additional Context

relates to #8931

---------

Co-authored-by: Tim Möhlmann <[email protected]>
2024-12-04 20:17:49 +00:00
Livio SpringandGitHub ffe9570776 fix(saml): improve error handling (#8928)
# Which Problems Are Solved

There are multiple issues with the metadata and error handling of SAML:
- When providing a SAML metadata for an IdP, which cannot be processed,
the error will only be noticed once a user tries to use the IdP.
- Parsing for metadata with any other encoding than UTF-8 fails.
- Metadata containing an enclosing EntitiesDescriptor around
EntityDescriptor cannot be parsed.
- Metadata's `validUntil` value is always set to 48 hours, which causes
issues on external providers, if processed from a manual down/upload.
- If a SAML response cannot be parsed, only a generic "Authentication
failed" error is returned, the cause is hidden to the user and also to
actions.

# How the Problems Are Solved

- Return parsing errors after create / update and retrieval of an IdP in
the API.
- Prevent the creation and update of an IdP in case of a parsing
failure.
- Added decoders for encodings other than UTF-8 (including ASCII,
windows and ISO, [currently
supported](https://github.com/golang/text/blob/efd25daf282ae4d20d3625f1ccb4452fe40967ae/encoding/ianaindex/ianaindex.go#L156))
- Updated parsing to handle both `EntitiesDescriptor` and
`EntityDescriptor` as root element
- `validUntil` will automatically set to the certificate's expiration
time
- Unwrapped the hidden error to be returned. The Login UI will still
only provide a mostly generic error, but action can now access the
underlying error.

# Additional Changes

None

# Additional Context

reported by a customer
2024-12-03 10:38:28 +00:00
Livio SpringandGitHub 8537805ea5 feat(notification): use event worker pool (#8962)
# Which Problems Are Solved

The current handling of notification follows the same pattern as all
other projections:
Created events are handled sequentially (based on "position") by a
handler. During the process, a lot of information is aggregated (user,
texts, templates, ...).
This leads to back pressure on the projection since the handling of
events might take longer than the time before a new event (to be
handled) is created.

# How the Problems Are Solved

- The current user notification handler creates separate notification
events based on the user / session events.
- These events contain all the present and required information
including the userID.
- These notification events get processed by notification workers, which
gather the necessary information (recipient address, texts, templates)
to send out these notifications.
- If a notification fails, a retry event is created based on the current
notification request including the current state of the user (this
prevents race conditions, where a user is changed in the meantime and
the notification already gets the new state).
- The retry event will be handled after a backoff delay. This delay
increases with every attempt.
- If the configured amount of attempts is reached or the message expired
(based on config), a cancel event is created, letting the workers know,
the notification must no longer be handled.
- In case of successful send, a sent event is created for the
notification aggregate and the existing "sent" events for the user /
session object is stored.
- The following is added to the defaults.yaml to allow configuration of
the notification workers:
```yaml

Notifications:
  # The amount of workers processing the notification request events.
  # If set to 0, no notification request events will be handled. This can be useful when running in
  # multi binary / pod setup and allowing only certain executables to process the events.
  Workers: 1 # ZITADEL_NOTIFIACATIONS_WORKERS
  # The amount of events a single worker will process in a run.
  BulkLimit: 10 # ZITADEL_NOTIFIACATIONS_BULKLIMIT
  # Time interval between scheduled notifications for request events
  RequeueEvery: 2s # ZITADEL_NOTIFIACATIONS_REQUEUEEVERY
  # The amount of workers processing the notification retry events.
  # If set to 0, no notification retry events will be handled. This can be useful when running in
  # multi binary / pod setup and allowing only certain executables to process the events.
  RetryWorkers: 1 # ZITADEL_NOTIFIACATIONS_RETRYWORKERS
  # Time interval between scheduled notifications for retry events
  RetryRequeueEvery: 2s # ZITADEL_NOTIFIACATIONS_RETRYREQUEUEEVERY
  # Only instances are projected, for which at least a projection-relevant event exists within the timeframe
  # from HandleActiveInstances duration in the past until the projection's current time
  # If set to 0 (default), every instance is always considered active
  HandleActiveInstances: 0s # ZITADEL_NOTIFIACATIONS_HANDLEACTIVEINSTANCES
  # The maximum duration a transaction remains open
  # before it spots left folding additional events
  # and updates the table.
  TransactionDuration: 1m # ZITADEL_NOTIFIACATIONS_TRANSACTIONDURATION
  # Automatically cancel the notification after the amount of failed attempts
  MaxAttempts: 3 # ZITADEL_NOTIFIACATIONS_MAXATTEMPTS
  # Automatically cancel the notification if it cannot be handled within a specific time
  MaxTtl: 5m  # ZITADEL_NOTIFIACATIONS_MAXTTL
  # Failed attempts are retried after a confogired delay (with exponential backoff).
  # Set a minimum and maximum delay and a factor for the backoff
  MinRetryDelay: 1s  # ZITADEL_NOTIFIACATIONS_MINRETRYDELAY
  MaxRetryDelay: 20s # ZITADEL_NOTIFIACATIONS_MAXRETRYDELAY
  # Any factor below 1 will be set to 1
  RetryDelayFactor: 1.5 # ZITADEL_NOTIFIACATIONS_RETRYDELAYFACTOR
```


# Additional Changes

None

# Additional Context

- closes #8931
2024-11-27 15:01:17 +00:00
Livio SpringandGitHub 374b9a7f66 fix(saml): provide option to get internal as default ACS (#8888)
# Which Problems Are Solved

Some SAML IdPs including Google only allow to configure a single
AssertionConsumerService URL.
Since the current metadata provides multiple and the hosted login UI is
not published as neither the first nor with `isDefault=true`, those IdPs
take another and then return an error on sign in.

# How the Problems Are Solved

Allow to reorder the ACS URLs using a query parameter
(`internalUI=true`) when retrieving the metadata endpoint.
This will list the `ui/login/login/externalidp/saml/acs` first and also
set the `isDefault=true`.

# Additional Changes

None

# Additional Context

Reported by a customer
2024-11-15 07:19:43 +01:00
Livio SpringandGitHub 85bdf01505 fix(actions): preserve order of execution (#8895)
# Which Problems Are Solved

The order of actions on a trigger was not respected on the execution and
not correctly returned when retrieving the flow, for example in Console.
The supposed correction of the order (e.g. in the UI) would then return
a "no changes" error since the order was already as desired.

# How the Problems Are Solved

- Correctly order the actions of a trigger based on their configuration
(`trigger_sequence`).

# Additional Changes

- replaced a `reflect.DeepEqual` with `slices.Equal` for checking the
action list

# Additional Context

- reported by a customer
- requires backports
2024-11-14 14:04:39 +00:00
Livio SpringandGitHub ecbf0db15b fix(setup): improve search query to use index (#8898)
# Which Problems Are Solved

The setup filter for previous steps and kept getting slower. This is due
to the filter, which did not provide any instanceID and thus resulting
in a full table scan.

# How the Problems Are Solved

- Added an empty instanceID filter (since it's on system level)

# Additional Changes

None

# Additional Context

Noticed internally and during migrations on some regions
2024-11-13 07:50:23 +00:00
Livio SpringandGitHub fb6579e456 fix(milestones): use previous spelling for milestone types (#8886)
# Which Problems Are Solved

https://github.com/zitadel/zitadel/pull/8788 accidentally changed the
spelling of milestone types from PascalCase to snake_case. This breaks
systems where `milestone.pushed` events already exist.

# How the Problems Are Solved

- Use PascalCase again
- Prefix event types with v2. (Previous pushed event type was anyway
ignored).
- Create `milstones3` projection

# Additional Changes

None

# Additional Context

relates to #8788
2024-11-11 11:28:27 +00:00
041af26917 feat(OIDC): add back channel logout (#8837)
# Which Problems Are Solved

Currently ZITADEL supports RP-initiated logout for clients. Back-channel
logout ensures that user sessions are terminated across all connected
applications, even if the user closes their browser or loses
connectivity providing a more secure alternative for certain use cases.

# How the Problems Are Solved

If the feature is activated and the client used for the authentication
has a back_channel_logout_uri configured, a
`session_logout.back_channel` will be registered. Once a user terminates
their session, a (notification) handler will send a SET (form POST) to
the registered uri containing a logout_token (with the user's ID and
session ID).

- A new feature "back_channel_logout" is added on system and instance
level
- A `back_channel_logout_uri` can be managed on OIDC applications
- Added a `session_logout` aggregate to register and inform about sent
`back_channel` notifications
- Added a `SecurityEventToken` channel and `Form`message type in the
notification handlers
- Added `TriggeredAtOrigin` fields to `HumanSignedOut` and
`TerminateSession` events for notification handling
- Exported various functions and types in the `oidc` package to be able
to reuse for token signing in the back_channel notifier.
- To prevent that current existing session termination events will be
handled, a setup step is added to set the `current_states` for the
`projections.notifications_back_channel_logout` to the current position

- [x] requires https://github.com/zitadel/oidc/pull/671

# Additional Changes

- Updated all OTEL dependencies to v1.29.0, since OIDC already updated
some of them to that version.
- Single Session Termination feature is correctly checked (fixed feature
mapping)

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/8467
- TODO:
  - Documentation
  - UI to be done: https://github.com/zitadel/zitadel/issues/8469

---------

Co-authored-by: Hidde Wieringa <[email protected]>
2024-10-31 15:57:17 +01:00
Livio SpringandGitHub 0b5079c11f fix: correctly search for verified domain (#8820)
# Which Problems Are Solved

Searching orgs by domain currently only looked for the primary domain,
but should be possible with all verified domains (as documented)

# How the Problems Are Solved

- fixed the search query

# Additional Changes

None

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/8749
2024-10-25 17:51:44 +02:00
Livio SpringandGitHub 79fb4cc1cc fix: correctly check denied domains and ips for actions (#8810)
# Which Problems Are Solved

System administrators can block hosts and IPs for HTTP calls in actions.
Using DNS, blocked IPs could be bypassed.

# How the Problems Are Solved

- Hosts are resolved (DNS lookup) to check whether their corresponding
IP is blocked.

# Additional Changes

- Added complete lookup ip address range and "unspecified" address to
the default `DenyList`
2024-10-22 16:16:44 +02:00
Livio SpringandGitHub df2033253d fix: only allow domain discovery if no organization was preselected (#8748)
# Which Problems Are Solved

If an organization was preselected using an orgID or primaryDomain
scope, users could still switch to another organization, if the latter
allowed domain discovery and the entered username / or login_hint
included the corresponding domain suffix.

# How the Problems Are Solved

Domain discovery will only be done in case no org was preselected.

# Additional Changes

None

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/8464
- closes https://github.com/zitadel/zitadel/issues/8588
2024-10-10 15:29:53 +00:00
Livio SpringandGitHub 16171ce3b9 fix: pass sessionID to OTP email link (#8745)
# Which Problems Are Solved

OTP Email links currently could not use / include the sessionID they
belong to. This prevents an easy use for redirecting and handling OTP
via email through the session API.

# How the Problems Are Solved

Added the sessionID as placeholder for the OTP Email link template.

# Additional Changes

List all available placeholders in the url_templates of V2 endpoints.

# Additional Context

- discussed in a customer meeting
2024-10-10 13:53:32 +00:00
Livio SpringandGitHub 94f267d475 fix: respect PrivateLabelingSetting on project for login texts (#8744)
# Which Problems Are Solved

Admins can set the branding to be used from the project's organization.
Until now, only the branding (colors) were respected, but texts were
still loaded from the user's organization.

# How the Problems Are Solved

Respect the setting when loading the texts for the login pages.

# Additional Changes

None

# Additional Context

- closes #8502
2024-10-09 09:02:28 +00:00
Livio SpringandGitHub 91b2d67325 fix: correctly compute user state (#8740)
# Which Problems Are Solved

A customer reported, that after a created user (in initial state) got
(manually) locked and a new initial code would be created, the user
could not be locked again.

# How the Problems Are Solved

Query for the initial code added event

# Additional Changes

None

# Additional Context

- reported by a customer
2024-10-07 16:50:17 +02:00
f653589609 fix: twilio code generation and verification (#8728)
# Which Problems Are Solved

The recently added possibility to generate and verify codes through
Twilio verification service did failed on checking OTP SMS code through
the session API. Additionally, password codes generated by the V2 API
and sent through phone would always use the internal generator and
verification mechanism rather than the configured.

# How the Problems Are Solved

- Correctly set the verifier for OTP SMS for the session API
  - Always use the internal verifier for OTP Email (for now)
- Select the generator / verifier based on the configuration for
password codes with notification type SMS for V2 APIs

# Additional Changes

None

# Additional Context

- relates to #8678 
- reported by customer

---------

Co-authored-by: Stefan Benz <[email protected]>
2024-10-07 07:12:44 +02:00
Livio SpringandGitHub bee0744d46 fix: update logging to allow slog field overwrite (#8720)
# Which Problems Are Solved

When using slog (e.g. in OIDC) the logs field name can not be
overwritten.
This is necessary for example to change log level to severity.

# How the Problems Are Solved

- Update logging library

# Additional Changes

None

# Additional Context

None
2024-10-04 12:48:00 +00:00
Livio SpringandGitHub 18499274dd fix(SAML): check on empty nameID (#8714)
# Which Problems Are Solved

If a SAML IdP did not send a `NameID` (even though required by the
specification), ZITADEL would crash.

# How the Problems Are Solved

- Check specifically if the `Subject` and its `NameID` is passed

# Additional Changes

None

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/8654
2024-10-03 08:17:33 +00:00
Livio SpringandGitHub dc7330f251 fix(API): select org context by domain (#8706)
# Which Problems Are Solved

V2 and V3 APIs allow setting the organization context by providing the
organization domain in the request. Users currently experience the
following error: "rpc error: code = Unauthenticated desc = context
missing (AUTH-rKLWEH)"

# How the Problems Are Solved

Correctly check the org domain when set.

# Additional Changes

None

# Additional Context

- support request
2024-10-02 06:38:54 +00:00
Livio SpringandGitHub c347e75485 fix: ignore projectID and origin check for service accounts (#8704)
# Which Problems Are Solved

Calls with tokens issued through JWT Profile or Client Credentials
Grants were no longer possible and threw a "could not read projectid by
clientid (AUTH-GHpw2)" error.
ZITADEL checks the allowed origins of an application and load its
projectID into the context on any API call.
Tokens from service accounts did not contain any clientID and therefore
never did that check.
But due to a change in https://github.com/zitadel/zitadel/pull/8580,
were the service user id was set as client_id in the OIDC session to fix
the introspection response
(https://github.com/zitadel/zitadel/issues/8590).

# How the Problems Are Solved

- Check if the project and origin were retrieved and only then check the
origins

# Additional Changes

None.

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/8676
- relates to https://github.com/zitadel/zitadel/pull/8580 (released on
2.62.0)
- relates to https://github.com/zitadel/zitadel/issues/8590
2024-10-01 16:38:28 +02:00
14e2aba1bc feat: Add Twilio Verification Service (#8678)
# Which Problems Are Solved
Twilio supports a robust, multi-channel verification service that
notably supports multi-region SMS sender numbers required for our use
case. Currently, Zitadel does much of the work of the Twilio Verify (eg.
localization, code generation, messaging) but doesn't support the pool
of sender numbers that Twilio Verify does.

# How the Problems Are Solved
To support this API, we need to be able to store the Twilio Service ID
and send that in a verification request where appropriate: phone number
verification and SMS 2FA code paths.

This PR does the following: 
- Adds the ability to use Twilio Verify of standard messaging through
Twilio
- Adds support for international numbers and more reliable verification
messages sent from multiple numbers
- Adds a new Twilio configuration option to support Twilio Verify in the
admin console
- Sends verification SMS messages through Twilio Verify
- Implements Twilio Verification Checks for codes generated through the
same

# Additional Changes

# Additional Context
- base was implemented by @zhirschtritt in
https://github.com/zitadel/zitadel/pull/8268 ❤️
- closes https://github.com/zitadel/zitadel/issues/8581

---------

Co-authored-by: Zachary Hirschtritt <[email protected]>
Co-authored-by: Joey Biscoglia <[email protected]>
2024-09-26 09:14:33 +02:00
Livio SpringandGitHub 5b40af79f0 fix: correctly check user state (#8631)
# Which Problems Are Solved

ZITADEL's user account deactivation mechanism did not work correctly
with service accounts. Deactivated service accounts retained the ability
to request tokens, which could lead to unauthorized access to
applications and resources.

# How the Problems Are Solved

Additionally to checking the user state on the session API and login UI,
the state is checked on all oidc session methods resulting in a new
token or when returning the user information (userinfo, introspection,
id_token / access_token and saml attributes)
2024-09-17 13:21:49 +00:00
Livio SpringandGitHub ca1914e235 fix: user grants deactivation (#8634)
# Which Problems Are Solved

ZITADEL's user grants deactivation mechanism did not work correctly.
Deactivated user grants were still provided in token, which could lead
to unauthorized access to applications and resources.
Additionally, the management and auth API always returned the state as
active or did not provide any information about the state.

# How the Problems Are Solved

- Correctly check the user grant state on active for tokens and user
information (userinfo, introspection, saml attributes)
- Map state in API and display in Console
2024-09-17 12:18:29 +00:00
Livio SpringandGitHub d01bd1c51a fix: correctly check app state on authentication (#8630)
# Which Problems Are Solved

In Zitadel, even after an organization is deactivated, associated
projects, respectively their applications remain active. Users across
other organizations can still log in and access through these
applications, leading to unauthorized access.
Additionally, if a project was deactivated access to applications was
also still possible.

# How the Problems Are Solved

- Correctly check the status of the organization and related project. 
(Corresponding functions have been renamed to `Active...`)
2024-09-17 11:34:14 +00:00
Livio SpringandGitHub 8831b33013 fix: prevent password init on email verification for external users (#8602)
# Which Problems Are Solved

If a user is created through the login UI with an IdP and an unverified
email, the user is prompted with the email verification and to set a
password.
This was unintentionally changed with #8291

# How the Problems Are Solved

- Check if the user has any IdP linked when computing the
EmailVerification step.

# Additional Changes

None

# Additional Context

- reported in a support request
2024-09-13 15:51:06 +00:00
Livio SpringandGitHub 14383cfe8c fix: race condition in system jwt signature check (#8618) 2024-09-13 17:18:37 +02:00
Livio SpringandGitHub a07b2f4677 feat: invite user link (#8578)
# Which Problems Are Solved

As an administrator I want to be able to invite users to my application
with the API V2, some user data I will already prefil, the user should
add the authentication method themself (password, passkey, sso).

# How the Problems Are Solved

- A user can now be created with a email explicitly set to false.
- If a user has no verified email and no authentication method, an
`InviteCode` can be created through the User V2 API.
  - the code can be returned or sent through email
- additionally `URLTemplate` and an `ApplicatioName` can provided for
the email
- The code can be resent and verified through the User V2 API
- The V1 login allows users to verify and resend the code and set a
password (analog user initialization)
- The message text for the user invitation can be customized

# Additional Changes

- `verifyUserPasskeyCode` directly uses `crypto.VerifyCode` (instead of
`verifyEncryptedCode`)
- `verifyEncryptedCode` is removed (unnecessarily queried for the code
generator)

# Additional Context

- closes #8310
- TODO: login V2 will have to implement invite flow:
https://github.com/zitadel/typescript/issues/166
2024-09-11 10:53:55 +00:00
Livio SpringandGitHub 650c21f18a fix: check if pw login allowed (#8584)
# Which Problems Are Solved

When checking for the next step for the login UI and a user did not yet
have an IdP linked, they would always be presented the password check
screen, even if the local authentication was disabled.

# How the Problems Are Solved

- Correctly check the login policy for the `Allow Username Password`
option
- In case the user has no IdP linked yet, fallback to the organizations
configuration (and redirect if possible)
- the user can be auto-linked based on the username / email after
successfully authenticating at the IdP

# Additional Changes

None

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/5106
- closes https://github.com/zitadel/zitadel/issues/7502
2024-09-10 12:55:32 +02:00