# Which Problems Are Solved
Zitadel did not provide easy correlation between errors, logs, traces
and metrics. The configuration for those instrumentations was also not
consistent, with some supporting different exporters then others.
Implementation and parsing of config was also spaghettified over
multiple packages, with awkward parsing and inconsistent naming of
options.
# How the Problems Are Solved
All telemetry is now merged under the name "instrumentation". Why?
1. We thought it was a good idea in the past to call the milestone
exporter `Telemtry` in the runtime config. Calling this `TelemetryV2`
looks weird.
2. Not everything is a meter and not everything is sent (tele...).
3. It's also
[defined](https://opentelemetry.io/docs/concepts/instrumentation/) as
such by the OTEL documentation.
## New features
- Adds structured, context based logging with trace-ID awareness
- Static log fields are added to the context, such as service and
request path
- Static log fields are injected in each logline emitted by the
application
- Structured logs can also be send to an otel exporter
- Structured logs can be printed to StdErr in text and JSON format
- Error sinks make sure every error is logged at the correct level:
- Warnings for client side errors (HTTP 400 range, Invalid request etc)
- Error for server side errors (Internal server errors)
- Metrics can now also be send to a OTEL collector. (previously they
could only be scraped from `/debug/metrics` with prometheus)
## Exporters
This change adds all the exporters supported by OTEL upstream and some
google specific exporters for our cloud deployment.
- StdOut / StdErr: all instrumentations
- OTEL gRPC / HTTP: all instrumentations
- Google: all instrumentations except logging
- Prometheus (pull-based): only metrics
The exception is profiling, which only supports the google exporting due
to lack of support by OTEL upstream.
## Configuration and structure
- All instrumentation is moved into the new `backend/v3/instrumentation`
package. It reuses configuration types, so both code and runtime
configuration are easier to understand.
- The `internal/telemetry` packages are removed.
- Instrumentation is started with a single function and a proper
shutdown function is now provided.
- Legacy configuration is still parsed from the runtime config, as long
as the new configuration is disabled. This allows backporting this
feature to v4 without breaking existing configurations.
# Additional Changes
- Devcontainer: set `$PATH` variable so installed go binaries can be run
individually, without NX.
- NX: install GCI tool to fix imports
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/8408
- Closes https://github.com/zitadel/zitadel/issues/6664
- Backport to v4
<!--
Please inform yourself about the contribution guidelines on submitting a
PR here:
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#submit-a-pull-request-pr.
Take note of how PR/commit titles should be written and replace the
template texts in the sections below. Don't remove any of the sections.
It is important that the commit history clearly shows what is changed
and why.
Important: By submitting a contribution you agree to the terms from our
Licensing Policy as described here:
https://github.com/zitadel/zitadel/blob/main/LICENSING.md#community-contributions.
-->
One-time recovery codes are a common multi-factor authentication (MFA)
backup method, letting users access their account if they lose other MFA
devices. Support for recovery codes can also reduce support burden for
users locked out of their accounts and provide a more secure and
reliable form of verification than security questions.
# Which Problems Are Solved
Zitadel currently lacks support for recovery codes.
# How the Problems Are Solved
This PR partially addresses recovery code support in Zitadel.
Importantly, it adds recovery codes as a new 2FA `Factor` and an
additional `Check` type for the Session API.
```
Example recovery code flow:
1. User generates N new recovery codes using `POST /v2/users/{user_id}/recovery_codes`
2. Zitadel hashes and stores these codes and returns the un-hashed codes in the response
3. User creates new session with an additional check: `recoveryCode`
4. Code is checked against hash and, if valid, cannot be used again
5. User attempts to adds N more codes using the same endpoint
6. If `remaining_codes + N <= RecoveryCodes->MaxCount` config value, then recovery codes are added in addition to original codes
7. User can remove all recovery codes using `DEL /v2/users/:userId/recovery_codes`
```
This PR adds:
- [x] Session recovery_code check support on `POST+PATCH /v2/sessions`
endpoints
- [x] Adds `mfa_recovery_code_checked_at` column (default null) to
`projections.sessions8` table
- [x] Support for `SECOND_FACTOR_TYPE_RECOVERY_CODES` as available 2FA
method on login policy
- [x] Support for importing recovery codes in /import code
Missing, will _not_ implement in this PR:
- [ ] Admin console support for displaying Recovery Code settings for
user(s)
- [ ] Zitadel Typescript login support for recovery codes
TODO:
- [x] Additional unit and integration tests
- [x] Error translations
# Additional Changes
None
# Additional Context
- Closes#6898
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Some configuration can not be set directly through environment
variables.
# How the Problems Are Solved
Comment in the attributes, so that Zitadel uses the environment
variables.
# Additional Changes
None
# Additional Context
Closes#10913
# Which Problems Are Solved
1. Adding users to user groups and removing users from user groups.
2. Searching for users in user groups by group IDs or user IDs
# How the Problems Are Solved
By adding:
1. The API definitions to manage users in users groups
3. The command-layer implementation of adding users/removing users
to/from user groups.
4. The projection table group_users1
5. Query-side implementation to search for users in user groups
# Additional Changes
1. Remove debug statements from unit tests.
2. Fix removal of groups when orgs are removed
3. Add unit tests for groups projection
# Additional Context
* Related to #9702
* Follow-up for PRs
* https://github.com/zitadel/zitadel/pull/10455
* https://github.com/zitadel/zitadel/pull/10758
* https://github.com/zitadel/zitadel/pull/10853
# Which Problems Are Solved
Ensuring the user group resource is managed with appropriate
permissions.
# How the Problems Are Solved
By configuring and checking for the relevant permissions needed to
create, read, update, and delete the user groups resource.
# Additional Changes
N/A
# Additional Context
- Related to #9702
- Follow-up for PRs #10455, #10758
# Which Problems Are Solved
`ZITADEL_DEFAULTINSTANCE_FEATURES_LOGINV2_BASEURI` env variable had no
impact on configuration.
# How the Problems Are Solved
Attribute in the configuration file has to be set as empty so that you
can change it through an env variable.
# Additional Changes
None
# Additional Context
Closes#10405
# Which Problems Are Solved
When Postgres was not ready when the API was started, the API failed
immediately.
This made task orchestration hard, especially in a platform agnostic
way:
- The current health check in the Nx target `@zitadel/api:prod` uses the
timeout command, which is not installed on all platforms and behaves
unpredictably
- The current health check in the Nx target `@zitadel/api:prod` requires
the DB to have been started using `@zitadel/zitadel:db`
# How the Problems Are Solved
- Additional configuration option `Database.Postgres.AwaitInitialConn`
is added and defaults to *0m* for backwards compatibility.
- If a duration is configured, the API retries to ping the database
until it succeeds
- The API sleeps for a second between each ping.
- It emits an info-level log with the error on each try.
- When the configured duration times out before the ping is successful,
the error is returned and the command exits with a failure code.
- When the ping succeeds within the configured duration, the API goes on
with the init, setup or start phase.
# Additional Context
- Relates to internally reported problems with the current DB health
check command
[here](https://zitadel.slack.com/archives/C07EUL5H83A/p1759915009839269?thread_ts=1759912259.410789&cid=C07EUL5H83A)
and
[here](https://zitadel.slack.com/archives/C07EUL5H83A/p1759918324246249?thread_ts=1759912259.410789&cid=C07EUL5H83A).
# Which Problems Are Solved
The current service ping reports can run into body size limit errors and
there's no way of knowing how big the current size is.
# How the Problems Are Solved
Log the current size to have at least some insights and possibly change
bulk size.
# Additional Changes
None
# Additional Context
- noticed internally
- backport to v4.x
# Which Problems Are Solved
Starting with Zitadel v4, the new login UI is enabled by default (for
new instances) through the corresponding feature flag.
There's an additional flag to use the V2 API in console, which is mostly
required to use the login V2 without problems, but was not yet activated
by default (for new instances).
# How the Problems Are Solved
- Enabled the `ConsoleUseV2UserApi` feature flag on the
`defaultInstance`
# Additional Changes
- Cleaned up removed flags on the `defaultInstance`
# Additional Context
- noticed internally
- backport to v4.x
This PR overhauls our event projection system to make it more robust and
prevent skipped events under high load. The core change replaces our
custom, transaction-based locking with standard PostgreSQL advisory
locks. We also introduce a worker pool to manage concurrency and prevent
database connection exhaustion.
### Key Changes
* **Advisory Locks for Projections:** Replaces exclusive row locks and
inspection of `pg_stat_activity` with PostgreSQL advisory locks for
managing projection state. This is a more reliable and standard approach
to distributed locking.
* **Simplified Await Logic:** Removes the complex logic for awaiting
open transactions, simplifying it to a more straightforward time-based
filtering of events.
* **Projection Worker Pool:** Implements a worker pool to limit
concurrent projection triggers, preventing connection exhaustion and
improving stability under load. A new `MaxParallelTriggers`
configuration option is introduced.
### Problem Solved
Under high throughput, a race condition could cause projections to miss
events from the eventstore. This led to inconsistent data in projection
tables (e.g., a user grant might be missing). This PR fixes the
underlying locking and concurrency issues to ensure all events are
processed reliably.
### How it Works
1. **Event Writing:** When writing events, a *shared* advisory lock is
taken. This signals that a write is in progress.
2. **Event Handling (Projections):**
* A projection worker attempts to acquire an *exclusive* advisory lock
for that specific projection. If the lock is already held, it means
another worker is on the job, so the current one backs off.
* Once the lock is acquired, the worker briefly acquires and releases
the same *shared* lock used by event writers. This acts as a barrier,
ensuring it waits for any in-flight writes to complete.
* Finally, it processes all events that occurred before its transaction
began.
### Additional Information
* ZITADEL no longer modifies the `application_name` PostgreSQL variable
during event writes.
* The lock on the `current_states` table is now `FOR NO KEY UPDATE`.
* Fixes https://github.com/zitadel/zitadel/issues/8509
---------
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Tim Möhlmann <[email protected]>
# Which Problems Are Solved
This PR introduces a new feature flag `EnableRelationalTables` that will
be used in following implementations to decide whether Zitadel should
use the relational model or the event sourcing one.
# TODO
- [x] Implement flag at system level
- [x] Display the flag on console:
https://github.com/zitadel/zitadel/pull/10615
# How the Problems Are Solved
- Implement loading the flag from config
- Add persistence of the flag through gRPC endpoint
(SetInstanceFeatures)
- Implement reading of the flag through gRPC endpoint
(GetInstanceFeatures)
# Additional Changes
Some minor refactoring to remove un-needed generics annotations
# Additional Context
- Closes#10574
---------
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
Emails are still send only with URLs to login v1.
# How the Problems Are Solved
Add configuration for URLs as URL templates, so that links can point at
Login v2.
# Additional Changes
None
# Additional Context
Closes#10236
---------
Co-authored-by: Marco A. <[email protected]>
# Which Problems Are Solved
Some events that are now unused are clogging the event queue from time
to time.
# How the Problems Are Solved
Remove the events described in #10458
# Additional Changes
- Updated `stringer` and `enumer` in Makefile target `core_generate_all`
to resolve generated files compilation issues
# Notes
It looks like there are a lot of changes, but most of it is fixing
translation files. I suggest doing a review per-commit
# Additional Context
- Closes#10458
- Depends on https://github.com/zitadel/zitadel/pull/10513
# Which Problems Are Solved
Typo in environment variable reference for
OIDC:DeviceAuth:UserCode:CharAmount config
`ZITADEL_OIDC_DEVICEAUTH_USERCODE_CHARARMOUNT` - _CHARA_**~~R~~**_MOUNT_
# How the Problems Are Solved
Fixed the typo `ZITADEL_OIDC_DEVICEAUTH_USERCODE_CHARAMOUNT`
Co-authored-by: Stefan Benz <[email protected]>
# Which Problems Are Solved
The production endpoint of the service ping was wrong.
Additionally we discussed in the sprint review, that we could randomize
the default interval to prevent all systems to report data at the very
same time and also require a minimal interval.
# How the Problems Are Solved
- fixed the endpoint
- If the interval is set to @daily (default), we generate a random time
(minute, hour) as a cron format.
- Check if the interval is more than 30min and return an error if not.
- Fixed yaml indent on `ResourceCount`
# Additional Changes
None
# Additional Context
as discussed internally
This PR is still WIP and needs changes to at least the tests.
# Which Problems Are Solved
To be able to report analytical / telemetry data from deployed Zitadel
systems back to a central endpoint, we designed a "service ping"
functionality. See also https://github.com/zitadel/zitadel/issues/9706.
This PR adds the first implementation to allow collection base data as
well as report amount of resources such as organizations, users per
organization and more.
# How the Problems Are Solved
- Added a worker to handle the different `ReportType` variations.
- Schedule a periodic job to start a `ServicePingReport`
- Configuration added to allow customization of what data will be
reported
- Setup step to generate and store a `systemID`
# Additional Changes
None
# Additional Context
relates to #9869
# Which Problems Are Solved
We provide a seamless way to initialize Zitadel and the login together.
# How the Problems Are Solved
Additionally to the `IAM_OWNER` role, a set up admin user also gets the
`IAM_LOGIN_CLIENT` role if it is a machine user with a PAT.
# Additional Changes
- Simplifies the load balancing example, as the intermediate
configuration step is not needed anymore.
# Additional Context
- Depends on #10116
- Contributes to https://github.com/zitadel/zitadel-charts/issues/332
- Contributes to https://github.com/zitadel/zitadel/issues/10016
---------
Co-authored-by: Stefan Benz <[email protected]>
# Which Problems Are Solved
Currently if a user signs in using an IdP, once they sign out of
Zitadel, the corresponding IdP session is not terminated. This can be
the desired behavior. In some cases, e.g. when using a shared computer
it results in a potential security risk, since a follower user might be
able to sign in as the previous using the still open IdP session.
# How the Problems Are Solved
- Admins can enabled a federated logout option on SAML IdPs through the
Admin and Management APIs.
- During the termination of a login V1 session using OIDC end_session
endpoint, Zitadel will check if an IdP was used to authenticate that
session.
- In case there was a SAML IdP used with Federated Logout enabled, it
will intercept the logout process, store the information into the shared
cache and redirect to the federated logout endpoint in the V1 login.
- The V1 login federated logout endpoint checks every request on an
existing cache entry. On success it will create a SAML logout request
for the used IdP and either redirect or POST to the configured SLO
endpoint. The cache entry is updated with a `redirected` state.
- A SLO endpoint is added to the `/idp` handlers, which will handle the
SAML logout responses. At the moment it will check again for an existing
federated logout entry (with state `redirected`) in the cache. On
success, the user is redirected to the initially provided
`post_logout_redirect_uri` from the end_session request.
# Additional Changes
None
# Additional Context
- This PR merges the https://github.com/zitadel/zitadel/pull/9841 and
https://github.com/zitadel/zitadel/pull/9854 to main, additionally
updating the docs on Entra ID SAML.
- closes#9228
- backport to 3.x
---------
Co-authored-by: Silvan <[email protected]>
Co-authored-by: Zach Hirschtritt <[email protected]>
# Which Problems Are Solved
- Allow users to use SHA-256 and SHA-512 hashing algorithms. These
algorithms are used by Linux's crypt(3) function.
- Allow users to import passwords using the PHPass algorithm. This
algorithm is used by older PHP systems, WordPress in particular.
# How the Problems Are Solved
- Upgrade passwap to
[v0.9.0](https://github.com/zitadel/passwap/releases/tag/v0.9.0)
- Add sha2 and phpass as a new verifier option in defaults.yaml
# Additional Changes
- Updated docs to explain the two algorithms
# Additional Context
Implements the changes in the passwap library from
https://github.com/zitadel/passwap/pull/59 and
https://github.com/zitadel/passwap/pull/60
# Which Problems Are Solved
The execution handler projection handles all events to check if an
execution has to be provided to the worker to execute.
In this logic all events would be processed from the beginning which is
not necessary.
# How the Problems Are Solved
Add the current state to the execution handler projection, to avoid
processing all existing events.
# Additional Changes
Add custom configuration to the default, so that the transactions are
limited to some events.
# Additional Context
None
# Which Problems Are Solved
Instance that had improved performance flags set, got event errors when
getting instance features. This is because the improved performance
flags were marshalled using the enumerated integers, but now needed to
be unmashalled using the added UnmarshallText method.
# How the Problems Are Solved
- Remove emnumer generation
# Additional Changes
- none
# Additional Context
- reported on QA
- Backport to next-rc / v3
# Which Problems Are Solved
Webkeys were not generated with new instances when the webkey feature
flag was enabled for instance defaults. This would cause a redirect loop
with console for new instances on QA / coud.
# How the Problems Are Solved
- uncomment the webkeys section on defaults.yaml
- Fix field naming of webkey config
# Additional Changes
- Add all available features as comments.
- Make the improved performance type enum parsable from the config,
untill now they were just ints.
- Running of the enumer command created missing enum entries for feature
keys.
# Additional Context
- Needs to be back-ported to v3 / next-rc
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
If I start a fresh instance and do not overwrite `SystemAPIUsers` I get
an error during startup `error="decoding failed due to the following
error(s):\n\n'SystemAPIUsers[0][path]' expected a map, got
'string'\n'SystemAPIUsers[0][memberships]' expected a map, got 'slice'"`
# How the Problems Are Solved
the configuration is commented so that the example is still there
# Additional Changes
-
# Additional Context
was added in https://github.com/zitadel/zitadel/pull/9757
# Which Problems Are Solved
Add the possibility to filter project resources based on project member
roles.
# How the Problems Are Solved
Extend and refactor existing Pl/PgSQL functions to implement the
following:
- Solve O(n) complexity in returned resources IDs by returning a boolean
filter for instance level permissions.
- Individually permitted orgs are returned only if there was no instance
permission
- Individually permitted projects are returned only if there was no
instance permission
- Because of the multiple filter terms, use `INNER JOIN`s instead of
`WHERE` clauses.
# Additional Changes
- system permission function no longer query the organization view and
therefore can be `immutable`, giving big performance benefits for
frequently reused system users. (like our hosted login in Zitadel cloud)
- The permitted org and project functions are now defined as `stable`
because the don't modify on-disk data. This might give a small
performance gain
- The Pl/PgSQL functions are now tested using Go unit tests.
# Additional Context
- Depends on https://github.com/zitadel/zitadel/pull/9677
- Part of https://github.com/zitadel/zitadel/issues/9188
- Closes https://github.com/zitadel/zitadel/issues/9190
# Which Problems Are Solved
Allow verification of imported salted passwords hashed with plain md5.
# How the Problems Are Solved
- Upgrade passwap to
[v0.7.0](https://github.com/zitadel/passwap/releases/tag/v0.7.0)
- Add md5salted as a new verifier option in `defaults.yaml`
# Additional Changes
- go version and libraries updated (required by passkey v0.7.0)
- secrets.md verifiers updated
- configuration verifiers updated
- added MD5salted and missing MD5Plain to test cases
# Which Problems Are Solved
The service name is hardcoded in the metrics code. Making the service
name to be configurable helps when running multiple instances of
Zitadel.
The defaults remain unchanged, the service name will be defaulted to
ZITADEL.
# How the Problems Are Solved
Add a config option to override the name in defaults.yaml and pass it
down to the corresponding metrics or tracing module (google or otel)
# Additional Changes
NA
# Additional Context
NA
# Which Problems Are Solved
Currently I am not able to run the new login with a service account with
an IAM_OWNER role.
As the role is missing some permissions which the LOGIN_CLIENT role does
have
# How the Problems Are Solved
Added session permissions to the IAM_OWNER
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
The recently introduced notification queue have potential race conditions.
# How the Problems Are Solved
Current code is refactored to use the queue package, which is safe in
regards of concurrency.
# Additional Changes
- the queue is included in startup
- improved code quality of queue
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/9278
# Which Problems Are Solved
* Adds support for the service provider configuration SCIM v2 endpoints
# How the Problems Are Solved
* Adds support for the service provider configuration SCIM v2 endpoints
* `GET /scim/v2/{orgId}/ServiceProviderConfig`
* `GET /scim/v2/{orgId}/ResourceTypes`
* `GET /scim/v2/{orgId}/ResourceTypes/{name}`
* `GET /scim/v2/{orgId}/Schemas`
* `GET /scim/v2/{orgId}/Schemas/{id}`
# Additional Context
Part of #8140
Co-authored-by: Stefan Benz <[email protected]>
# Which Problems Are Solved
* Adds support for the bulk SCIM v2 endpoint
# How the Problems Are Solved
* Adds support for the bulk SCIM v2 endpoint under `POST
/scim/v2/{orgID}/Bulk`
# Additional Context
Part of #8140
Co-authored-by: Stefan Benz <[email protected]>
# Which Problems Are Solved
Zitadel currently uses 3 database pool, 1 for queries, 1 for pushing
events and 1 for scheduled projection updates. This defeats the purpose
of a connection pool which already handles multiple connections.
During load tests we found that the current structure of connection
pools consumes a lot of database resources. The resource usage dropped
after we reduced the amount of database pools to 1 because existing
connections can be used more efficiently.
# How the Problems Are Solved
Removed logic to handle multiple connection pools and use a single one.
# Additional Changes
none
# Additional Context
part of https://github.com/zitadel/zitadel/issues/8352
# Which Problems Are Solved
Currently ZITADEL defines organization and instance member roles and
permissions in defaults.yaml. The permission check is done on API call
level. For example: "is this user allowed to make this call on this
org". This makes sense on the V1 API where the API is permission-level
shaped. For example, a search for users always happens in the context of
the organization. (Either the organization the calling user belongs to,
or through member ship and the x-zitadel-orgid header.
However, for resource based APIs we must be able to resolve permissions
by object. For example, an IAM_OWNER listing users should be able to get
all users in an instance based on the query filters. Alternatively a
user may have user.read permissions on one or more orgs. They should be
able to read just those users.
# How the Problems Are Solved
## Role permission mapping
The role permission mappings defined from `defaults.yaml` or local
config override are synchronized to the database on every run of
`zitadel setup`:
- A single query per **aggregate** builds a list of `add` and `remove`
actions needed to reach the desired state or role permission mappings
from the config.
- The required events based on the actions are pushed to the event
store.
- Events define search fields so that permission checking can use the
indices and is strongly consistent for both query and command sides.
The migration is split in the following aggregates:
- System aggregate for for roles prefixed with `SYSTEM`
- Each instance for roles not prefixed with `SYSTEM`. This is in
anticipation of instance level management over the API.
## Membership
Current instance / org / project membership events now have field table
definitions. Like the role permissions this ensures strong consistency
while still being able to use the indices of the fields table. A
migration is provided to fill the membership fields.
## Permission check
I aimed keeping the mental overhead to the developer to a minimal. The
provided implementation only provides a permission check for list
queries for org level resources, for example users. In the `query`
package there is a simple helper function `wherePermittedOrgs` which
makes sure the underlying database function is called as part of the
`SELECT` query and the permitted organizations are part of the `WHERE`
clause. This makes sure results from non-permitted organizations are
omitted. Under the hood:
- A Pg/PlSQL function searches for a list of organization IDs the passed
user has the passed permission.
- When the user has the permission on instance level, it returns early
with all organizations.
- The functions uses a number of views. The views help mapping the
fields entries into relational data and simplify the code use for the
function. The views provide some pre-filters which allow proper index
usage once the final `WHERE` clauses are set by the function.
# Additional Changes
# Additional Context
Closes#9032
Closes https://github.com/zitadel/zitadel/issues/9014https://github.com/zitadel/zitadel/issues/9188 defines follow-ups for
the new permission framework based on this concept.
# Which Problems Are Solved
- Adds infrastructure code (basic implementation, error handling,
middlewares, ...) to implement the SCIM v2 interface
- Adds support for the user create SCIM v2 endpoint
# How the Problems Are Solved
- Adds support for the user create SCIM v2 endpoint under `POST
/scim/v2/{orgID}/Users`
# Additional Context
Part of #8140
# Which Problems Are Solved
It was possible to set a diffent algorithm for the legacy signer. This
is not supported howerver and breaks the token endpoint.
# How the Problems Are Solved
Remove the OIDC.SigningKeyAlgorithm config option and hard-code RS256
for the legacy signer.
# Additional Changes
- none
# Additional Context
Only RS256 is supported by the legacy signer. It was mentioned in the
comment of the config not to use it and use the webkeys resource
instead.
- closes#9121
# Which Problems Are Solved
The console has no information about where and how to send PostHog
events.
# How the Problems Are Solved
A PostHog API URL and token are passed through as plain text from the
Zitadel runtime config to the environment.json. By default, no values
are configured and the keys in the environment.json are omitted.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9070
- Complements https://github.com/zitadel/zitadel/pull/9077
# Which Problems Are Solved
Some IdP callbacks use HTTP form POST to return their data on callbacks.
For handling CSRF in the login after such calls, a 302 Found to the
corresponding non form callback (in ZITADEL) is sent. Depending on the
size of the initial form body, this could lead to ZITADEL terminating
the connection, resulting in the user not getting a response or an
intermediate proxy to return them an HTTP 502.
# How the Problems Are Solved
- the form body is parsed and stored into the ZITADEL cache (using the
configured database by default)
- the redirect (302 Found) is performed with the request id
- the callback retrieves the data from the cache instead of the query
parameters (will fallback to latter to handle open uncached requests)
# Additional Changes
- fixed a typo in the default (cache) configuration: `LastUsage` ->
`LastUseAge`
# Additional Context
- reported by a customer
- needs to be backported to current cloud version (2.66.x)
---------
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
It is currently not possible to use SAML with the Session API.
# How the Problems Are Solved
Add SAML service, to get and resolve SAML requests.
Add SAML session and SAML request aggregate, which can be linked to the
Session to get back a SAMLResponse from the API directly.
# Additional Changes
Update of dependency zitadel/saml to provide all functionality for
handling of SAML requests and responses.
# Additional Context
Closes#6053
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
To be able to migrate or test the new login UI, admins might want to
(temporarily) switch individual apps.
At a later point admin might want to make sure all applications use the
new login UI.
# How the Problems Are Solved
- Added a feature flag `` on instance level to require all apps to use
the new login and provide an optional base url.
- if the flag is enabled, all (OIDC) applications will automatically use
the v2 login.
- if disabled, applications can decide based on their configuration
- Added an option on OIDC apps to use the new login UI and an optional
base url.
- Removed the requirement to use `x-zitadel-login-client` to be
redirected to the login V2 and retrieve created authrequest and link
them to SSO sessions.
- Added a new "IAM_LOGIN_CLIENT" role to allow management of users,
sessions, grants and more without `x-zitadel-login-client`.
# Additional Changes
None
# Additional Context
closes https://github.com/zitadel/zitadel/issues/8702
# Which Problems Are Solved
Scheduled handlers use `eventstore.InstanceIDs` to get the all active
instances within a given timeframe. This function scrapes through all
events written within that time frame which can cause heavy load on the
database.
# How the Problems Are Solved
A new query cache `activeInstances` is introduced which caches the ids
of all instances queried by id or host within the configured timeframe.
# Additional Changes
- Changed `default.yaml`
- Removed `HandleActiveInstances` from custom handler configs
- Added `MaxActiveInstances` to define the maximal amount of cached
instance ids
- fixed start-from-init and start-from-setup to start auth and admin
projections twice
- fixed org cache invalidation to use correct index
# Additional Context
- part of #8999
# Which Problems Are Solved
There are some problems related to the use of CockroachDB with the new
notification handling (#8931).
See #9002 for details.
# How the Problems Are Solved
- Brought back the previous notification handler as legacy mode.
- Added a configuration to choose between legacy mode and new parallel
workers.
- Enabled legacy mode by default to prevent issues.
# Additional Changes
None
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/9002
- relates to #8931
# Which Problems Are Solved
While running the latest RC / main, we noticed some errors including
context timeouts and rollback issues.
# How the Problems Are Solved
- The transaction context is passed and used for any event being written
and for handling savepoints to be able to handle context timeouts.
- The user projection is not triggered anymore. This will reduce
unnecessary load and potential timeouts if lot of workers are running.
In case a user would not be projected yet, the request event will log an
error and then be skipped / retried on the next run.
- Additionally, the context is checked if being closed after each event
process.
- `latestRetries` now correctly only returns the latest retry events to
be processed
- Default values for notifications have been changed to run workers less
often, more retry delay, but less transaction duration.
# Additional Changes
None
# Additional Context
relates to #8931
---------
Co-authored-by: Tim Möhlmann <[email protected]>
# Which Problems Are Solved
The action v2 messages were didn't contain anything providing security
for the sent content.
# How the Problems Are Solved
Each Target now has a SigningKey, which can also be newly generated
through the API and returned at creation and through the Get-Endpoints.
There is now a HTTP header "Zitadel-Signature", which is generated with
the SigningKey and Payload, and also contains a timestamp to check with
a tolerance if the message took to long to sent.
# Additional Changes
The functionality to create and check the signature is provided in the
pkg/actions package, and can be reused in the SDK.
# Additional Context
Closes#7924
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
The current handling of notification follows the same pattern as all
other projections:
Created events are handled sequentially (based on "position") by a
handler. During the process, a lot of information is aggregated (user,
texts, templates, ...).
This leads to back pressure on the projection since the handling of
events might take longer than the time before a new event (to be
handled) is created.
# How the Problems Are Solved
- The current user notification handler creates separate notification
events based on the user / session events.
- These events contain all the present and required information
including the userID.
- These notification events get processed by notification workers, which
gather the necessary information (recipient address, texts, templates)
to send out these notifications.
- If a notification fails, a retry event is created based on the current
notification request including the current state of the user (this
prevents race conditions, where a user is changed in the meantime and
the notification already gets the new state).
- The retry event will be handled after a backoff delay. This delay
increases with every attempt.
- If the configured amount of attempts is reached or the message expired
(based on config), a cancel event is created, letting the workers know,
the notification must no longer be handled.
- In case of successful send, a sent event is created for the
notification aggregate and the existing "sent" events for the user /
session object is stored.
- The following is added to the defaults.yaml to allow configuration of
the notification workers:
```yaml
Notifications:
# The amount of workers processing the notification request events.
# If set to 0, no notification request events will be handled. This can be useful when running in
# multi binary / pod setup and allowing only certain executables to process the events.
Workers: 1 # ZITADEL_NOTIFIACATIONS_WORKERS
# The amount of events a single worker will process in a run.
BulkLimit: 10 # ZITADEL_NOTIFIACATIONS_BULKLIMIT
# Time interval between scheduled notifications for request events
RequeueEvery: 2s # ZITADEL_NOTIFIACATIONS_REQUEUEEVERY
# The amount of workers processing the notification retry events.
# If set to 0, no notification retry events will be handled. This can be useful when running in
# multi binary / pod setup and allowing only certain executables to process the events.
RetryWorkers: 1 # ZITADEL_NOTIFIACATIONS_RETRYWORKERS
# Time interval between scheduled notifications for retry events
RetryRequeueEvery: 2s # ZITADEL_NOTIFIACATIONS_RETRYREQUEUEEVERY
# Only instances are projected, for which at least a projection-relevant event exists within the timeframe
# from HandleActiveInstances duration in the past until the projection's current time
# If set to 0 (default), every instance is always considered active
HandleActiveInstances: 0s # ZITADEL_NOTIFIACATIONS_HANDLEACTIVEINSTANCES
# The maximum duration a transaction remains open
# before it spots left folding additional events
# and updates the table.
TransactionDuration: 1m # ZITADEL_NOTIFIACATIONS_TRANSACTIONDURATION
# Automatically cancel the notification after the amount of failed attempts
MaxAttempts: 3 # ZITADEL_NOTIFIACATIONS_MAXATTEMPTS
# Automatically cancel the notification if it cannot be handled within a specific time
MaxTtl: 5m # ZITADEL_NOTIFIACATIONS_MAXTTL
# Failed attempts are retried after a confogired delay (with exponential backoff).
# Set a minimum and maximum delay and a factor for the backoff
MinRetryDelay: 1s # ZITADEL_NOTIFIACATIONS_MINRETRYDELAY
MaxRetryDelay: 20s # ZITADEL_NOTIFIACATIONS_MAXRETRYDELAY
# Any factor below 1 will be set to 1
RetryDelayFactor: 1.5 # ZITADEL_NOTIFIACATIONS_RETRYDELAYFACTOR
```
# Additional Changes
None
# Additional Context
- closes#8931
Tim MöhlmannandGitHub
Stefan BenzandGitHub
Gayathri VijayanandGitHub
Elio BischofandGitHub
Juriaan KennedyandGitHub
Miro TriscandGitHub
SilvanandGitHub
