# Which Problems Are Solved
Milestones used existing events from a number of aggregates. OIDC
session is one of them. We noticed in load-tests that the reduction of
the oidc_session.added event into the milestone projection is a costly
business with payload based conditionals. A milestone is reached once,
but even then we remain subscribed to the OIDC events. This requires the
projections.current_states to be updated continuously.
# How the Problems Are Solved
The milestone creation is refactored to use dedicated events instead.
The command side decides when a milestone is reached and creates the
reached event once for each milestone when required.
# Additional Changes
In order to prevent reached milestones being created twice, a migration
script is provided. When the old `projections.milestones` table exist,
the state is read from there and `v2` milestone aggregate events are
created, with the original reached and pushed dates.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/8800
# Which Problems Are Solved
Since the optiimzation of the token endpoint, we longer push the
`user.token.added` event. However, the milestone projection keeps
quering for it, including a payload query.
This incured a static waste of DB resources.
# How the Problems Are Solved
Remove the `user.token.added` event reducer from the milestone
projection
# Additional Changes
- none
# Additional Context
- Related to https://github.com/zitadel/zitadel/issues/8742. Other
changes ommitted so this PR can be backported to stable.
# Which Problems Are Solved
Optimize the query that checks for terminated sessions in the access
token verifier. The verifier is used in auth middleware, userinfo and
introspection.
# How the Problems Are Solved
The previous implementation built a query for certain events and then
appended a single `PositionAfter` clause. This caused the postgreSQL
planner to use indexes only for the instance ID, aggregate IDs,
aggregate types and event types. Followed by an expensive sequential
scan for the position. This resulting in internal over-fetching of rows
before the final filter was applied.
Furthermore, the query was searching for events which are not always
applicable. For example, there was always a session ID search and if
there was a user ID, we would also search for a browser fingerprint in
event payload (expensive). Even if those argument string would be empty.
This PR changes:
1. Nest the position query, so that a full `instance_id, aggregate_id,
aggregate_type, event_type, "position"` index can be matched.
2. Redefine the `es_wm` index to include the `position` column.
3. Only search for events for the IDs that actually have a value. Do not
search (noop) if none of session ID, user ID or fingerpint ID are set.
New query plan:
# Additional Changes
- cleanup how we load multi-statement migrations and make that a bit
more reusable.
# Additional Context
- Related to https://github.com/zitadel/zitadel/issues/7639
# Which Problems Are Solved
Cache implementation using a PGX connection pool.
# How the Problems Are Solved
Defines a new schema `cache` in the zitadel database.
A table for string keys and a table for objects is defined.
For postgreSQL, tables are unlogged and partitioned by cache name for
performance.
Cockroach does not have unlogged tables and partitioning is an
enterprise feature that uses alternative syntax combined with sharding.
Regular tables are used here.
# Additional Changes
- `postgres.Config` can return a pxg pool. See following discussion
# Additional Context
- Part of https://github.com/zitadel/zitadel/issues/8648
- Closes https://github.com/zitadel/zitadel/issues/8647
---------
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
When executing many concurrent authentication requests on a single
machine user, there were performance issues. As the same aggregate is
being searched and written to concurrently, we traced it down to a
locking issue on the used index.
We already optimized the token endpoint by creating a separate OIDC
aggregate.
At the time we decided to push a single event to the user aggregate, for
the user audit log. See [technical advisory
10010](https://zitadel.com/docs/support/advisory/a10010) for more
details.
However, a recent security fix introduced an additional search query on
the user aggregate, causing the locking issue we found.
# How the Problems Are Solved
Add a feature flag which disables pushing of the `user.token.v2.added`.
The event has no importance and was only added for informational
purposes on the user objects. The `oidc_session.access_token.added` is
the actual payload event and is pushed on the OIDC session aggregate and
can still be used for audit trail.
# Additional Changes
- Fix an event mapper type for
`SystemOIDCSingleV1SessionTerminationEventType`
# Additional Context
- Reported by support request
- https://github.com/zitadel/zitadel/pull/7822 changed the token
aggregate
- https://github.com/zitadel/zitadel/pull/8631 introduced user state
check
Load test trace graph with `user.token.v2.added` **enabled**. Query
times are steadily increasing:
Load test trace graph with `user.token.v2.added` **disabled**. Query
times constant:
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
We identified the need of caching.
Currently we have a number of places where we use different ways of
caching, like go maps or LRU.
We might also want shared chaches in the future, like Redis-based or in
special SQL tables.
# How the Problems Are Solved
Define a generic Cache interface which allows different implementations.
- A noop implementation is provided and enabled as.
- An implementation using go maps is provided
- disabled in defaults.yaml
- enabled in integration tests
- Authz middleware instance objects are cached using the interface.
# Additional Changes
- Enabled integration test command raceflag
- Fix a race condition in the limits integration test client
- Fix a number of flaky integration tests. (Because zitadel is super
fast now!) 🎸🚀
# Additional Context
Related to https://github.com/zitadel/zitadel/issues/8648
# Which Problems Are Solved
Reduce the chance for projection dead-locks. Increasing or disabling the
projection transaction duration solved dead-locks in all reported cases.
# How the Problems Are Solved
Increase the default transaction duration to 1 minute.
Due to the high value it is functionally similar to disabling,
however it still provides a safety net for transaction that do freeze,
perhaps due to connection issues with the database.
# Additional Changes
- Integration test uses default.
- Technical advisory
# Additional Context
- Related to https://github.com/zitadel/zitadel/issues/8517
---------
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
When the `openid` scope was not requested, as is possible in machine
authentication, we didn't set the `sub` (subject) claim to tokens and
possibly also userInfo and introspection.
This fix always sets the `sub` claim for all cases.
# How the Problems Are Solved
Set the `Subject` field to regardless of passed scopes.
# Additional Changes
- none
# Additional Context
According to standards:
- [RFC9068 - JSON Web Token (JWT) Profile for OAuth 2.0 Access
Tokens](https://datatracker.ietf.org/doc/html/rfc9068#name-data-structure)
this claim is **required**.
- [RFC7667 - OAuth 2.0 Token
Introspection](https://datatracker.ietf.org/doc/html/rfc7662#section-2.2)
the claim is optional, however there is no correlation to the `openid`
or OpenID Connect. Therefore it doesn't harm to always return this
claim.
- [OpenID connect, User Info
Response](https://openid.net/specs/openid-connect-core-1_0.html#UserInfoResponse):
"The sub (subject) Claim **MUST** always be returned in the UserInfo
Response."
Closes https://github.com/zitadel/zitadel/issues/8591
# Which Problems Are Solved
Upload the integration test server logs as artifacts, even if the tests
fail.
Before this change logs where printed through the Makefile.
However if a test would fail, the logs wouldn't get printed.
# How the Problems Are Solved
- Add an extra build step that pushes `tmp/zitadel.log` and
`tmp/race.log.$pid` to artificats storage.
- Logs are no longer printed in the `core_integration_reports` Makefile
recipe.
# Additional Changes
Do not remove coverage data when generating the coverage report in
`core_integration_reports`. This is to prevent future "File not found"
erros when running the command repeatedly.
# Additional Context
Reported as internal feedback
# Which Problems Are Solved
In actions/v3 there was no check for the feature flag on any of the:
- ListExecutionFunctions
- ListExecutionMethods
- ListExecutionServices
In the integration tests `ensureFeatureEnabled` relies on
`ListExecutionMethods` to return an error if the feature is not enabled.
This fix makes the test wait untill the feature is fully projected.
# How the Problems Are Solved
Add the feature check to all of the above methods.
# Additional Changes
- none
# Additional Context
Flaky introduced in https://github.com/zitadel/zitadel/pull/8407
# Which Problems Are Solved
Improve performance by removing a GetUserByID call. The call also
executed a Trigger on projections, which significantly impacted
concurrent requests.
# How the Problems Are Solved
Token creation needs information from the user, such as the resource
owner and access token type.
For client credentials this is solved in a single search. By getting the
user by username (`client_id`), the user details and secret were
obtained in a single query. After that verification and token creation
can proceed. For JWT profile it is a bit more complex. We didn't know
anything about the user until after JWT verification.
The verification did a query for the AuthN key and after that we did a
GetUserByID to get remaining details.
This change uses a joined query when the OIDC library calls the
`GetKeyByIDAndClientID` method on the token storage. The found user
details are set to the verifieer object and returned after verification
is completed.
It is safe because the `jwtProfileKeyStorage` is a single-use object as
a wrapper around `query.Queries`.
This way getting the public key and user details are obtained in a
single query.
# Additional Changes
- Correctly set the `client_id` field with machine's username.
# Additional Context
- Related to: https://github.com/zitadel/zitadel/issues/8352
# Which Problems Are Solved
Add a debug API which allows pushing a set of events to be reduced in a
dedicated projection.
The events can carry a sleep duration which simulates a slow query
during projection handling.
# How the Problems Are Solved
- `CreateDebugEvents` allows pushing multiple events which simulate the
lifecycle of a resource. Each event has a `projectionSleep` field, which
issues a `pg_sleep()` statement query in the projection handler :
- Add
- Change
- Remove
- `ListDebugEventsStates` list the current state of the projection,
optionally with a Trigger
- `GetDebugEventsStateByID` get the current state of the aggregate ID in
the projection, optionally with a Trigger
# Additional Changes
- none
# Additional Context
- Allows reproduction of https://github.com/zitadel/zitadel/issues/8517
# Which Problems Are Solved
defaults.yaml only specifies defaults for cockroach. Therefore, options
omitted for postgresql are actually set to `0`.
This means that the connections timeouts are set to `0` and connections
were not reused, resulting in a performance penalty while running the
integration tests.
# How the Problems Are Solved
Set MaxConnLifeTime and MaxConnIdleTime options in postgres
# Additional Changes
- none
# Additional Context
- none
Co-authored-by: Silvan <[email protected]>
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Return the user's project roles when the
`urn:zitadel:iam:org:projects:roles` scope is requested.
We alreayd returned it for access tokens, now also ID tokens.
# How the Problems Are Solved
Set `idTokenRoleAssertion` to `true` when calling
`accessTokenResponseFromSession` for service users. This parameter is
normally set to the client config. However, service user authentication
does not have a client.
# Additional Changes
- none
# Additional Context
- Introduced in https://github.com/zitadel/zitadel/pull/8046
- Closes https://github.com/zitadel/zitadel/issues/8107
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Deal with eventual consistency in the webkey integration tests.
# How the Problems Are Solved
Use an eventual with T for the list state check.
# Additional Changes
- none
# Additional Context
- none
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Use a single server instance for API integration tests. This optimizes
the time taken for the integration test pipeline,
because it allows running tests on multiple packages in parallel. Also,
it saves time by not start and stopping a zitadel server for every
package.
# How the Problems Are Solved
- Build a binary with `go build -race -cover ....`
- Integration tests only construct clients. The server remains running
in the background.
- The integration package and tested packages now fully utilize the API.
No more direct database access trough `query` and `command` packages.
- Use Makefile recipes to setup, start and stop the server in the
background.
- The binary has the race detector enabled
- Init and setup jobs are configured to halt immediately on race
condition
- Because the server runs in the background, races are only logged. When
the server is stopped and race logs exist, the Makefile recipe will
throw an error and print the logs.
- Makefile recipes include logic to print logs and convert coverage
reports after the server is stopped.
- Some tests need a downstream HTTP server to make requests, like quota
and milestones. A new `integration/sink` package creates an HTTP server
and uses websockets to forward HTTP request back to the test packages.
The package API uses Go channels for abstraction and easy usage.
# Additional Changes
- Integration test files already used the `//go:build integration`
directive. In order to properly split integration from unit tests,
integration test files need to be in a `integration_test` subdirectory
of their package.
- `UseIsolatedInstance` used to overwrite the `Tester.Client` for each
instance. Now a `Instance` object is returned with a gRPC client that is
connected to the isolated instance's hostname.
- The `Tester` type is now `Instance`. The object is created for the
first instance, used by default in any test. Isolated instances are also
`Instance` objects and therefore benefit from the same methods and
values. The first instance and any other us capable of creating an
isolated instance over the system API.
- All test packages run in an Isolated instance by calling
`NewInstance()`
- Individual tests that use an isolated instance use `t.Parallel()`
# Additional Context
- Closes#6684
- https://go.dev/doc/articles/race_detector
- https://go.dev/doc/build-cover
---------
Co-authored-by: Stefan Benz <[email protected]>
# Which Problems Are Solved
Do not push secret succeeded and failed events for API and OIDC clients
on the introspection endpoint.
On instances where introspection was fequently called, the pushed events
created issues on duplicate primary keys, due to collisions on the
`sequence` column in the eventstore. As the event pusher retries on this
collision and we pushed above mentioned events async, it would create a
backpressure of concurrent pushers and effectively cripple an instance.
We considered that pushing these events have little value with regards
to the audit trail, as we do not push similar events when client
assertion is used. Also, before #7657 the events were defined, but not
pushed.
# How the Problems Are Solved
- Removed API secret check succeeded and faild event definitions
- Removed OIDC secret check succeeded and faild event definitions
- Push only Hash Updated event when needed
# Additional Changes
- None
# Additional Context
- Fixes https://github.com/zitadel/zitadel/issues/8479
- Closes https://github.com/zitadel/zitadel/issues/8430
- Intoduced in https://github.com/zitadel/zitadel/pull/7657
# Which Problems Are Solved
Use web keys, managed by the `resources/v3alpha/web_keys` API, for OIDC
token signing and verification,
as well as serving the public web keys on the jwks / keys endpoint.
Response header on the keys endpoint now allows caching of the response.
This is now "safe" to do since keys can be created ahead of time and
caches have sufficient time to pickup the change before keys get
enabled.
# How the Problems Are Solved
- The web key format is used in the `getSignerOnce` function in the
`api/oidc` package.
- The public key cache is changed to get and store web keys.
- The jwks / keys endpoint returns the combined set of valid "legacy"
public keys and all available web keys.
- Cache-Control max-age default to 5 minutes and is configured in
`defaults.yaml`.
When the web keys feature is enabled, fallback mechanisms are in place
to obtain and convert "legacy" `query.PublicKey` as web keys when
needed. This allows transitioning to the feature without invalidating
existing tokens. A small performance overhead may be noticed on the keys
endpoint, because 2 queries need to be run sequentially. This will
disappear once the feature is stable and the legacy code gets cleaned
up.
# Additional Changes
- Extend legacy key lifetimes so that tests can be run on an existing
database with more than 6 hours apart.
- Discovery endpoint returns all supported algorithms when the Web Key
feature is enabled.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/8031
- Part of https://github.com/zitadel/zitadel/issues/7809
- After https://github.com/zitadel/oidc/pull/637
- After https://github.com/zitadel/oidc/pull/638
# Which Problems Are Solved
Currently the OIDC API of ZITADEL only prints parent errors to the logs.
Where 4xx status are typically warn level and 5xx error level. This
makes it hard to debug certain errors for client in multi-instance
environments like ZITADEL cloud, where there is no direct access to
logs. In case of support requests we often can't correlate past
log-lines to the error that was reported.
This change adds the possibility to return the parent error in the
response to the OIDC client. For the moment this only applies to JSON
body responses, not error redirects to the RP.
# How the Problems Are Solved
- New instance-level feature flag: `debug_oidc_parent_error`
- Use the new `WithReturnParentToClient()` function from the oidc lib
introduced in https://github.com/zitadel/oidc/pull/629 for all cases
where `WithParent` was already used and the request context is
available.
# Additional Changes
none
# Additional Context
- Depends on: https://github.com/zitadel/oidc/pull/629
- Related to: https://github.com/zitadel/zitadel/issues/8362
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Sometimes integrations tests are failing with an error `http: no
location header in response`. The underlying cause was hidden, as in
some tests we assumed a 3xx range response but got a 4xx response
instead. No assertion on the status code was made, resulting in the
above error message on calling `resp.Location()`.
The underlying issue, the application not found in the projection, is
also fixed.
# How the Problems Are Solved
This change adds a check for the status code and returns the response
body if the response is not in the 3xx status code range.
Helper function that create applications now do an additional
`GetAppByID` in a retry loop to ensure consitency in the projection
before proceeding with tests.
# Additional Changes
- none
# Additional Context
- Pipline failures were observed, no issue was created
- Cherry-picked form WIP #8407
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
When Web keys with a large RSA Bitsize were generated, the new web key
did not get projected.
Zitadel logs printed the following errors:
```
time="2024-08-15T09:18:47Z" level=debug msg="execution of statements failed" caller="/home/tim/Repositories/zitadel/zitadel/internal/eventstore/handler/v2/handler.go:518" error="statement failed: ID=CRDB-pKtsr Message=exec failed Parent=(ERROR: index row size 3400 exceeds btree version 4 maximum 2704 for index \"web_keys_web_key_state_idx\" (SQLSTATE 54000))" lastProcessedIndex=-1 projection=projections.web_keys
```
# How the Problems Are Solved
Remove the the `WithInclude` option with the Private key from the
`web_keys` projection index and bump the projection version.
# Additional Changes
- Correct RPC summaries for documentation
- Fix docs slug in sidebar.js
# Additional Context
- Found during release review.
# Which Problems Are Solved
Implement a new API service that allows management of OIDC signing web
keys.
This allows users to manage rotation of the instance level keys. which
are currently managed based on expiry.
The API accepts the generation of the following key types and
parameters:
- RSA keys with 2048, 3072 or 4096 bit in size and:
- Signing with SHA-256 (RS256)
- Signing with SHA-384 (RS384)
- Signing with SHA-512 (RS512)
- ECDSA keys with
- P256 curve
- P384 curve
- P512 curve
- ED25519 keys
# How the Problems Are Solved
Keys are serialized for storage using the JSON web key format from the
`jose` library. This is the format that will be used by OIDC for
signing, verification and publication.
Each instance can have a number of key pairs. All existing public keys
are meant to be used for token verification and publication the keys
endpoint. Keys can be activated and the active private key is meant to
sign new tokens. There is always exactly 1 active signing key:
1. When the first key for an instance is generated, it is automatically
activated.
2. Activation of the next key automatically deactivates the previously
active key.
3. Keys cannot be manually deactivated from the API
4. Active keys cannot be deleted
# Additional Changes
- Query methods that later will be used by the OIDC package are already
implemented. Preparation for #8031
- Fix indentation in french translation for instance event
- Move user_schema translations to consistent positions in all
translation files
# Additional Context
- Closes#8030
- Part of #7809
---------
Co-authored-by: Elio Bischof <[email protected]>
# Which Problems Are Solved
Log some details when a secret generator is not found.
This should help us debugging such issue.
# How the Problems Are Solved
When a secret generator by type query fails,
we log the generator type and instance id for which
the generator was requested.
# Additional Changes
- none
# Additional Context
- Related to https://github.com/zitadel/zitadel/issues/8379
- Also encountered in https://github.com/zitadel/zitadel/pull/8407
# Which Problems Are Solved
We noticed logging where 500: Internal Server errors were returned from
the token endpoint, mostly for the `refresh_token` grant. The error was
thrown by the database as it received non-UTF8 strings for token IDs
Zitadel uses symmetric encryption for opaque tokens, including refresh
tokens. Encrypted values are base64 encoded. It appeared to be possible
to send garbage base64 to the token endpoint, which will pass decryption
and string-splitting. In those cases the resulting ID is not a valid
UTF-8 string.
Invalid non-UTF8 strings are now rejected during token decryption.
# How the Problems Are Solved
- `AESCrypto.DecryptString()` checks if the decrypted bytes only contain
valid UTF-8 characters before converting them into a string.
- `AESCrypto.Decrypt()` is unmodified and still allows decryption on
non-UTF8 byte strings.
- `FromRefreshToken` now uses `DecryptString` instead of `Decrypt`
# Additional Changes
- Unit tests added for `FromRefreshToken` and
`AESCrypto.DecryptString()`.
- Fuzz tests added for `FromRefreshToken` and
`AESCrypto.DecryptString()`. This was to pinpoint the problem
- Testdata with values that resulted in invalid strings are committed.
In the pipeline this results in the Fuzz tests to execute as regular
unit-test cases. As we don't use the `-fuzz` flag in the pipeline no
further fuzzing is performed.
# Additional Context
- Closes#7765
- https://go.dev/doc/tutorial/fuzz
# Which Problems Are Solved
Improve the performance of human imports by optimizing the query that
finds domains claimed by other organizations.
# How the Problems Are Solved
Use the fields search table introduced in
https://github.com/zitadel/zitadel/pull/8191 by storing each
organization domain as Object ID and the verified status as field value.
# Additional Changes
- Feature flag for this optimization
# Additional Context
- Performance improvements for import are evaluated and acted upon
internally at the moment
---------
Co-authored-by: adlerhurst <[email protected]>
# Which Problems Are Solved
Imporve the performance of user grant addition, especially for import.
# How the Problems Are Solved
Use the search table to query for the project grant state.
This could easily be done by making the search used in
`checkProjectGrantPreCondition` reusable.
# Additional Changes
Chanded event declerations to `const` in the
`internal/repository/project` package.
# Additional Context
- Performance improvements for import are evaluated and acted upon
internally at the moment
---------
Co-authored-by: adlerhurst <[email protected]>
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Allow verification of imported passwords hashed with plain md5, without
salt. These are password digests typically created by one of:
- `printf "password" | md5sum` on most linux systems.
- PHP's `md5("password")`
- Python3's `hashlib.md5(b"password").hexdigest()`
# How the Problems Are Solved
- Upgrade passwap to
[v0.6.0](https://github.com/zitadel/passwap/releases/tag/v0.6.0)
- Add md5plain as a new verfier option in `defaults.yaml`
# Additional Changes
- Updated documentation to explain difference between `md5` (crypt) and
`md5plain` verifiers.
# Additional Context
- Requested by customer for import case
# Which Problems Are Solved
Improve the performance of the `admin/v1/import` API endpoint.
Specifaclly the import of large amount of project grants.
# How the Problems Are Solved
`AddProjectGrantWithID` and `AddProjectGrantMember` methods of
`Commands` used to get the current state of the Writemodel to check if
the current GrantID or the combination of GrantID & UserID wasn't
already used. However, the Added events already have protection against
duplication by the `UniqueConstaint` methods.
The queries become very slow when there is a great amount of project
grants. Because all the events are pushed to the aggregate ID of the
project, we had to obtain all related project events, including events
of grantIDs we do not care about. This O(n) duration for bached import
jobs adding many organization granted to a single project.
This change removes the unnecesary state query to improve performance.
# Additional Changes
- Add integration tests for import
# Additional Context
- reported internally
# Which Problems Are Solved
This fix adds tracing spans to all V1 API import related functions. This
is to troubleshoot import related performance issues reported to us.
# How the Problems Are Solved
Add a tracing span to `api/grpc/admin/import.go` and all related
functions that are called in the `command` package.
# Additional Changes
- none
# Additional Context
- Reported by internal communication
# Which Problems Are Solved
Zitadel never stored or returned the requested `response_mode` in oidc
Auth Requests. This caused the oidc library to fallback to the default
based on the response_type.
# How the Problems Are Solved
- Store the `response_mode` in the Auth request repo
- Store the `response_mode` in the Auth request v2 events
- Return the `resonse_mode` from the Auth Request v1 and v2
`ResponseMode()` methods. (Was hard-coded to an empty string)
# Additional Changes
- Populate the `response_modes_supported` to the oidc Discovery
Configuration. When it was empty, the standard specifies the default of
`query` and `fragment`. However, our oidc library also supports
`form_post` and by this fix, zitadel now also supports this.
# Additional Context
- Closes#6586
- Reported
https://discord.com/channels/927474939156643850/1151508313717084220
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
An admin / application might want to be able to reduce the amount of
roles returned in the token, for example if a user is granted to many
organizations or for specific cases where the application want to narrow
down the access for that token to a specific organization or multiple.
This can now be achieved by providing a scope with the id of the
organization, resp. multiple scopes for every organization, which should
be included.
```
urn:zitadel:iam:org:roles🆔{orgID}
```
**Note:** the new scope does not work when Introspection / Userinfo are
set to legacy mode.
# How the Problems Are Solved
The user info query now has two variants:
1. Variant that returns all organization authorization grants if the new
scope wasn't provided for backward compatibility.
2. Variant that filters the organizations based on the IDs passed in one
or more of the above scopes and returns only those authorization grants.
The query is defined as a `text/template` and both variants are rendered
once in package `init()`.
# Additional Changes
- In the integration tests `assertProjectRoleClaims` now also checks the
org IDs in the roles.
# Additional Context
- Closes#7996
# Which Problems Are Solved
After https://github.com/zitadel/zitadel/pull/7822 was merged we
discovered that
v2 tokens that where obtained through an IDP using the v1 login, can't
be used for
zitadel API calls.
- Because we used to store the AMR claim on the auth request, but
internally use the domain.UserAuthMethod type. AMR has no notion of an
IDP login, so that "factor" was lost
during conversion. Rendering those v2 tokens invalid on the zitadel API.
- A wrong check on machine user tokens falsly allowed some tokens to be
valid
- The client ID was set to tokens from client credentials and JWT
profile, which made client queries fail in the validation middleware.
The middleware expects client ID unset for machine users.
# How the Problems Are Solved
Store the domain.AuthMethods directly in the auth requests and session,
instead of using AMR claims with lossy conversion.
- IDPs have seperate auth method, which is not an AMR claim
- Machine users are treated specialy, eg auth methods are not required.
- Do not set the client ID for client credentials and JWT profile
# Additional Changes
Cleaned up mostly unused `oidc.getInfoFromRequest()`.
# Additional Context
- Bugs were introduced in https://github.com/zitadel/zitadel/pull/7822
and not yet part of a release.
- Reported internally.
# Which Problems Are Solved
The session update requires the current session token as argument.
Since this adds extra complexity but no real additional security and
prevents case like magic links, we want to remove this requirement.
We still require the session token on other resouces / endpoints, e.g.
for finalizing the auth request or on idp intents.
# How the Problems Are Solved
- Removed the session token verifier in the Update Session GRPc call.
- Removed the session token from login UI examples session update calls
# Additional Changes
- none
# Additional Context
- Closes#7883
* implement code exchange
* port tokenexchange to v2 tokens
* implement refresh token
* implement client credentials
* implement jwt profile
* implement device token
* cleanup unused code
* fix current unit tests
* add user agent unit test
* unit test domain package
* need refresh token as argument
* test commands create oidc session
* test commands device auth
* fix device auth build error
* implicit for oidc session API
* implement authorize callback handler for legacy implicit mode
* upgrade oidc module to working draft
* add missing auth methods and time
* handle all errors in defer
* do not fail auth request on error
the oauth2 Go client automagically retries on any error. If we fail the auth request on the first error, the next attempt will always fail with the Errors.AuthRequest.NoCode, because the auth request state is already set to failed.
The original error is then already lost and the oauth2 library does not return the original error.
Therefore we should not fail the auth request.
Might be worth discussing and perhaps send a bug report to Oauth2?
* fix code flow tests by explicitly setting code exchanged
* fix unit tests in command package
* return allowed scope from client credential client
* add device auth done reducer
* carry nonce thru session into ID token
* fix token exchange integration tests
* allow project role scope prefix in client credentials client
* gci formatting
* do not return refresh token in client credentials and jwt profile
* check org scope
* solve linting issue on authorize callback error
* end session based on v2 session ID
* use preferred language and user agent ID for v2 access tokens
* pin oidc v3.23.2
* add integration test for jwt profile and client credentials with org scopes
* refresh token v1 to v2
* add user token v2 audit event
* add activity trigger
* cleanup and set panics for unused methods
* use the encrypted code for v1 auth request get by code
* add missing event translation
* fix pipeline errors (hopefully)
* fix another test
* revert pointer usage of preferred language
* solve browser info panic in device auth
* remove duplicate entries in AMRToAuthMethodTypes to prevent future `mfa` claim
* revoke v1 refresh token to prevent reuse
* fix terminate oidc session
* always return a new refresh toke in refresh token grant
---------
Co-authored-by: Livio Spring <[email protected]>
We are trying to reproduce a few 500 responses we observe on zitadel cloud's token endpoint.
As in the past these were caused by wrongly encoded or encrypted refresh tokens, I created a integration test which tries to reproduce 500 errors by sending invalid refresh tokens.
The added test does not reproduce 500s, all returned errors are in the 400 range as they should. However, as the test is already written, we might as well include them.
Related to #7765
Co-authored-by: Livio Spring <[email protected]>
* fix(oidc): roles in userinfo for client credentials token
When tokens were obtained using the client credentials grant,
with audience and role scopes, userinfo would not return the role claims. This had multiple causes:
1. There is no auth request flow, so for legacy userinfo project data was never attached to the token
2. For optimized userinfo, there is no client ID that maps to an application. The client ID for client credentials is the machine user's name. There we can't obtain a project ID. When the project ID remained empty, we always ignored the roleAudience.
This PR fixes situation 2, by always taking the roleAudience into account, even when the projectID is empty. The code responsible for the bug is also refactored to be more readable and understandable, including additional godoc.
The fix only applies to the optimized userinfo code introduced in #7706 and released in v2.50 (currently in RC). Therefore it can't be back-ported to earlier versions.
Fixes#6662
* chore(deps): update all go deps (#7764)
This change updates all go modules, including oidc, a major version of go-jose and the go 1.22 release.
* Revert "chore(deps): update all go deps" (#7772)
Revert "chore(deps): update all go deps (#7764)"
This reverts commit 6893e7d060a953d595a18ff8daa979834c4324d5.
---------
Co-authored-by: Livio Spring <[email protected]>
This fixes the projection of events that have a null audience or scope.
As audience was added in v2.50, legacy events do not have an audience, this made replay of the old events not possible after an upgrade.
* feat(oidc): optimize the userinfo endpoint
* store project ID in the access token
* query for projectID if not in token
* add scope based tests
* Revert "store project ID in the access token"
This reverts commit 5f0262f23988e7f62d415d0e4a02a705f6ad5197.
* query project role assertion
* use project role assertion setting to return roles
* workaround eventual consistency and handle PAT
* do not append empty project id
* fix(oidc): return bad request for base64 errors
We've recently noticed an increased amount of 500: internal server error status returns on zitadel cloud.
The source of these errors appear to be erroneous input in fields that are supposed to be bas64 formatted.
```
time=2024-04-08T14:05:47.600Z level=ERROR msg="request error" oidc_error.parent="ID=OIDC-AhX2u Message=Errors.Internal Parent=(illegal base64 data at input byte 8)" oidc_error.description=Errors.Internal oidc_error.type=server_error status_code=500
```
Within the possible code paths of the token endpoint there are a couple of uses of base64.Encoding.DecodeString of which a returned error was not properly wrapped, but returned as-is.
This causes the oidc error handler to return a 500 with the `OIDC-AhX2u` ID.
We were not able to pinpoint the exact errors that are happening to any one call of `DecodeString`.
This fix wraps all errors from `DecodeString` so that proper 400: bad request is returned with information about the error. Each wrapper now has an unique error ID, so that logs will contain the source of the error as well.
This bug was reported internally by the ops team.
* catch op.ErrInvalidRefreshToken
* fix(idp): do not call userinfo when mapping from ID token is configured
This change prevents the call of the Userinfo endpoint of a OIDC IDP if the IDP is configured to use the ID token for user information instead.
A unit test has been added to confirm the corrected behavior.
Closes#7373
* video for e2e
---------
Co-authored-by: Livio Spring <[email protected]>
chore(fmt): run gci on complete project
Fix global import formatting in go code by running the `gci` command. This allows us to just use the command directly, instead of fixing the import order manually for the linter, on each PR.
Co-authored-by: Elio Bischof <[email protected]>
feat(db): wrap BeginTx in spans to get acquire metrics
This changes adds a span around most db.BeginTx calls so we can get tracings about the connection pool acquire process.
This might help us pinpoint why sometimes some query package traces show longer execution times, while this was not reflected on database side execution times.
Co-authored-by: Silvan <[email protected]>
* docs(oidc): token exchange beta feature info
This change adds an info box to the token exchange documentation, informing the reader of the beta state of the feature and how to enable it.
* Update docs/docs/apis/openidoauth/endpoints.mdx
Co-authored-by: Fabi <[email protected]>
* Update docs/docs/guides/integrate/token-exchange.mdx
Co-authored-by: Fabi <[email protected]>
---------
Co-authored-by: Fabi <[email protected]>
* docs(oidc): token exchange guide
This change adds a token exchange guide which includes "simple" and impersonation examples.
The endpoint, claims and grant type documentation also has been amended with token exchange specifics.
* solve suggestions
* fix impersonated event type
* add link to event store concept
* fix links build error
* add to sidebar and update some info boxes
fix(oidc): define audience inside auth request instead off token creation
When using the v1 OIDC Code flow, tokens would not carry the correct audience when returned as JWT. This applies to access tokens as JWT and ID tokens.
Introspection would still show the correct audience.
This happened because project audience was appended at token creation time. This stored the appended audience, used later in introspection or token refresh. However, the OIDC library still only had a view of the original auth request with the original audience.
When signing JWTs it would use this outdated information.
This change moves audience modifications to the auth request creation. This is was already the way it was done for v2 login and now v1 follows the same method.
Co-authored-by: Livio Spring <[email protected]>
* add token exchange feature flag
* allow setting reason and actor to access tokens
* impersonation
* set token types and scopes in response
* upgrade oidc to working draft state
* fix tests
* audience and scope validation
* id toke and jwt as input
* return id tokens
* add grant type token exchange to app config
* add integration tests
* check and deny actors in api calls
* fix instance setting tests by triggering projection on write and cleanup
* insert sleep statements again
* solve linting issues
* add translations
* pin oidc v3.15.0
* resolve comments, add event translation
* fix refreshtoken test
* use ValidateAuthReqScopes from oidc
* apparently the linter can't make up its mind
* persist actor thru refresh tokens and check in tests
* remove unneeded triggers
fix(query): optimize instance by domain query
On zitadel cloud we noticed an increase in database CPU usage and slightly higher response times.
By analyzes we found that the instance by domain query was wrongly joining all instance_feature rows against all instances.
This PR adds an additional CTE to limit the join set to only the features that apply to the found instance.
The query was introduced with https://github.com/zitadel/zitadel/pull/7356 and part of the v2.47 release.
* fix: assign instance ID to aggregate ID when converting from v1 to v2 feature
This change fixes a mismatch between v1 and v2 aggregate IDs for instance feature events.
The old v1 used a random aggregate ID, while v2 uses the instance ID as aggregate ID.
The adapter was not correctly mapping, which resulted in the projections.instance_features table being filled with wrong instance IDs.
Closes#7501
* fix unit test
* feat(api): feature API proto definitions
* update proto based on discussion with @livio-a
* cleanup old feature flag stuff
* authz instance queries
* align defaults
* projection definitions
* define commands and event reducers
* implement system and instance setter APIs
* api getter implementation
* unit test repository package
* command unit tests
* unit test Get queries
* grpc converter unit tests
* migrate the V1 features
* migrate oidc to dynamic features
* projection unit test
* fix instance by host
* fix instance by id data type in sql
* fix linting errors
* add system projection test
* fix behavior inversion
* resolve proto file comments
* rename SystemDefaultLoginInstanceEventType to SystemLoginDefaultOrgEventType so it's consistent with the instance level event
* use write models and conditional set events
* system features integration tests
* instance features integration tests
* error on empty request
* documentation entry
* typo in feature.proto
* fix start unit tests
* solve linting error on key case switch
* remove system defaults after discussion with @eliobischof
* fix system feature projection
* resolve comments in defaults.yaml
---------
Co-authored-by: Livio Spring <[email protected]>
* fix(oidc): ignore public key expiry for ID Token hints
This splits the key sets used for access token and ID token hints.
ID Token hints should be able to be verified by with public keys that are already expired.
However, we do not want to change this behavior for Access Tokens,
where an error for an expired public key is still returned.
The public key cache is modified to purge public keys based on last use,
instead of expiry.
The cache is shared between both verifiers.
* resolve review comments
* pin oidc 3.11
* chore(deps): upgrade all go deps
Also `go mod tidy`.
Added comments with URLs for package version lists to makefile commands.
* Update Makefile
Co-authored-by: Livio Spring <[email protected]>
---------
Co-authored-by: Livio Spring <[email protected]>
fix(oidc); refresh token for device authorization
Due to a mis-alignment of OIDC interface and concrete implementations in zitadel, requesting a refresh token for device authorization would fail.
This change adds the possibility to to use the op.IDTokenRequest directly.
Also, the UserAgentID is dropped as required parameter, as devices do not have a user agent.
* fix(db): add additional connection pool for projection spooling
* use correct connection pool for projections
---------
Co-authored-by: Livio Spring <[email protected]>
* cleanup todo
* pass id token details to oidc
* feat(oidc): id token for device authorization
This changes updates to the newest oidc version,
so the Device Authorization grant can return ID tokens when
the scope `openid` is set.
There is also some refactoring done, so that the eventstore can be
queried directly when polling for state.
The projection is cleaned up to a minimum with only data required for the login UI.
* try to be explicit wit hthe timezone to fix github
* pin oidc v3.8.0
* remove TBD entry
* fix(authz): add logging to access token verification errors
Related to #6949
* use logging fields
---------
Co-authored-by: Livio Spring <[email protected]>
* chore: add database questions to bug report template
Sometimes we get bug reports that are only reproducible when zitadel is running against a certain database.
This change adds database related questions to the issue template, as it is something people don't tend to describe in the detail fields.
* fix syntax error
This change adds a core_generate_all make target.
It installs the required tools and runs generate on the complete project.
`golang/mock` is no longer maintained and a fork is available
from the Uber folks. So the latter is used as tool.
All the mock files have been regenerated and are part of the PR.
The obsolete `tools` directory has been removed,
as all the tools are now part of specific make targets.
Co-authored-by: Silvan <[email protected]>
* get key by id and cache them
* userinfo from events for v2 tokens
* improve keyset caching
* concurrent token and client checks
* client and project in single query
* logging and otel
* drop owner_removed column on apps and authN tables
* userinfo and project roles in go routines
* get oidc user info from projections and add actions
* add avatar URL
* some cleanup
* pull oidc work branch
* remove storage from server
* add config flag for experimental introspection
* legacy introspection flag
* drop owner_removed column on user projections
* drop owner_removed column on useer_metadata
* query userinfo unit test
* query introspection client test
* add user_grants to the userinfo query
* handle PAT scopes
* bring triggers back
* test instance keys query
* add userinfo unit tests
* unit test keys
* go mod tidy
* solve some bugs
* fix missing preferred login name
* do not run triggers in go routines, they seem to deadlock
* initialize the trigger handlers late with a sync.OnceValue
* Revert "do not run triggers in go routines, they seem to deadlock"
This reverts commit 2a03da2127b7dc74552ec25d4772282a82cc1cba.
* add missing translations
* chore: update go version for linting
* pin oidc version
* parse a global time location for query test
* fix linter complains
* upgrade go lint
* fix more linting issues
---------
Co-authored-by: Stefan Benz <[email protected]>
* chore(deps): upgrade all go modules
This change upgrades all go.mod dependecies. As well as Makefile tools.
There where some imports that still used the old and deprecated
`github.com/golang/protobuf/ptypes` package.
These have been moved to the equivelant
`google.golang.org/protobuf/types/known` package.
The `internal/proto` package is removed as was only used once.
With a simple refactor in the Validator it became completely obsolete.
* fix validate unit test
* cleanup merge
* update otel
---------
Co-authored-by: Livio Spring <[email protected]>
* feat(oidc): use the new oidc server interface
* rename from provider to server
* pin logging and oidc packages
* use oidc introspection fix branch
* add overloaded methods with tracing
* cleanup unused code
* include latest oidc fixes
---------
Co-authored-by: Livio Spring <[email protected]>
* fix(eventstore): prevent allocation of filtered events
Directly reduce each event obtained from a sql.Rows scan,
so that we do not have to allocate all events in a slice.
* reinstate the mutex as RWMutex
* scan data directly
* add todos
* fix(writemodels): add reduce of parent
* test: remove comment
* update comments
---------
Co-authored-by: adlerhurst <[email protected]>
This pr upgrades oidc to v3 . Function signature changes have been migrated as well. Specifically there are more client calls that take a context now. Where feasable a context is added to those calls. Where a context is not (easily) available context.TODO() is used as a reminder for when it does.
Related to #6619
This change adds the ability to set and get user agent data, such as fingerprint, IP, request headers and a description to the session. All fields are optional.
Closes#6028
When doing local builds, the `VERSION` was always set to `development`.
When running this local build, database migration would be skipped,
as the version did not change.
This change suffixes `developement` with the dynamic timestamp from `now`,
so that migrations are always executed
when developing.
Tim MöhlmannandGitHub
