# Which Problems Are Solved
ZITADEL cannot currently be used as an OAuth 2.0 Authorization Server by
clients that need to register themselves at runtime:
- There is no [OAuth 2.0 Dynamic Client Registration (RFC
7591)](https://datatracker.ietf.org/doc/html/rfc7591) endpoint, and no
`registration_endpoint` is advertised in the discovery document.
- This blocks [Model Context Protocol
(MCP)](https://modelcontextprotocol.io) clients (Claude Desktop,
claude.ai, Cursor, the MCP SDKs), whose OAuth 2.1 authorization profile
expects an Authorization Server that supports dynamic client
registration. They self-register before any user context exists and only
then start the authorization code + PKCE flow.
# How the Problems Are Solved
## Control plane: instance security settings
No feature flag and no runtime config. A
`DynamicClientRegistrationSettings` message is nested under the instance
`SecuritySettings` in `settings/v2`, next to `embedded_iframe` and
`enable_impersonation`:
| `enabled` | `allow_unauthenticated` | Behaviour |
| --- | --- | --- |
| `false` (default) | — | `POST /oauth/v2/register` returns `404`,
`registration_endpoint` absent from discovery |
| `true` | `false` (default) | Token-gated registration, client homed in
the token's organization |
| `true` | `true` | Open registration for MCP hosts, client homed in the
instance default organization |
Both values are carried on `authz.Instance` and loaded with the
instance, exactly like `EnableImpersonation`, so the discovery document
and the endpoint read them per request without an extra query.
`allow_unauthenticated` implies `enabled` in both implementations, so a
single setting closes the endpoint.
Adding the two columns bumps the projection table to
`projections.security_policies3`, following what the impersonation
setting did for `security_policies2`.
## Authorization: a dedicated permission, not `project.app.write`
- New org-scoped permission **`project.app.register_dynamic`**, granted
by default to `ORG_OWNER`, `IAM_OWNER` and `IAM_ORG_MANAGER` — in both
`InternalAuthZ` and `SystemAuthZ`, mirroring where `project.app.write`
already lives.
- New built-in role **`ORG_DYNAMIC_CLIENT_REGISTRAR`** carrying only
that permission, so a service user can self-register clients without
gaining write access to the organization's existing applications.
- Open mode (`allow_unauthenticated = true`) requires no token and no
permission: that is the MCP self-register path.
- A valid token whose user lacks the permission is answered with **`403
insufficient_scope`** (RFC 6750 §3.1) rather than `401 invalid_token`,
so a caller can tell "your token is bad" from "your token may not do
this". `verifyAccessToken` also reports invalid tokens as permission
denied, so the authorization failure is marked explicitly to keep the
two apart.
The check runs once, at the endpoint, right after the token is verified
and before any state is created, so it also gates the auto-provisioning
of the organization's DCR project. The permission itself lives in the
command layer next to the others
(`Commands.CheckPermissionRegisterDynamicClient`). As the OIDC endpoints
are not behind the authorization interceptor, the caller identity is
derived from the verified token, the way token exchange does for the
impersonating actor.
## Registration itself
- **Registered clients are ordinary OIDC applications.** The endpoint
reuses the existing application-creation events
(`NewApplicationAddedEvent` and `NewOIDCConfigAddedEvent`), so the whole
token, authorization and introspection flow keeps working unchanged.
They are stored in a dedicated, auto-provisioned project named `ZITADEL
DCR` per organization (name lookup), so they do not pollute the
`IAMProject` or other projects.
- Input is validated with the existing `GetOIDCV1Compliance` rules;
`private_key_jwt` and `jwks`/`jwks_uri` are rejected with
`invalid_client_metadata`.
- **Endpoint wiring without forking the library.** `NewServer` calls
`op.RegisterServer` instead of `op.RegisterLegacyServer`, so the new
route can be added through the same `op.WithSetRouter` hook as the
authorize callback. `op.RegisterLegacyServer` appends a middleware after
the caller options, which chi forbids once a route has been registered;
`op.NewIssuerInterceptor` reproduces the issuer middleware it would
otherwise add. The `/oauth/v2` routing prefix is unchanged.
# Additional Changes
- **`application_type` inference for custom-scheme redirects** (last
commit, isolated). `application_type` is an OpenID Connect member; RFC
7591 does not define it, so clients that only implement RFC 7591 omit
it. Native MCP hosts doing so register custom-scheme redirect URIs,
which ZITADEL reserves for native applications, and the OIDC default of
`web` rejected them with `invalid_redirect_uri`. Inference is limited to
custom schemes and to an absent `application_type`, so every request
accepted today keeps its current application type and auth method — an
`http` loopback redirect still yields a web application. Happy to drop
this commit if you would rather keep it out of this PR.
- **Docs**: a Dynamic Client Registration integration guide, a
`registration_endpoint` section on the OpenID Connect endpoints page,
and the new role in the administrators table (plus its description in
the Console translations).
# Additional Context
- Part of #9810. This is **Phase 1 (DCR)**; #12315 adds RFC 7592 and
#12316 adds CIMD, both stacked on this branch and rebased once this
lands.
- Implements the control-plane and authorization model requested in the
review of this PR, itself a refinement of the direction in
https://github.com/zitadel/zitadel/issues/9810#issuecomment-4891176997.
- **Out of scope (deliberately)**: RFC 7592, CIMD, persisting free-form
metadata (`logo_uri`, `contacts`, and similar), and `private_key_jwt` /
`jwks_uri` (rejected cleanly). No new event types.
- Follow-ups explicitly left out, per the review: DCR client lifecycle
and richer audit, consent and custom scopes, JWT / `aud` (RFC 8707), and
RFC 8414 AS metadata (currently absent from both ZITADEL and
`zitadel/oidc`).
# Which Problems Are Solved
While Zitadel provides a possibility to restrict certain languages to be
used, the corresponding list could not be retrieved in the settings
service (v2). This blocked login v2 implementations from respecting the
list and they would always use all available languages.
# How the Problems Are Solved
- Retrieve the list when checking the instance and pass it into the
context.
- Return it as part of the existing `GetGeneralSettingsResponse`
- This allows us to remove an additional query in some other cases /
endpoints.
# Additional Changes
none
# Additional Context
- required for https://github.com/zitadel/zitadel/pull/11372
- backport to v4.x
---------
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Marco A. <[email protected]>
# Which Problems Are Solved
In many cases IAM was used where instance was meant.
# How the Problems Are Solved
- Rename wrong usage of IAM to instance all over the code base.
- Add an agents.md which explains the terms used in Zitadel's
architecture.
# Additional Context
- closes#11280
---------
Co-authored-by: Copilot <[email protected]>
Co-authored-by: muhlemmer <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Marco A. <[email protected]>
# Which Problems Are Solved
The following terms have all been renamed to management console:
- Customer Portal (when used to mean the console)
- Console
- Admin Console
# How the Problems Are Solved
- Search & Replace smartly
- Use Copilot for translation files
Changes done to: backend + frontend codebase, docs, translations and
protobufs (descriptions only)
# Additional Context
- Partially Closes#11279
# Which Problems Are Solved
A customer noted that one some of their environments actions were never
executed at all.
During the investigation it was discovered, that these systems did not
have any features set.
# How the Problems Are Solved
Fix the `scanAuthzInstance` function to handle queries without features
returned correctly.
# Additional Changes
None
# Additional Context
- reported through support
- requires backport to v4.x and v3.x
# Which Problems Are Solved
Cached object may have a different schema between Zitadel versions.
# How the Problems Are Solved
Use the curent build version in DB based cache connectors PostgreSQL and
Redis.
# Additional Changes
- Cleanup the ZitadelVersion field from the authz Instance
- Solve potential race condition on global variables in build package.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/10648
- Obsoletes https://github.com/zitadel/zitadel/pull/10646
- Needs to be back-ported to v4 over
https://github.com/zitadel/zitadel/pull/10645
# Which Problems Are Solved
The event execution system currently uses a projection handler that
subscribes to and processes all events for all instances. This creates a
high static cost because the system over-fetches event data, handling
many events that are not needed by most instances. This inefficiency is
also reflected in high "rows returned" metrics in the database.
# How the Problems Are Solved
Eliminate the use of a project handler. Instead, events for which
"execution targets" are defined, are directly pushed to the queue by the
eventstore. A Router is populated in the Instance object in the authz
middleware.
- By joining the execution targets to the instance, no additional
queries are needed anymore.
- As part of the instance object, execution targets are now cached as
well.
- Events are queued within the same transaction, giving transactional
guarantees on delivery.
- Uses the "insert many fast` variant of River. Multiple jobs are queued
in a single round-trip to the database.
- Fix compatibility with PostgreSQL 15
# Additional Changes
- The signing key was stored as plain-text in the river job payload in
the DB. This violated our [Secrets
Storage](https://zitadel.com/docs/concepts/architecture/secrets#secrets-storage)
principle. This change removed the field and only uses the encrypted
version of the signing key.
- Fixed the target ordering from descending to ascending.
- Some minor linter warnings on the use of `io.WriteString()`.
# Additional Context
- Introduced in https://github.com/zitadel/zitadel/pull/9249
- Closes https://github.com/zitadel/zitadel/issues/10553
- Closes https://github.com/zitadel/zitadel/issues/9832
- Closes https://github.com/zitadel/zitadel/issues/10372
- Closes https://github.com/zitadel/zitadel/issues/10492
---------
Co-authored-by: Stefan Benz <[email protected]>
<!--
Please inform yourself about the contribution guidelines on submitting a
PR here:
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#submit-a-pull-request-pr.
Take note of how PR/commit titles should be written and replace the
template texts in the sections below. Don't remove any of the sections.
It is important that the commit history clearly shows what is changed
and why.
Important: By submitting a contribution you agree to the terms from our
Licensing Policy as described here:
https://github.com/zitadel/zitadel/blob/main/LICENSING.md#community-contributions.
-->
# Which Problems Are Solved
These changes introduce resource-based API endpoints for managing
instances and custom domains.
There are 4 types of changes:
- Endpoint implementation: consisting of the protobuf interface and the
implementation of the endpoint. E.g:
606439a17227b629c1d018842dc3f1c569e4627a
- (Integration) Tests: testing the implemented endpoint. E.g:
cdfe1f0372b30cb74e34f0f23c6ada776e4477e9
- Fixes: Bugs found during development that are being fixed. E.g:
acbbeedd3259b785948c1d702eb98f5810b3e60a
- Miscellaneous: code needed to put everything together or that doesn't
fit any of the above categories. E.g:
529df92abce1ffd69c0b3214bd835be404fd0de0 or
6802cb5468fbe24664ae6639fd3a40679222a2fd
# How the Problems Are Solved
_Ticked checkboxes indicate that the functionality is complete_
- [x] Instance
- [x] Create endpoint
- [x] Create endpoint tests
- [x] Update endpoint
- [x] Update endpoint tests
- [x] Get endpoint
- [x] Get endpoint tests
- [x] Delete endpoint
- [x] Delete endpoint tests
- [x] Custom Domains
- [x] Add custom domain
- [x] Add custom domain tests
- [x] Remove custom domain
- [x] Remove custom domain tests
- [x] List custom domains
- [x] List custom domains tests
- [x] Trusted Domains
- [x] Add trusted domain
- [x] Add trusted domain tests
- [x] Remove trusted domain
- [x] Remove trusted domain tests
- [x] List trusted domains
- [x] List trusted domains tests
# Additional Changes
When looking for instances (through the `ListInstances` endpoint)
matching a given query, if you ask for the results to be order by a
specific column, the query will fail due to a syntax error. This is
fixed in acbbeedd3259b785948c1d702eb98f5810b3e60a . Further explanation
can be found in the commit message
# Additional Context
- Relates to #9452
- CreateInstance has been excluded:
https://github.com/zitadel/zitadel/issues/9930
- Permission checks / instance retrieval (middleware) needs to be
changed to allow context based permission checks
(https://github.com/zitadel/zitadel/issues/9929), required for
ListInstances
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Scheduled handlers use `eventstore.InstanceIDs` to get the all active
instances within a given timeframe. This function scrapes through all
events written within that time frame which can cause heavy load on the
database.
# How the Problems Are Solved
A new query cache `activeInstances` is introduced which caches the ids
of all instances queried by id or host within the configured timeframe.
# Additional Changes
- Changed `default.yaml`
- Removed `HandleActiveInstances` from custom handler configs
- Added `MaxActiveInstances` to define the maximal amount of cached
instance ids
- fixed start-from-init and start-from-setup to start auth and admin
projections twice
- fixed org cache invalidation to use correct index
# Additional Context
- part of #8999
# Which Problems Are Solved
Organizations are ofter searched for by ID or primary domain. This
results in many redundant queries, resulting in a performance impact.
# How the Problems Are Solved
Cache Organizaion objects by ID and primary domain.
# Additional Changes
- Adjust integration test config to use all types of cache.
- Adjust integration test lifetimes so the pruner has something to do
while the tests run.
# Additional Context
- Closes#8865
- After #8902
# Which Problems Are Solved
Add a cache implementation using Redis single mode. This does not add
support for Redis Cluster or sentinel.
# How the Problems Are Solved
Added the `internal/cache/redis` package. All operations occur
atomically, including setting of secondary indexes, using LUA scripts
where needed.
The [`miniredis`](https://github.com/alicebob/miniredis) package is used
to run unit tests.
# Additional Changes
- Move connector code to `internal/cache/connector/...` and remove
duplicate code from `query` and `command` packages.
- Fix a missed invalidation on the restrictions projection
# Additional Context
Closes#8130
# Which Problems Are Solved
Cache implementation using a PGX connection pool.
# How the Problems Are Solved
Defines a new schema `cache` in the zitadel database.
A table for string keys and a table for objects is defined.
For postgreSQL, tables are unlogged and partitioned by cache name for
performance.
Cockroach does not have unlogged tables and partitioning is an
enterprise feature that uses alternative syntax combined with sharding.
Regular tables are used here.
# Additional Changes
- `postgres.Config` can return a pxg pool. See following discussion
# Additional Context
- Part of https://github.com/zitadel/zitadel/issues/8648
- Closes https://github.com/zitadel/zitadel/issues/8647
---------
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
We identified the need of caching.
Currently we have a number of places where we use different ways of
caching, like go maps or LRU.
We might also want shared chaches in the future, like Redis-based or in
special SQL tables.
# How the Problems Are Solved
Define a generic Cache interface which allows different implementations.
- A noop implementation is provided and enabled as.
- An implementation using go maps is provided
- disabled in defaults.yaml
- enabled in integration tests
- Authz middleware instance objects are cached using the interface.
# Additional Changes
- Enabled integration test command raceflag
- Fix a race condition in the limits integration test client
- Fix a number of flaky integration tests. (Because zitadel is super
fast now!) 🎸🚀
# Additional Context
Related to https://github.com/zitadel/zitadel/issues/8648
# Which Problems Are Solved
The current v3alpha actions APIs don't exactly adhere to the [new
resources API
design](https://zitadel.com/docs/apis/v3#standard-resources).
# How the Problems Are Solved
- **Improved ID access**: The aggregate ID is added to the resource
details object, so accessing resource IDs and constructing proto
messages for resources is easier
- **Explicit Instances**: Optionally, the instance can be explicitly
given in each request
- **Pagination**: A default search limit and a max search limit are
added to the defaults.yaml. They apply to the new v3 APIs (currently
only actions). The search query defaults are changed to ascending by
creation date, because this makes the pagination results the most
deterministic. The creation date is also added to the object details.
The bug with updated creation dates is fixed for executions and targets.
- **Removed Sequences**: Removed Sequence from object details and
ProcessedSequence from search details
# Additional Changes
Object details IDs are checked in unit test only if an empty ID is
expected. Centralizing the details check also makes this internal object
more flexible for future evolutions.
# Additional Context
- Closes#8169
- Depends on https://github.com/zitadel/zitadel/pull/8225
---------
Co-authored-by: Silvan <[email protected]>
Co-authored-by: Stefan Benz <[email protected]>
# Which Problems Are Solved
ZITADEL currently selects the instance context based on a HTTP header
(see https://github.com/zitadel/zitadel/issues/8279#issue-2399959845 and
checks it against the list of instance domains. Let's call it instance
or API domain.
For any context based URL (e.g. OAuth, OIDC, SAML endpoints, links in
emails, ...) the requested domain (instance domain) will be used. Let's
call it the public domain.
In cases of proxied setups, all exposed domains (public domains) require
the domain to be managed as instance domain.
This can either be done using the "ExternalDomain" in the runtime config
or via system API, which requires a validation through CustomerPortal on
zitadel.cloud.
# How the Problems Are Solved
- Two new headers / header list are added:
- `InstanceHostHeaders`: an ordered list (first sent wins), which will
be used to match the instance.
(For backward compatibility: the `HTTP1HostHeader`, `HTTP2HostHeader`
and `forwarded`, `x-forwarded-for`, `x-forwarded-host` are checked
afterwards as well)
- `PublicHostHeaders`: an ordered list (first sent wins), which will be
used as public host / domain. This will be checked against a list of
trusted domains on the instance.
- The middleware intercepts all requests to the API and passes a
`DomainCtx` object with the hosts and protocol into the context
(previously only a computed `origin` was passed)
- HTTP / GRPC server do not longer try to match the headers to instances
themself, but use the passed `http.DomainContext` in their interceptors.
- The `RequestedHost` and `RequestedDomain` from authz.Instance are
removed in favor of the `http.DomainContext`
- When authenticating to or signing out from Console UI, the current
`http.DomainContext(ctx).Origin` (already checked by instance
interceptor for validity) is used to compute and dynamically add a
`redirect_uri` and `post_logout_redirect_uri`.
- Gateway passes all configured host headers (previously only did
`x-zitadel-*`)
- Admin API allows to manage trusted domain
# Additional Changes
None
# Additional Context
- part of #8279
- open topics:
- "single-instance" mode
- Console UI
* feat: improve instance not found error
* unit tests
* check if is templatable
* lint
* assert
* compile tests
* remove error templates
* link to instance not found page
* fmt
* cleanup
* lint
* feat(api): feature API proto definitions
* update proto based on discussion with @livio-a
* cleanup old feature flag stuff
* authz instance queries
* align defaults
* projection definitions
* define commands and event reducers
* implement system and instance setter APIs
* api getter implementation
* unit test repository package
* command unit tests
* unit test Get queries
* grpc converter unit tests
* migrate the V1 features
* migrate oidc to dynamic features
* projection unit test
* fix instance by host
* fix instance by id data type in sql
* fix linting errors
* add system projection test
* fix behavior inversion
* resolve proto file comments
* rename SystemDefaultLoginInstanceEventType to SystemLoginDefaultOrgEventType so it's consistent with the instance level event
* use write models and conditional set events
* system features integration tests
* instance features integration tests
* error on empty request
* documentation entry
* typo in feature.proto
* fix start unit tests
* solve linting error on key case switch
* remove system defaults after discussion with @eliobischof
* fix system feature projection
* resolve comments in defaults.yaml
---------
Co-authored-by: Livio Spring <[email protected]>
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and [06](https://zitadel.com/docs/support/advisory/a10006).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
feat(storage): read only transactions for queries (#6415)
* fix: tests
* bastle wie en grosse
* fix(database): scan as callback
* fix tests
* fix merge failures
* remove as of system time
* refactor: remove unused test
* refacotr: remove unused lines
* reproduce #5808
Add an integration test that imports and gets N amount of human users.
- With N set to 1-10 the operation seems to succeed always
- With N set to 100 the operation seems to fail between 1 and 7 times.
* fix merge issue
* fix: reset the call timestamp after a bulk trigger
With the use of `AS OF SYSTEM TIME` in queries,
there was a change for the query package not
finding the latest projection verson after
a bulk trigger.
If events where processed in the bulk trigger,
the resulting row timestamp would be after the call
start timestamp.
This sometimes resulted in consistency issues when
Set and Get API methods are called in short succession.
For example a Import and Get user could sometimes result in a Not Found
error.
Although the issue was reported for the Management API user import,
it is likely this bug contributed to the flaky integration and e2e tests.
Fixes#5808
* trigger bulk action in GetSession
* don't use the new context in handler schedule
* disable reproduction test
---------
Co-authored-by: Livio Spring <[email protected]>
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: rename orgiampolicy to domain policy
* fix: merge conflicts
* fix: protos
* fix: md files
* implement deprecated org iam policy again
Co-authored-by: Livio Amstutz <[email protected]>
jmaretteandGitHub
SilvanandGitHub
Elio BischofandGitHub
