chore!: Introduce ZITADEL v3 (#9645)

This PR summarizes multiple changes specifically only available with
ZITADEL v3:

- feat: Web Keys management
(https://github.com/zitadel/zitadel/pull/9526)
- fix(cmd): ensure proper working of mirror
(https://github.com/zitadel/zitadel/pull/9509)
- feat(Authz): system user support for permission check v2
(https://github.com/zitadel/zitadel/pull/9640)
- chore(license): change from Apache to AGPL
(https://github.com/zitadel/zitadel/pull/9597)
- feat(console): list v2 sessions
(https://github.com/zitadel/zitadel/pull/9539)
- fix(console): add loginV2 feature flag
(https://github.com/zitadel/zitadel/pull/9682)
- fix(feature flags): allow reading "own" flags
(https://github.com/zitadel/zitadel/pull/9649)
- feat(console): add Actions V2 UI
(https://github.com/zitadel/zitadel/pull/9591)

BREAKING CHANGE
- feat(webkey): migrate to v2beta API
(https://github.com/zitadel/zitadel/pull/9445)
- chore!: remove CockroachDB Support
(https://github.com/zitadel/zitadel/pull/9444)
- feat(actions): migrate to v2beta API
(https://github.com/zitadel/zitadel/pull/9489)

---------

Co-authored-by: Livio Spring <[email protected]>
Co-authored-by: Stefan Benz <[email protected]>
Co-authored-by: Silvan <[email protected]>
Co-authored-by: Ramon <[email protected]>
Co-authored-by: Elio Bischof <[email protected]>
Co-authored-by: Kenta Yamaguchi <[email protected]>
Co-authored-by: Harsha Reddy <[email protected]>
Co-authored-by: Livio Spring <[email protected]>
Co-authored-by: Max Peintner <[email protected]>
Co-authored-by: Iraq <[email protected]>
Co-authored-by: Florian Forster <[email protected]>
Co-authored-by: Tim Möhlmann <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Max Peintner <[email protected]>
This commit is contained in:
Fabienne Bühler
2025-04-02 16:53:06 +02:00
committed by GitHub
co-authored by Livio Spring Stefan Benz Silvan Ramon Elio Bischof Kenta Yamaguchi Harsha Reddy Livio Spring Max Peintner Iraq Florian Forster Tim Möhlmann Copilot Max Peintner
parent d14a23ae7e
commit 07ce3b6905
559 changed files with 14578 additions and 7622 deletions
@@ -20,7 +20,6 @@ The following table shows the available integration patterns for streaming audit
| | Description | Self-hosting | ZITADEL Cloud |
|-------------------------------------|----------------------------------------------------------------------------------------------------------------|-------------|---------------|
| Events-API | Pulling events of all ZITADEL resources such as Users, Projects, Apps, etc. (Events = Change Log of Resources) | ✅ | ✅ |
| Cockroach Change Data Capture | Sending events of all ZITADEL resources such as Users, Projects, Apps, etc. (Events = Change Log of Resources) | ✅ | ❌ |
| ZITADEL Actions Log to Stdout | Custom log to messages possible on predefined triggers during login / register Flow | ✅ | ❌ |
| ZITADEL Actions trigger API/Webhook | Custom API/Webhook request on predefined triggers during login / register | ✅ | ✅ |
@@ -34,71 +33,6 @@ This API offers granular control through various filters, enabling you to:
You can find a comprehensive guide on how to use the events API for different use cases here: [Get Events from ZITADEL](/docs/guides/integrate/zitadel-apis/event-api)
### Cockroach Change Data Capture
For self-hosted ZITADEL deployments utilizing CockroachDB as the database, [CockroachDB's built-in Change Data Capture (CDC)](https://www.cockroachlabs.com/docs/stable/change-data-capture-overview) functionality provides a streamlined approach to integrate ZITADEL audit logs with external systems.
CDC captures row-level changes in your database and streams them as messages to a configurable destination, such as Google BigQuery or a SIEM/SOC solution. This real-time data stream enables:
- **Continuous monitoring**: Receive near-instantaneous updates on ZITADEL activity, facilitating proactive threat detection and response.
- **Simplified integration**: Leverage CockroachDB's native capabilities for real-time data transfer, eliminating the need for additional tools or configurations.
This approach is limited to self-hosted deployments using CockroachDB and requires expertise in managing the database and CDC configuration.
#### Sending events to Google Cloud Storage using Change Data Capture
This example will show you how you can utilize CDC for sending all ZITADEL events to Google Cloud Storage.
For a detailed description please read [CockroachLab's Get Started Guide](https://www.cockroachlabs.com/docs/v23.2/create-and-configure-changefeeds) and [Cloud Storage Authentication](https://www.cockroachlabs.com/docs/v23.2/cloud-storage-authentication?filters=gcs#set-up-google-cloud-storage-assume-role) from Cockroach.
You will need a Google Cloud Storage Bucket and a service account.
1. [Create Google Cloud Storage Bucket](https://cloud.google.com/storage/docs/creating-buckets)
2. [Create Service Account](https://cloud.google.com/iam/docs/service-accounts-create)
3. Create a role with the `storage.objects.create` permission
4. Grant service account access to the bucket
5. Create key for service account and download it
Now we need to enable and create the changefeed in the cockroach DB.
1. [Enable rangefeeds on cockroach cluster](https://www.cockroachlabs.com/docs/v23.2/create-and-configure-changefeeds#enable-rangefeeds)
```bash
SET CLUSTER SETTING kv.rangefeed.enabled = true;
```
2. Encode the keyfile from the service account with base64 and replace the placeholder it in the script below
3. Create Changefeed to send data into Google Cloud Storage
The following example sends all events without payload to Google Cloud Storage
Per default we do not want to send the payload of the events, as this could potentially include personally identifiable information (PII)
If you want to include the payload, you can just add `payload` to the select list in the query.
```sql
CREATE CHANGEFEED INTO 'gs://gc-storage-zitadel-data/events?partition_format=flat&AUTH=specified&CREDENTIALS=base64encodedkey'
AS SELECT instance_id, aggregate_type, aggregate_id, owner, event_type, sequence, created_at
FROM eventstore.events2;
```
In some cases you might want the payload of only some specific events.
This example shows you how to get all events and the instance domain events with the payload:
```sql
CREATE CHANGEFEED INTO 'gs://gc-storage-zitadel-data/events?partition_format=flat&AUTH=specified&CREDENTIALS=base64encodedkey'
AS SELECT instance_id, aggregate_type, aggregate_id, owner, event_type, sequence, created_at
CASE WHEN event_type IN ('instance.domain.added', 'instance.domain.removed', 'instance.domain.primary.set' )
THEN payload END AS payload
FROM eventstore.events2;
```
The partition format in the example above is flat, this means that all files for each timestamp will be created in the same folder.
You will have files for different timestamps including the output for the events created in that time.
Each event is represented as a json row.
Example Output:
```json lines
{
"aggregate_id": "26553987123463875",
"aggregate_type": "user",
"created_at": "2023-12-25T10:01:45.600913Z",
"event_type": "user.human.added",
"instance_id": "123456789012345667",
"payload": null,
"sequence": 1
}
```
## ZITADEL Actions
ZITADEL [Actions](/docs/concepts/features/actions) offer a powerful mechanism for extending the platform's capabilities and integrating with external systems tailored to your specific requirements.
@@ -138,6 +138,10 @@ In this initial release, the new login is available for self-hosting only. We'll
### Current State
:::info
The documentation describes the state of the feature in ZITADEL V3.
:::
Our primary goal for the TypeScript login system is to replace the existing login functionality within Zitadel Core, which is shipped with Zitadel automatically. This will allow us to leverage the benefits of the new system, including its modular architecture and enhanced security features.
To achieve this, we are actively working on implementing the core features currently available in Zitadel Core, such as:
@@ -160,15 +164,10 @@ As we continue to develop the TypeScript login system, we will provide regular u
For the first implementation we have excluded the following features:
- SAML (SP & OP)
- Generic JWT IDP
- LDAP IDP
- Device Authorization Grants
- Timebased features
- Lockout Settings
- Password Expiry Settings
- Login Settings - Multifactor init prompt
- Force MFA on external authenticated users
- Force MFA on external authenticated users
- Passkey/U2F Setup
- As passkey and u2f is bound to a domain, it is important to notice, that setting up the authentication possibility in the ZITADEL management console (Self-service), will not work if the login runs on a different domain
- Custom Login Texts
@@ -23,6 +23,7 @@ endpoints are called with a JWT access token.
:::info
Web keys are an [experimental](/docs/support/software-release-cycles-support#beta) feature. Be sure to enable the `web_key` [feature](/docs/apis/resources/feature_service_v2/feature-service-set-instance-features) before using it.
The documentation describes the state of the feature in ZITADEL V3.
Test the feature and add improvement or bug reports directly to the [github repository](https://github.com/zitadel/zitadel) or let us know your general feedback in the [discord thread](https://discord.com/channels/927474939156643850/1329100936127320175/threads/1332344892629717075)!
:::
@@ -112,7 +113,7 @@ When the request does not contain any specific configuration,
[RSA](#rsa) is used as default with the default options as described below:
```bash
curl -L 'https://$CUSTOM-DOMAIN/resources/v3alpha/web_keys' \
curl -L 'https://$CUSTOM-DOMAIN/v2beta/web_keys' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-H 'Authorization: Bearer <TOKEN>' \
@@ -136,7 +137,7 @@ The RSA generator config takes two enum values.
For example, to create a RSA web key with the size of 3072 bits and the SHA512 algorithm (RS512):
```bash
curl -L 'https://$CUSTOM-DOMAIN/resources/v3alpha/web_keys' \
curl -L 'https://$CUSTOM-DOMAIN/v2beta/web_keys' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-H 'Authorization: Bearer <TOKEN>' \
@@ -160,7 +161,7 @@ The ECDSA generator config takes a single `curve` enum value which determines bo
For example, to create a ECDSA web key with a P-256 curve and the SHA256 algorithm:
```bash
curl -L 'https://$CUSTOM-DOMAIN/resources/v3alpha/web_keys' \
curl -L 'https://$CUSTOM-DOMAIN/v2beta/web_keys' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-H 'Authorization: Bearer <TOKEN>' \
@@ -184,7 +185,7 @@ Clients which support both curves must inspect `crv` header value to assert the
For example, to create a ed25519 web key:
```bash
curl -L 'https://$CUSTOM-DOMAIN/resources/v3alpha/web_keys' \
curl -L 'https://$CUSTOM-DOMAIN/v2beta/web_keys' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-H 'Authorization: Bearer <TOKEN>' \
@@ -246,11 +247,11 @@ For the sake of this example we will use simplified IDs and restrict timestamps
After one month, on 2025-02-01, we wish to activate the next available key and create a new key to be available for activation next month. This fulfills requirements 1 and 2.
```bash
curl -L -X POST 'https://$CUSTOM-DOMAIN/resources/v3alpha/web_keys/2/_activate' \
curl -L -X POST 'https://$CUSTOM-DOMAIN/v2beta/web_keys/2/_activate' \
-H 'Accept: application/json' \
-H 'Authorization: Bearer <TOKEN>'
curl -L 'https://$CUSTOM-DOMAIN/resources/v3alpha/web_keys' \
curl -L 'https://$CUSTOM-DOMAIN/v2beta/web_keys' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-H 'Authorization: Bearer <TOKEN>' \
@@ -282,7 +283,7 @@ In addition to the activate and create calls we made on this iteration,
we can now safely delete the oldest key, as both requirement 3 and 4 are now fulfilled:
```bash
curl -L -X DELETE 'https://$CUSTOM-DOMAIN/resources/v3alpha/web_keys/1' \
curl -L -X DELETE 'https://$CUSTOM-DOMAIN/v2beta/web_keys/1' \
-H 'Accept: application/json' \
-H 'Authorization: Bearer <TOKEN>'
```
@@ -16,6 +16,8 @@ To authenticate the user a self-signed JWT will be created and utilized.
You can define any id for your user. This guide will assume it's `system-user-1`.
**NOTE:** system user id cannot contain capital letters
## Generate an RSA keypair
Generate an RSA private key with 2048 bit modulus:
@@ -3,9 +3,34 @@ To create a new user, go to Users and click on **New**. Enter the required conta
import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
:::note
If you started with Zitadel before version 3, you might have the "Human User [deprecated]" UI.
In this case please enable the Feature Flag "Use V2 Api in Console for User creation" in the Default Settings.
:::
<Tabs>
<TabItem value="human" label="Human User" default>
<img src="/docs/img/guides/console/invitehuman.png" width="680px" alt="Invite Human" />
When creating a new user you have different options.
First add the email, and select if the email address should be added automatically as "verified".
In the last section you can choose the authentication options:
- **Setup authentication later for this user**: This flow might be useful if an employee starts at a later point but you already want to prepare the account. The user will not have an authentication method, before they will be able to login, they need to setup a method.
- **Send an invitation E-Mail for authentication setup and E-Mail verification**: The user will receive an email and be able to setup an authentication method (e.g Password, Passkey, External SSO).
- When using the [Zitadel Login V1](/docs/guides/integrate/login/hosted-login) the user will be prompted to setup a password
- When using the [Zitadel Login V2](/docs/guides/integrate/login/hosted-login#hosted-login-version-2-beta) the user has the option to choose the authentication method (password, passkey, identity provider), based on the configuration of the organization
<img src="/docs/img/guides/console/setupauthmethod.png" width="680px" alt="Invite Human - Setup authentication method" />
- **Set an initial password for the user**: The user will receive an email and be able to setup an authentication method (e.g Password, Passkey, External SSO)
</TabItem>
<TabItem value="human_deprecated" label="Human User [deprecated]">
<img src="/docs/img/guides/console/addhuman.png" width="680px" alt="Add Human" />
After a human user is created, by default, an initialization mail with a code is sent to the registered email. This code then has to be verified on first login.
If you want to omit this mail, you can check the **email verified** and **set initial password** toggle.
If no password is set initially, the initialization mail prompting the user to set his password is sent.
</TabItem>
<TabItem value="service" label="Service User">
<img
@@ -16,10 +41,6 @@ import TabItem from "@theme/TabItem";
</TabItem>
</Tabs>
After a human user is created, by default, an initialization mail with a code is sent to the registered email. This code then has to be verified on first login.
If you want to omit this mail, you can check the **email verified** and **set initial password** toggle.
If no password is set initially, the initialization mail prompting the user to set his password is sent.
You can prompt the user to add a second factor method too by checking the **Force MFA** toggle in [Login behaviour settings](/docs/guides/manage/console/default-settings#login-behavior-and-access).
When logged in, a user can then manage the profile in the console, adding a profile picture, external IDPs and Passwordless authentication devices.