mirror of
https://github.com/andwati/zitadel.git
synced 2026-11-05 13:52:15 +00:00
feat(cmd): mirror (#7004)
# Which Problems Are Solved Adds the possibility to mirror an existing database to a new one. For that a new command was added `zitadel mirror`. Including it's subcommands for a more fine grained mirror of the data. Sub commands: * `zitadel mirror eventstore`: copies only events and their unique constraints * `zitadel mirror system`: mirrors the data of the `system`-schema * `zitadel mirror projections`: runs all projections * `zitadel mirror auth`: copies auth requests * `zitadel mirror verify`: counts the amount of rows in the source and destination database and prints the diff. The command requires one of the following flags: * `--system`: copies all instances of the system * `--instance <instance-id>`, `--instance <comma separated list of instance ids>`: copies only the defined instances The command is save to execute multiple times by adding the `--replace`-flag. This replaces currently existing data except of the `events`-table # Additional Changes A `--for-mirror`-flag was added to `zitadel setup` to prepare the new database. The flag skips the creation of the first instances and initial run of projections. It is now possible to skip the creation of the first instance during setup by setting `FirstInstance.Skip` to true in the steps configuration. # Additional info It is currently not possible to merge multiple databases. See https://github.com/zitadel/zitadel/issues/7964 for more details. It is currently not possible to use files. See https://github.com/zitadel/zitadel/issues/7966 for more information. closes https://github.com/zitadel/zitadel/issues/7586 closes https://github.com/zitadel/zitadel/issues/7486 ### Definition of Ready - [x] I am happy with the code - [x] Short description of the feature/issue is added in the pr description - [x] PR is linked to the corresponding user story - [x] Acceptance criteria are met - [x] All open todos and follow ups are defined in a new ticket and justified - [x] Deviations from the acceptance criteria and design are agreed with the PO and documented. - [x] No debug or dead code - [x] My code has no repetitions - [x] Critical parts are tested automatically - [ ] Where possible E2E tests are implemented - [x] Documentation/examples are up-to-date - [x] All non-functional requirements are met - [x] Functionality of the acceptance criteria is checked manually on the dev system. --------- Co-authored-by: Livio Spring <[email protected]>
This commit is contained in:
Silvan
Livio Spring
@@ -131,9 +131,9 @@ In the guides below, some of which utilize the Generic OIDC or SAML templates fo
|
||||
|
||||
If ZITADEL doesn't offer a specific template for your Identity Provider (IdP) and your IdP is fully compliant with OpenID Connect (OIDC), you have the option to use the generic OIDC provider configuration.
|
||||
|
||||
For those utilizing a SAML Service Provider, the SAML Service Provider option is available. You can learn how to set up a SAML Service Provider with our [MockSAML example](https://zitadel.com/docs/guides/integrate/identity-providers/mocksaml).
|
||||
For those utilizing a SAML Service Provider, the SAML Service Provider option is available. You can learn how to set up a SAML Service Provider with our [MockSAML example](/docs/guides/integrate/identity-providers/mocksaml).
|
||||
|
||||
Should you wish to transition from a generic OIDC provider to Entra ID (formerly Azure Active Directory) or Google, consider following this [guide](https://zitadel.com/docs/guides/integrate/identity-providers/migrate).
|
||||
Should you wish to transition from a generic OIDC provider to Entra ID (formerly Azure Active Directory) or Google, consider following this [guide](/docs/guides/integrate/identity-providers/migrate).
|
||||
|
||||
|
||||
|
||||
@@ -176,6 +176,6 @@ Deciding whether to configure an external Identity Provider (IdP) at the organiz
|
||||
|
||||
## References
|
||||
|
||||
- [Identity brokering in ZITADEL](https://zitadel.com/docs/concepts/features/identity-brokering)
|
||||
- [The ZITADEL API reference for managing external IdPs](https://zitadel.com/docs/category/apis/resources/admin/identity-providers)
|
||||
- [Handle external logins in a custom login UI](https://zitadel.com/docs/guides/integrate/login-ui/external-login)
|
||||
- [Identity brokering in ZITADEL](/docs/concepts/features/identity-brokering)
|
||||
- [The ZITADEL API reference for managing external IdPs](/docs/category/apis/resources/admin/identity-providers)
|
||||
- [Handle external logins in a custom login UI](/docs/guides/integrate/login-ui/external-login)
|
||||
@@ -1,5 +1,5 @@
|
||||
When your user is done using your application and clicks on the logout button, you have to send a request to the terminate session endpoint.
|
||||
[Terminate Session Documentation](https://zitadel.com/docs/apis/resources/session_service/session-service-delete-session)
|
||||
[Terminate Session Documentation](/docs/apis/resources/session_service/session-service-delete-session)
|
||||
|
||||
Sessions can be terminated by either:
|
||||
- the authenticated user
|
||||
|
||||
@@ -3,7 +3,7 @@ If you want to build your own select account/account picker, you have to cache t
|
||||
We recommend storing a list of the session Ids with the corresponding session token in a cookie.
|
||||
The list of session IDs can be sent in the “search sessions” request to get a detailed list of sessions for the account selection.
|
||||
|
||||
[Search Sessions Documentation](https://zitadel.com/docs/apis/resources/session_service/session-service-list-sessions)
|
||||
[Search Sessions Documentation](/docs/apis/resources/session_service/session-service-list-sessions)
|
||||
|
||||
### Request
|
||||
|
||||
|
||||
@@ -115,7 +115,7 @@ We do have a guide series on how to build your own login ui, which also includes
|
||||
- Passkeys
|
||||
- External Login Providers
|
||||
|
||||
You can find all the guides here: [Build your own login UI](https://zitadel.com/docs/guides/integrate/login-ui)
|
||||
You can find all the guides here: [Build your own login UI](/docs/guides/integrate/login-ui)
|
||||
|
||||
The create user request also allows you to add metadata (key, value) to the user.
|
||||
This gives you the possibility to collect additional data from your users during the registration process and store it directly to the user in ZITADEL.
|
||||
|
||||
@@ -5,7 +5,7 @@ sidebar_label: Basic Authentication
|
||||
|
||||
import IntrospectionResponse from './_introspection-response.mdx';
|
||||
|
||||
This is a guide on how to secure your API using [Basic Authentication](https://zitadel.com/docs/apis/openidoauth/authn-methods#client-secret-basic).
|
||||
This is a guide on how to secure your API using [Basic Authentication](/docs/apis/openidoauth/authn-methods#client-secret-basic).
|
||||
|
||||
## Register the API in ZITADEL
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ ZITADEL leverages the power of eventsourcing, meaning every action and change wi
|
||||
To provide you with greater flexibility and access to these events, ZITADEL has introduced an Event API.
|
||||
This API allows you to easily retrieve and utilize the events generated within the system, enabling you to integrate them into your own system and respond to specific events as they occur.
|
||||
|
||||
You need to give a user the [manager role](https://zitadel.com/docs/guides/manage/console/managers) IAM_OWNER_VIEWER or IAM_OWNER to access the Event API.
|
||||
You need to give a user the [manager role](/docs/guides/manage/console/managers) IAM_OWNER_VIEWER or IAM_OWNER to access the Event API.
|
||||
|
||||
If you like to know more about eventsourcing/eventstore and how this works in ZITADEL, head over to our [concepts](/docs/concepts/eventstore/overview).
|
||||
## Request Events
|
||||
|
||||
@@ -19,7 +19,7 @@ You can subscribe and unsubscribe to notifications and newsletters:
|
||||
- Security: Receive notifications related to security issues
|
||||
|
||||
:::info Technical Advisories
|
||||
If you want to stay up to date on our technical advisories, we recommend [subscribing here to the mailing list](https://zitadel.com/docs/support/technical_advisory#subscribe-to-our-mailing-list).
|
||||
If you want to stay up to date on our technical advisories, we recommend [subscribing here to the mailing list](/docs/support/technical_advisory#subscribe-to-our-mailing-list).
|
||||
|
||||
Technical advisories are notices that report major issues with ZITADEL Self-Hosted or the ZITADEL Cloud platform that could potentially impact security or stability in production environments.
|
||||
:::
|
||||
|
||||
@@ -5,7 +5,7 @@ sidebar_label: From Keycloak
|
||||
|
||||
## Migrating from Keycloak to ZITADEL
|
||||
|
||||
This guide will use [Docker installation](https://www.docker.com/) to run Keycloak and ZITADEL. However, both Keycloak and ZITADEL offer different installation methods. As a result, this guide won't include any required production tuning or security hardening for either system. However, it's advised you follow [recommended guidelines](https://zitadel.com/docs/guides/manage/self-hosted/production) before putting those systems into production. You can skip setting up Keycloak and ZITADEL if you already have running instances.
|
||||
This guide will use [Docker installation](https://www.docker.com/) to run Keycloak and ZITADEL. However, both Keycloak and ZITADEL offer different installation methods. As a result, this guide won't include any required production tuning or security hardening for either system. However, it's advised you follow [recommended guidelines](/docs/self-hosting/manage/production) before putting those systems into production. You can skip setting up Keycloak and ZITADEL if you already have running instances.
|
||||
|
||||
## Set up Keycloak
|
||||
### Run Keycloak
|
||||
@@ -77,7 +77,7 @@ docker cp <keycloak container ID>:/tmp/my-realm-users-0.json .
|
||||
|
||||
## Set up ZITADEL
|
||||
|
||||
After creating a sample application that connects to Keycloak, you need to set up ZITADEL in order to migrate the application and users from Keycloak to ZITADEL. For this, ZITADEL offers a [Docker Compose](https://zitadel.com/docs/self-hosting/deploy/compose) installation guide. Follow the instructions under the [Docker compose](https://zitadel.com/docs/self-hosting/deploy/compose#docker-compose) section to run a ZITADEL instance locally.
|
||||
After creating a sample application that connects to Keycloak, you need to set up ZITADEL in order to migrate the application and users from Keycloak to ZITADEL. For this, ZITADEL offers a [Docker Compose](/docs/self-hosting/deploy/compose) installation guide. Follow the instructions under the [Docker compose](/docs/self-hosting/deploy/compose#docker-compose) section to run a ZITADEL instance locally.
|
||||
|
||||
Next, the application will be available at [http://localhost:8080/ui/console/](http://localhost:8080/ui/console/).
|
||||
|
||||
@@ -91,13 +91,13 @@ Now you can access the console with the following default credentials:
|
||||
|
||||
## Import Keycloak users into ZITADEL
|
||||
|
||||
As explained in this [ZITADEL user migration guide](https://zitadel.com/docs/guides/migrate/users), you can import users individually or in bulk. Since we are looking at importing a single user from Keycloak, migrating that individual user to ZITADEL can be done with the [ImportHumanUser](https://zitadel.com/docs/apis/resources/mgmt/management-service-import-human-user) endpoint.
|
||||
As explained in this [ZITADEL user migration guide](/docs/guides/migrate/users), you can import users individually or in bulk. Since we are looking at importing a single user from Keycloak, migrating that individual user to ZITADEL can be done with the [ImportHumanUser](/docs/apis/resources/mgmt/management-service-import-human-user) endpoint.
|
||||
|
||||
> With this endpoint, an email will only be sent to the user if the email is marked as not verified or if there's no password set.
|
||||
|
||||
### Create a service user to consume ZITADEL API
|
||||
|
||||
But first of all, in order to use this ZITADEL API, you need to create a [service user](https://zitadel.com/docs/guides/integrate/service-users/authenticate-service-users#exercise-create-a-service-user).
|
||||
But first of all, in order to use this ZITADEL API, you need to create a [service user](/docs/guides/integrate/service-users/authenticate-service-users#exercise-create-a-service-user).
|
||||
|
||||
Go to the **Users** menu and select the **Service Users** tab. And click the **+ New** button.
|
||||
|
||||
@@ -167,7 +167,7 @@ if your Keycloak Realm has a single user, your `my-realm-users-0.json` file, int
|
||||
}
|
||||
```
|
||||
|
||||
Now, you need to transform the JSON to the ZITADEL data format by adhering to the ZITADEL API [specification](https://zitadel.com/docs/apis/resources/mgmt/management-service-import-human-user) to import a user. The minimal format would be as shown below:
|
||||
Now, you need to transform the JSON to the ZITADEL data format by adhering to the ZITADEL API [specification](/docs/apis/resources/mgmt/management-service-import-human-user) to import a user. The minimal format would be as shown below:
|
||||
|
||||
```js
|
||||
{
|
||||
|
||||
@@ -42,7 +42,7 @@ Please also consult our [guide](/docs/guides/manage/user/reg-create-user) on how
|
||||
|
||||
## Bulk import
|
||||
|
||||
For bulk import use the [import endpoint](https://zitadel.com/docs/apis/resources/admin/admin-service-import-data) on the admin API:
|
||||
For bulk import use the [import endpoint](/docs/apis/resources/admin/admin-service-import-data) on the admin API:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -191,7 +191,7 @@ Currently it is not possible to migrate passkeys directly from another system.
|
||||
|
||||
## Users linked to an external IDP
|
||||
|
||||
A users `sub` is bound to the external [IDP's Client ID](https://zitadel.com/docs/guides/manage/console/default-settings#identity-providers).
|
||||
A users `sub` is bound to the external [IDP's Client ID](/docs/guides/manage/console/default-settings#identity-providers).
|
||||
This means that the IDP Client ID configured in ZITADEL must be the same ID as in the legacy system.
|
||||
|
||||
Users should be imported with their `externalUserId`.
|
||||
@@ -211,7 +211,7 @@ _snippet from [bulk-import](#bulk-import) example:_
|
||||
}
|
||||
```
|
||||
|
||||
You can use an Action with [post-creation flow](https://zitadel.com/docs/apis/actions/external-authentication#post-creation) to pull information such as roles from the old system and apply them to the user in ZITADEL.
|
||||
You can use an Action with [post-creation flow](/docs/apis/actions/external-authentication#post-creation) to pull information such as roles from the old system and apply them to the user in ZITADEL.
|
||||
|
||||
## Metadata
|
||||
|
||||
@@ -220,7 +220,7 @@ Use metadata to store additional attributes of the users, such as organizational
|
||||
|
||||
:::info
|
||||
Metadata must be added to users after the users were created. Currently metadata can't be added during user creation.
|
||||
[API reference: User Metadata](https://zitadel.com/docs/category/apis/resources/mgmt/user-metadata)
|
||||
[API reference: User Metadata](/docs/category/apis/resources/mgmt/user-metadata)
|
||||
:::
|
||||
|
||||
Request metadata from the userinfo endpoint by passing the required [reserved scope](/docs/apis/openidoauth/scopes#reserved-scopes) in your auth request.
|
||||
@@ -232,5 +232,5 @@ You can assign roles from owned or granted projects to a user.
|
||||
|
||||
:::info
|
||||
Authorizations must be added to users after the users were created. Currently metadata can't be added during user creation.
|
||||
[API reference: User Authorization / Grants](https://zitadel.com/docs/category/apis/resources/auth/user-authorizations-grants)
|
||||
[API reference: User Authorization / Grants](/docs/category/apis/resources/auth/user-authorizations-grants)
|
||||
:::
|
||||
@@ -353,7 +353,7 @@ The provided config extends the `UserManagerSettings` of the `oidc-client-ts` li
|
||||
- redirect_uri (the URL to redirect to after the authorization flow is complete)
|
||||
- post_logout_redirect_uri (the URL to redirect to after the user logs out)
|
||||
- scope (the permissions requested from the user)
|
||||
- project_resource_id (To add a ZITADEL project scope. `urn:zitadel:iam:org:project:id:[projectId]:aud` and `urn:zitadel:iam:org:projects:roles` [scopes](https://zitadel.com/docs/apis/openidoauth/scopes#reserved-scopes).)
|
||||
- project_resource_id (To add a ZITADEL project scope. `urn:zitadel:iam:org:project:id:[projectId]:aud` and `urn:zitadel:iam:org:projects:roles` [scopes](/docs/apis/openidoauth/scopes#reserved-scopes).)
|
||||
- prompt ([the OIDC prompt parameter](/apis/openidoauth/endpoints#additional-parameters))
|
||||
|
||||
2. Create a folder named components in the src directory. Create two files named Login.js and Callback.js.
|
||||
@@ -412,4 +412,4 @@ And this brings us to the end of this quick start guide!
|
||||
|
||||
This tutorial covered how to configure ZITADEL and how to use React to build an app that communicates with ZITADEL to access secured resources.
|
||||
|
||||
We hope you enjoyed the tutorial and encourage you to check out the ZITADEL [documentation](https://zitadel.com/docs) for more information on how to use the ZITADEL platform to its full potential. Thanks for joining us!
|
||||
We hope you enjoyed the tutorial and encourage you to check out the ZITADEL [documentation](/docs) for more information on how to use the ZITADEL platform to its full potential. Thanks for joining us!
|
||||
|
||||
Reference in New Issue
Block a user