docs: introduce versioned docs and migrate to fuma (#11166)

## Todos for release

- [x] Configure Env in docs project on vercel
- [x] Configure Root Path in the docs project on vercel
- [ ] Remove old CSP https://github.com/zitadel/website/pull/1592

## What we did

This pull request migrates the project documentation from the old
`docs/` directory to the new `apps/docs/` directory, introduces a new
documentation system built with Next.js and Fumadocs, and updates all
relevant references, configuration files, and documentation to reflect
this change. It also adds new configuration and ignore files for the new
documentation app, updates CI and linting to exclude the new docs from
certain checks, and revises the contributing guidelines accordingly.

**Documentation System Migration and New Docs App**

* Migrated all documentation from `docs/` to `apps/docs/`, and updated
all references in `README.md`, `CONTRIBUTING.md`, and other files to
point to the new location.
[[1]](diffhunk://#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055L585-L640)
[[2]](diffhunk://#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055L660-R607)
[[3]](diffhunk://#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055L740-R687)
[[4]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L2-R3)
[[5]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L30-R30)
* Added a new Next.js/Fumadocs-based documentation app under
`apps/docs/`, including core app files, layouts, routing, search API,
and a comprehensive `README.md` with development and contribution
instructions.
[[1]](diffhunk://#diff-5a1b07344a2c1b4d3f37b23ff1388b62cd9f57dea3c1cd23d8a0412b7602b132R1-R74)
[[2]](diffhunk://#diff-462b9ad1eabbb7d1c29bb9c36e4931eb180fd7190900e6d2babf8f4d66ad1c28R1-R39)
[[3]](diffhunk://#diff-e16ae25660ded787b10ac35dea96d5ecaacf895dae0afc8a9bd4382dc79a8c87R1-R7)
[apps/docs/app/[[...slug]]/layout.tsxR1-R81](diffhunk://#diff-59e08acde4e805b7aeccef1dcf98f1d71dfc550777e6b9402085cee0e9fa4e0aR1-R81),
[apps/docs/app/[[...slug]]/page.tsxR1-R79](diffhunk://#diff-e5df3f80d0fa01e12d63d81f29c57a9d14e846c78fd3beabb2ef768e38fd9580R1-R79),
[[4]](diffhunk://#diff-389b34918e040cacaa87cd7201ffa462cc2b0b716736f537e3d3c660ac69353bR1-R7)
[[5]](diffhunk://#diff-d3b03416d1c457b19f1c27b26f2db741412df841b875c26ccadc36e4522247f4R1-R29)
[[6]](diffhunk://#diff-c8fb8339570a5305809be7c618e14705fd86390dc278e6ce8ba224a7bc8b0c3cR1-R25)

**Configuration and Tooling Updates**

* Updated `.github/workflows/codeql.yml`, `.golangci.yaml`, and
`.github/dependabot.yml` to properly handle the new docs app: excluded
`apps/docs` from certain checks, added npm dependency updates for the
docs app, and excluded generated content.
[[1]](diffhunk://#diff-12783128521e452af0cfac94b99b8d250413c516ec71fe6d97dbea666ff7ba27L8-R14)
[[2]](diffhunk://#diff-9917ddc9f1c3304218f7269265b746d997c5c0615478177b5fceecd33ef47cb5R5-R6)
[[3]](diffhunk://#diff-9917ddc9f1c3304218f7269265b746d997c5c0615478177b5fceecd33ef47cb5R126-R129)
[[4]](diffhunk://#diff-dd4fbda47e51f1e35defb9275a9cd9c212ecde0b870cba89ddaaae65c5f3cd28R89-R106)
* Updated `.devcontainer/devcontainer.json` to use the latest Go 1.25.3
version for consistency.

**Licensing and Miscellaneous**

* Added `apps/docs/` to the list of licensed directories in
`LICENSING.md`.

These changes ensure the documentation is now maintained in a modern,
scalable system and all project tooling is updated to support the new
structure.

---------

Co-authored-by: Federico Coppede <[email protected]>
This commit is contained in:
Florian Forster
2026-01-29 08:38:36 -08:00
committed by GitHub
co-authored by Federico Coppede
parent 7b6395c348
commit d8ea041c56
2826 changed files with 78739 additions and 23571 deletions
@@ -0,0 +1 @@
*.pat
@@ -0,0 +1 @@
Visit http://localhost:8080/ui/[email protected] and enter `Password1!` to log in.
@@ -0,0 +1,8 @@
## Disclaimer
This guide is for development / demonstration purpose only and does NOT reflect a production setup.
Things such as TLS termination and email verification will not be available unless you
- Use an API gateway with valid certificates in front of the service
- Configure an appropriate email server
@@ -0,0 +1,8 @@
## What's next
For running a production grade ZITADEL instance in your environment, go on with the [configure ZITADEL](/self-hosting/manage/configure) section.
<Callout type="warn">
The ZITADEL management console [requires end-to-end HTTP/2 support](/self-hosting/manage/http2)
</Callout>
@@ -0,0 +1,89 @@
---
title: Set up Zitadel with Docker Compose
sidebar_label: Docker Compose
---
import CodeBlock from '@/components/docusaurus/code-block';
import NoteInstanceNotFound from './troubleshooting/_note_instance_not_found.mdx';
import DockerComposeSource from "./docker-compose.yaml"
import Disclaimer from './_disclaimer.mdx'
import DefaultUser from './_defaultuser.mdx'
import Next from './_next.mdx'
This guide is the entrypoint for running the Zitadel platform locally for the first time.
It is for demonstration and development purposes only and does not set up a production-ready and security hardened instance of Zitadel.
Once Zitadel up is and running, learn more about a production setup in the [What's Next section](#next-steps).
## Prerequisites
The setup is likely to work with other software versions too, but it is tested against the following environment:
- Ubuntu 24.04.1
- Docker 28.3.2
- Docker Compose v2.38.2
- curl 8.5.0
## Docker compose
The following commands set up services for a PostgreSQL database, a Zitadel API and a Zitadel login:
1. Download the `docker-compose.yaml` file from the Zitadel repository:
```shell
curl -L https://raw.githubusercontent.com/zitadel/zitadel/main/docs/self-hosting/deploy/docker-compose.yaml -o docker-compose.yaml
```
2. Make sure the containers use the latest image versions.
```shell
docker compose pull
```
3. Run the PostgreSQL database, the Zitadel API, and the Zitadel login.
```shell
docker compose up --detach --wait
```
4. Verify the containers are running and healthy.
```shell
docker compose ps
```
<DefaultUser components={props.components} />
## What's next
Before proceeding, review the downloaded `docker-compose.yaml` file:
The comments give context to the used variables and show examples for commonly used configuration variants.
<details>
<summary>docker-compose.yaml</summary>
<CodeBlock language="yaml">{DockerComposeSource}</CodeBlock>
</details>
Here are some natural steps forward to go from the current setup to production:
- **Use a different master key**: Set up an instance from scratch and use a different master key. For example, generate one by running `tr -dc A-Za-z0-9 </dev/urandom | head -c 32`
- **Use files for secrets instead of environment variables, and don't commit them to Git**: Learn how to [configure the platform](/self-hosting/manage/configure)
- **Run the containers as non-root users**: Remove the `user: "0"` lines from the `docker-compose.yaml` file.
- **Configure a different external domain or IP**: Beware that the login only works with HTTPS on non-localhost domains. [Run and configure a reverse proxy](/self-hosting/manage/reverseproxy/reverse_proxy)
- **Serve the API and the UI together on the same port**: [Run and configure a reverse proxy](/self-hosting/manage/reverseproxy/reverse_proxy)
- **Encrypt Traffic**: [Run and configure a reverse proxy](/self-hosting/manage/reverseproxy/reverse_proxy)
For more detailed recommendations, read the [Zitadel Production Setup Guide](/self-hosting/manage/production).
<details>
<summary>Read this if the environment uses the login v1</summary>
<p>
The login v2 is the next generation of the Zitadel login.
Its code and deployment are easily customizable.
Unlike the login v1, it runs in its own process.
The login v2 is enabled by default in new installations.
But if an existing `docker-compose.yaml` file is updated by following this guide,
an additional `zitadel-login` service is now running which is actually not doing anything, yet.
The following steps give guidance on how to enable the login v2 in an environment.
</p>
<ol>
<li>Read the login v2 related comments in the docker-compose.yaml.</li>
<li><a href="../../self-hosting/manage/login-client#create-login-client">Manually create a login client personal access token (PAT) for the now running v2 login</a>.</li>
<li>Move the PAT to `./login-client.pat`</li>
<li>Restart the login process: `docker compose restart login`</li>
<li><a href="../../self-hosting/manage/login-client#require-login-v2">Enable the Login UI for all users</a></li>
</ol>
</details>
@@ -0,0 +1,32 @@
---
title: Developing ZITADEL with Dev Containers
sidebar_label: Dev Containers
---
Dev containers provide a convenient way to set up a development environment for ZITADEL with all the necessary dependencies pre-configured. This allows you to start contributing or working on ZITADEL locally with minimal setup.
## Prerequisites
- Docker installed on your machine. You can find installation instructions for Docker on their official website: https://docs.docker.com/engine/install/
- A code editor or IDE with remote container development capabilities (optional, but recommended). [Visual Studio Code](https://code.visualstudio.com) with the [Remote Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) is a popular option.
## Setting Up Dev Container
ZITADEL provides a `.devcontainer` folder that configures the development environment within a container. Here's how to get started:
1. Clone the ZITADEL repository from GitHub:
```bash
git clone https://github.com/zitadel/zitadel.git
```
2. Navigate to the project directory:
```bash
cd zitadel
```
3. Open the project in your code editor or IDE (if using one with remote container support).
4. Follow the instructions provided by your code editor/IDE to initiate the development container. This typically involves opening the "Command Palette" or similar functionality and searching for commands related to "Dev Containers" or "Remote Containers". The quick start guide for VS Code can found [here](https://code.visualstudio.com/docs/devcontainers/containers#_quick-start-open-an-existing-folder-in-a-container)
**Note**: The first time you run this command, it might take some time to download the container image.
## Using Dev Container
Once the container is running, you will have a development environment set up with all the necessary dependencies pre-installed. You can then follow the instructions in the ZITADEL [contribution guide](https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#developing-zitadel-with-dev-containers) to build and run ZITADEL, or develop the ZITADEL Management Console application.
@@ -0,0 +1,136 @@
services:
zitadel:
restart: unless-stopped
image: ghcr.io/zitadel/zitadel:latest
command: start-from-init --masterkey "MasterkeyNeedsToHave32Characters"
environment:
# See "What's next" to learn about how to serve Zitadel on a different domain or IP.
ZITADEL_EXTERNALDOMAIN: localhost
# See "What's next" to learn about how to enable TLS.
ZITADEL_EXTERNALSECURE: false
ZITADEL_TLS_ENABLED: false
# Database connection settings.
ZITADEL_DATABASE_POSTGRES_HOST: db
ZITADEL_DATABASE_POSTGRES_PORT: 5432
# The database is created by the init job if it does not exist.
ZITADEL_DATABASE_POSTGRES_DATABASE: zitadel
# The admin user must already exist in the database.
ZITADEL_DATABASE_POSTGRES_ADMIN_USERNAME: postgres
ZITADEL_DATABASE_POSTGRES_ADMIN_PASSWORD: postgres
ZITADEL_DATABASE_POSTGRES_ADMIN_SSL_MODE: disable
# The zitadel user is created by the init job if it does not exist.
ZITADEL_DATABASE_POSTGRES_USER_USERNAME: zitadel
ZITADEL_DATABASE_POSTGRES_USER_PASSWORD: zitadel
ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE: disable
# By configuring a login client, the setup job creates a user of type machine with the role IAM_LOGIN_CLIENT.
# It writes a PAT to the path specified in ZITADEL_FIRSTINSTANCE_LOGINCLIENTPATPATH.
# The PAT is passed to the login container via the environment variable ZITADEL_SERVICE_USER_TOKEN_FILE.
ZITADEL_FIRSTINSTANCE_LOGINCLIENTPATPATH: /current-dir/login-client.pat
ZITADEL_FIRSTINSTANCE_ORG_HUMAN_PASSWORDCHANGEREQUIRED: false
ZITADEL_FIRSTINSTANCE_ORG_LOGINCLIENT_MACHINE_USERNAME: login-client
ZITADEL_FIRSTINSTANCE_ORG_LOGINCLIENT_MACHINE_NAME: Automatically Initialized IAM_LOGIN_CLIENT
ZITADEL_FIRSTINSTANCE_ORG_LOGINCLIENT_PAT_EXPIRATIONDATE: '2029-01-01T00:00:00Z'
# Activate the login v2 on an installation from scratch.
# To activate the login v2 on an existing installation, read the "What's next" section.
ZITADEL_DEFAULTINSTANCE_FEATURES_LOGINV2_REQUIRED: true # To use the login v1, set this to false.
ZITADEL_DEFAULTINSTANCE_FEATURES_LOGINV2_BASEURI: http://localhost:3000/ui/v2/login
# Configure the redirection paths to the login v2.
ZITADEL_OIDC_DEFAULTLOGINURLV2: http://localhost:3000/ui/v2/login/login?authRequest=
ZITADEL_OIDC_DEFAULTLOGOUTURLV2: http://localhost:3000/ui/v2/login/logout?post_logout_redirect=
ZITADEL_SAML_DEFAULTLOGINURLV2: http://localhost:3000/ui/v2/login/login?samlRequest=
# By configuring a machine, the setup job creates a user of type machine with the role IAM_OWNER.
# It writes a personal access token (PAT) to the path specified in ZITADEL_FIRSTINSTANCE_PATPATH.
# The PAT can be used to provision resources with [Terraform](/guides/manage/terraform-provider), for example.
# ZITADEL_FIRSTINSTANCE_PATPATH: /current-dir/admin.pat
# ZITADEL_FIRSTINSTANCE_ORG_MACHINE_MACHINE_USERNAME: admin
# ZITADEL_FIRSTINSTANCE_ORG_MACHINE_MACHINE_NAME: Automatically Initialized IAM_OWNER
# ZITADEL_FIRSTINSTANCE_ORG_MACHINE_PAT_EXPIRATIONDATE: '2029-01-01T00:00:00Z'
# To change the initial human admin users username and password, uncomment the following lines.
# The first login name is formatted like this: <username>@<org_name>.<external_domain>
# With the following incommented configuration, this would be [email protected]
# Visit http://localhost:8080/ui/console to check if the login name works.
# If you can't log in, check the available login names:
# echo "select * from projections.login_names3;" | psql -h localhost -U postgres -d zitadel
# The postgres users password is postgres.
# ZITADEL_FIRSTINSTANCE_ORG_NAME: My Organization
# ZITADEL_FIRSTINSTANCE_ORG_HUMAN_USERNAME: root
# ZITADEL_FIRSTINSTANCE_ORG_HUMAN_PASSWORD: RootPassword1!
# Enable debug logs
# ZITADEL_LOG_LEVEL: debug
# Write Access Logs to stdout.
# ZITADEL_LOGSTORE_ACCESS_STDOUT_ENABLED: true
healthcheck:
test:
- CMD
- /app/zitadel
- ready
interval: 10s
timeout: 60s
retries: 5
start_period: 10s
user: "0"
volumes:
- .:/current-dir:delegated
ports:
- 8080:8080
- 3000:3000
networks:
- zitadel
depends_on:
db:
condition: service_healthy
login:
restart: unless-stopped
image: ghcr.io/zitadel/zitadel-login:latest
# If you can't use the network_mode service:zitadel, you can pass the environment variables ZITADEL_API_URL=http://zitadel:8080 and CUSTOM_REQUEST_HEADERS=Host:localhost instead.
environment:
- ZITADEL_API_URL=http://localhost:8080
- NEXT_PUBLIC_BASE_PATH=/ui/v2/login
- ZITADEL_SERVICE_USER_TOKEN_FILE=/current-dir/login-client.pat
network_mode: service:zitadel
user: "0"
volumes:
- .:/current-dir:ro
depends_on:
zitadel:
condition: service_healthy
restart: false
db:
restart: unless-stopped
image: postgres:17
environment:
PGUSER: postgres
POSTGRES_PASSWORD: postgres
healthcheck:
test:
- CMD-SHELL
- pg_isready
- -d
- zitadel
- -U
- postgres
interval: 10s
timeout: 30s
retries: 5
start_period: 20s
networks:
- zitadel
ports:
- 5432:5432
volumes:
- 'data:/var/lib/postgresql/data:rw'
networks:
zitadel:
volumes:
data:
@@ -0,0 +1,8 @@
---
title: Set up ZITADEL on Kubernetes
sidebar_label: Kubernetes
---
For getting started with an easily testable insecure setup with Postgres, follow the [Insecure Postgres Example](https://github.com/zitadel/zitadel-charts/tree/main/examples/1-postgres-insecure).
For more information and configuration examples, go to the [ZITADEL charts repo](https://github.com/zitadel/zitadel-charts).
@@ -0,0 +1,73 @@
---
title: Install Zitadel on Linux
sidebar_label: Linux
---
import Disclaimer from "./_disclaimer.mdx";
import DefaultUser from "./_defaultuser.mdx";
import Next from "./_next.mdx";
import NoteInstanceNotFound from "./troubleshooting/_note_instance_not_found.mdx";
<Callout type="warn">
Be aware! This guide does not work for the newly updated version of Zitadel 4! [Learn more](https://github.com/zitadel/zitadel/issues/10526)
Please use Docker version if you want to use Zitadel 4.
</Callout>
## Install PostgreSQL
Download a `postgresql` binary as described [in the PostgreSQL docs](https://www.postgresql.org/download/linux/).
Zitadel is tested against PostgreSQL latest stable tag and latest Ubuntu LTS.
## Run PostgreSQL
```bash
sudo systemctl start postgresql
sudo systemctl enable postgresql
```
## Install Zitadel
Download the Zitadel release according to your architecture from [Github](https://github.com/zitadel/zitadel/releases/latest), unpack the archive and copy zitadel binary to /usr/local/bin
```bash
LATEST=$(curl -i https://github.com/zitadel/zitadel/releases/latest | grep location: | cut -d '/' -f 8 | tr -d '\r'); ARCH=$(uname -m); case $ARCH in armv5*) ARCH="armv5";; armv6*) ARCH="armv6";; armv7*) ARCH="arm";; aarch64) ARCH="arm64";; x86) ARCH="386";; x86_64) ARCH="amd64";; i686) ARCH="386";; i386) ARCH="386";; esac; wget -c https://github.com/zitadel/zitadel/releases/download/$LATEST/zitadel-linux-$ARCH.tar.gz -O - | tar -xz && sudo mv zitadel-linux-$ARCH/zitadel /usr/local/bin
```
## Run Zitadel
```bash
ZITADEL_DATABASE_POSTGRES_HOST=localhost ZITADEL_DATABASE_POSTGRES_PORT=5432 ZITADEL_DATABASE_POSTGRES_DATABASE=zitadel ZITADEL_DATABASE_POSTGRES_USER_USERNAME=zitadel ZITADEL_DATABASE_POSTGRES_USER_PASSWORD=zitadel ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE=disable ZITADEL_DATABASE_POSTGRES_ADMIN_USERNAME=root ZITADEL_DATABASE_POSTGRES_ADMIN_PASSWORD=postgres ZITADEL_DATABASE_POSTGRES_ADMIN_SSL_MODE=disable ZITADEL_EXTERNALSECURE=false zitadel start-from-init --masterkey "MasterkeyNeedsToHave32Characters" --tlsMode disabled
```
<DefaultUser components={props.components} />
<NoteInstanceNotFound components={props.components} />
## VideoGuide
<iframe
width="100%"
height="315"
src="https://www.youtube.com/embed/YVLua-q7dbs"
title="YouTube video player"
frameborder="0"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
allowfullscreen
></iframe>
### Setup Zitadel with a service account
```bash
ZITADEL_DATABASE_POSTGRES_HOST=localhost ZITADEL_DATABASE_POSTGRES_PORT=5432 ZITADEL_DATABASE_POSTGRES_DATABASE=zitadel ZITADEL_DATABASE_POSTGRES_USER_USERNAME=zitadel ZITADEL_DATABASE_POSTGRES_USER_PASSWORD=zitadel ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE=disable ZITADEL_DATABASE_POSTGRES_ADMIN_USERNAME=root ZITADEL_DATABASE_POSTGRES_ADMIN_SSL_MODE=disable ZITADEL_EXTERNALSECURE=false ZITADEL_FIRSTINSTANCE_MACHINEKEYPATH=/tmp/zitadel-admin-sa.json ZITADEL_FIRSTINSTANCE_ORG_MACHINE_MACHINE_USERNAME=zitadel-admin-sa ZITADEL_FIRSTINSTANCE_ORG_MACHINE_MACHINE_NAME=Admin ZITADEL_FIRSTINSTANCE_ORG_MACHINE_MACHINEKEY_TYPE=1 zitadel start-from-init --masterkey "MasterkeyNeedsToHave32Characters" --tlsMode disabled
# then you can move your machine key
mv /tmp/zitadel-admin-sa.json $HOME/zitadel-admin-sa.json
```
This key can be used to provision resources with for example [Terraform](/guides/manage/terraform-provider).
<Next components={props.components} />
<Disclaimer components={props.components} />
@@ -0,0 +1,68 @@
---
title: Install ZITADEL on MacOS
sidebar_label: MacOS
---
import NoteInstanceNotFound from './troubleshooting/_note_instance_not_found.mdx';
import Disclaimer from './_disclaimer.mdx'
import DefaultUser from './_defaultuser.mdx'
import Next from './_next.mdx'
## Install PostgreSQL
Download a `postgresql` binary as described [in the PostgreSQL docs](https://www.postgresql.org/download/macosx/).
ZITADEL is tested against PostgreSQL latest stable tag and latest Ubuntu LTS.
## Run PostgreSQL
```bash
sudo pg_ctl init -D /tmp/postgresql
sudo pg_ctl start -D /tmp/postgresql
```
## Install ZITADEL
### Homebrew
```bash
brew install zitadel/tap/zitadel
```
### Download from GitHub
Download the ZITADEL release according to your architecture from [Github](https://github.com/zitadel/zitadel/releases/latest)
## Unpack the archive
move to your download location and unpack the archive
```bash
#unpack and copy to /usr/local/bin
LATEST=$(curl -i https://github.com/zitadel/zitadel/releases/latest | grep location: | cut -d '/' -f 8 | tr -d '\r'); wget -qO- https://github.com/zitadel/zitadel/releases/download/$LATEST/zitadel_Darwin_$(uname -m).tar.gz | tar -xJ zitadel && sudo mv zitadel /usr/local/bin
```
## Run ZITADEL
```bash
ZITADEL_DATABASE_POSTGRES_HOST=localhost ZITADEL_DATABASE_POSTGRES_PORT=5432 ZITADEL_DATABASE_POSTGRES_DATABASE=zitadel ZITADEL_DATABASE_POSTGRES_USER_USERNAME=zitadel ZITADEL_DATABASE_POSTGRES_USER_PASSWORD=zitadel ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE=disable ZITADEL_DATABASE_POSTGRES_ADMIN_USERNAME=$(whoami) ZITADEL_DATABASE_POSTGRES_ADMIN_PASSWORD=postgres ZITADEL_DATABASE_POSTGRES_ADMIN_SSL_MODE=disable ZITADEL_EXTERNALSECURE=false zitadel start-from-init --masterkey "MasterkeyNeedsToHave32Characters" --tlsMode disabled
```
<DefaultUser components={props.components} />
<NoteInstanceNotFound components={props.components} />
## VideoGuide
<iframe width="100%" height="315" src="https://www.youtube.com/embed/Ob-kBzmezeg" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
### Setup ZITADEL with a service account
```bash
ZITADEL_DATABASE_POSTGRES_HOST=localhost ZITADEL_DATABASE_POSTGRES_PORT=5432 ZITADEL_DATABASE_POSTGRES_DATABASE=zitadel ZITADEL_DATABASE_POSTGRES_USER_USERNAME=zitadel ZITADEL_DATABASE_POSTGRES_USER_PASSWORD=zitadel ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE=disable ZITADEL_DATABASE_POSTGRES_ADMIN_USERNAME=$(whoami) ZITADEL_DATABASE_POSTGRES_ADMIN_SSL_MODE=disable ZITADEL_EXTERNALSECURE=false ZITADEL_FIRSTINSTANCE_MACHINEKEYPATH=/tmp/zitadel-admin-sa.json ZITADEL_FIRSTINSTANCE_ORG_MACHINE_MACHINE_USERNAME=zitadel-admin-sa ZITADEL_FIRSTINSTANCE_ORG_MACHINE_MACHINE_NAME=Admin ZITADEL_FIRSTINSTANCE_ORG_MACHINE_MACHINEKEY_TYPE=1 zitadel start-from-init --masterkey "MasterkeyNeedsToHave32Characters" --tlsMode disabled
# then you can move your machine key
mv /tmp/zitadel-admin-sa.json $HOME/zitadel-admin-sa.json
```
This key can be used to provision resources with for example [Terraform](/guides/manage/terraform-provider).
<Next components={props.components} />
<Disclaimer components={props.components} />
@@ -0,0 +1,33 @@
---
title: Deploy ZITADEL
sidebar_label: Overview
---
Choose your platform and run ZITADEL with the most minimal configuration possible.
- [Linux](./linux)
- [MacOS](./macos)
- [Docker Compose](./compose)
- [Kubernetes](./kubernetes)
## Prerequisites
- For test environments, ZITADEL does not need many resources, 1 CPU and 512MB memory are more than enough. (With more CPU, the password hashing might be faster)
- A PostgreSQL as only needed storage. Make sure to read our [Production Guide](/self-hosting/manage/production#prefer-postgre-sql) before you decide to use Postgresql.
## Releases
The easiest way to use ZITADEL is to run one of our container releases
- ZITADEL does provide latest and stable [container images](https://github.com/zitadel/zitadel/pkgs/container/zitadel)
- **stable** is the current **production** release of ZITADEL.
- **latest** is the **last created** release from our pipelines that gets updated in a high frequency.
# Production Setup
After you have successfully created your first test environment using one of the deployment guides in this section,
you might want to configure ZITADEL for production and embed it into your system landscape.
To do so, jump straight to the [production setup guide](/self-hosting/manage/production).
To achieve high availability, we recommend using a [Kubernetes](https://kubernetes.io/docs/home/) Cluster.
We have an official [Helm chart](https://artifacthub.io/packages/helm/zitadel/zitadel) for easy deployment and maintenance.
@@ -0,0 +1,5 @@
`ID=QUERY-n0wng Message=Instance not found`
If you're self hosting with a Custom Domain, you need to instruct ZITADEL to use the `ExternalDomain`.
You can find further instructions in our guide about [Custom Domains](/self-hosting/manage/custom-domain).
We also provide a guide on how to [configure](/self-hosting/manage/configure) ZITADEL with variables from files or environment variables.
@@ -0,0 +1,5 @@
<Callout>
This guide is based on a local setup.
If you encounter an error "Instance Not Found" please read the following section:
[Instance not found](/self-hosting/deploy/troubleshooting#instance-not-found)
</Callout>
@@ -0,0 +1,37 @@
If you operate Zitadel behind a Reverse Proxy or Ingress inside a Kubernetes cluster,
you may encounter an Error like `upstream sent too big header while reading response header from upstream`
in your NGINX Logs and receive a 403 Error when accessing NGINX.
you can solve it by increasing the grpc buffer size in your nginx config:
### Ingress NGINX
```yaml
ingress:
enabled: true
annotations:
nginx.ingress.kubernetes.io/modsecurity-snippet: |
SecRuleRemoveById 949110
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
nginx.ingress.kubernetes.io/configuration-snippet: |
grpc_set_header Host $host;
more_clear_input_headers "Host" "X-Forwarded-Host";
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $http_x_forwarded_host;
# highlight-next-line
nginx.ingress.kubernetes.io/server-snippet: "grpc_buffer_size 8k;"
```
### NGINX Config
```nginx
http {
server {
listen 80;
http2 on;
location / {
grpc_pass grpc://zitadel-disabled-tls:8080;
grpc_set_header Host $host:$server_port;
# highlight-next-line
grpc_buffer_size 8k;
}
}
}
```
@@ -0,0 +1,15 @@
---
title: Troubleshoot ZITADEL
---
import InstanceNotFound from "../../../self-hosting/deploy/troubleshooting/_instance_not_found.mdx";
import UpstreamHeader from "../../../self-hosting/deploy/troubleshooting/_upstream_header.mdx"
## Instance not found
<InstanceNotFound components={props.components} />
## upstream sent too big header while reading response header from upstream
<UpstreamHeader components={props.components} />
@@ -0,0 +1,3 @@
**/*.pat
**/selfsigned.crt
**/selfsigned.key
@@ -0,0 +1,254 @@
---
title: Caches
sidebar_label: Caches [Beta]
---
ZITADEL supports the use of a caches to speed up the lookup of frequently needed objects. As opposed to HTTP caches which might reside between ZITADEL and end-user applications, the cache build into ZITADEL uses active invalidation when an object gets updated. Another difference is that HTTP caches only cache the result of a complete request and the built-in cache stores objects needed for the internal business logic. For example, each request made to ZITADEL needs to retrieve and set [instance](/concepts/structure/instance) information in middleware.
<Callout type="info">
Caches is currently an [experimental beta](https://help.zitadel.com/zitadel-software-release-cycle#beta) feature.
Test the feature and add improvement or bug reports directly to the [github repository](https://github.com/zitadel/zitadel) or let us know your general feedback in the [discord thread](https://discord.com/channels/927474939156643850/1332343909900222506)!
</Callout>
## Configuration
The `Caches` configuration entry defines *connectors* which can be used by several objects. It is possible to mix *connectors* with different objects based on operational needs.
```yaml
Caches:
Connectors:
SomeConnector:
Enabled: true
SomeOption: foo
SomeObject:
# Connector must be enabled above.
# When connector is empty, this cache will be disabled.
Connector: "SomeConnector"
MaxAge: 1h
LastUsage: 10m
# Log enables cache-specific logging. Default to error log to stderr when omitted.
Log:
Level: error
```
For a full configuration reference, please see the [runtime configuration file](/self-hosting/manage/configure/configure#runtime-configuration-file) section's `defaults.yaml`.
## Connectors
ZITADEL supports a number of *connectors*. Connectors integrate a cache with a storage backend. Users can combine connectors with the type of object cache depending on their operational and performance requirements.
When no connector is specified for an object cache, then no caching is performed. This is the current default.
### Auto prune
Some connectors take an `AutoPrune` option. This is provided for caches which don't have built-in expiry and cleanup routines. The auto pruner is a routine launched by ZITADEL and scans and removes outdated objects in the cache. Pruning can take a cost as they typically involve some kind of scan. However, using a long interval can cause higher storage utilization.
```yaml
Caches:
Connectors:
Memory:
Enabled: true
# AutoPrune removes invalidated or expired object from the cache.
AutoPrune:
Interval: 1m
TimeOut: 5s
```
### Redis cache
Redis is supported in simple mode. Cluster and Sentinel are not yet supported. There is also a circuit-breaker provided which prevents a single point of failure, should the single Redis instance become unavailable.
Benefits:
- Centralized cache with single source of truth
- Consistent invalidation
- Very fast when network latency is kept to a minimum
- Built-in object expiry, no pruner required
Drawbacks:
- Increased operational overhead: need to run a Redis instance as part of your infrastructure.
- When running multiple servers of ZITADEL in different regions, network roundtrip time might impact performance, neutralizing the benefit of a cache.
#### Circuit breaker
A [circuit breaker](https://learn.microsoft.com/en-us/previous-versions/msp-n-p/dn589784(v=pandp.10)?redirectedfrom=MSDN) is provided for the Redis connector, to prevent a single point of failure in the case persistent errors. When the circuit breaker opens, the cache is temporary disabled and ignored. ZITADEL will continue to operate using queries to the database.
```yaml
Caches:
Connectors:
Redis:
Enabled: true
Addr: localhost:6379
# Many other options...
CircuitBreaker:
# Interval when the counters are reset to 0.
# 0 interval never resets the counters until the CB is opened.
Interval: 0
# Amount of consecutive failures permitted
MaxConsecutiveFailures: 5
# The ratio of failed requests out of total requests
MaxFailureRatio: 0.1
# Timeout after opening of the CB, until the state is set to half-open.
Timeout: 60s
# The allowed amount of requests that are allowed to pass when the CB is half-open.
MaxRetryRequests: 1
```
### PostgreSQL cache
PostgreSQL can be used to store objects in unlogged tables. [Unlogged tables](https://www.postgresql.org/docs/current/sql-createtable.html#SQL-CREATETABLE-UNLOGGED) do not write to the WAL log and are therefore faster than regular tables. If the PostgreSQL server crashes, the data from those tables are lost. ZITADEL always creates the cache schema in the `zitadel` database during [setup](./updating_scaling#the-setup-phase). This connector requires a [pruner](#auto-prune) routine.
Benefits:
- Centralized cache with single source of truth
- No operational overhead. Reuses the query connection pool and the existing `zitadel` database.
- Consistent invalidation
- Faster than regular queries which often contain `JOIN` clauses.
Drawbacks:
- Slowest of the available caching options
- Might put additional strain on the database server, limiting horizontal scalability
### Local memory cache
ZITADEL is capable of caching object in local application memory, using hash-maps. Each ZITADEL server manages its own copy of the cache. This connector requires a [pruner](#auto-prune) routine.
Benefits:
- Fastest of the available caching options
- No operational overhead
Drawbacks:
- Inconsistent invalidation. An object validated in one ZITADEL server will not get invalidated in other servers.
- There's no single source of truth. Different servers may operate on a different version of an object
- Data is duplicated in each server, consuming more total memory inside a deployment.
The drawbacks restricts its usefulness in distributed deployments. However simple installations running a single server can benefit greatly from this type of cache. For example test, development or home deployments.
If inconsistency is acceptable for short periods of time, one can choose to use this type of cache in distributed deployments with short max age configuration.
**For example**: A ZITADEL deployment with 2 servers is serving 1000 req/sec total. The installation only has one instance[^1]. There is only a small amount of data cached (a few kB) so duplication is not a problem in this case. It is acceptable for [instance level setting](/guides/manage/console/default-settings) to be out-dated for a short amount of time. When the memory cache is enabled for the instance objects, with a max age of 1 second, the instance only needs to be obtained from the database 2 times per second (once for each server). Saving 998 of redundant queries. Once an instance level setting is changed, it takes up to 1 second for all the servers to get the new state.
## Objects
The following section describes the type of objects ZITADEL can currently cache. Objects are actively invalidated at the cache backend when one of their properties is changed. Each object cache defines:
- `Connector`: Selects the used [connector](#connectors) back-end. Must be activated first.
- `MaxAge`: the amount of time that an object is considered valid. When this age is passed the object is ignored (cache miss) and possibly cleaned up by the [pruner](#auto-prune) or other built-in garbage collection.
- `LastUsage`: defines usage based lifetime. Each time an object is used, its usage timestamp is updated. Popular objects remain cached, while unused objects are cleaned up. This option can be used to indirectly limit the size of the cache.
- `Log`: allows specific log settings for the cache. This can be used to debug a certain cache without having to change the global log level.
```yaml
Caches:
SomeObject:
# Connector must be enabled above.
# When connector is empty, this cache will be disabled.
Connector: ""
MaxAge: 1h
LastUsage: 10m
# Log enables cache-specific logging. Default to error log to stderr when omitted.
Log:
Level: error
AddSource: true
Formatter:
Format: text
```
### Instance
All HTTP and gRPC requests sent to ZITADEL receive an instance context. The instance is usually resolved by the domain from the request. In some cases, like the [system service](/reference/api/system), the instance can be resolved by its ID. An instance object contains many of the [default settings](/reference/api/system), the instance can be resolved by its ID. An instance object contains many of the [default settings](/guides/manage/console/default-settings):
- Instance [features](/guides/manage/console/default-settings#features)
- Custom Domains: generated and [custom](/guides/manage/cloud/instances#add-custom-domain)
- [Trusted domains](/apis/resources/admin/admin-service-add-instance-trusted-domain)
- Security settings ([IFrame policy](/guides/solution-scenarios/configurations#embedding-zitadel-in-an-iframe))
- Limits[^2]
- [Allowed languages](/guides/manage/console/default-settings#languages)
These settings typically change infrequently in production. ***Every*** request made to ZITADEL needs to query for the instance. This is a typical case of set once, get many times where a cache can provide a significant optimization.
### Milestones
Milestones are used to track the administrator's progress in setting up their instance. Milestones are used to render *your next steps* in the [console](/guides/manage/console/console-overview) landing page.
Milestones are reached upon the first time a certain action is performed. For example the first application created or the first human login. In order to push a "reached" event only once, ZITADEL must keep track of the current state of milestones by an eventstore query every time an eligible action is performed. This can cause an unwanted overhead on production servers, therefore they are cached.
As an extra optimization, once all milestones are reached by the instance, an in-memory flag is set and the milestone state is never queried again from the database nor cache.
For single instance setups which fulfilled all milestone (*your next steps* in the management console) it is not needed to enable this cache. We mainly use it for ZITADEL cloud where there are many instances with *incomplete* milestones.
### Organization
Most resources like users, project and applications are part of an [organization](/guides/manage/console/organizations-overview). Therefore many parts of the ZITADEL logic search for an organization by ID or by their primary domain.
Organization objects are quite small and receive infrequent updates after they are created:
- Change of organization name
- Deactivation / Reactivation
- Change of primary domain
- Removal
## Examples
Currently caches are in beta and disabled by default. However, if you want to give caching a try, the following sections contains some suggested configurations for different setups.
The following configuration is recommended for single instance setups with a single ZITADEL server:
```yaml
Caches:
Memory:
Enabled: true
Instance:
Connector: "memory"
MaxAge: 1h
Organization:
Connector: "memory"
MaxAge: 1h
```
The following configuration is recommended for single instance setups with high traffic on multiple servers, where Redis is not available:
```yaml
Caches:
Memory:
Enabled: true
Postgres:
Enabled: true
Instance:
Connector: "memory"
MaxAge: 1s
Milestones:
Connector: "postgres"
MaxAge: 1h
LastUsage: 10m
Organization:
Connector: "memory"
MaxAge: 1s
```
When running many instances on multiple servers:
```yaml
Caches:
Connectors:
Redis:
Enabled: true
# Other connection options
Instance:
Connector: "redis"
MaxAge: 1h
LastUsage: 10m
Milestones:
Connector: "redis"
MaxAge: 1h
LastUsage: 10m
Organization:
Connector: "redis"
MaxAge: 1h
LastUsage: 10m
```
----
[^1]: Many deployments of ZITADEL have only one or few [instances](/concepts/structure/instance). Multiple instances are mostly used for ZITADEL cloud, where each customer gets at least one instance.
[^2]: Limits are imposed by the system API, usually when customers exceed their subscription in ZITADEL cloud.
@@ -0,0 +1,230 @@
---
title: Mirror data from cockroach to postgres
sidebar_label: Mirror command
---
The `mirror` command allows you to do database to database migrations. This functionality is useful to copy data from one database to another.
The data can be mirrored to multiple database without influencing each other.
## Use cases
Migrate from cockroachdb to postgres.
Replicate data to a secondary environment for testing.
## Prerequisites
You need an existing source database, most probably the database Zitadel currently serves traffic from.
To mirror the data, the destination database needs to be initialized and set up without an instance. You can find the commands to start an empty Zitadel deployment in [the example section](#prepare-the-destination-database).
### Start the destination database
Follow one of the following guides to start the database:
* [Linux](/self-hosting/deploy/linux#run-postgre-sql)
* [MacOS](/self-hosting/deploy/macos#install-postgre-sql)
Or use the following commands for [Docker Compose](/self-hosting/deploy/compose)
```bash
# Download the docker compose example configuration.
wget https://raw.githubusercontent.com/zitadel/zitadel/main/docs/self-hosting/deploy/docker-compose.yaml
# Run the database and application containers.
docker compose up db --detach
```
## Example
### Prepare the destination database
The following commands setup the database without an instance.
```bash
zitadel init --config /path/to/your/new/config.yaml
zitadel setup --for-mirror --config /path/to/your/new/config.yaml # make sure to set --tlsMode and masterkey analog to your current deployment
```
### Mirror the data
The next step is to copy the data from the source to the destination database. For detailed configuration options, please refer to the [configuration section](#configuration).
```bash
zitadel mirror --system --config /path/to/your/mirror/config.yaml # make sure to set --tlsMode and masterkey analog to your current deployment
```
### Initialize the data and verify
The last step is to setup the permissions and verify the data, there might be differences between source and destination, refer to [`zitadel mirror verify`](#zitadel-mirror-verify) to get an overview of possible diffs.
```bash
zitadel setup --for-mirror --config /path/to/your/new/config.yaml # make sure to set --tlsMode and masterkey analog to your current deployment
zitadel mirror verify --system --config /path/to/your/mirror/config.yaml # make sure to set --tlsMode and masterkey analog to your current deployment
```
## Usage
The general syntax for the mirror command is:
```bash
zitadel mirror [flags]
Flags:
-h, --help help for mirror
--config stringArray path to config file to overwrite system defaults
--ignore-previous ignores previous migrations of the events table. This flag should be used if you manually dropped previously mirrored events.
--replace replaces all data of the following tables for the provided instances or all if the `--system`-flag is set:
* system.assets
* auth.auth_requests
* eventstore.unique_constraints
The should be provided if you want to execute the mirror command multiple times so that the static data are also mirrored to prevent inconsistent states.
--instance strings id or comma separated ids of the instance(s) to migrate. Either this or the `--system`-flag must be set. Make sure to always use the same flag if you execute the command multiple times.
--system migrates the whole system. Either this or the `--instance`-flag must be set. Make sure to always use the same flag if you execute the command multiple times.
# For the flags below use the same configuration you also use in the current deployment
--masterkey string masterkey as argument for en/decryption keys
-m, --masterkeyFile string path to the masterkey for en/decryption keys
--masterkeyFromEnv read masterkey for en/decryption keys from environment variable (ZITADEL_MASTERKEY)
--tlsMode externalSecure start Zitadel with (enabled), without (disabled) TLS or external component e.g. reverse proxy (external) terminating TLS, this flag will overwrite externalSecure and `tls.enabled` in configs files
```
## Configuration
```yaml
# The source database the data are copied from. Use either cockroach or postgres, by default cockroach is used
Source:
cockroach:
Host: localhost # ZITADEL_SOURCE_COCKROACH_HOST
Port: 26257 # ZITADEL_SOURCE_COCKROACH_PORT
Database: zitadel # ZITADEL_SOURCE_COCKROACH_DATABASE
MaxOpenConns: 6 # ZITADEL_SOURCE_COCKROACH_MAXOPENCONNS
MaxIdleConns: 6 # ZITADEL_SOURCE_COCKROACH_MAXIDLECONNS
MaxConnLifetime: 30m # ZITADEL_SOURCE_COCKROACH_MAXCONNLIFETIME
MaxConnIdleTime: 5m # ZITADEL_SOURCE_COCKROACH_MAXCONNIDLETIME
Options: "" # ZITADEL_SOURCE_COCKROACH_OPTIONS
User:
Username: zitadel # ZITADEL_SOURCE_COCKROACH_USER_USERNAME
Password: "" # ZITADEL_SOURCE_COCKROACH_USER_PASSWORD
SSL:
Mode: disable # ZITADEL_SOURCE_COCKROACH_USER_SSL_MODE
RootCert: "" # ZITADEL_SOURCE_COCKROACH_USER_SSL_ROOTCERT
Cert: "" # ZITADEL_SOURCE_COCKROACH_USER_SSL_CERT
Key: "" # ZITADEL_SOURCE_COCKROACH_USER_SSL_KEY
# Postgres is used as soon as a value is set
# The values describe the possible fields to set values
postgres:
Host: # ZITADEL_SOURCE_POSTGRES_HOST
Port: # ZITADEL_SOURCE_POSTGRES_PORT
Database: # ZITADEL_SOURCE_POSTGRES_DATABASE
MaxOpenConns: # ZITADEL_SOURCE_POSTGRES_MAXOPENCONNS
MaxIdleConns: # ZITADEL_SOURCE_POSTGRES_MAXIDLECONNS
MaxConnLifetime: # ZITADEL_SOURCE_POSTGRES_MAXCONNLIFETIME
MaxConnIdleTime: # ZITADEL_SOURCE_POSTGRES_MAXCONNIDLETIME
Options: # ZITADEL_SOURCE_POSTGRES_OPTIONS
User:
Username: # ZITADEL_SOURCE_POSTGRES_USER_USERNAME
Password: # ZITADEL_SOURCE_POSTGRES_USER_PASSWORD
SSL:
Mode: # ZITADEL_SOURCE_POSTGRES_USER_SSL_MODE
RootCert: # ZITADEL_SOURCE_POSTGRES_USER_SSL_ROOTCERT
Cert: # ZITADEL_SOURCE_POSTGRES_USER_SSL_CERT
Key: # ZITADEL_SOURCE_POSTGRES_USER_SSL_KEY
# The destination database the data are copied to. Use either cockroach or postgres, by default cockroach is used
Destination:
postgres:
Host: localhost # ZITADEL_DATABASE_POSTGRES_HOST
Port: 5432 # ZITADEL_DATABASE_POSTGRES_PORT
Database: zitadel # ZITADEL_DATABASE_POSTGRES_DATABASE
MaxOpenConns: 5 # ZITADEL_DATABASE_POSTGRES_MAXOPENCONNS
MaxIdleConns: 2 # ZITADEL_DATABASE_POSTGRES_MAXIDLECONNS
MaxConnLifetime: 30m # ZITADEL_DATABASE_POSTGRES_MAXCONNLIFETIME
MaxConnIdleTime: 5m # ZITADEL_DATABASE_POSTGRES_MAXCONNIDLETIME
Options: "" # ZITADEL_DATABASE_POSTGRES_OPTIONS
User:
Username: zitadel # ZITADEL_DATABASE_POSTGRES_USER_USERNAME
Password: "" # ZITADEL_DATABASE_POSTGRES_USER_PASSWORD
SSL:
Mode: disable # ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE
RootCert: "" # ZITADEL_DATABASE_POSTGRES_USER_SSL_ROOTCERT
Cert: "" # ZITADEL_DATABASE_POSTGRES_USER_SSL_CERT
Key: "" # ZITADEL_DATABASE_POSTGRES_USER_SSL_KEY
# As cockroachdb first copies the data into memory this parameter is used to iterate through the events table and fetch only the given amount of events per iteration
EventBulkSize: 10000 # ZITADEL_EVENTBULKSIZE
# The maximum duration an auth request was last updated before it gets ignored.
# Default is 30 days
MaxAuthRequestAge: 720h # ZITADEL_MAXAUTHREQUESTAGE
Projections:
# Defines how many projections are allowed to run in parallel
ConcurrentInstances: 7 # ZITADEL_PROJECTIONS_CONCURRENTINSTANCES
# Limits the amount of events projected by each iteration
EventBulkLimit: 1000 # ZITADEL_PROJECTIONS_EVENTBULKLIMIT
Auth:
Spooler:
# Limits the amount of events projected by each iteration
BulkLimit: 1000 #ZITADEL_AUTH_SPOOLER_BULKLIMIT
Admin:
Spooler:
# Limits the amount of events projected by each iteration
BulkLimit: 10 #ZITADEL_ADMIN_SPOOLER_BULKLIMIT
Log:
Level: info
```
## Sub commands
The provided sub commands allow more fine grained execution of copying the data.
The following commands are safe to execute multiple times by adding the `--replace`-flag which replaces the data not provided by the events in the destination database.
### `zitadel mirror auth`
Copies the auth requests to the destination database.
### `zitadel mirror eventstore`
Copies the events since the last migration and unique constraints to the destination database.
### `zitadel mirror projections`
Executes all projections in the destination database.
It is NOOP if the projections are already up-to-date.
### `zitadel mirror system`
Copies encryption keys and assets to the destination database.
### `zitadel mirror verify`
Prints the amount of rows of the source and destination database and the diff. Positive numbers indicate more rows in the destination table that in the source, negative numbers the opposite.
The following tables will likely have an unequal count:
* **projections.current_states**: If your deployment was upgraded several times, the number of entries in the destination will be lower
* **projections.locks**: If your deployment was upgraded several times, the number of entries in the destination will be lower
* **projections.keys4\***: Only not expired keys are inserted, the number of entries in the destination will be lower
* **projections.failed_events**: Should be lower or equal.
* **auth.users2**: Was replaced with auth.users3, the number of entries in the destination will be 0
* **auth.users3**: Is the replacement of auth.users2, the number of entries in the destination will be equal or higher
## Limitations
It is not possible to use files as source or destination. See github issue [here](https://github.com/zitadel/zitadel/issues/7966)
Currently the encryption keys of the source database must be copied to the destination database. See github issue [here](https://github.com/zitadel/zitadel/issues/7964)
It is not possible to change the domain of the Zitadel deployment.
Once you mirrored an instance using the `--instance` flag, you have to make sure you don't mirror other preexisting instances. This means for example, you cannot mirror a few instances and then pass the `--system` flag. You have to pass all remaining instances explicitly, once you used the `--instance` flag
@@ -0,0 +1,34 @@
---
title: ZITADEL Command Line Interface
sidebar_label: Overview
---
This documentation serves as your guide to interacting with Zitadel through the command line interface (CLI). The Zitadel CLI empowers you to manage various aspects of your Zitadel system efficiently from your terminal.
This introductory section provides a brief overview of what the Zitadel CLI offers and who can benefit from using it.
Let's dive in!
## Download the CLI
Download the CLI for [Linux](/self-hosting/deploy/linux#install-zitadel) or [MacOS](/self-hosting/deploy/macos#install-zitadel).
## Quick start
The easiest way to start ZITADEL is by following the [docker compose example](/self-hosting/deploy/compose) which executes the commands for you.
## Initialize the database
The `zitadel init`-command sets up the zitadel database. The statements executed need a user with `ADMIN`-privilege. See [init phase](/self-hosting/manage/updating_scaling#the-init-phase) for more information.
## Setup ZITADEL
The `zitadel setup`-command further sets up the database created using `zitadel init`. This command only requires the user created in the previous step. See [setup phase](/self-hosting/manage/updating_scaling#the-setup-phase) for more information.
## Start ZITADEL
The `zitadel start`-command runs the ZITADEL server. See [runtime phase](/self-hosting/manage/updating_scaling#the-runtime-phase) for more information.
The `zitadel start-from-setup`-command first executes [the setup phase](#setup-zitadel) and afterwards runs the ZITADEL server.
The `zitadel start-from-init`-command first executes [the init phase](#Initialize-the-database), afterwards [the setup phase](#setup-zitadel) and lastly runs the ZITADEL server.
@@ -0,0 +1,2 @@
defaults.yaml
steps.yaml
@@ -0,0 +1,56 @@
---
title: Download the docker compose example configuration.
---
import CodeBlock from '@/components/docusaurus/code-block';
import DockerComposeSource from "./docker-compose.yaml"
import ExampleZITADELConfigSource from "./example-zitadel-config.yaml"
import ExampleZITADELSecretsSource from "./example-zitadel-secrets.yaml"
import ExampleZITADELInitStepsSource from "./example-zitadel-init-steps.yaml"
The docker compose example mounts the example zitadel configuration files to the ZITADEL container.
By executing the commands below, you will download the following files:
<details>
<summary>docker-compose.yaml</summary>
<CodeBlock language="yaml">{DockerComposeSource}</CodeBlock>
</details>
<details>
<summary>example-zitadel-config.yaml</summary>
<CodeBlock language="yaml">{ExampleZITADELConfigSource}</CodeBlock>
</details>
<details>
<summary>example-zitadel-secrets.yaml</summary>
<CodeBlock language="yaml">{ExampleZITADELSecretsSource}</CodeBlock>
</details>
<details>
<summary>example-zitadel-init-steps.yaml</summary>
<CodeBlock language="yaml">{ExampleZITADELInitStepsSource}</CodeBlock>
</details>
```bash
# Download the docker compose example configuration.
wget https://raw.githubusercontent.com/zitadel/zitadel/main/docs/self-hosting/manage/configure/docker-compose.yaml
# Download and adjust the example configuration file containing standard configuration.
wget https://raw.githubusercontent.com/zitadel/zitadel/main/docs/self-hosting/manage/configure/example-zitadel-config.yaml
# Download and adjust the example configuration file containing secret configuration.
wget https://raw.githubusercontent.com/zitadel/zitadel/main/docs/self-hosting/manage/configure/example-zitadel-secrets.yaml
# Download and adjust the example configuration file containing database initialization configuration.
wget https://raw.githubusercontent.com/zitadel/zitadel/main/docs/self-hosting/manage/configure/example-zitadel-init-steps.yaml
# A single ZITADEL instance always needs the same 32 bytes long masterkey
# Generate one to a file if you haven't done so already and pass it as environment variable
LC_ALL=C tr -dc '[:graph:]' </dev/urandom | head -c 32 > ./zitadel-masterkey
export ZITADEL_MASTERKEY="$(cat ./zitadel-masterkey)"
# Run the database and application containers
docker compose up --detach
```
@@ -0,0 +1,6 @@
To run Zitadel on Kubernetes, use [the official Zitadel Helm chart](https://github.com/zitadel/zitadel-charts).
Configure Zitadel using native Helm values.
You can manage secrets through Helm values, letting Helm create Kubernetes secrets.
Alternatively, reference existing Kubernetes secrets managed outside of Helm.
See the [referenced secrets example](https://github.com/zitadel/zitadel-charts/tree/main/examples/3-referenced-secrets) in the charts */examples* folder.
For a quick setup, check out the [insecure Postgres example](https://github.com/zitadel/zitadel-charts/tree/main/examples/1-postgres-insecure).
@@ -0,0 +1,74 @@
---
title: Download and adjust the example configuration file containing standard configuration
---
import CodeBlock from '@/components/docusaurus/code-block';
import ExampleZITADELConfigSource from "./example-zitadel-config.yaml"
import ExampleZITADELSecretsSource from "./example-zitadel-secrets.yaml"
import ExampleZITADELInitStepsSource from "./example-zitadel-init-steps.yaml"
### Configure by Files
By executing the commands below, you will download the following files:
<details>
<summary>example-zitadel-config.yaml</summary>
<CodeBlock language="yaml">{ExampleZITADELConfigSource}</CodeBlock>
</details>
<details>
<summary>example-zitadel-secrets.yaml</summary>
<CodeBlock language="yaml">{ExampleZITADELSecretsSource}</CodeBlock>
</details>
<details>
<summary>example-zitadel-init-steps.yaml</summary>
<CodeBlock language="yaml">{ExampleZITADELInitStepsSource}</CodeBlock>
</details>
```bash
# Download and adjust the example configuration file containing standard configuration
wget https://raw.githubusercontent.com/zitadel/zitadel/main/docs/self-hosting/manage/configure/example-zitadel-config.yaml
# Download and adjust the example configuration file containing secret configuration
wget https://raw.githubusercontent.com/zitadel/zitadel/main/docs/self-hosting/manage/configure/example-zitadel-secrets.yaml
# Download and adjust the example configuration file containing database initialization configuration
wget https://raw.githubusercontent.com/zitadel/zitadel/main/docs/self-hosting/manage/configure/example-zitadel-init-steps.yaml
# A single ZITADEL instance always needs the same 32 characters long masterkey
# If you haven't done so already, you can generate a new one
# The key must be passed as argument
ZITADEL_MASTERKEY="$(LC_ALL=C tr -dc '[:graph:]' </dev/urandom | head -c 32)"
# Pass zitadel configuration by configuration files
zitadel start-from-init \
--config ./example-zitadel-config.yaml \
--config ./example-zitadel-secrets.yaml \
--steps ./example-zitadel-init-steps.yaml \
--masterkey "${ZITADEL_MASTERKEY}"
```
### Configure by Environment Variables
```bash
# Set runtime environment variables
export ZITADEL_DATABASE_POSTGRES_HOST=my.database
export ZITADEL_DATABASE_POSTGRES_PORT=5432
export ZITADEL_DATABASE_POSTGRES_DATABASE=zitadel
export ZITADEL_DATABASE_POSTGRES_USER_USERNAME=my_zitadel_db_user
export ZITADEL_DATABASE_POSTGRES_USER_PASSWORD=Secret_DB_User_Password
export ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE=disable
export ZITADEL_DATABASE_POSTGRES_ADMIN_USERNAME=root
export ZITADEL_DATABASE_POSTGRES_ADMIN_SSL_MODE=disable
export ZITADEL_FIRSTINSTANCE_ORG_HUMAN_USERNAME="root"
export ZITADEL_FIRSTINSTANCE_ORG_HUMAN_PASSWORD="RootPassword1!"
# A single ZITADEL instance always needs the same 32 bytes long masterkey
# Generate one to a file if you haven't done so already and pass it as environment variable
LC_ALL=C tr -dc '[:graph:]' </dev/urandom | head -c 32 > ./zitadel-masterkey
# Let the zitadel binary read configuration from environment variables
zitadel start-from-init --masterkey "${ZITADEL_MASTERKEY}" --tlsMode disabled --masterkeyFile ./zitadel-masterkey
```
@@ -0,0 +1 @@
Open your favorite internet browser at http://localhost:8080/ui/[email protected] and use the password _RootPassword1!_
@@ -0,0 +1,125 @@
---
title: Configuration Options in ZITADEL
sidebar_label: Configuration
---
import LinuxUnix from "./_linuxunix.mdx";
import Compose from "./_compose.mdx";
import Helm from "./_helm.mdx";
import Login from "./_login.mdx";
import CodeBlock from "@/components/docusaurus/code-block";
import DefaultsYamlSource from "../../../../../../cmd/defaults.yaml";
import StepsYamlSource from "../../../../../../cmd/setup/steps.yaml";
import SkipSuperUserDatabaseQueries from "./example-skip-superuser-queries.yaml";
This guide assumes you are familiar with [running ZITADEL using the least amount of configuration possible](/self-hosting/deploy/overview).
## Configuration files
### Runtime configuration file
You can configure the runtime using the `--config` flag of the `zitadel` binary.
Also, you can use the environment variables listed in the defaults.yaml.
<Callout>
For overwriting the default configuration for the first instance created by zitadel setup, use the FirstInstance section in the [database initialization file](#database-initialization-file).
</Callout>
<details>
<summary>defaults.yaml</summary>
<CodeBlock language="yaml">{DefaultsYamlSource}</CodeBlock>
</details>
### Database initialization file
ZITADEL uses a [different configuration file](https://github.com/zitadel/zitadel/blob/main/cmd/setup/steps.yaml) for _database initialization steps_.
Use the `--steps` flag of the `zitadel` binary to provide this configuration file.
Also, you can use the environment variables listed in the steps.yaml.
<Callout>
By using the FirstInstance section, you can overwrite the [DefaultInstance configuration](#runtime-configuration-file) for the first instance created by zitadel setup.
</Callout>
<details>
<summary>steps.yaml</summary>
<CodeBlock language="yaml">{StepsYamlSource}</CodeBlock>
</details>
### Pre-existing Database and User
By default, when you run `zitadel init` or `zitadel start-from-init`, ZITADEL checks if the specified database and user exist. If they don't, ZITADEL creates them and grants the necessary permissions.
If you manage the database and user manually, you can instruct ZITADEL to skip these checks. This is useful if the ZITADEL user has limited database permissions, for example, if it is not a `SUPERUSER`. This scenario is most common on cloud providers.
To skip the checks, you must ensure the database objects (database, user, grant) exist before running the commands and then set the `Database.postgres.Admin.ExistingDatabase` field in your configuration as shown below. This tells ZITADEL to connect to the existing database as an admin user, bypassing the creation and permission steps.
<details>
<summary>skip-superuser-queries-config.yaml</summary>
<CodeBlock language="yaml">{SkipSuperUserDatabaseQueries}</CodeBlock>
</details>
### Multiple configuration files
ZITADEL merges configuration files when multiple `--config` and `--steps` flags are provided.
You can use these flags to handle standard configuration files differently from secret configuration files.
For example, standard configuration files stored in git may contain public information such as a database hostname.
To use private information — such as a database admin credential — without storing it in git, use an extra `--config` or `--steps` flag that requests the private information from a secret manager.
## Environment variables
All configuration properties are configurable using environment variables.
ZITADEL environment variable keys are prefixed with `ZITADEL_`.
For example, to configure the default ZITADEL IAM admin username and password set the `zitadel` binary runtime environment variables `ZITADEL_FIRSTINSTANCE_ORG_HUMAN_USERNAME` and `ZITADEL_FIRSTINSTANCE_ORG_HUMAN_PASSWORD`.
All supported environment variables are listed in the [runtime configuration file](#runtime-configuration-file) and the [database initialization file](#database-initialization-file).
### Proxy configuration
A proxy for outgoing connections can be configured using the environment variables: Use `HTTP_PROXY` for outgoing HTTP requests, and `HTTPS_PROXY` for outgoing HTTPS requests.
These environment variables are used as a proxy URL.
To exclude specific hosts from proxying, set the `NO_PROXY` environment variable: The value is interpreted as a comma-separated string.
For more information on the `NO_PROXY` environment variable, read the [`httpproxy` Go doc](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config).
## Masterkey
The masterkey is used to AES256-encrypt other generated encryption keys.
It must be 32 bytes.
There are three ways to pass the masterkey to the `zitadel` binary:
- By value: Use the flag `--masterkey My_Master_Key_Which_Has_32_Bytes`
- By environment variable `ZITADEL_MASTERKEY`: Use the flag `--masterkeyFromEnv`
- By file: Use the flag `--masterkeyFile /path/to/file`
## Passing the configuration
<Tabs
groupId="installation-target"
default="linuxunix"
values={[
{ label: "Linux and Unix", value: "linuxunix" },
{ label: "Docker Compose", value: "compose" },
{ label: "Kubernetes (Helm)", value: "k8s" },
]}
>
<Tab value="linuxunix">
<LinuxUnix components={props.components} />
<Login components={props.components} />
</Tab>
<Tab value="compose">
<Compose components={props.components} />
<Login components={props.components} />
</Tab>
<Tab value="k8s">
<Helm components={props.components} />
</Tab>
</Tabs>
## What's next
- Read more about [the login process](/guides/integrate/login/login-users).
- If you are running ZITADEL in production, you need to [customize your own domain](../custom-domain).
- Check out all possible [runtime configuration properties and their defaults in the source code](https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml)
- Check out all possible [setup step configuration properties and their defaults in the source code](https://github.com/zitadel/zitadel/blob/main/cmd/setup/steps.yaml)
<Callout type="warn">
The ZITADEL management console [requires end-to-end HTTP/2 support](/self-hosting/manage/http2)
</Callout>
@@ -0,0 +1,40 @@
services:
zitadel:
restart: "always"
networks:
- "zitadel"
image: "ghcr.io/zitadel/zitadel:latest"
command: 'start-from-init --config /example-zitadel-config.yaml --config /example-zitadel-secrets.yaml --steps /example-zitadel-init-steps.yaml --masterkey "${ZITADEL_MASTERKEY}" --tlsMode disabled'
ports:
- "8080:8080"
volumes:
- "./example-zitadel-config.yaml:/example-zitadel-config.yaml:ro"
- "./example-zitadel-secrets.yaml:/example-zitadel-secrets.yaml:ro"
- "./example-zitadel-init-steps.yaml:/example-zitadel-init-steps.yaml:ro"
depends_on:
db:
condition: "service_healthy"
db:
image: postgres:17-alpine
restart: always
environment:
- POSTGRES_USER=root
- POSTGRES_PASSWORD=postgres
networks:
- 'zitadel'
healthcheck:
test: ["CMD-SHELL", "pg_isready", "-d", "db_prod"]
interval: 10s
timeout: 60s
retries: 5
start_period: 10s
volumes:
- 'data:/var/lib/postgresql/data:rw'
networks:
zitadel:
volumes:
data:
@@ -0,0 +1,10 @@
Database:
postgres:
Database: existing_database
# Admin credentials should be the same as the application user credentials
Admin:
Username: zitadel-user
ExistingDatabase: existing_database # this is used to connect the admin to the previously created database
# Application user credentials
User:
Username: zitadel-user
@@ -0,0 +1,19 @@
# All possible options and their defaults: https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
Log:
Level: 'info'
# Make ZITADEL accessible over HTTP, not HTTPS
ExternalSecure: false
# If not using the docker compose example, adjust these values for connecting ZITADEL to your PostgreSQL
Database:
postgres:
Host: 'db'
Port: 5432
Database: zitadel
User:
SSL:
Mode: 'disable'
Admin:
SSL:
Mode: 'disable'
@@ -0,0 +1,7 @@
# All possible options and their defaults: https://github.com/zitadel/zitadel/blob/main/cmd/setup/steps.yaml
FirstInstance:
Org:
Human:
# use the loginname [email protected]
Username: 'root'
Password: 'RootPassword1!'
@@ -0,0 +1,12 @@
# All possible options and their defaults: https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
# If not using the docker compose example, adjust these values for connecting ZITADEL to your PostgreSQL
Database:
postgres:
User:
# If the user doesn't exist already, it is created
Username: 'zitadel_user'
Password: 'zitadel_pw'
Admin:
Username: 'root'
Password: 'postgres'
@@ -0,0 +1,7 @@
---
title: Configure ZITADEL
---
Detailed configuration options for ZITADEL.
- [Configuration Guide](/self-hosting/manage/configure/configure)
@@ -0,0 +1,64 @@
---
title: External ZITADEL Access
sidebar_label: Instance Not Found
---
## Why do I get an "Instance not found" error?
Also, ZITADEL has the [concept of virtual instances](/concepts/structure/instance#multiple-virtual-instances).
It uses a requests Host header to determine which virtual instance to use.
This is useful for multi-tenancy and resource sharing, for example in SaaS scenarios.
For most cases however, ZITADEL should run on exactly one domain.
This guide assumes you are already familiar with [configuring ZITADEL](./configure).
## Standard Config
ZITADEL only serves requests sent to the expected protocol, host and port.
For local testing purposes, you can use following configuration:
```yaml
ExternalDomain: localhost
ExternalPort: 8080
ExternalSecure: false
```
For productive setups however, we recommend using HTTPS and a Custom Domain:
```yaml
ExternalDomain: 'zitadel.my.domain'
ExternalPort: 443
ExternalSecure: true
```
## Changing ExternalDomain, ExternalPort or ExternalSecure
You can change the ExternalDomain, ExternalPort and ExternalSecure configuration options at any time.
However, for ZITADEL to be able to pick up the changes, [you need to rerun ZITADELs setup phase](/self-hosting/manage/updating_scaling#the-setup-phase).
## Running ZITADEL behind a Reverse Proxy
If you run ZITADEL behind a reverse proxy, you need to ensure that it sends the correct request headers to ZITADEL.
The proxy must either ensure that
- the original *Host* header value is assigned to the *Forwarded* headers host directive.
- the original requests *Host* header value is unchanged by the proxy.
Check out the [reverse proxy configuration examples](/self-hosting/manage/reverseproxy/reverse_proxy) for more information.
## Organization Domains
Note that by default, you cannot access ZITADEL at an organizations domain.
Organization level domains [are intended for routing users by their login methods to their correct organization](/guides/solution-scenarios/domain-discovery).
However, if you want to access ZITADEL at an organization domain, [you can add additional domains using the System API](/reference/api/system/zitadel.system.v1.SystemService.AddDomain).
Be aware that you won't automatically have the organizations context when you access ZITADEL like this.
## Generated Subdomains
ZITADEL creates random subdomains for [each new virtual instance](/concepts/structure/instance#multiple-virtual-instances).
You can immediately access the ZITADEL Management Console and APIs using these subdomains without further actions.
## More Information
- [Check out the production-near loadbalancing example with Traefik](/self-hosting/manage/reverseproxy/traefik)
- [Explore some concrete proxy configuration examples for ZITADEL using the domain 127.0.0.1.sslip.io](/self-hosting/manage/reverseproxy/reverse_proxy)
@@ -0,0 +1,56 @@
## ZITADEL with Postgres
PostgreSQL is the default database for ZITADEL due to its reliability, robustness, and adherence to SQL standards. It is well-suited for handling the complex data requirements of an identity management system.
If you are using Zitadel v2 and want to use a PostgreSQL database, you can [overwrite the default configuration](/self-hosting/manage/configure/configure).
Currently, versions 14 to 17 are supported.
Postgres can be configured as follows:
```yaml
Database:
postgres:
Host: localhost
Port: 5432
Database: zitadel
AwaitInitialConn: 5m
MaxOpenConns: 15
MaxIdleConns: 10
MaxConnLifetime: 1h
MaxConnIdleTime: 5m
Options:
User:
Username: zitadel
Password: zitadel
SSL:
Mode: disable
RootCert:
Cert:
Key:
Admin:
Username: postgres
Password: postgres
SSL:
Mode: disable
RootCert:
Cert:
Key:
```
The admin user is only needed for the installation step but can also be avoided
by preparing the necessary requirements:
* the zitadel user
* the zitadel database
* granting required permissions to the zitadel user
```sql
CREATE ROLE zitadel LOGIN;
CREATE DATABASE zitadel;
GRANT CONNECT, CREATE ON DATABASE zitadel TO zitadel;
```
Don't forget to adjust `pg_hba.conf` and set a password for the zitadel user.
With the setup done, follow the [phases guide](/self-hosting/manage/updating_scaling#separating-init-and-setup-from-the-runtime)
to run the init and then setup phase to get all necessary tables and data set up.
@@ -0,0 +1,31 @@
---
title: Database
---
import Postgres from './_postgres.mdx'
# Database Configuration
<Postgres components={props.components} />
## Zitadel Credentials
The [init phase](/self-hosting/manage/updating_scaling#separating-init-and-setup-from-the-runtime) of ZITADEL creates the ZITADEL database user (`Database.*.User.Username` & `Database.*.User.Password`) if it does not exist and admin credentials are provided. **Init does not update or deprecate existing users or passwords.** If you need to change the ZITADEL database user or rotate its password, you must do this manually.
<Callout type="warn">
The `init` command is designed to be run **only once**, during the initial setup of ZITADEL. Running `init` again after changing the database user or credentials does **not** migrate database object ownerships, nor does it reassign permissions on schemas, tables, or other objects to a new user. You must update these permissions manually if you switch users.
</Callout>
### Rotating Database Credentials
If you must rotate the credentials:
- You can change the password for the existing ZITADEL user, but you must manually update the password wherever ZITADEL needs it.
- Creating a new database user requires manual reassignment of ownership and privileges for all required database objects (schemas, tables, etc.) to the new user.
- After migration, remove or deprecate the old user using admin access and commands appropriate for your database provider.
Attempting to run `init` a second time with new credentials will fail if the user already exists, and will not change existing permissions or migrate data.
<Callout type="warn">
Dropping and recreating the database does **not** automatically remove existing users. Always verify and clean up old users if necessary.
</Callout>
@@ -0,0 +1,16 @@
---
title: HTTP/2 Support in ZITADEL
sidebar_label: HTTP/2 Support
---
ZITADEL follows a strict API first approach and makes heavy use of the modern API framework called [gRPC](https://grpc.io/).
Besides gRPC all APIs are also available in an openapi Rest fashion as well as in gRPC-web for compatibility towards browser integrations.
To make use of gRPC it is vital to allow your clients to communicate with ZITADEL with [HTTP/2](https://en.wikipedia.org/wiki/HTTP/2).
Sometimes you need to configure explicitly that you want to use HTTP/2 if you run ZITADEL behind a reverse proxy and below you should find examples for different vendors and projects.
Furthermore it is important to notice that by default HTTP/2 is always encrypted, but if you want to run ZITADEL without TLS from your reverse proxy or service mesh this is possible through [h2c](https://httpd.apache.org/docs/2.4/howto/http2.html).
Oftentimes when you run ZITADEL inside a service mesh, or a serverless offering (e.g. Google Cloud Run, Knative, ...) you will need h2c.
You can read more about ZITADEL's [TLSs modes here](/self-hosting/manage/tls_modes).
@@ -0,0 +1,47 @@
---
title: Connect your Self-Hosted Login UI to Zitadel
sidebar_label: Create a Login Client
---
To enable your self-hosted Login UI to connect to the Zitadel API, it needs a token for a user with the IAM_LOGIN_CLIENT role.
On new installations, the Zitadel setup job can be configured to automatically write a Personal Access Token (PAT) for the login client.
Check out [one of the deployment examples](https://zitadel.com/docs/self-hosting/deploy/overview) to learn how to do this.
However, if you want to replace the v1 login of an existing installation by a self-hosted v2 login, the setup job won't execute these steps.
In that case, you can create a new PAT for the login client manually.
## Create a Login Client User
In the following URLs, replace the base URL and the user ID according to your environment.
1. Create a new machine user, for example at http://localhost:8080/ui/console/users/create-machine
2. Create a PAT, for example at http://localhost:8080/ui/console/users/332169800719532035?new=true&id=pat
3. Save the PAT to a file, for example `/path/on/your/host/login-client.pat`
4. Make sure the user has the `Iam Login Client` role (internally called `IAM_LOGIN_CLIENT`), for example at http://localhost:8080/ui/console/instance/members
# Configure the Login UI
Make sure your Login UI has the environment variable `ZITADEL_SERVICE_USER_TOKEN` set with your PAT.
If you run the Login UI with Docker, you can also mount the file into the container and reference it by passing the environment variable `ZITADEL_SERVICE_USER_TOKEN_FILE`.
For example:
```bash
docker run -p 3000:3000 -v /path/on/your/host/login-client.pat:/path/in/container/login-client.pat:ro -e ZITADEL_SERVICE_USER_TOKEN_FILE=/path/in/container/login-client.pat ghcr.io/zitadel/zitadel-login:latest
```
# Enable the Login UI for all users
<Callout type="warn">
Before doing this, make sure you have a working PAT for an Iam Owner user.
In case something goes wrong and you lock yourself out from the login screen, you can revert the changes.
Create a machine user PAT like you created the [login client PAT above](#create-login-client), but give the user the Iam Owner role (internally called `IAM_OWNER`).
</Callout>
Enable the `Login V2` feature flag, for example at the bottom of http://localhost:8080/ui/console/instance?id=features.
Enter the base URI of your Login UI, for example `http://localhost:3000/ui/v2/login`.
# Test
That's it!
Click your users avatar in the top right corner of the management console and select `Log in With Another Account`.
You should see the new Login UI.
@@ -0,0 +1,66 @@
---
title: Metrics
description: Overview of ZITADEL metrics, exposure endpoint, and compatibility with monitoring systems.
---
# Metrics
ZITADEL exposes operational and performance metrics to support monitoring, alerting, and capacity planning.
Metrics are **enabled by default** in standard deployments and are exposed via an HTTP endpoint that can be consumed by Prometheus and other monitoring systems that understand the Prometheus exposition format or use an OpenTelemetry pipeline.
## Metrics endpoint
ZITADEL exposes metrics at the following endpoint:
`/debug/metrics`
This endpoint returns a snapshot of the current metrics in **Prometheus-compatible exposition format**.
If you changed ports, paths, or network exposure as part of your deployment (for example via Helm values, reverse proxies, or ingress), adjust the endpoint accordingly.
## Instrumentation model
Internally, ZITADEL instruments metrics using **OpenTelemetry (OTel)**.
The exposed metrics include:
- **Process and runtime metrics**
(for example Go runtime, memory usage, goroutines)
- **HTTP and gRPC server metrics**
(request counts, latencies, error rates)
- **ZITADEL-specific metrics**
(server behavior and internal components available in your version)
The exact set of metrics and labels may evolve between ZITADEL releases.
## Collection and compatibility
The `/debug/metrics` endpoint can be consumed in multiple ways:
- **Prometheus** can scrape the endpoint directly.
- **OpenTelemetry Collector** can scrape the endpoint using a Prometheus receiver and forward metrics to other backends.
- **Managed monitoring platforms** (for example Grafana Cloud, Amazon Managed Service for Prometheus, Datadog, New Relic) can ingest the same endpoint via Prometheus-compatible agents or collectors.
ZITADEL does **not** require a specific metrics backend. Any system capable of scraping or ingesting Prometheus-format metrics can be used.
## Configuration
Metrics are enabled by default in standard ZITADEL deployments.
If metrics were explicitly disabled in your configuration, re-enable them before proceeding. The default configuration is defined in the project's `defaults.yaml` file:
- https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
Refer to the configuration documentation for details on enabling or adjusting metrics behavior.
## Next steps
Choose how you want to collect and store metrics:
- **Prometheus** — scrape metrics directly from ZITADEL
- **OpenTelemetry Collector** — aggregate and forward metrics to another backend
- **Managed observability platforms** — use their Prometheus-compatible agents
The following sections describe concrete setups, starting with **Prometheus**.
@@ -0,0 +1,306 @@
---
title: Prometheus
description: Configure Prometheus to scrape ZITADEL metrics from /debug/metrics on Kubernetes and Docker.
---
# Prometheus
This page shows platform-agnostic ways to scrape ZITADEL metrics using:
- Kubernetes **annotation-based autodiscovery** (no Operator required)
- Kubernetes with **Prometheus Operator** using **ServiceMonitor**
- Standalone Prometheus on a VM or using **Docker**
- How to verify scraping, plus a minimal alert example
> ZITADEL’s endpoint path is `/debug/metrics`. If you changed ports or paths via your deployment tooling, adjust the examples accordingly.
## Before you begin
- You have a running ZITADEL instance reachable from your monitoring stack.
- You know the base URL and port to reach ZITADEL (e.g., `http://zitadel.zitadel.svc:8080` in Kubernetes or `http://localhost:8080` locally).
- Metrics are enabled in your runtime [configuration](https://zitadel.com/docs/self-hosting/manage/configure) (they are enabled by default in standard setups). If you explicitly disabled metrics in your configuration, re-enable them before proceeding. The (default) configuration is located in the [defaults.yaml](https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml).
## Option A — Kubernetes (annotation-based autodiscovery)
This approach is common when you **don’t use the Prometheus Operator**. Vanilla Prometheus can **auto-discover** scrape targets by reading standard annotations on Pods/Services:
```yaml
metadata:
annotations:
prometheus.io/scrape: "true"
prometheus.io/path: "/debug/metrics"
prometheus.io/port: "8080"
```
When your Prometheus server is configured with **Kubernetes service discovery** and **relabeling rules** that honor these annotations, it will automatically discover and scrape ZITADEL without any per-target `scrape_configs`.
### 1) Ensure Prometheus has Kubernetes discovery and relabeling
Your Prometheus configuration (often installed via Helm) should include jobs like the following. These are canonical examples that keep annotated Pods and map the annotated path/port to the actual metrics endpoint:
```yaml
scrape_configs:
- job_name: "kubernetes-pods"
kubernetes_sd_configs:
- role: pod
relabel_configs:
# Only keep pods with prometheus.io/scrape: "true"
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: true
# Use prometheus.io/path for the metrics path
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
# Replace the address with <pod_ip>:<prometheus.io/port>
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
target_label: __address__
```
> Most Prometheus Helm charts already ship with similar discovery jobs and relabeling rules. If you installed Prometheus via Helm, you likely already have these in place.
### 2) Annotate ZITADEL Pods (or the Service)
If you deploy ZITADEL via Helm and the chart emits scrape annotations on the Deployment/Pods, no extra work is needed. Otherwise, add the annotations yourself (via values override or a strategic patch):
```yaml
spec:
template:
metadata:
annotations:
prometheus.io/scrape: "true"
prometheus.io/path: "/debug/metrics"
prometheus.io/port: "8080"
```
If you prefer annotating the Service and your Prometheus config uses `role: service` discovery, add the same keys to the Service metadata.
### 3) RBAC
Prometheus must have permission to **list/watch** Pods/Endpoints in the target namespaces. Ensure its ServiceAccount has the standard ClusterRole/ClusterRoleBinding for discovery. Missing RBAC typically shows up as discovery errors in the Prometheus logs.
## Option B — Kubernetes with Prometheus Operator (ServiceMonitor)
If you run **kube-prometheus-stack** or the **Prometheus Operator**, use a **ServiceMonitor** (or **PodMonitor**). ZITADEL’s Helm chart provides **out-of-the-box** [ServiceMonitor](https://github.com/zitadel/zitadel-charts/blob/main/charts/zitadel/templates/servicemonitor.yaml) support that you can enable via values—no manual YAML is required.
### Enable the built-in ServiceMonitor (recommended)
```bash
helm repo add zitadel https://charts.zitadel.com
helm upgrade --install zitadel zitadel/zitadel \
--namespace zitadel --create-namespace \
--set metrics.enabled=true \
--set metrics.serviceMonitor.enabled=true
```
**Optional but common settings:**
* Make the ServiceMonitor discoverable by your Prometheus (Operator) instance (many stacks match a label like `release=kube-prometheus-stack`):
```bash
--set metrics.serviceMonitor.additionalLabels.release=kube-prometheus-stack
```
* Place the ServiceMonitor in a central monitoring namespace (default is the ZITADEL release namespace):
```bash
--set metrics.serviceMonitor.namespace=monitoring
```
* Tune scraping:
```bash
--set metrics.serviceMonitor.scrapeInterval=15s \
--set metrics.serviceMonitor.scrapeTimeout=10s
```
* Network/TLS customization (use only if you need them):
```bash
--set metrics.serviceMonitor.scheme=https \
--set metrics.serviceMonitor.tlsConfig.insecureSkipVerify=true
```
* Relabeling:
```bash
--set metrics.serviceMonitor.relabellings[0].action=replace \
--set metrics.serviceMonitor.metricRelabellings[0].action=drop
```
### What the chart generates (explained)
The chart renders a `ServiceMonitor` roughly equivalent to:
```yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: <release-name> # {{ include "zitadel.fullname" . }}
# namespace: <metrics.serviceMonitor.namespace> # only if you set it
labels:
# Standard chart labels + any you add:
# {{- include "zitadel.start.labels" . | nindent 4 }}
# {{- toYaml .Values.metrics.serviceMonitor.additionalLabels | nindent 4 }}
spec:
jobLabel: <release-name> # {{ include "zitadel.fullname" . }}
namespaceSelector:
matchNames:
- "<release-namespace>" # defaults to the Helm release namespace
selector:
matchLabels:
# Matches the ZITADEL Service created by the chart
# {{- include "zitadel.service.selectorLabels" . | nindent 6 }}
endpoints:
- port: "<protocol>-server" # e.g., "http-server" or "https-server"
path: /debug/metrics
# Optional tunables below are included only if set:
interval: <metrics.serviceMonitor.scrapeInterval>
scrapeTimeout: <metrics.serviceMonitor.scrapeTimeout>
scheme: <metrics.serviceMonitor.scheme> # http|https
tlsConfig: # metrics.serviceMonitor.tlsConfig
# ...
proxyUrl: <metrics.serviceMonitor.proxyUrl>
honorLabels: <metrics.serviceMonitor.honorLabels>
honorTimestamps: <metrics.serviceMonitor.honorTimestamps>
relabelings: # metrics.serviceMonitor.relabellings
# ...
metricRelabelings: # metrics.serviceMonitor.metricRelabellings
# ...
```
**Details that matter:**
* **Port name**: The chart uses
`port: {{ regexReplaceAll "\\W+" .Values.service.protocol "-" }}-server`
which resolves to **`http-server`** when `service.protocol=http` (default) or **`https-server`** when `service.protocol=https`. You do **not** need to edit this—just make sure you didn’t rename the ZITADEL Service port.
* **Namespace selection**: By default, the ServiceMonitor targets the **ZITADEL release namespace** via:
```yaml
namespaceSelector:
matchNames:
- "<release-namespace>"
```
Set `metrics.serviceMonitor.namespace` if you want the ServiceMonitor object itself to live elsewhere (e.g., `monitoring`). The `selector.matchLabels` still points to the ZITADEL **Service** labels.
* **Labels for discovery**: If your Prometheus (Operator) instance selects ServiceMonitors by label (common in kube-prometheus-stack), add those under `metrics.serviceMonitor.additionalLabels`—for example:
```yaml
metrics:
serviceMonitor:
additionalLabels:
release: kube-prometheus-stack
```
* **Path**: The chart fixes the metrics path to **`/debug/metrics`** (matches ZITADEL’s endpoint).
### Manual ServiceMonitor (only if you don’t want Helm to render it)
If you prefer to manage the `ServiceMonitor` yourself, keep it aligned with the chart’s conventions:
* Target the ZITADEL **Service** (not pods) using the same selector labels the chart adds.
* Use the correct **port name** (`http-server` or `https-server`) and **path** (`/debug/metrics`).
* Ensure your Prometheus (Operator) selects this ServiceMonitor by label/namespace.
> **Tip**
> If you already run the **Prometheus Operator**, prefer this ServiceMonitor approach. If you run **vanilla Prometheus without the Operator**, consider the **annotation-based discovery** method instead (Option A).
## Option C — Standalone Prometheus (VM / Docker)
If you run Prometheus outside of Kubernetes, add a static job pointing at ZITADEL’s metrics endpoint:
```yaml
global:
scrape_interval: 15s
scrape_configs:
- job_name: "zitadel"
metrics_path: "/debug/metrics"
scheme: "http" # use https if TLS is enabled for Zitadel
static_configs:
- targets: ["<ZITADEL_HOST>:8080"] # e.g., "localhost:8080", "zitadel.internal:8080" or "host.docker.internal:8080"
```
In this snippet, replace `<ZITADEL_HOST>:8080` with the appropriate address. This could be localhost:8080 for local deployments, or a DNS name / IP of the server or Kubernetes service where ZITADEL is running. If ZITADEL is behind a reverse proxy or ingress, ensure that `/debug/metrics` is reachable (you might expose it internally only). The metrics_path is set to /debug/metrics to match ZITADEL’s endpoint. We use http scheme assuming an internal/non-TLS endpoint; if you have enabled TLS on ZITADEL, use https and the appropriate port (e.g., `443`) and adjust any hostname (like zitadel.yourdomain.com).
> When running **Prometheus in Docker** on your workstation:
>
> - **macOS/Windows**: if ZITADEL runs on your host, use `host.docker.internal:8080`.
> - **Linux**: either add `--add-host=host.docker.internal:host-gateway` to `docker run`, attach Prometheus to the same Docker network as ZITADEL and use the service name (e.g., `zitadel:8080`), or run Prometheus with `--network host` (Linux only).
## Verify scraping
Use Prometheus’s built-in UI to confirm your target is up:
1. Open the UI (default `http://localhost:9090`).
2. Go to **Status → Targets** and find the `zitadel` target. It should be **UP**.
3. Go to **Graph** and run:
```yaml
up{job="zitadel"}
```
Value **1** indicates successful scrapes.
To explore ZITADEL metrics, type `zitadel` in the expression box and pick from auto-complete. Common families include:
- **Go runtime / process** (e.g., `go_goroutines`, `process_resident_memory_bytes`)
- **HTTP / gRPC** request counters/latencies (names vary by build and exporters)
- **ZITADEL server / DB** metrics (e.g., connection pool counts or other instrumentation available in your version)
> Metric names and labels can change between releases. Use the **Graph → Insert metric at cursor** dropdown to discover what your instance exposes.
## Minimal alerting example
Create a basic rule to alert when ZITADEL stops scraping:
```yaml
# /etc/prometheus/alerts/zitadel.yml
groups:
- name: zitadel-basic
rules:
- alert: ZitadelTargetDown
expr: up{job="zitadel"} == 0
for: 5m
labels:
severity: critical
annotations:
summary: "ZITADEL metrics target is down"
description: "Prometheus has not scraped ZITADEL successfully for 5 minutes."
```
Reference the rule file in your Prometheus config:
```yaml
rule_files:
- "/etc/prometheus/alerts/*.yml"
```
(Configure Alertmanager routing according to your environment.)
## Troubleshooting
**Target is DOWN / connection refused**
- In Docker: remember **`localhost` inside the Prometheus container is the container itself**, not your host. Use `host.docker.internal:8080` (plus `--add-host=host.docker.internal:host-gateway` on Linux), or join Prometheus to the **same Docker network** as ZITADEL and use the **service name** (e.g., `zitadel:8080`), or run with `--network host` on Linux.
- In Kubernetes: verify the Service port name (for example, `http-server` or `https-server`) and path (`/debug/metrics`) match your ServiceMonitor (or annotations). Check that Prometheus has RBAC to list Pods/Endpoints.
**Metrics path mismatch**
- ZITADEL uses **`/debug/metrics`**. If you see 404s, confirm your Prometheus job or annotations aren’t still using `/metrics`.
**No targets discovered (Kubernetes)**
- If using annotations, make sure your Prometheus config has **Kubernetes discovery & relabeling** rules that honor `prometheus.io/*` annotations and that the Pods/Service are annotated.
- If using ServiceMonitor, ensure your Prometheus Operator instance **selects** the ServiceMonitor by label/namespace.
**Nothing shows up in the Graph dropdown**
- First confirm `up{job="zitadel"}` returns **1**. If yes, metrics are being scraped—start typing generic prefixes like `go_` or `process_` to explore. ZITADEL’s exported metric set can evolve; check the raw output at `/debug/metrics` to see exactly what is exposed by your version.
## Alternatives that understand OTEL/Prometheus
While Prometheus is the most common choice, other collectors and services can ingest the same endpoint:
- **Amazon Managed Service for Prometheus (AMP)** — managed, Prometheus-compatible backend on AWS.
- **AWS CloudWatch via ADOT/OTel Collector** — scrape with the OpenTelemetry Collector and export to CloudWatch Metrics.
- **Grafana Cloud / VictoriaMetrics / Thanos** — remote-write targets or managed TSDBs for Prometheus data.
- **Datadog / New Relic / Splunk Observability** — agents or OTel pipelines can ingest Prometheus-format metrics.
If you already operate one of these platforms, you can point their agents/collectors at `/debug/metrics` or use an OTel Collector with a **Prometheus receiver** and the appropriate exporter.
@@ -0,0 +1,249 @@
---
title: ZITADEL Production Setup
sidebar_lable: Production Setup
---
As soon as you successfully deployed ZITADEL as a proof of concept using one of our [deployment guides](/self-hosting/deploy/overview),
you are ready to configure ZITADEL for production usage.
## TL;DR
We created a [Production Checklist](./productionchecklist) as a technical step by step guide.
## High Availability
We recommend running ZITADEL highly available using an orchestrator that schedules ZITADEL on multiple servers,
like [Kubernetes](/self-hosting/deploy/kubernetes).
For keeping startup times fast when scaling ZITADEL,
you should also consider using separate jobs with `zitadel init` and `zitadel setup`,
so your workload containers just have to execute `zitadel start`.
Read more about separating the init and setup phases on the [Updating and Scaling page](/self-hosting/manage/updating_scaling).
## Configuration
Read [on the configure page](/self-hosting/manage/configure) about the available options you have to configure ZITADEL.
Prefer passing .yaml files to the ZITADEL binary instead of environment variables.
Restricting access to these files to avoid leaking sensitive information is easier than restricting access to environment variables.
Also, not all configuration options are available as environment variables.
## Networking
- To make ZITADEL available at the domain of your choice, [you need to configure the ExternalDomain property](/self-hosting/manage/custom-domain).
- To enable and restrict access to **HTTPS**, head over to [the description of your TLS options](/self-hosting/manage/tls_modes).
- If you want to front ZITADEL with a reverse proxy, web application firewall or content delivery network, make sure to support **[HTTP/2](/self-hosting/manage/http2)**.
- You can also refer to some **[example reverse proxy configurations](/self-hosting/manage/reverseproxy/reverse_proxy)**.
- The ZITADEL Management Console web GUI uses many gRPC-Web stubs. This results in a fairly big JavaScript bundle. You might want to compress it using [Gzip](/self-hosting/manage/custom-domain) or [Brotli](https://www.gnu.org/software/gzip/) or [Brotli](https://github.com/google/brotli).
- Serving and caching the assets using a content delivery network could improve network latencies and shield your ZITADEL runtime.
## Monitoring
By default, [**metrics**](/apis/observability/metrics) are exposed at /debug/metrics in OpenTelemetry (otel) format.
Also, you can enable **tracing** in the ZITADEL configuration.
```yaml
Tracing:
# Choose one in "otel", "google", "log" and "none"
Type: google
Fraction: 1
MetricPrefix: zitadel
```
## Logging
ZITADEL follows the principles that guide cloud-native and twelve factor applications.
Logs are a stream of time-ordered events collected from all running processes.
[ZITADEL is configurable](#default-zitadel-logging-config) to write the following events to the standard output:
- Runtime Logs: Define the log level and record format in the `Log` configuration section.
- Access Logs: Enable logging all HTTP and gRPC responses from the ZITADEL binary by setting `LogStore.Access.Stdout.Enabled` to true.
- Actions Execution Logs: Actions can emit custom logs at different levels. For example, a log record can be emitted each time a user is created or authenticated. If you don't want to have these logs in STDOUT, you can disable this by setting `LogStore.Execution.Stdout.Enabled` to true.
### Default ZITADEL Logging Config
```yaml
Log:
Level: info # ZITADEL_LOG_LEVEL
Formatter:
Format: text # ZITADEL_LOG_FORMATTER_FORMAT
LogStore:
Access:
Stdout:
# If enabled, all access logs are printed to the binary's standard output
Enabled: false # ZITADEL_LOGSTORE_ACCESS_STDOUT_ENABLED
Execution:
Stdout:
# If enabled, all execution logs are printed to the binary's standard output
Enabled: true # ZITADEL_LOGSTORE_EXECUTION_STDOUT_ENABLED
```
### Why ZITADEL does not write logs to files
Log file management should not be in each business apps responsibility.
Instead, your execution environment should provide tooling for managing logs in a generic way.
This includes tasks like rotating files, routing, collecting, archiving and cleaning-up.
For example, systemd has journald and kubernetes has fluentd and fluentbit.
## Telemetry
If you want to have some data about reached usage milestones pushed to external systems, enable telemetry in the ZITADEL configuration.
The following table describes the milestones that are sent to the endpoints:
| Trigger | Description |
|-----------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------|
| A virtual instance is created. | This data point is also sent when the first instance is automatically created during the ZITADEL binaries setup phase in a self-hosting scenario. |
| An authentication succeeded for the first time on an instance. | This is the first authentication with the instances automatically created admin user during the instance setup, which can be a human or a machine. |
| A project is created for the first time in a virtual instance. | The ZITADEL project that is automatically created during the instance setup is omitted. |
| An application is created for the first time in a virtual instance. | The applications in the ZITADEL project that are automatically created during the instance setup are omitted. |
| An authentication succeeded for the first time in a virtual instances application. | This is the first authentication using a ZITADEL application that is not created during the instance setup phase. |
| A virtual instance is deleted. | This data point is sent when a virtual instance is deleted via ZITADELs system API |
ZITADEL pushes the metrics by projecting certain events.
Therefore, you can configure delivery guarantees not in the Telemetry section of the ZITADEL configuration,
but in the Projections.Customizations.Telemetry section
## Database
### Prefer PostgreSQL
ZITADEL supports [PostgreSQL](https://www.postgresql.org/).
<Callout>
ZITADEL supports PostgreSQL. For migrating from CockroachDB to PostgreSQL, please refer to [the mirror guide](/self-hosting/manage/cli/mirror).
</Callout>
The indexes for the database are optimized using load tests from [ZITADEL Cloud](https://zitadel.com),
which runs with PostgreSQL.
If you identify problems with your database during load tests that indicate that the indexes are not optimized,
please create an issue in our [github repository](https://github.com/zitadel/zitadel).
### Configure ZITADEL
Depending on your environment, you maybe would want to tweak some settings about how ZITADEL interacts with the database in the database section of your ZITADEL configuration. Read more about your [database configuration options](/self-hosting/manage/database).
```yaml
Database:
postgres:
Host: localhost
Port: 5432
Database: zitadel
//highlight-start
MaxOpenConns: 10
MaxIdleConns: 5
MaxConnLifetime: 30m
MaxConnIdleTime: 5m
//highlight-end
Options: ""
```
You also might want to configure how [projections](/concepts/eventstore/implementation#projections) are computed. These are the default values:
```yaml
# The Projections section defines the behavior for the scheduled and synchronous events projections.
Projections:
# Time interval between scheduled projections
RequeueEvery: 60s
# Time between retried database statements resulting from projected events
RetryFailedAfter: 1s
# Retried execution number of database statements resulting from projected events
MaxFailureCount: 5
# Number of concurrent projection routines. Values of 0 and below are overwritten to 1
ConcurrentInstances: 1
# Limit of returned events per query
BulkLimit: 200
# Only instance are projected, for which at least a projection relevant event exists withing the timeframe
# from HandleActiveInstances duration in the past until the projections current time
# Defaults to twice the RequeueEvery duration
HandleActiveInstances: 120s
# In the Customizations section, all settings from above can be overwritten for each specific projection
Customizations:
Projects:
BulkLimit: 2000
# The Notifications projection is used for sending emails and SMS to users
Notifications:
# As notification projections don't result in database statements, retries don't have an effect
MaxFailureCount: 0
# The NotificationsQuotas projection is used for calling quota webhooks
NotificationsQuotas:
# Delivery guarantee requirements are probably higher for quota webhooks
# Defaults to 45 days
HandleActiveInstances: 1080h
# As quota notification projections don't result in database statements, retries don't have an effect
MaxFailureCount: 0
# Quota notifications are not so time critical. Setting RequeueEvery every five minutes doesn't annoy the db too much.
RequeueEvery: 300s
```
### Manage your data
When designing your backup strategy,
it is worth knowing that
[ZITADEL is event sourced](/concepts/eventstore/overview).
That means, ZITADEL itself is able to recompute its
whole state from the records in the table eventstore.events.
The timestamp of your last record in the events table
defines up to which point in time ZITADEL can restore its state.
The ZITADEL binary itself is stateless,
so there is no need for a special backup job.
Generally, for maintaining your database management system in production,
please refer to the corresponding docs [for PostgreSQL](https://www.postgresql.org/docs/current/admin.html).
## Data initialization
- You can configure instance defaults in the DefaultInstance section.
If you plan to eventually create [multiple virtual instances](/concepts/structure/instance#multiple-virtual-instances), these defaults take effect.
Also, these configurations apply to the first instance, that ZITADEL automatically creates for you.
Especially the following properties are of special interest for your production setup.
```yaml
DefaultInstance:
OIDCSettings:
AccessTokenLifetime: 12h
IdTokenLifetime: 12h
RefreshTokenIdleExpiration: 720h #30d
RefreshTokenExpiration: 2160h #90d
# this configuration sets the default email configuration
SMTPConfiguration:
# configuration of the host
SMTP:
#for example smtp.mailtrap.io:2525
Host:
User:
Password:
TLS:
# if the host of the sender is different from ExternalDomain set DefaultInstance.DomainPolicy.SMTPSenderAddressMatchesInstanceDomain to false
From:
FromName:
ReplyToAddress:
```
- If you don't want to use the DefaultInstance configuration for the first instance that ZITADEL automatically creates for you during the [setup phase](/self-hosting/manage/configure/configure#database-initialization-file), you can provide a FirstInstance YAML section using the --steps argument.
- Learn how to configure ZITADEL via the [Console user interface](/guides/manage/console/console-overview).
- Probably, you also want to [apply your custom branding](/guides/manage/customize/branding), [hook into certain events](/guides/manage/customize/behavior), [customize texts](/guides/manage/customize/texts) or [add metadata to your users](/guides/manage/customize/user-metadata).
- If you want to automatically create ZITADEL resources, you can use the [ZITADEL Terraform Provider](/guides/manage/terraform-provider).
## Limits and Quotas
If you host ZITADEL as a service,
you might want to [limit usage and/or execute tasks on certain usage units and levels](/self-hosting/manage/usage_control).
## Minimum system requirements
### General resource usage
ZITADEL itself requires approximately 512MB of RAM and can operate with less than one CPU core. The database component, under typical conditions, utilizes about one CPU core per 100 requests per second (req/s) and 4GB of RAM per core, which includes some caching.
<Callout title="Password hashing">
Be aware of CPU spikes when hashing passwords. We recommend to have 4 CPU cores available for this purpose.
</Callout>
### Production HA cluster
For a minimal high-availability setup, we recommend a cluster of 3 nodes, each with 4 CPU cores and 16GB of memory.
If you exclude non-essential services like log collection and metrics, you can reduce the resources to approximately 4 CPU cores and 8GB of memory per node.
@@ -0,0 +1,68 @@
---
title: ZITADEL Production Checklist
sidebar_label: Production Checklist
---
To apply best practices to your production setup we created a step by step checklist you may wish to follow.
### Infrastructure Configuration
- [ ] Make use of configuration management tools such as Terraform to provision all of the below
- [ ] Use a secrets manager to store your confidential information
- [ ] Reduce the manual interaction with your platform to an absolute minimum
#### HA Setup
- [ ] High Availability for ZITADEL containers
- [ ] Use a container orchestrator such as Kubernetes
- [ ] Use serverless platform such as Knative or a hyperscaler equivalent (e.g. CloudRun from Google)
- [ ] Split `zitadel init` and `zitadel setup` for fast start-up times when [scaling](/self-hosting/manage/updating_scaling) ZITADEL
- [ ] High Availability for database
- [ ] Follow [this guide](https://www.postgresql.org/docs/current/high-availability.html) to set up the database.
- [ ] Configure logging
- [ ] Configure timeouts
- [ ] Configure backups on a regular basis for the database
- [ ] Test the restore scenarios before going live
- [ ] Secure database connections from outside your network and/or use an internal subnet for database connectivity
- [ ] High Availability for critical infrastructure components (depending on your setup)
- [ ] Loadbalancer
- [ ] [Reverse Proxies](/self-hosting/manage/reverseproxy/reverse_proxy)
- [ ] Web Application Firewall
#### Networking
- [ ] Use a Layer 7 Web Application Firewall to secure ZITADEL that supports **[HTTP/2](/self-hosting/manage/http2)**
- [ ] Limit the access by IP addresses if needed
- [ ] Secure the access by rate limits for specific endpoints (e.g. API vs frontend) to secure availability on high load. See the [ZITADEL Cloud rate limits](/legal/policies/rate-limit-policy) for reference.
- [ ] Check that your firewall also filters IPv6 traffic
### ZITADEL configuration
- [ ] Configure a valid [SMTP Server](/guides/manage/console/default-settings#smtp) and test the email delivery
- [ ] Add [Custom Branding](/guides/manage/customize/branding) if required
- [ ] Configure a valid [SMS Service](/guides/manage/console/default-settings#sms) such as Twilio if needed
- [ ] Configure your privacy policy, terms of service and a help Link if needed
- [ ] Keep your [masterkey](/self-hosting/manage/configure) in a secure storage
- [ ] Declare and apply zitadel configuration using the zitadel terraform [provider](https://github.com/zitadel/terraform-provider-zitadel)
### Security
- [ ] Ensure that your ZITADEL does not use [the default, example or _easy-to-guess_ credentials](/self-hosting/manage/database#zitadel-credentials)
- [ ] Use a FQDN and a trusted valid certificate for external [TLS](/self-hosting/manage/tls_modes) connections
- [ ] Create service accounts for applications that interact with ZITADEL's APIs
- [ ] Make use of a CDN service to decrease the load for static assets served by ZITADEL
- [ ] Make use of a [security scanner](https://owasp.org/www-community/Vulnerability_Scanning_Tools) to test your application and deployment environment
### Monitoring
Use an appropriate monitoring solution to have an overview about your ZITADEL instance. In particular you may want to watch out for things like:
- [ ] CPU and memory of ZITADEL and the database
- [ ] Open database connections
- [ ] Running instances of ZITADEL and the database
- [ ] Latency of requests
- [ ] Requests per second
- [ ] Requests by URL/endpoint
- [ ] Lifetime of TLS certificates
- [ ] ZITADEL and database logs
- [ ] ZITADEL [metrics](/apis/observability/metrics)
@@ -0,0 +1,2 @@
- [Read more about Zitadel's TLS Modes](/self-hosting/manage/tls_modes)
- [Read more about how Zitadel uses HTTP/2](/self-hosting/manage/http2)
@@ -0,0 +1,16 @@
import CodeBlock from '@/components/docusaurus/code-block';
import ComposeYaml from "./docker-compose.yaml";
<>With these examples, you create and run a minimal {props.link} configuration for Zitadel with <a href="https://docs.docker.com/compose">Docker Compose</a>.
Whereas the guide focuses on the configuration for {props.name}, you can inspect the configurations for the Zitadel API, the Zitadel login and the database in the base Docker Compose file.</>
<details>
<summary>base docker-compose.yaml</summary>
<CodeBlock language="yaml">{ComposeYaml}</CodeBlock>
</details>
<>For running {props.name}, you will extend the base Docker Compose file with the {props.name} specific Docker Compose file.</>
<details>
<summary>specific docker-compose.yaml</summary>
<CodeBlock language="yaml">{props.compose}</CodeBlock>
</details>
@@ -0,0 +1,90 @@
import CodeBlock from '@/components/docusaurus/code-block';
export const Description = ({mode, name}) => {
let desc
switch (mode) {
case "disabled":
desc = <>Neither {name} nor Zitadel terminates TLS.
Nevertheless, {name} forwards unencrypted HTTP/2 traffic, aka h2c, to Zitadel.</>;
break;
case "external":
desc = <>{name} terminates TLS and forwards the requests to Zitadel via unencrypted h2c.
This example uses an unsafe self-signed certificate for {name}.</>;
break;
case "enabled":
desc = <>{name} terminates TLS and forwards the requests to Zitadel via encrypted HTTP/2.
This example uses an unsafe self-signed certificate for {name} and the same for Zitadel.</>;
break;
}
return (
<>
{desc}
<> By executing the commands below, you will download the files necessary to run Zitadel behind {name} with the following config:</>
</>)
}
export const Commands = ({mode, name, lower, configfilename}) => {
let genCert = '# Generate a self signed certificate and key.\nopenssl req -x509 -batch -subj "/CN=127.0.0.1.sslip.io/O=Zitadel Demo" -nodes -newkey rsa:2048 -keyout ./selfsigned.key -out ./selfsigned.crt 2>/dev/null\n\n';
let connPort = "443"
let connInsecureFlag = "--insecure "
let connScheme = "https"
let grpcPlainTextFlag = ""
if (mode === "disabled") {
genCert = ''
connPort = "80"
grpcPlainTextFlag = "--plaintext "
connScheme = "http"
// We only need that flag for TLS connections with the self-signed cert
connInsecureFlag = ""
}
return (
<div>
<CodeBlock language="bash">
{'# Download the configuration files.'}{'\n'}
{'export ZITADEL_CONFIG_FILES=https://raw.githubusercontent.com/zitadel/zitadel/main/docs/self-hosting/manage/reverseproxy\n'}
{'wget ${ZITADEL_CONFIG_FILES}/docker-compose.yaml -O docker-compose-base.yaml --quiet \n'}
{'wget ${ZITADEL_CONFIG_FILES}/'}{lower}{'/docker-compose.yaml -O docker-compose-'}{lower}{'.yaml --quiet \n'}
{'wget ${ZITADEL_CONFIG_FILES}/'}{lower}{'/'}{configfilename}{' -O '}{configfilename}{' --quiet \n'}
{'\n'}
{genCert}
{'# Run the database, and '}{name}{'.'}{'\n'}
{'docker compose --file docker-compose-base.yaml --file docker-compose-'}{lower}{'.yaml up --detach --wait db zitadel-init zitadel-'}{mode}{'-tls login-'}{mode}{'-tls proxy-'}{mode}{'-tls'}{'\n'}
{'\n'}
{'# Test that gRPC and HTTP APIs work. Empty brackets like {} means success.\n'}
{'# Make sure you have the grpcurl cli installed on your machine https://github.com/fullstorydev/grpcurl?tab=readme-ov-file#installation\n'}
{'grpcurl '}{connInsecureFlag}{grpcPlainTextFlag}{'127.0.0.1.sslip.io:'}{connPort}{' zitadel.admin.v1.AdminService/Healthz\n'}
{'curl '}{connInsecureFlag}{connScheme}{'://127.0.0.1.sslip.io:'}{connPort}{'/admin/v1/healthz\n'}
</CodeBlock>
</div>
)
}
export const LoginURL = ({mode}) => {
let scheme = "https";
if (mode === "disabled") {
scheme = "http"
}
const url = scheme + "://127.0.0.1.sslip.io/ui/console/[email protected]";
return <a href={url}>{url}</a>
}
<Description mode={props.mode} name={props.providername} link={props.link}/>
<details open>
<summary>{props.configfilename}</summary>
<CodeBlock >{props.configfilecontent}</CodeBlock>
</details>
<Commands mode={props.mode} name={props.providername} lower={props.lower} configfilename={props.configfilename}/>
<>When the docker compose command exits successfully, go to <LoginURL mode={props.mode}/> and log in:</>
- **username**: *zitadel-admin@<span></span>zitadel.127.0.0.1.sslip.io*
- **password**: *Password1!*
If the management console loads normally, you know that the HTTP and gRPC-Web and gRPC APIs are working correctly.
<CodeBlock language="bash">
{'# You can now stop the database, the Zitadel API, the Zitadel login and '}{props.providername}{'.'}{'\n'}
{'docker compose --file docker-compose-base.yaml --file docker-compose-'}{props.lower}{'.yaml down'}{'\n'}
</CodeBlock>
@@ -0,0 +1,4 @@
http://127.0.0.1.sslip.io {
reverse_proxy /ui/v2/login/* http://login-disabled-tls:3000
reverse_proxy h2c://zitadel-disabled-tls:8080
}
@@ -0,0 +1,41 @@
services:
proxy-disabled-tls:
image: "caddy:latest"
volumes:
- "./disabled-tls.Caddyfile:/etc/caddy/Caddyfile:ro"
ports:
- "80:80"
networks:
- app
depends_on:
zitadel-disabled-tls:
condition: 'service_healthy'
proxy-external-tls:
image: "caddy:latest"
volumes:
- "./external-tls.Caddyfile:/etc/caddy/Caddyfile:ro"
- "./selfsigned.crt:/etc/certs/selfsigned.crt:ro"
- "./selfsigned.key:/etc/certs/selfsigned.key:ro"
ports:
- "443:443"
networks:
- app
depends_on:
zitadel-external-tls:
condition: 'service_healthy'
proxy-enabled-tls:
image: "caddy:latest"
volumes:
- "./enabled-tls.Caddyfile:/etc/caddy/Caddyfile:ro"
- "./selfsigned.crt:/etc/certs/selfsigned.crt:ro"
- "./selfsigned.key:/etc/certs/selfsigned.key:ro"
ports:
- "443:443"
networks:
- app
depends_on:
zitadel-enabled-tls:
condition: 'service_healthy'
@@ -0,0 +1,9 @@
https://127.0.0.1.sslip.io {
tls /etc/certs/selfsigned.crt /etc/certs/selfsigned.key
reverse_proxy /ui/v2/login/* http://login-enabled-tls:3000
reverse_proxy https://zitadel-enabled-tls:8080 {
transport http {
tls_insecure_skip_verify
}
}
}
@@ -0,0 +1,5 @@
https://127.0.0.1.sslip.io {
tls /etc/certs/selfsigned.crt /etc/certs/selfsigned.key
reverse_proxy /ui/v2/login/* http://login-external-tls:3000
reverse_proxy h2c://zitadel-external-tls:8080
}
@@ -0,0 +1,32 @@
---
title: Configure Zitadel with Caddy
sidebar_label: Caddy
---
import ProxyGuideOverview from '../_proxy_guide_overview.mdx';
import ProxyGuideTLSMode from '../_proxy_guide_tls_mode.mdx';
import ProxyGuideMore from '../_proxy_guide_more.mdx';
import Compose from "./docker-compose.yaml";
import ConfigDisabled from "./disabled-tls.Caddyfile";
import ConfigExternal from "./external-tls.Caddyfile";
import ConfigEnabled from "./enabled-tls.Caddyfile";
export const providername = 'Caddy';
export const lower = "caddy";
export const link = <a href="https://caddyserver.com/">{providername}</a>
<ProxyGuideOverview components={props.components} name={providername} link={link} compose={Compose}></ProxyGuideOverview>
You can either setup your environment for <a href={'#tls-mode-external'}>TLS mode external</a> or <a href={'#tls-mode-enabled'}>TLS mode enabled</a>.
## TLS mode external
<ProxyGuideTLSMode components={props.components} mode="external" configfilename="external-tls.Caddyfile" configfilecontent={ConfigExternal} providername={providername} link={link} lower={lower}></ProxyGuideTLSMode>
## TLS mode enabled
<ProxyGuideTLSMode components={props.components} mode="enabled" configfilename="enabled-tls.Caddyfile" configfilecontent={ConfigEnabled} providername={providername} link={link} lower={lower}></ProxyGuideTLSMode>
## More Information
<ProxyGuideMore components={props.components} ></ProxyGuideMore>
@@ -0,0 +1,23 @@
---
title: Configure Zitadel with Cloudflare
sidebar_label: Cloudflare
---
## Settings
- [Make sure HTTP/2 is enabled](https://support.cloudflare.com/hc/en-us/articles/200168076-Understanding-Cloudflare-HTTP-2-and-HTTP-3-Support)
- [Verify that gRPC is enabled](https://support.cloudflare.com/hc/en-us/articles/360050483011-Understanding-Cloudflare-gRPC-support)
- [Verify that traffic is proxied through cloudflare](https://developers.cloudflare.com/dns/manage-dns-records/reference/proxied-dns-records/)
- [Configure Zitadel to use the TLS Mode enabled](/self-hosting/manage/tls_modes#enabled)
<Callout>
[Cloudflare does only support gRPC with TLS!](https://support.cloudflare.com/hc/en-us/articles/360050483011-Understanding-Cloudflare-gRPC-support)
</Callout>
## Troubleshooting
If something is not working please check the cloudflare WAF rules for potential violations.
These two rules are known to be triggered:
- 100001 Anomaly:Header:User-Agent - Missing Cloudflare Specials
- 100004 Anomaly:Header:User-Agent, Anomaly:Header:Referer - Missing or empty
@@ -0,0 +1,9 @@
---
title: Configure Zitadel with Cloudflare Tunnel
sidebar_label: Cloudflare Tunnel
---
<Callout type="warn">
[The Cloudflare tunnel client currently has an issue which disallows it to force HTTP/2 usage towards the origin.](https://github.com/cloudflare/cloudflared/issues/682)
</Callout>
@@ -0,0 +1,166 @@
services:
zitadel-disabled-tls:
extends:
service: zitadel-init
command: 'start-from-setup --masterkey "MasterkeyNeedsToHave32Characters"'
environment:
ZITADEL_EXTERNALPORT: 80
ZITADEL_EXTERNALSECURE: false
ZITADEL_TLS_ENABLED: false
networks:
- app
- db
depends_on:
zitadel-init:
condition: 'service_completed_successfully'
db:
condition: 'service_healthy'
zitadel-external-tls:
extends:
service: zitadel-init
command: 'start-from-setup --masterkey "MasterkeyNeedsToHave32Characters"'
environment:
ZITADEL_EXTERNALPORT: 443
ZITADEL_EXTERNALSECURE: true
ZITADEL_TLS_ENABLED: false
networks:
- app
- db
depends_on:
db:
condition: 'service_healthy'
zitadel-init:
condition: 'service_completed_successfully'
zitadel-enabled-tls:
extends:
service: zitadel-init
command: 'start-from-setup --masterkey "MasterkeyNeedsToHave32Characters"'
environment:
ZITADEL_EXTERNALPORT: 443
ZITADEL_EXTERNALSECURE: true
ZITADEL_TLS_ENABLED: true
ZITADEL_TLS_CERTPATH: /etc/certs/selfsigned.crt
ZITADEL_TLS_KEYPATH: /etc/certs/selfsigned.key
volumes:
- ./selfsigned.crt:/etc/certs/selfsigned.crt
- ./selfsigned.key:/etc/certs/selfsigned.key
networks:
- app
- db
depends_on:
zitadel-init:
condition: 'service_completed_successfully'
db:
condition: 'service_healthy'
zitadel-init:
image: '${ZITADEL_IMAGE:-ghcr.io/zitadel/zitadel:latest}'
command: 'init'
depends_on:
db:
condition: 'service_healthy'
environment:
# Using an external domain other than localhost proofs, that the proxy configuration works.
# If Zitadel can't resolve a requests original host to this domain,
# it will return a 404 Instance not found error.
ZITADEL_EXTERNALDOMAIN: 127.0.0.1.sslip.io
# In case something doesn't work as expected,
# it can be handy to be able to read the access logs.
ZITADEL_LOGSTORE_ACCESS_STDOUT_ENABLED: true
# For convenience, ZITADEL should not ask to change the initial admin users password.
ZITADEL_FIRSTINSTANCE_ORG_HUMAN_PASSWORDCHANGEREQUIRED: false
# database configuration
ZITADEL_DATABASE_POSTGRES_HOST: db
ZITADEL_DATABASE_POSTGRES_USER_PASSWORD: zitadel_pw
# Set up a service account with IAM_LOGIN_CLIENT role and write the PAT to the file ./login-client.pat
ZITADEL_FIRSTINSTANCE_LOGINCLIENTPATPATH: /current-dir/login-client.pat
ZITADEL_FIRSTINSTANCE_ORG_LOGINCLIENT_MACHINE_USERNAME: login-client
ZITADEL_FIRSTINSTANCE_ORG_LOGINCLIENT_MACHINE_NAME: Automatically Initialized IAM Login Client
ZITADEL_FIRSTINSTANCE_ORG_LOGINCLIENT_PAT_EXPIRATIONDATE: '2029-01-01T00:00:00Z'
# The master key is used to
networks:
- db
healthcheck:
test: [ "CMD", "/app/zitadel", "ready" ]
interval: '10s'
timeout: '5s'
retries: 5
start_period: '10s'
volumes:
- '.:/current-dir:rw'
db:
restart: 'always'
image: postgres:17-alpine
environment:
POSTGRES_PASSWORD: postgres
healthcheck:
test: [ "CMD-SHELL", "pg_isready" ]
interval: 5s
timeout: 60s
retries: 10
start_period: 5s
networks:
- db
volumes:
- 'data:/var/lib/postgresql/data:rw'
login-disabled-tls:
restart: 'unless-stopped'
image: 'ghcr.io/zitadel/zitadel-login:latest'
environment:
- ZITADEL_API_URL=http://zitadel-disabled-tls:8080
- NEXT_PUBLIC_BASE_PATH=/ui/v2/login
- ZITADEL_SERVICE_USER_TOKEN_FILE=/current-dir/login-client.pat
- CUSTOM_REQUEST_HEADERS=Host:127.0.0.1.sslip.io
- NODE_ENV=test
volumes:
- '.:/current-dir:ro'
networks:
- app
depends_on:
zitadel-disabled-tls:
condition: 'service_healthy'
login-external-tls:
restart: 'unless-stopped'
image: 'ghcr.io/zitadel/zitadel-login:latest'
environment:
- ZITADEL_API_URL=http://zitadel-external-tls:8080
- NEXT_PUBLIC_BASE_PATH=/ui/v2/login
- ZITADEL_SERVICE_USER_TOKEN_FILE=/current-dir/login-client.pat
- CUSTOM_REQUEST_HEADERS=Host:127.0.0.1.sslip.io
volumes:
- '.:/current-dir:ro'
networks:
- app
depends_on:
zitadel-external-tls:
condition: 'service_healthy'
login-enabled-tls:
restart: 'unless-stopped'
image: 'ghcr.io/zitadel/zitadel-login:latest'
environment:
- ZITADEL_API_URL=https://zitadel-enabled-tls:8080
- NEXT_PUBLIC_BASE_PATH=/ui/v2/login
- ZITADEL_SERVICE_USER_TOKEN_FILE=/current-dir/login-client.pat
- CUSTOM_REQUEST_HEADERS=Host:127.0.0.1.sslip.io
- NODE_TLS_REJECT_UNAUTHORIZED=0
volumes:
- '.:/current-dir:ro'
networks:
- app
depends_on:
zitadel-enabled-tls:
condition: 'service_healthy'
networks:
app:
db:
volumes:
data:
@@ -0,0 +1,41 @@
services:
proxy-disabled-tls:
image: "httpd:latest"
volumes:
- "./httpd-disabled-tls.conf:/usr/local/apache2/conf/httpd.conf"
ports:
- "80:80"
networks:
- app
depends_on:
zitadel-disabled-tls:
condition: 'service_healthy'
proxy-external-tls:
image: "httpd:latest"
volumes:
- "./httpd-external-tls.conf:/usr/local/apache2/conf/httpd.conf"
- "./selfsigned.crt:/etc/certs/selfsigned.crt:ro"
- "./selfsigned.key:/etc/certs/selfsigned.key:ro"
ports:
- "443:443"
networks:
- app
depends_on:
zitadel-external-tls:
condition: 'service_healthy'
proxy-enabled-tls:
image: "httpd:latest"
volumes:
- "./httpd-enabled-tls.conf:/usr/local/apache2/conf/httpd.conf"
- "./selfsigned.crt:/etc/certs/selfsigned.crt:ro"
- "./selfsigned.key:/etc/certs/selfsigned.key:ro"
ports:
- "443:443"
networks:
- app
depends_on:
zitadel-enabled-tls:
condition: 'service_healthy'
@@ -0,0 +1,52 @@
LoadModule mpm_event_module modules/mod_mpm_event.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule reqtimeout_module modules/mod_reqtimeout.so
LoadModule filter_module modules/mod_filter.so
LoadModule mime_module modules/mod_mime.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule headers_module modules/mod_headers.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule proxy_http2_module modules/mod_proxy_http2.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule dir_module modules/mod_dir.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so
ServerRoot "/usr/local/apache2"
LogLevel debug
ErrorLog /proc/self/fd/2
CustomLog /proc/self/fd/1 "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
Listen 80
<VirtualHost *:80>
# Enable HTTP/2 for gRPC support (cleartext)
Protocols h2c http/1.1
# Proxy settings
ProxyPreserveHost on
ProxyAddHeaders on
RequestHeader set X-Forwarded-Proto "http"
# Route login UI first (more specific path)
ProxyPass /ui/v2/login http://login-disabled-tls:3000/ui/v2/login
ProxyPassReverse /ui/v2/login http://login-disabled-tls:3000/ui/v2/login
# Route everything else to Zitadel
ProxyPass / h2c://zitadel-disabled-tls:8080/
ProxyPassReverse / h2c://zitadel-disabled-tls:8080/
</VirtualHost>
@@ -0,0 +1,77 @@
LoadModule mpm_event_module modules/mod_mpm_event.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule reqtimeout_module modules/mod_reqtimeout.so
LoadModule filter_module modules/mod_filter.so
LoadModule mime_module modules/mod_mime.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule headers_module modules/mod_headers.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule proxy_http2_module modules/mod_proxy_http2.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule dir_module modules/mod_dir.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule http2_module modules/mod_http2.so
ServerRoot "/usr/local/apache2"
LogLevel debug
ErrorLog /proc/self/fd/2
CustomLog /proc/self/fd/1 "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
Listen 443
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
<VirtualHost *:443>
# Enable HTTP/2 and configure ALPN for gRPC
Protocols h2 http/1.1
H2Direct on
# SSL Configuration
SSLEngine on
SSLProxyEngine on
SSLCertificateFile /etc/certs/selfsigned.crt
SSLCertificateKeyFile /etc/certs/selfsigned.key
# Modern SSL settings for gRPC compatibility
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
SSLCipherSuite ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305
SSLHonorCipherOrder off
# Relax SSL Proxy Configuration to debug
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
# Add timeouts to prevent hanging
ProxyTimeout 60
# Proxy settings
ProxyPreserveHost on
ProxyAddHeaders on
RequestHeader set X-Forwarded-Proto "https"
# Route login UI
ProxyPass /ui/v2/login http://login-enabled-tls:3000/ui/v2/login
ProxyPassReverse /ui/v2/login http://login-enabled-tls:3000/ui/v2/login
# Route to Zitadel with TLS
ProxyPass / https://zitadel-enabled-tls:8080/
ProxyPassReverse / https://zitadel-enabled-tls:8080/
</VirtualHost>
@@ -0,0 +1,63 @@
LoadModule mpm_event_module modules/mod_mpm_event.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule reqtimeout_module modules/mod_reqtimeout.so
LoadModule filter_module modules/mod_filter.so
LoadModule mime_module modules/mod_mime.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule headers_module modules/mod_headers.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule proxy_http2_module modules/mod_proxy_http2.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule dir_module modules/mod_dir.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule http2_module modules/mod_http2.so
LogLevel debug
ErrorLog /proc/self/fd/2
CustomLog /proc/self/fd/1 "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
Listen 443
<VirtualHost *:443>
# Enable HTTP/2 and configure ALPN
Protocols h2 http/1.1
H2Direct on
# Configure SSL for gRPC
SSLEngine on
SSLCertificateFile /etc/certs/selfsigned.crt
SSLCertificateKeyFile /etc/certs/selfsigned.key
# Enable ALPN protocols for gRPC
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
SSLCipherSuite ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305
SSLHonorCipherOrder off
SSLSessionTickets off
ProxyPreserveHost on
ProxyAddHeaders on
RequestHeader set X-Forwarded-Proto "https"
# Route login requests
ProxyPass /ui/v2/login http://login-external-tls:3000/ui/v2/login
ProxyPassReverse /ui/v2/login http://login-external-tls:3000/ui/v2/login
# Route gRPC/API requests with HTTP/2 support
ProxyPass / h2c://zitadel-external-tls:8080/
ProxyPassReverse / h2c://zitadel-external-tls:8080/
</VirtualHost>
@@ -0,0 +1,28 @@
---
title: Configure Zitadel with Apache httpd
sidebar_label: Apache httpd
---
import ProxyGuideOverview from '../_proxy_guide_overview.mdx';
import ProxyGuideTLSMode from '../_proxy_guide_tls_mode.mdx';
import ProxyGuideMore from '../_proxy_guide_more.mdx';
import Compose from "./docker-compose.yaml";
import ConfigDisabled from "./httpd-disabled-tls.conf";
import ConfigExternal from "./httpd-external-tls.conf";
import ConfigEnabled from "./httpd-enabled-tls.conf";
export const providername = "Apache httpd";
export const lower = "httpd";
export const link = <a href="https://httpd.apache.org//">{providername}</a>
<ProxyGuideOverview components={props.components} name={providername} link={link} compose={Compose}></ProxyGuideOverview>
You can either setup your environment for <a href={'#tls-mode-disabled'}>TLS mode disabled</a>, <a href={'#tls-mode-external'}>TLS mode external</a> or <a href={'#tls-mode-enabled'}>TLS mode enabled</a>.
## TLS mode external
<ProxyGuideTLSMode components={props.components} mode="external" configfilename="httpd-external-tls.conf" configfilecontent={ConfigExternal} providername={providername} link={link} lower={lower}></ProxyGuideTLSMode>
## More Information
<ProxyGuideMore components={props.components} ></ProxyGuideMore>
@@ -0,0 +1,41 @@
services:
proxy-disabled-tls:
image: "nginx:mainline-alpine"
volumes:
- "./nginx-disabled-tls.conf:/etc/nginx/nginx.conf:ro"
ports:
- "80:80"
networks:
- app
depends_on:
zitadel-disabled-tls:
condition: 'service_healthy'
proxy-external-tls:
image: "nginx:mainline-alpine"
volumes:
- "./nginx-external-tls.conf:/etc/nginx/nginx.conf:ro"
- "./selfsigned.crt:/etc/certs/selfsigned.crt:ro"
- "./selfsigned.key:/etc/certs/selfsigned.key:ro"
ports:
- "443:443"
networks:
- app
depends_on:
zitadel-external-tls:
condition: 'service_healthy'
proxy-enabled-tls:
image: "nginx:mainline-alpine"
volumes:
- "./nginx-enabled-tls.conf:/etc/nginx/nginx.conf:ro"
- "./selfsigned.crt:/etc/certs/selfsigned.crt:ro"
- "./selfsigned.key:/etc/certs/selfsigned.key:ro"
ports:
- "443:443"
networks:
- app
depends_on:
zitadel-enabled-tls:
condition: 'service_healthy'
@@ -0,0 +1,32 @@
---
title: Configure Zitadel with NGINX
sidebar_label: NGINX
---
import ProxyGuideOverview from '../_proxy_guide_overview.mdx';
import ProxyGuideTLSMode from '../_proxy_guide_tls_mode.mdx';
import ProxyGuideMore from '../_proxy_guide_more.mdx';
import Compose from "./docker-compose.yaml";
import ConfigDisabled from "./nginx-disabled-tls.conf";
import ConfigExternal from "./nginx-external-tls.conf";
import ConfigEnabled from "./nginx-enabled-tls.conf";
export const providername = 'NGINX';
export const lower = "nginx";
export const link = <a href="https://nginx.com/">{providername}</a>;
<ProxyGuideOverview components={props.components} name={providername} link={link} compose={Compose}></ProxyGuideOverview>
You can either setup your environment for <a href={'#tls-mode-disabled'}>TLS mode disabled</a>, <a href={'#tls-mode-external'}>TLS mode external</a> or <a href={'#tls-mode-enabled'}>TLS mode enabled</a>.
## TLS mode external
<ProxyGuideTLSMode components={props.components} mode="external" configfilename="nginx-external-tls.conf" configfilecontent={ConfigExternal} providername={providername} link={link} lower={lower}></ProxyGuideTLSMode>
## TLS mode enabled
<ProxyGuideTLSMode components={props.components} mode="enabled" configfilename="nginx-enabled-tls.conf" configfilecontent={ConfigEnabled} providername={providername} link={link} lower={lower}></ProxyGuideTLSMode>
## More Information
<ProxyGuideMore components={props.components} ></ProxyGuideMore>
@@ -0,0 +1,17 @@
events {
worker_connections 1024;
}
http {
server {
listen 80;
http2 on;
location /ui/v2/login {
proxy_pass http://login-disabled-tls:3000;
proxy_set_header Host $host;
}
location / {
grpc_pass grpc://zitadel-disabled-tls:8080;
grpc_set_header Host $host;
}
}
}
@@ -0,0 +1,20 @@
events {
worker_connections 1024;
}
http {
server {
listen 443 ssl;
http2 on;
ssl_certificate /etc/certs/selfsigned.crt;
ssl_certificate_key /etc/certs/selfsigned.key;
location /ui/v2/login {
proxy_pass http://login-enabled-tls:3000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto https;
}
location / {
grpc_pass grpcs://zitadel-enabled-tls:8080;
grpc_set_header Host $host;
}
}
}
@@ -0,0 +1,21 @@
events {
worker_connections 1024;
}
http {
server {
listen 443 ssl;
http2 on;
ssl_certificate /etc/certs/selfsigned.crt;
ssl_certificate_key /etc/certs/selfsigned.key;
location /ui/v2/login {
proxy_pass http://login-external-tls:3000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto https;
}
location / {
grpc_pass grpc://zitadel-external-tls:8080;
grpc_set_header Host $host;
grpc_set_header X-Forwarded-Proto https;
}
}
}
@@ -0,0 +1,13 @@
---
title: Reverse Proxy Configuration
---
Check out one of the following guides to configure your favorite reverse proxy:
- [Traefik](/self-hosting/manage/reverseproxy/traefik)
- [NGINX](/self-hosting/manage/reverseproxy/nginx)
- [Caddy](/self-hosting/manage/reverseproxy/caddy)
- [Apache httpd](/self-hosting/manage/reverseproxy/httpd)
- [Cloudflare](/self-hosting/manage/reverseproxy/cloudflare)
- [Cloudflare Tunnel](/self-hosting/manage/reverseproxy/cloudflare_tunnel)
- [Fronting Zitadel Cloud](/self-hosting/manage/reverseproxy/zitadel_cloud)
@@ -0,0 +1,41 @@
services:
proxy-disabled-tls:
image: "traefik:latest"
volumes:
- "./traefik-disabled-tls.yaml:/etc/traefik/traefik.yaml:ro"
ports:
- "80:80"
networks:
- app
depends_on:
zitadel-disabled-tls:
condition: 'service_healthy'
proxy-external-tls:
image: "traefik:latest"
volumes:
- "./traefik-external-tls.yaml:/etc/traefik/traefik.yaml:ro"
- "./selfsigned.crt:/etc/certs/selfsigned.crt:ro"
- "./selfsigned.key:/etc/certs/selfsigned.key:ro"
ports:
- "443:443"
networks:
- app
depends_on:
zitadel-external-tls:
condition: 'service_healthy'
proxy-enabled-tls:
image: "traefik:latest"
volumes:
- "./traefik-enabled-tls.yaml:/etc/traefik/traefik.yaml:ro"
- "./selfsigned.crt:/etc/certs/selfsigned.crt:ro"
- "./selfsigned.key:/etc/certs/selfsigned.key:ro"
ports:
- "443:443"
networks:
- app
depends_on:
zitadel-enabled-tls:
condition: 'service_healthy'
@@ -0,0 +1,32 @@
---
title: Configure Zitadel with Traefik
sidebar_label: Traefik
---
import ProxyGuideOverview from '../_proxy_guide_overview.mdx';
import ProxyGuideTLSMode from '../_proxy_guide_tls_mode.mdx';
import ProxyGuideMore from '../_proxy_guide_more.mdx';
import Compose from "./docker-compose.yaml";
import ConfigDisabled from "./traefik-disabled-tls.yaml";
import ConfigExternal from "./traefik-external-tls.yaml";
import ConfigEnabled from "./traefik-enabled-tls.yaml";
export const providername = 'Traefik';
export const lower = "traefik";
export const link = <a href="https://doc.traefik.io/traefik/">{providername}</a>;
<ProxyGuideOverview components={props.components} name={providername} link={link} compose={Compose}></ProxyGuideOverview>
You can either setup your environment for <a href={'#tls-mode-disabled'}>TLS mode disabled</a>, <a href={'#tls-mode-external'}>TLS mode external</a> or <a href={'#tls-mode-enabled'}>TLS mode enabled</a>.
## TLS mode external
<ProxyGuideTLSMode components={props.components} mode="external" configfilename="traefik-external-tls.yaml" configfilecontent={ConfigExternal} providername={providername} link={link} lower={lower}></ProxyGuideTLSMode>
## TLS mode enabled
<ProxyGuideTLSMode components={props.components} mode="enabled" configfilename="traefik-enabled-tls.yaml" configfilecontent={ConfigEnabled} providername={providername} link={link} lower={lower}></ProxyGuideTLSMode>
## More Information
<ProxyGuideMore components={props.components} ></ProxyGuideMore>
@@ -0,0 +1,31 @@
log:
level: "DEBUG"
providers:
file:
filename: "/etc/traefik/traefik.yaml"
entrypoints:
web:
address: ":80"
http:
routers:
zitadel:
entryPoints:
- "web"
service: "zitadel"
rule: '!PathPrefix(`/ui/v2/login`)'
login:
entryPoints:
- "web"
service: "login"
rule: 'PathPrefix(`/ui/v2/login`)'
services:
zitadel:
loadBalancer:
passHostHeader: true
servers:
- url: "h2c://zitadel-disabled-tls:8080"
login:
loadBalancer:
passHostHeader: true
servers:
- url: "http://login-disabled-tls:3000"
@@ -0,0 +1,46 @@
log:
level: "DEBUG"
providers:
file:
filename: "/etc/traefik/traefik.yaml"
entrypoints:
web:
address: ":443"
http:
routers:
zitadel:
entryPoints:
- "web"
service: "zitadel"
rule: '!PathPrefix(`/ui/v2/login`)'
tls: {}
login:
entryPoints:
- "web"
service: "login"
rule: 'PathPrefix(`/ui/v2/login`)'
tls: {}
services:
zitadel:
loadBalancer:
serversTransport: "zitadel"
passHostHeader: true
servers:
- url: "https://zitadel-enabled-tls:8080"
login:
loadBalancer:
serversTransport: "login"
passHostHeader: true
servers:
- url: "http://login-enabled-tls:3000"
serversTransports:
zitadel:
insecureSkipVerify: true
login:
insecureSkipVerify: true
tls:
stores:
default:
defaultCertificate:
certFile: /etc/certs/selfsigned.crt
keyFile: /etc/certs/selfsigned.key
@@ -0,0 +1,39 @@
log:
level: "DEBUG"
providers:
file:
filename: "/etc/traefik/traefik.yaml"
entrypoints:
web:
address: ":443"
http:
routers:
zitadel:
entryPoints:
- "web"
service: "zitadel"
rule: '!PathPrefix(`/ui/v2/login`)'
tls: {}
login:
entryPoints:
- "web"
service: "login"
rule: 'PathPrefix(`/ui/v2/login`)'
tls: {}
services:
zitadel:
loadBalancer:
passHostHeader: true
servers:
- url: "h2c://zitadel-external-tls:8080"
login:
loadBalancer:
passHostHeader: true
servers:
- url: "http://login-external-tls:3000"
tls:
stores:
default:
defaultCertificate:
certFile: /etc/certs/selfsigned.crt
keyFile: /etc/certs/selfsigned.key
@@ -0,0 +1,16 @@
---
title: Front Zitadel Cloud with a CDN, WAF or Reverse Proxy
sidebar_label: Fronting Zitadel Cloud
---
## Fronting Zitadel Cloud
You can use your reverseproxy, content delivery network (CDN) or web application firewall (WAF) to front Zitadel Cloud.
However we currently do not recommend this for production settings.
To configure your service that fronts Zitadel please have a look at the vendors in this page.
## Things to look out for when fronting Zitadel Cloud
- Cache-control - Zitadel Cloud uses a CDN to globally distribute data. Please try to avoid overriding this header as it may lead to sideeffects
- Rate Limits - Zitadel Cloud uses a combination of static and dynamic rate limits. If you receive occasional 429 headers you are rate limited.
@@ -0,0 +1,96 @@
---
title: Service Ping
sidebar_label: Service Ping
---
Service Ping is a feature that periodically sends anonymized analytics and usage data from your ZITADEL system to a central endpoint.
This data helps improve ZITADEL by providing insights into its usage patterns.
The feature is enabled by default, but can be disabled either completely or for specific reports.
Checkout the configuration options below.
## Data Sent by Service Ping
### Base Information
If the feature is enabled, the base information will always be sent. To prevent that, you can opt out by disabling the entire Service Ping:
```yaml
ServicePing:
Enabled: false # ZITADEL_SERVICEPING_ENABLED
```
The base information sent back includes the following:
- your systemID
- the currently run version of ZITADEL
- information on all instances
- id
- creation date
- domains
### Resource Counts
Resource counts is a report that provides us with information about the number of resources in your ZITADEL instances.
The following resources are counted:
- Instances
- Organizations
- Projects per organization
- Users per organization
- Users of type machine per organization
- SCIM provisioned users per organization
- Instance Administrators
- Identity Providers
- LDAP Identity Providers
- Actions (V1)
- Targets and set up executions
- Login Policies
- MFA enforcement (if either MFA is required for local or all users through the login policy)
- Password Complexity Policies
- Password Expiry Policies
- Lockout Policies
- Notification Policies with option "Password changed" enabled
The list might be extended in the future to include more resources.
To disable this report, set the following in your configuration file:
```yaml
ServicePing:
Telemetry:
ResourceCounts:
Enabled: false # ZITADEL_SERVICEPING_TELEMETRY_RESOURCECOUNT_ENABLED
```
## Configuration
The Service Ping feature can be configured through the runtime configuration. Please check out the configuration file
for all available options. Below is a list of the most important options:
### Interval
This defines at which interval the Service Ping feature sends data to the central endpoint. It supports the extended cron syntax
and by default is set to `@daily`, which means it will send data every day. The time is randomized on startup to prevent
all systems from sending data at the same time.
You can adjust it to your needs to make sure there is no performance impact on your system.
For example, if you already have some scheduled job syncing data in and out of ZITADEL around a specific time or have regularly a
lot of traffic during the day, you might want to change it to a different time, e.g. `15 4 * * *` to send it every day at 4:15 AM.
The interval must be at least 30 minutes to prevent too frequent requests to the central endpoint.
### MaxAttempts
This defines how many attempts the Service Ping feature will make to send data to the central endpoint before giving up
for a specific interval and report. If one report fails, it will be retried up to this number of times.
Other reports will still be handled in parallel and have their own retry count. This means if the base information
only succeeded after three attempts, the resource count still has five attempts to be sent.
### BulkSize
Certain reports, like the resource counts, can generate a lot of data. To prevent sending too much data in one request,
the data is split into smaller chunks. This setting defines the maximum number of items that will be
sent in one request. If there are more items, they will be sent in multiple requests.
The size of the request is limited by the maximum request size of the central endpoint.
Each report will log its size before sending it, so you can adjust the bulk size if needed.
@@ -0,0 +1,61 @@
---
title: TLS Modes
---
To run ZITADEL on any kind of infrastructure, you can configure on how to handle TLS connections.
There are three modes of operation: `disabled`, `external`, `enabled`.
Generally this command is set as argument while starting ZITADEL. For example like this:
```bash
zitadel start-from-init --masterkey "MasterkeyNeedsToHave32Characters" --tlsMode disabled
```
## Disabled
With the mode `disabled`, you instruct ZITADEL to await all connections with plain http without TLS.
<Callout type="warn">
Be aware this is not a secure setup and should only be used for test systems!
</Callout>
## External
The mode `external` allows you to configure ZITADEL in such a way that it will instruct its clients to use https.
However ZITADEL delegates the management of TLS connections to a reverseproxy, web application firewall or a service mesh.
## Enabled
When using the mode `enabled` ZITADEL is setup to await incoming connections in an encrypted fashion.
Whether it is from a client directly, a reverse proxy or web application firewall.
This allows HTTP connections to be secured at the transport level the whole way.
If you use the mode `enabled` you need to configure ZITADEL with the necessary TLS settings.
```yaml
TLS:
# if enabled, ZITADEL will serve all traffic over TLS (HTTPS and gRPC)
# you must then also provide a private key and certificate to be used for the connection
# either directly or by a path to the corresponding file
Enabled: true
# Path to the private key of the TLS certificate, it will be loaded into the Key
# and overwrite any existing value
KeyPath: #/path/to/key/file.pem
# Private key of the TLS certificate (KeyPath will this overwrite, if specified)
Key: #<bas64 encoded content of a pem file>
# Path to the certificate for the TLS connection, it will be loaded into the Cert
# and overwrite any existing value
CertPath: #/path/to/cert/file.pem
# Certificate for the TLS connection (CertPath will this overwrite, if specified)
Cert: #<bas64 encoded content of a pem file>
```
## More Information
Beware that ZITADEL uses HTTP/2 for all its connections.
If you are using the mode `external` or `disabled` make sure to verify h2c compatibility.
- [Read more about how ZITADEL utilizes HTTP/2](/self-hosting/manage/http2).
- [Explore some concrete proxy configuration examples for ZITADEL](/self-hosting/manage/reverseproxy/reverse_proxy).
@@ -0,0 +1,88 @@
---
title: Update and Scale ZITADEL
sidebar_label: Update and Scale
---
## TL;DR
For getting started easily, you can just combine the init, setup and runtime phases
by executing the `zitadel` binary with the argument `start-from-init`.
However, to improve day-two-operations,
we recommend you run the init phase and the setup phase
separately using `zitadel init` and `zitadel setup` respectively.
Both the init and the setup phases are idempotent,
meaning you can run them as many times as you want without unexpectedly changing any state.
Nevertheless, they just __need__ to be run at certain points in time, whereas further runs are obsolete.
The init phase has to run once over the full ZITADEL lifecycle, before the setup and runtime phases run first.
The setup phase has to run each time a new ZITADEL version is deployed.
After init and setup is done, you can start the runtime phase, that immediately accepts traffic, using `zitadel start`.
## Scaling ZITADEL
For production setups, we recommend that you [run ZITADEL in a highly available mode](/self-hosting/manage/production).
As all shared state is managed in the database,
the ZITADEL binary itself is stateless.
You can easily run as many ZITADEL binaries in parallel as you want.
If you use [Knative](https://knative.dev/docs/)
or [Google Cloud Run](https://cloud.google.com/run) (which uses Knative, too),
you can even scale to zero.
Especially if you use an autoscaler that scales to zero,
it is crucial that you minimize startup times.
## Updating ZITADEL
When want to deploy a new ZITADEL version,
the new versions setup phase takes care of database migrations.
You generally want to run the job in a controlled manner,
so multiple executions don’t interfere with each other.
Also, after the setup is done,
rolling out a new ZITADEL version is much faster
when the runtime processes are just executed with `zitadel start`.
## Separating Init and Setup from the Runtime
If you use the [official ZITADEL Helm chart](/self-hosting/deploy/kubernetes),
then you can stop reading now.
The init and setup phases are already separated and executed in dedicated Kubernetes Jobs.
If you use another orchestrator and want to separate the phases yourself,
you should know what happens during these phases.
### The Init Phase
The command `zitadel init` ensures the database connection is ready to use for the subsequent phases.
It just needs to be executed once over ZITADELs full life cycle,
when you install ZITADEL from scratch.
During `zitadel init`, for connecting to your database,
ZITADEL uses the privileged and preexisting database user configured in `Database.postgres.Admin.Username`.
, `zitadel init` ensures the following:
- If it doesn’t exist already, it creates a database with the configured database name.
- If it doesn’t exist already, it creates the unprivileged user use configured in `Database.postgres.User.Username`.
Subsequent phases connect to the database with this user's credentials only.
- If not already done, it grants the necessary permissions ZITADEL needs to the non privileged user.
- If they don’t exist already, it creates all schemas and some basic tables.
The init phase is idempotent if executed with the same binary version.
### The Setup Phase
During `zitadel setup`, ZITADEL creates projection tables and migrates existing data, if `--init-projections=true` is set.
Depending on the ZITADEL version and the runtime resources,
this step can take several minutes.
When deploying a new ZITADEL version,
make sure the setup phase runs before you roll out the new `zitadel start` processes.
The setup phase is executed in subsequent steps
whereas a new version's execution takes over where the last execution stopped.
Some configuration changes are only applied during the setup phase, like ExternalDomain, ExternalPort and ExternalSecure.
The setup phase is idempotent if executed with the same binary version.
### The Runtime Phase
The `zitadel start` command assumes the database is already initialized and set up.
It starts serving requests within fractions of a second.
Beware, in the background, out-of-date projections
[recompute their state by replaying all missed events](/concepts/eventstore/implementation#projections).
If a new ZITADEL version is deployed, this can take quite a long time,
depending on the amount of events to catch up.
You probably should consider providing `--init-projections=true`-flag to the [Setup Phase](#the-setup-phase) to shift the synchronization time to previous steps and delay the startup phase until events are caught up.
@@ -0,0 +1,137 @@
---
title: Usage Control
sidebar_label: Usage Control
---
If you have a self-hosted ZITADEL environment, you can limit the usage of your [instances](/concepts/structure/instance).
For example, if you provide your customers [their own virtual instances](/concepts/structure/instance#multiple-virtual-instances) with access on their own domains, you can design a pricing model based on the usage of their instances.
The usage control features are currently limited to the instance level only.
## Block Instances
You can block an instance using the [system API](/reference/api/system).
Most requests to a blocked instance are rejected with the HTTP status *429 Too Many Requests* or the gRPC status *8 Resource Exhausted*.
However, requests to the [system API](/apis/introduction#system) are still allowed.
Requests to paths with the prefix */ui/login* return a redirect with HTTP status *302 Found* to */ui/console*, where the user is guided to *InstanceManagementURL*.
Blocked HTTP requests additionally set a cookie to make it easy to block traffic before it reaches your ZITADEL runtime, for example with a WAF rule.
You can block new instances by default using the *DefaultInstance.Limits.Block* runtime configuration.
The following snippets shows the default YAML:
```yaml
DefaultInstance:
Limits:
# If Block is true, all requests except to /ui/console or the system API are blocked and /ui/login is redirected to /ui/console.
# /ui/console shows a message that the instance is blocked with a link to Console.InstanceManagementURL
Block: # ZITADEL_DEFAULTINSTANCE_LIMITS_BLOCK
```
## Limit Audit Trails
You can restrict the maximum age of events returned by the following APIs:
- [Events Search](/reference/api/admin/zitadel.admin.v1.AdminService.ListEvents), See also the [Event API guide](/guides/integrate/zitadel-apis/event-api)
- [My User History](/reference/api/auth/zitadel.auth.v1.AuthService.ListMyUserChanges)
- [A Users History](/reference/api/management/zitadel.management.v1.ManagementService.ListUserChanges)
- [A Applications History](/reference/api/management/zitadel.management.v1.ManagementService.ListAppChanges)
- [A Organizations History](/reference/api/management/zitadel.management.v1.ManagementService.ListOrgChanges)
- [A Projects History](/reference/api/management/zitadel.management.v1.ManagementService.ListProjectChanges)
- [A Project Grants History](/reference/api/management/zitadel.management.v1.ManagementService.ListProjectGrantChanges)
You can set a global default limit as well as a default limit [for new virtual instances](/concepts/structure/instance#multiple-virtual-instances) in the ZITADEL configuration.
The following snippets shows the defaults:
```yaml
# AuditLogRetention limits the number of events that can be queried via the events API by their age.
# A value of "0s" means that all events are available.
# If an audit log retention is set using an instance limit, it will overwrite the system default.
AuditLogRetention: 0s # ZITADEL_AUDITLOGRETENTION
DefaultInstance:
Limits:
# AuditLogRetention limits the number of events that can be queried via the events API by their age.
# A value of "0s" means that all events are available.
# If this value is set, it overwrites the system default unless it is not reset via the admin API.
AuditLogRetention: # ZITADEL_DEFAULTINSTANCE_LIMITS_AUDITLOGRETENTION
```
You can also set a limit for [a specific virtual instance](/concepts/structure/instance#multiple-virtual-instances) using the [system API](/reference/api/system).
## Quotas
Quotas enables you to limit usage and/or register webhooks that trigger on configurable usage levels for certain units.
For example, you might want to report usage to an external billing tool and notify users when 80 percent of a quota is exhausted.
ZITADEL supports limiting authenticated requests and action run seconds with quotas.
For using the quotas feature you have to activate it in your ZITADEL configurations *Quotas* section.
The following snippets shows the defaults:
```yaml
Quotas:
Access:
# If enabled, authenticated requests are counted and potentially limited depending on the configured quota of the instance
Enabled: false # ZITADEL_QUOTAS_ACCESS_ENABLED
Debounce:
MinFrequency: 0s # ZITADEL_QUOTAS_ACCESS_DEBOUNCE_MINFREQUENCY
MaxBulkSize: 0 # ZITADEL_QUOTAS_ACCESS_DEBOUNCE_MAXBULKSIZE
ExhaustedCookieKey: "zitadel.quota.exhausted" # ZITADEL_QUOTAS_ACCESS_EXHAUSTEDCOOKIEKEY
ExhaustedCookieMaxAge: "300s" # ZITADEL_QUOTAS_ACCESS_EXHAUSTEDCOOKIEMAXAGE
Execution:
# If enabled, all action executions are counted and potentially limited depending on the configured quota of the instance
Enabled: false # ZITADEL_QUOTAS_EXECUTION_DATABASE_ENABLED
Debounce:
MinFrequency: 0s # ZITADEL_QUOTAS_EXECUTION_DEBOUNCE_MINFREQUENCY
MaxBulkSize: 0 # ZITADEL_QUOTAS_EXECUTION_DEBOUNCE_MAXBULKSIZE
```
Once you have activated the quotas feature, you can configure quotas [for your virtual instances](/concepts/structure/instance#multiple-virtual-instances) using the [system API](/reference/api/system) or the *DefaultInstances.Quotas* section.
The following snippets shows the defaults:
```yaml
DefaultInstance:
Quotas:
# Items take a slice of quota configurations, whereas, for each unit type and instance, one or zero quotas may exist.
# The following unit types are supported
# "requests.all.authenticated"
# The sum of all requests to the ZITADEL API with an authorization header,
# excluding the following exceptions
# - Calls to the System API
# - Calls that cause internal server errors
# - Failed authorizations
# - Requests after the quota already exceeded
# "actions.all.runs.seconds"
# The sum of all actions run durations in seconds
Items:
# - Unit: "requests.all.authenticated"
# # From defines the starting time from which the current quota period is calculated.
# # This is relevant for querying the current usage.
# From: "2023-01-01T00:00:00Z"
# # ResetInterval defines the quota periods duration
# ResetInterval: 720h # 30 days
# # Amount defines the number of units for this quota
# Amount: 25000
# # Limit defines whether ZITADEL should block further usage when the configured amount is used
# Limit: false
# # Notifications are emitted by ZITADEL when certain quota percentages are reached
# Notifications:
# # Percent defines the relative amount of used units, after which a notification should be emitted.
# - Percent: 100
# # Repeat defines, whether a notification should be emitted each time when a multitude of the configured Percent is used.
# Repeat: true
# # CallURL is called when a relative amount of the quota is used.
# CallURL: "https://httpbin.org/post"
```
### Exhausted Authenticated Requests
If a quota is configured to limit requests and the quotas amount is exhausted, all further authenticated requests are blocked except requests to the [system API](/apis/introduction#system).
Also, a cookie is set, to make it easier to block further traffic before it reaches your ZITADEL runtime, for example with a WAF rule.
The management console is still served, but it only shows a dialog that says that the instance is blocked with a link to *InstanceManagementURL*.
### Exhausted Action Run Seconds
If a quota is configured to limit action run seconds and the quotas amount is exhausted, all further actions will fail immediately with a context timeout exceeded error.
The action that runs into the limit also fails with the context timeout exceeded error.