feat(database): enhance PostgreSQL setup documentation and commands for non-admin access (#11631)

Users deploying ZITADEL against a managed PostgreSQL service (RDS, Cloud SQL, Azure Database, etc.) often do not have superuser access and cannot provide `Admin.*` credentials. The documented workaround — provisioning the user and database manually and then running `start-from-setup` — silently skips schema bootstrapping, causing `relation "eventstore.events" does not exist` errors with no clear recovery path. The root cause is that `zitadel init` conflates two steps that require different privileges without exposing them separately: - **Provisioning step** (`CREATE ROLE`, `CREATE DATABASE`, `GRANT`) — requires superuser. - **Schema bootstrapping step** (create `eventstore`/`projections`/`system` schemas and base tables) — requires only DB owner. Users who handle the provisioning step externally have no supported way to run schema bootstrapping alone. ## Changes - **`cmd/initialise/verify_schema.go`** (renamed from `verify_zitadel.go`): Rename `newZitadel()` → `newSchema()` (internal); rename the `init zitadel` sub-command to `zitadel init schema` (backwards-compatible alias kept) with a clear description that it bootstraps the ZITADEL database schema without admin/superuser privileges. Fix stale error log message to reference `init schema`. - **`cmd/initialise/verify_schema_test.go`** (renamed from `verify_zitadel_test.go`): Test file renamed to match source file. - **`cmd/initialise/init.go`**: Update call site to `newSchema()`. Add guidance in the `init` command's Long description about using `zitadel init schema` for users without admin credentials. - **`cmd/initialise/verify_database.go`**: Add a `pg_database` catalog pre-check before attempting `CREATE DATABASE`, so `zitadel init` with `ADMIN=service_user` no longer fails with `permission denied to create database` when the database was already provisioned externally. - **`cmd/initialise/verify_database_test.go`**: Add test cases covering the new catalog-check skip path, the existing error-skip path, and the error-propagation path when the `pg_database` query itself fails. - **`apps/docs/content/self-hosting/manage/database/index.mdx`**: Inline the `_postgres.mdx` partial (now only PostgreSQL is supported), add a top-level callout, and add a **Managed PostgreSQL / No Admin Access** section with explicit 3-step instructions and security guidance (strong passwords, SSL, TLS). Additional improvements: add inline `# Use 'require' or 'verify-full' for production` comments on `Mode: disable` lines in the YAML example; add clarifying comment to the redundant `GRANT` in the SQL snippet; replace admonition syntax with proper Fumadocs `<Callout>` components; add full database connection env vars to the `start-from-setup` example. - **`apps/docs/content/self-hosting/manage/updating_scaling.mdx`**: Rewrite the init phase description to clearly distinguish the provisioning and schema bootstrapping steps, and document `zitadel init schema` as the path for manual provisioning. Replace admonition syntax with proper Fumadocs `<Callout>` components. - **`apps/docs/content/self-hosting/manage/database/_postgres.mdx`**: Deleted (content merged into `index.mdx`). ## Problem This relates to https://github.com/zitadel/zitadel/discussions/9363 I think we can improve our UX in cases where a user wants to use an external DB and/or does not want to share too broad permissions with zitadel ## Related problems * https://github.com/zitadel/zitadel/issues/10432 * https://github.com/zitadel/zitadel/discussions/8583 * https://github.com/zitadel/zitadel/issues/7903 * https://github.com/zitadel/zitadel/issues/9718 * https://github.com/zitadel/zitadel/issues/8012 * https://github.com/zitadel/zitadel/issues/8558 ## Related PRs https://github.com/zitadel/zitadel/pull/11021
2026-10-18 05:52:51 +00:00 · 2026-03-03 07:05:32 +01:00
parent 9fb0eea125
commit 92a628d892
8 changed files with 195 additions and 78 deletions
@@ -1,56 +0,0 @@
-## ZITADEL with Postgres
-
-PostgreSQL is the default database for ZITADEL due to its reliability, robustness, and adherence to SQL standards. It is well-suited for handling the complex data requirements of an identity management system.
-
-If you are using Zitadel v2 and want to use a PostgreSQL database, you can [overwrite the default configuration](/self-hosting/manage/configure/configure).
-
-Currently, versions 14 to 18 are supported.
-
-Postgres can be configured as follows:
-```yaml
-Database:
-  postgres:
-    Host: localhost
-    Port: 5432
-    Database: zitadel
-    AwaitInitialConn: 5m
-    MaxOpenConns: 15
-    MaxIdleConns: 10
-    MaxConnLifetime: 1h
-    MaxConnIdleTime: 5m
-    Options:
-    User:
-      Username: zitadel
-      Password: zitadel
-      SSL:
-        Mode: disable
-        RootCert:
-        Cert:
-        Key:
-    Admin:
-      Username: postgres
-      Password: postgres
-      SSL:
-        Mode: disable
-        RootCert:
-        Cert:
-        Key:
-```
-
-The admin user is only needed for the installation step but can also be avoided
-by preparing the necessary requirements:
-
-* the zitadel user
-* the zitadel database
-* granting required permissions to the zitadel user
-
-```sql
-CREATE ROLE zitadel LOGIN;
-CREATE DATABASE zitadel;
-GRANT CONNECT, CREATE ON DATABASE zitadel TO zitadel;
-```
-
-Don't forget to adjust `pg_hba.conf` and set a password for the zitadel user.
-
-With the setup done, follow the [phases guide](/self-hosting/manage/updating_scaling#separating-init-and-setup-from-the-runtime)
-to run the init and then setup phase to get all necessary tables and data set up.
@@ -3,11 +3,117 @@ title: Database
 description: Configure and manage the ZITADEL database using PostgreSQL for persistent storage and audit logs
 ---

-import Postgres from './_postgres.mdx'
-
 ## Database Configuration

-<Postgres components={props.components} />
+<Callout type="info">
+Even if you provision the PostgreSQL user and database manually, you must still run `zitadel init schema`
+to bootstrap the required schemas and tables before running `start-from-setup` or `zitadel start`.
+Skipping this step causes `relation "eventstore.events" does not exist` errors.
+See the [Managed PostgreSQL / No Admin Access](#managed-postgresql--no-admin-access) section below for the exact steps.
+
+The command is `zitadel init schema` (alias: `zitadel init zitadel` for backwards compatibility).
+</Callout>
+
+## ZITADEL with Postgres
+
+PostgreSQL is the default database for ZITADEL due to its reliability, robustness, and adherence to SQL standards. It is well-suited for handling the complex data requirements of an identity management system.
+
+If you are using Zitadel v2 and want to use a PostgreSQL database, you can [overwrite the default configuration](/self-hosting/manage/configure/configure).
+
+Currently, versions 14 to 18 are supported.
+
+Postgres can be configured as follows:
+```yaml
+Database:
+  postgres:
+    Host: localhost
+    Port: 5432
+    Database: zitadel
+    AwaitInitialConn: 5m
+    MaxOpenConns: 15
+    MaxIdleConns: 10
+    MaxConnLifetime: 1h
+    MaxConnIdleTime: 5m
+    Options:
+    User:
+      Username: zitadel
+      Password: zitadel
+      SSL:
+        Mode: disable  # Use 'require' or 'verify-full' for production
+        RootCert:
+        Cert:
+        Key:
+    Admin:
+      Username: postgres
+      Password: postgres
+      SSL:
+        Mode: disable  # Use 'require' or 'verify-full' for production
+        RootCert:
+        Cert:
+        Key:
+```
+
+### Managed PostgreSQL / No Admin Access
+
+If you don't have a PostgreSQL superuser available (e.g. managed services like RDS, Cloud SQL, or Azure Database),
+you can pre-provision the user and database yourself and skip the admin-credential requirement.
+
+The `init` command performs two steps that require different privileges:
+
+- **Provisioning step** — `CREATE ROLE`, `CREATE DATABASE`, `GRANT` → requires a superuser/admin.
+- **Schema bootstrapping step** — creates schemas (`eventstore`, `projections`, `system`) and tables → requires only database owner privileges.
+
+If you handle the provisioning step manually, you still **must** run the schema bootstrapping step. Use the dedicated sub-command `zitadel init schema`, which performs only schema bootstrapping using the service user credentials — no admin privileges needed.
+
+**Provisioning step — Provision user and database (run as superuser)**
+
+<Callout type="warn">
+Replace `<strong-password>` with a strong, unique password and store it in a secret manager. Do not use trivial passwords in production.
+</Callout>
+
+```sql
+CREATE ROLE zitadel LOGIN PASSWORD '<strong-password>';
+CREATE DATABASE zitadel WITH OWNER zitadel;
+-- The GRANT below is redundant when zitadel is the database owner,
+-- but is included for clarity. Owners implicitly hold all privileges.
+GRANT CONNECT, CREATE ON DATABASE zitadel TO zitadel;
+```
+
+Adjust `pg_hba.conf` if needed to allow the zitadel user to connect.
+
+**Schema bootstrapping step — Bootstrap schemas and tables (run as the service user)**
+
+Set only the service user credentials — no `Admin.*` variables are needed for this command.
+
+For managed/remote PostgreSQL, set `SSL_MODE` to `require` or `verify-full` to encrypt the connection. `disable` is only appropriate for local development.
+
+```bash
+ZITADEL_DATABASE_POSTGRES_HOST=<your-host> \
+    ZITADEL_DATABASE_POSTGRES_PORT=5432 \
+    ZITADEL_DATABASE_POSTGRES_DATABASE=zitadel \
+    ZITADEL_DATABASE_POSTGRES_USER_USERNAME=zitadel \
+    ZITADEL_DATABASE_POSTGRES_USER_PASSWORD=<strong-password> \
+    ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE=require \
+    zitadel init schema
+```
+
+**Run setup and start**
+
+<Callout type="note">
+For production deployments, configure TLS to encrypt authentication traffic. See the [TLS configuration guide](/self-hosting/manage/tls_modes) for the available modes.
+</Callout>
+
+```bash
+ZITADEL_DATABASE_POSTGRES_HOST=<your-host> \
+    ZITADEL_DATABASE_POSTGRES_PORT=5432 \
+    ZITADEL_DATABASE_POSTGRES_DATABASE=zitadel \
+    ZITADEL_DATABASE_POSTGRES_USER_USERNAME=zitadel \
+    ZITADEL_DATABASE_POSTGRES_USER_PASSWORD=<strong-password> \
+    ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE=require \
+    zitadel start-from-setup --masterkey "<32-character-key>"
+```
+
+See the [phases guide](/self-hosting/manage/updating_scaling#separating-init-and-setup-from-the-runtime) for details on separating init, setup, and runtime for production deployments.

 ## Zitadel Credentials

@@ -53,14 +53,30 @@ you should know what happens during these phases.
 The command `zitadel init` ensures the database connection is ready to use for the subsequent phases.
 It just needs to be executed once over ZITADELs full life cycle,
 when you install ZITADEL from scratch.
-During `zitadel init`, for connecting to your database,
-ZITADEL uses the privileged and preexisting database user configured in `Database.postgres.Admin.Username`.
-, `zitadel init` ensures the following:
- If it doesn’t exist already, it creates a database with the configured database name.
- If it doesn’t exist already, it creates the unprivileged user use configured in `Database.postgres.User.Username`.
+
+`zitadel init` performs two steps that require different privileges:
+
+**Provisioning step** (requires a PostgreSQL superuser via `Database.postgres.Admin.Username`):
+- If it doesn't exist already, it creates the unprivileged user configured in `Database.postgres.User.Username`.
  Subsequent phases connect to the database with this user's credentials only.
- If not already done, it grants the necessary permissions ZITADEL needs to the non privileged user.
- If they don’t exist already, it creates all schemas and some basic tables.
+- If it doesn't exist already, it creates a database with the configured database name.
+- If not already done, it grants the necessary permissions to the unprivileged user.
+
+**Schema bootstrapping step** (requires database owner privileges, uses the service user):
+- If they don't exist already, it creates all schemas (`eventstore`, `projections`, `system`) and base tables.
+
+<Callout type="note">
+**Manual provisioning:** If you handle the provisioning step yourself (e.g. on a managed PostgreSQL service where you don't have superuser access),
+you can skip it entirely by running only the schema bootstrapping step:
+
+```bash
+zitadel init schema
+```
+
+This sub-command connects using the service user credentials and only performs schema bootstrapping.
+No `Admin.*` credentials are required.
+See the [database documentation](/self-hosting/manage/database) for full details.
+</Callout>

 The init phase is idempotent if executed with the same binary version.

@@ -44,6 +44,12 @@ The user provided by flags needs privileges to
 - create the database if it does not exist
 - see other users and create a new one if the user does not exist
 - grant all rights of the ZITADEL database to the user created if not yet set
+
+If you don't have admin/superuser credentials (e.g. on a managed PostgreSQL
+service), you can provision the database user and database manually and then
+run only 'zitadel init schema' to bootstrap the required schemas and tables
+using the service user credentials. No admin privileges are needed for that
+sub-command. See the database documentation for details.
 `,
 		RunE: func(cmd *cobra.Command, args []string) (err error) {
 			defer func() {
@@ -61,7 +67,7 @@ The user provided by flags needs privileges to
 		},
 	}

-	cmd.AddCommand(newZitadel(), newDatabase(), newUser(), newGrant())
+	cmd.AddCommand(newSchema(), newDatabase(), newUser(), newGrant())
 	return cmd
 }

@@ -58,6 +58,22 @@ func VerifyDatabase(databaseName string) func(context.Context, *database.DB) err
 			logging.Info(ctx, "database is same as config.database.postgres.admin.ExistingDatabase, skipping creation", "database", databaseName)
 			return nil
 		}
+
+		// Check if the database already exists in the catalog before attempting CREATE DATABASE.
+		// This handles the case where the database was provisioned externally and the admin
+		// credentials are the same as the service user, which lacks the CREATEDB privilege.
+		var exists bool
+		err = db.QueryRowContext(ctx, func(r *sql.Row) error {
+			return r.Scan(&exists)
+		}, "SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = $1)", databaseName)
+		if err != nil {
+			return fmt.Errorf("unable to check if database exists: %w", err)
+		}
+		if exists {
+			logging.Info(ctx, "database already exists, skipping creation", "database", databaseName)
+			return nil
+		}
+
 		logging.Info(ctx, "verify database", "database", databaseName)

 		return exec(ctx, db, fmt.Sprintf(databaseStmt, databaseName), []string{dbAlreadyExistsCode})
@@ -24,12 +24,15 @@ func Test_verifyDB(t *testing.T) {
 		targetErr error
 	}{
 		{
-			name: "doesn't exists, create fails",
+			name: "doesn't exist, create fails",
 			args: args{
 				db: prepareDB(t,
 					expectQuery("SELECT current_database()", nil, []string{"current_database"}, [][]driver.Value{
 						{"postgres"},
 					}),
+					expectQuery("SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = $1)", nil, []string{"exists"}, [][]driver.Value{
+						{false},
+					}, "zitadel"),
 					expectExec("-- replace zitadel with the name of the database\nCREATE DATABASE \"zitadel\"", sql.ErrTxDone),
 				),
 				database: "zitadel",
@@ -37,12 +40,15 @@ func Test_verifyDB(t *testing.T) {
 			targetErr: sql.ErrTxDone,
 		},
 		{
-			name: "doesn't exists, create successful",
+			name: "doesn't exist, create successful",
 			args: args{
 				db: prepareDB(t,
 					expectQuery("SELECT current_database()", nil, []string{"current_database"}, [][]driver.Value{
 						{"postgres"},
 					}),
+					expectQuery("SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = $1)", nil, []string{"exists"}, [][]driver.Value{
+						{false},
+					}, "zitadel"),
 					expectExec("-- replace zitadel with the name of the database\nCREATE DATABASE \"zitadel\"", nil),
 				),
 				database: "zitadel",
@@ -50,20 +56,35 @@ func Test_verifyDB(t *testing.T) {
 			targetErr: nil,
 		},
 		{
-			name: "already exists",
+			name: "already exists in catalog, skip creation",
 			args: args{
 				db: prepareDB(t,
 					expectQuery("SELECT current_database()", nil, []string{"current_database"}, [][]driver.Value{
 						{"postgres"},
 					}),
-					expectExec("-- replace zitadel with the name of the database\nCREATE DATABASE \"zitadel\"", nil),
+					expectQuery("SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = $1)", nil, []string{"exists"}, [][]driver.Value{
+						{true},
+					}, "zitadel"),
 				),
 				database: "zitadel",
 			},
 			targetErr: nil,
 		},
 		{
-			name: "same database, skip creation",
+			name: "catalog check fails",
+			args: args{
+				db: prepareDB(t,
+					expectQuery("SELECT current_database()", nil, []string{"current_database"}, [][]driver.Value{
+						{"postgres"},
+					}),
+					expectQuery("SELECT EXISTS(SELECT 1 FROM pg_database WHERE datname = $1)", sql.ErrConnDone, []string{"exists"}, [][]driver.Value{}, "zitadel"),
+				),
+				database: "zitadel",
+			},
+			targetErr: sql.ErrConnDone,
+		},
+		{
+			name: "same database as admin connection, skip creation",
 			args: args{
 				db: prepareDB(t,
 					expectQuery("SELECT current_database()", nil, []string{"current_database"}, [][]driver.Value{
@@ -15,18 +15,26 @@ import (
 	es_v3 "github.com/zitadel/zitadel/internal/eventstore/v3"
 )

-func newZitadel() *cobra.Command {
+func newSchema() *cobra.Command {
 	return &cobra.Command{
-		Use:   "zitadel",
-		Short: "initialize ZITADEL internals",
-		Long: `initialize ZITADEL internals.
+		Use:     "schema",
+		Aliases: []string{"zitadel"},
+		Short:   "bootstrap the ZITADEL database schema without admin/superuser privileges",
+		Long: `Bootstrap the ZITADEL database schema.
+
+Creates all required schemas (eventstore, projections, system) and base tables
+using the service user credentials. No admin/superuser privileges are required.
+
+Use this command when you have provisioned the database user and database
+yourself (e.g. on a managed PostgreSQL service) and want to skip the
+admin-credential requirement of the full 'zitadel init' command.

 Prerequisites:
- postgreSQL with user and database
+- PostgreSQL user exists and has ownership of the target database
 `,
 		RunE: func(cmd *cobra.Command, args []string) (err error) {
 			defer func() {
-				logging.OnError(cmd.Context(), err).Error("zitadel verify zitadel command failed")
+				logging.OnError(cmd.Context(), err).Error("zitadel init schema command failed")
 			}()
 			config, shutdown, err := NewConfig(cmd, viper.GetViper())
 			if err != nil {