<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
# Which Problems Are Solved
After #12740, setup on existing deployments (v4.17.x → v4.18.0) fails
before migration 78 can add `eventstore.unique_constraints.owners`.
Starting any setup step pushes `system.migration.started`, which inserts
a unique constraint that names the `owners` column. On upgraded
databases that column does not exist yet, so setup dies on
`76_users14_login_equality_indexes` with `SQLSTATE 42703`.
# How the Problems Are Solved
Empty-owner unique-constraint inserts no longer name the `owners`
column:
- `len(Owners) == 0` → `INSERT (instance_id, unique_type, unique_field)`
- tagged `WithOwners(...)` → unchanged `INSERT (..., owners)`
Setup started/done events never stamp owners, so 76–78 can start.
Migration 78 still adds the column; after that, omitted `owners` uses
`DEFAULT '{}'`. Both v2 and v3 pushers apply the same rule. Mixed pushes
split into two INSERT batches so tagged and empty rows do not share a
column list.
# Additional Changes
- v2 SQL mocks now expect the three-column INSERT for untagged adds,
plus tagged and mixed cases
- Pusher tests cover mixed tagged+empty adds and adds against a table
that does not have `owners` yet
- The missing-column pusher test drops `owners` and pushes in one
transaction (`PushWithClient`), then rolls back so the shared test
schema is not mutated. Other tests may stall on `AccessExclusiveLock`
until rollback.
# Additional Context
- Closes#12780
- Follow-up for PR #12740
- Operators do not need to re-run init or apply DDL outside setup
Verified locally:
- `go test ./internal/eventstore ./internal/v2/eventstore/postgres
./internal/eventstore/v3` (including `-race` on those packages)
- Empty-owner insert succeeds when the `owners` column is missing;
tagged insert still requires the column
<!-- CURSOR_AGENT_PR_BODY_END -->
<div><a
href="https://cursor.com/agents/bc-31bfb5c4-5888-43f0-85cf-af1af097ca30?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a
href="https://cursor.com/background-agent?bcId=bc-31bfb5c4-5888-43f0-85cf-af1af097ca30&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div>
---------
Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: Silvan <[email protected]>
<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
# Which Problems Are Solved
Bulk unique-constraint deletes (org, IDP with many links, user, project)
reconstruct every uniqueness key in Go and emit N `DELETE`s. That can
OOM on large tenants.
A one-shot owner backfill is also not enough: during a rolling deploy,
older binaries still insert `owners = '{}'`, so a later owner-only
delete would miss those rows.
Owner-only org/user/grant delete only matches rows whose `owners` bag
contains that tag. Several grant and member types stamped the aggregate
resource owner, which is not always the user’s home org or the granted
org. Username-scope rewrites also dropped `user:` tags, so owner-only
user delete could miss rewritten username rows.
Command must not switch to owner-only delete on the first 79 of a
version, because leftover `'{}'` rows from rolling-update overlap are
only backfilled on a later 79. A failed 79 must not turn owner-delete
on, and backfill joins must not stamp the wrong org (unverified domains,
org-scoped usernames colliding with instance-scoped ones).
# How the Problems Are Solved
Uniqueness stays the existing primary key `(instance_id, unique_type,
unique_field)`. `owners` is only a lifecycle bag of tags (`org:<id>`,
`user:<id>`, `idp:<id>`, `project:<id>`, `grant:<id>`).
- Add `owners TEXT[] NOT NULL DEFAULT '{}'` on
`eventstore.unique_constraints` (init SQL for greenfield, setup 78 for
existing clusters).
- Create `unique_constraints_owners_gin` with `CREATE INDEX
CONCURRENTLY` **outside a transaction** (same pattern as step 76).
Built-in `GIN (owners)` only; no `btree_gin`.
- Stamp `WithOwners` on **add** events. Field-level removes ignore
owners. `RemoveByOwner` stores the tag in `Owners` and both v1/v3 and v2
pushers run `DELETE … WHERE instance_id = $1 AND owners @>
ARRAY[$tag]::text[]`.
- Setup 79 backfills `owners` from projections via numbered
`cmd/setup/79/*.sql` files. Table names are interpolated from
`internal/query/projection` constants. Matches use `unique_field IN (x,
lower(x))` so the primary key can be used. Missing projection tables
(`42P01`) are skipped.
- Org domains are tagged only when `is_verified`. Instance-scoped
usernames join only users that are not org-scoped (organization settings
or domain policy `user_login_must_be_domain`). Every username row still
gets `org:` + `user:` of that user.
- 79 is a **repeatable** migration. `Check` re-runs when
`lastRun["version"]` changed or when setup-only `ForceFinalize` is set
and `lastRun["finalized"]` is still false. `Finalized` is written only
after Execute succeeds.
- Failed setup events keep the last successful `lastRun`. Command reads
`finalized` only from `system.migration.repeatable.done`.
- Org / user / project remove skip N-key event replay **when that
`finalized` bit is true**. Otherwise they keep today’s N-key lists
**and** still emit `RemoveByOwner`. The “ready” result is cached in
memory once true.
Setup writes `finalized` on the 79 repeatable.done payload:
- First 79 of a version: backfill only, `finalized=false` (unless
`ForceFinalize` is already set).
- Later 79: version change or
`BackfillUniqueConstraintOwners.ForceFinalize` /
`ZITADEL_BACKFILLUNIQUECONSTRAINTOWNERS_FORCEFINALIZE` after the rolling
update is done. That run backfills leftovers and writes
`finalized=true`.
Member and grant owner bags now include every org that should delete the
row:
- Instance, org, and project members also stamp
`org:<user_resource_owner>`.
- User grants stamp grant, user, project, and granted orgs.
- Project grant members stamp the project org, user org, and granted
org.
- Project grants stamp both orgs plus `grant:<id>`. Grant remove emits
`RemoveByOwner(grant)`.
- Username-scope rewrites keep `user:<id>` as well as `org:`.
- User/grant orgs are carried as unexported add-event fields (not
persisted JSON). `WithOwners` skips empty and duplicate tags.
# Additional Changes
- Table-driven owner-tag tests, including project grant (two `org:` tags
plus `grant:`), cross-org members, user grants, and username-scope
rewrite `user:` tags.
- Table-driven 79 `Check` + `finalizeAfterSuccess` tests for version
change, sequential version bumps, ForceFinalize restart, and
already-finalized skip.
- Command tests: a failed 79 event with `finalized=true` does not enable
owner-delete.
- Integration fixtures (verified vs unverified org domain;
instance-scoped `bob` vs org-scoped `bob`).
- Instance-only and global unique types stay untagged (`'{}'`). Instance
wipe still uses `UniqueConstraintInstanceRemove`.
- Pusher owner cases live in `eventstore_pusher_owners_test.go` so
`eventstore_pusher_test.go` stays under 1k lines.
# Additional Context
A later cleanup can delete the N-key replay helpers once a release that
ran 79 a second time (or an operator who set `ForceFinalize`) is in
production. That is not required for correctness.
<!-- CURSOR_AGENT_PR_BODY_END -->
<div><a
href="https://cursor.com/agents/bc-44f9bc9b-9a93-4cd9-8dd1-663a7b9aad2a?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a
href="https://cursor.com/background-agent?bcId=bc-44f9bc9b-9a93-4cd9-8dd1-663a7b9aad2a&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div>
---------
Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
- Typos
- Punctuation
- Markdown table formatting
- ...
(A bunch of issues my editor gave)
# How the Problems Are Solved
# Additional Changes
# Additional Context
# Which Problems Are Solved
The following terms have all been renamed to management console:
- Customer Portal (when used to mean the console)
- Console
- Admin Console
# How the Problems Are Solved
- Search & Replace smartly
- Use Copilot for translation files
Changes done to: backend + frontend codebase, docs, translations and
protobufs (descriptions only)
# Additional Context
- Partially Closes#11279
# Eventstore fixes
- `event.Position` used float64 before which can lead to [precision
loss](https://github.com/golang/go/issues/47300). The type got replaced
by [a type without precision
loss](https://github.com/jackc/pgx-shopspring-decimal)
- the handler reported the wrong error if the current state was updated
and therefore took longer to retry failed events.
# Mirror fixes
- max age of auth requests can be configured to speed up copying data
from `auth.auth_requests` table. Auth requests last updated before the
set age will be ignored. Default is 1 month
- notification projections are skipped because notifications should be
sent by the source system. The projections are set to the latest
position
- ensure that mirror can be executed multiple times
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Float64 which was used for the event.Position field is [not precise in
go and gets rounded](https://github.com/golang/go/issues/47300). This
can lead to unprecies position tracking of events and therefore
projections especially on cockcoachdb as the position used there is a
big number.
example of a unprecies position:
exact: 1725257931223002628
float64: 1725257931223002624.000000
# How the Problems Are Solved
The float64 was replaced by
[github.com/jackc/pgx-shopspring-decimal](https://github.com/jackc/pgx-shopspring-decimal).
# Additional Changes
Correct behaviour of makefile for load tests.
Rename `latestSequence`-queries to `latestPosition`
# Which Problems Are Solved
The mirror command used the wrong position to filter for events if
different database technologies for source and destination were used.
# How the Problems Are Solved
The statements which diverge are stored on the client so that different
technologies can use different statements.
# Additional Context
- https://discord.com/channels/927474939156643850/1256396896243552347
# Which Problems Are Solved
Postgres versions < 16 require an alias for subqueries. The query
executed in the new eventstore didn't add this alias.
# How the Problems Are Solved
Added the alias to the subquery
# Which Problems Are Solved
Adds the possibility to mirror an existing database to a new one.
For that a new command was added `zitadel mirror`. Including it's
subcommands for a more fine grained mirror of the data.
Sub commands:
* `zitadel mirror eventstore`: copies only events and their unique
constraints
* `zitadel mirror system`: mirrors the data of the `system`-schema
* `zitadel mirror projections`: runs all projections
* `zitadel mirror auth`: copies auth requests
* `zitadel mirror verify`: counts the amount of rows in the source and
destination database and prints the diff.
The command requires one of the following flags:
* `--system`: copies all instances of the system
* `--instance <instance-id>`, `--instance <comma separated list of
instance ids>`: copies only the defined instances
The command is save to execute multiple times by adding the
`--replace`-flag. This replaces currently existing data except of the
`events`-table
# Additional Changes
A `--for-mirror`-flag was added to `zitadel setup` to prepare the new
database. The flag skips the creation of the first instances and initial
run of projections.
It is now possible to skip the creation of the first instance during
setup by setting `FirstInstance.Skip` to true in the steps
configuration.
# Additional info
It is currently not possible to merge multiple databases. See
https://github.com/zitadel/zitadel/issues/7964 for more details.
It is currently not possible to use files. See
https://github.com/zitadel/zitadel/issues/7966 for more information.
closes https://github.com/zitadel/zitadel/issues/7586
closes https://github.com/zitadel/zitadel/issues/7486
### Definition of Ready
- [x] I am happy with the code
- [x] Short description of the feature/issue is added in the pr
description
- [x] PR is linked to the corresponding user story
- [x] Acceptance criteria are met
- [x] All open todos and follow ups are defined in a new ticket and
justified
- [x] Deviations from the acceptance criteria and design are agreed with
the PO and documented.
- [x] No debug or dead code
- [x] My code has no repetitions
- [x] Critical parts are tested automatically
- [ ] Where possible E2E tests are implemented
- [x] Documentation/examples are up-to-date
- [x] All non-functional requirements are met
- [x] Functionality of the acceptance criteria is checked manually on
the dev system.
---------
Co-authored-by: Livio Spring <[email protected]>
Wim Van LaerandGitHub
Marco A.andGitHub
Tim MöhlmannandGitHub
SilvanandGitHub
