fix(eventstore): add owners to unique constraints (#12740)

<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
# Which Problems Are Solved

Bulk unique-constraint deletes (org, IDP with many links, user, project)
reconstruct every uniqueness key in Go and emit N `DELETE`s. That can
OOM on large tenants.

A one-shot owner backfill is also not enough: during a rolling deploy,
older binaries still insert `owners = '{}'`, so a later owner-only
delete would miss those rows.

Owner-only org/user/grant delete only matches rows whose `owners` bag
contains that tag. Several grant and member types stamped the aggregate
resource owner, which is not always the user’s home org or the granted
org. Username-scope rewrites also dropped `user:` tags, so owner-only
user delete could miss rewritten username rows.

Command must not switch to owner-only delete on the first 79 of a
version, because leftover `'{}'` rows from rolling-update overlap are
only backfilled on a later 79. A failed 79 must not turn owner-delete
on, and backfill joins must not stamp the wrong org (unverified domains,
org-scoped usernames colliding with instance-scoped ones).

# How the Problems Are Solved

Uniqueness stays the existing primary key `(instance_id, unique_type,
unique_field)`. `owners` is only a lifecycle bag of tags (`org:<id>`,
`user:<id>`, `idp:<id>`, `project:<id>`, `grant:<id>`).

- Add `owners TEXT[] NOT NULL DEFAULT '{}'` on
`eventstore.unique_constraints` (init SQL for greenfield, setup 78 for
existing clusters).
- Create `unique_constraints_owners_gin` with `CREATE INDEX
CONCURRENTLY` **outside a transaction** (same pattern as step 76).
Built-in `GIN (owners)` only; no `btree_gin`.
- Stamp `WithOwners` on **add** events. Field-level removes ignore
owners. `RemoveByOwner` stores the tag in `Owners` and both v1/v3 and v2
pushers run `DELETE … WHERE instance_id = $1 AND owners @>
ARRAY[$tag]::text[]`.
- Setup 79 backfills `owners` from projections via numbered
`cmd/setup/79/*.sql` files. Table names are interpolated from
`internal/query/projection` constants. Matches use `unique_field IN (x,
lower(x))` so the primary key can be used. Missing projection tables
(`42P01`) are skipped.
- Org domains are tagged only when `is_verified`. Instance-scoped
usernames join only users that are not org-scoped (organization settings
or domain policy `user_login_must_be_domain`). Every username row still
gets `org:` + `user:` of that user.
- 79 is a **repeatable** migration. `Check` re-runs when
`lastRun["version"]` changed or when setup-only `ForceFinalize` is set
and `lastRun["finalized"]` is still false. `Finalized` is written only
after Execute succeeds.
- Failed setup events keep the last successful `lastRun`. Command reads
`finalized` only from `system.migration.repeatable.done`.
- Org / user / project remove skip N-key event replay **when that
`finalized` bit is true**. Otherwise they keep today’s N-key lists
**and** still emit `RemoveByOwner`. The “ready” result is cached in
memory once true.

Setup writes `finalized` on the 79 repeatable.done payload:

- First 79 of a version: backfill only, `finalized=false` (unless
`ForceFinalize` is already set).
- Later 79: version change or
`BackfillUniqueConstraintOwners.ForceFinalize` /
`ZITADEL_BACKFILLUNIQUECONSTRAINTOWNERS_FORCEFINALIZE` after the rolling
update is done. That run backfills leftovers and writes
`finalized=true`.

Member and grant owner bags now include every org that should delete the
row:

- Instance, org, and project members also stamp
`org:<user_resource_owner>`.
- User grants stamp grant, user, project, and granted orgs.
- Project grant members stamp the project org, user org, and granted
org.
- Project grants stamp both orgs plus `grant:<id>`. Grant remove emits
`RemoveByOwner(grant)`.
- Username-scope rewrites keep `user:<id>` as well as `org:`.
- User/grant orgs are carried as unexported add-event fields (not
persisted JSON). `WithOwners` skips empty and duplicate tags.

# Additional Changes

- Table-driven owner-tag tests, including project grant (two `org:` tags
plus `grant:`), cross-org members, user grants, and username-scope
rewrite `user:` tags.
- Table-driven 79 `Check` + `finalizeAfterSuccess` tests for version
change, sequential version bumps, ForceFinalize restart, and
already-finalized skip.
- Command tests: a failed 79 event with `finalized=true` does not enable
owner-delete.
- Integration fixtures (verified vs unverified org domain;
instance-scoped `bob` vs org-scoped `bob`).
- Instance-only and global unique types stay untagged (`'{}'`). Instance
wipe still uses `UniqueConstraintInstanceRemove`.
- Pusher owner cases live in `eventstore_pusher_owners_test.go` so
`eventstore_pusher_test.go` stays under 1k lines.

# Additional Context

A later cleanup can delete the N-key replay helpers once a release that
ran 79 a second time (or an operator who set `ForceFinalize`) is in
production. That is not required for correctness.

<!-- CURSOR_AGENT_PR_BODY_END -->

<div><a
href="https://cursor.com/agents/bc-44f9bc9b-9a93-4cd9-8dd1-663a7b9aad2a?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a>&nbsp;<a
href="https://cursor.com/background-agent?bcId=bc-44f9bc9b-9a93-4cd9-8dd1-663a7b9aad2a&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a>&nbsp;</div>

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: Silvan <[email protected]>

cmd/setup/78.go

@@ -0,0 +1,35 @@
+package setup
+
+import (
+	"context"
+	_ "embed"
+
+	"github.com/zitadel/zitadel/backend/v3/instrumentation/logging"
+	"github.com/zitadel/zitadel/internal/database"
+	"github.com/zitadel/zitadel/internal/eventstore"
+)
+
+var (
+	//go:embed 78_owners_column.sql
+	uniqueConstraintsOwnersColumn string
+	//go:embed 78_owners_gin.sql
+	uniqueConstraintsOwnersGIN string
+)
+
+type UniqueConstraintOwners struct {
+	dbClient *database.DB
+}
+
+func (mig *UniqueConstraintOwners) Execute(ctx context.Context, _ eventstore.Event) error {
+	logging.Info(ctx, "add unique constraint owners column", "migration", mig.String())
+	if _, err := mig.dbClient.ExecContext(ctx, uniqueConstraintsOwnersColumn); err != nil {
+		return err
+	}
+	logging.Info(ctx, "create unique constraint owners gin index concurrently", "migration", mig.String())
+	_, err := mig.dbClient.ExecContext(ctx, uniqueConstraintsOwnersGIN)
+	return err
+}
+
+func (mig *UniqueConstraintOwners) String() string {
+	return "78_unique_constraint_owners"
+}

cmd/setup/78_owners_column.sql

@@ -0,0 +1 @@
+ALTER TABLE eventstore.unique_constraints ADD COLUMN IF NOT EXISTS owners TEXT[] NOT NULL DEFAULT '{}';

cmd/setup/78_owners_gin.sql

@@ -0,0 +1 @@
+CREATE INDEX CONCURRENTLY IF NOT EXISTS unique_constraints_owners_gin ON eventstore.unique_constraints USING GIN (owners);

cmd/setup/78_test.go

@@ -0,0 +1,28 @@
+package setup
+
+import (
+	"context"
+	"regexp"
+	"testing"
+
+	"github.com/DATA-DOG/go-sqlmock"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+
+	"github.com/zitadel/zitadel/internal/database"
+)
+
+func TestUniqueConstraintOwners_Execute(t *testing.T) {
+	db, mock, err := sqlmock.New()
+	require.NoError(t, err)
+	defer db.Close()
+
+	mock.ExpectExec(regexp.QuoteMeta(uniqueConstraintsOwnersColumn)).
+		WillReturnResult(sqlmock.NewResult(0, 0))
+	mock.ExpectExec(regexp.QuoteMeta(uniqueConstraintsOwnersGIN)).
+		WillReturnResult(sqlmock.NewResult(0, 0))
+
+	mig := &UniqueConstraintOwners{dbClient: &database.DB{DB: db}}
+	assert.NoError(t, mig.Execute(context.Background(), nil))
+	assert.NoError(t, mock.ExpectationsWereMet())
+}

cmd/setup/79.go

@@ -0,0 +1,139 @@
+package setup
+
+import (
+	"context"
+	"database/sql"
+	"embed"
+	"errors"
+	"strings"
+
+	"github.com/jackc/pgx/v5/pgconn"
+
+	"github.com/zitadel/zitadel/backend/v3/instrumentation/logging"
+	"github.com/zitadel/zitadel/internal/database"
+	"github.com/zitadel/zitadel/internal/eventstore"
+	"github.com/zitadel/zitadel/internal/query/projection"
+)
+
+var (
+	//go:embed 79/*.sql
+	backfillUniqueConstraintOwnersFS embed.FS
+)
+
+type BackfillUniqueConstraintOwners struct {
+	dbClient *database.DB
+
+	Version       string `json:"version"`
+	Finalized     bool   `json:"finalized"`
+	ForceFinalize bool   `json:"-"` // YAML/env only; not persisted on lastRun
+
+	lastVersion   string `json:"-"`
+	lastFinalized bool   `json:"-"`
+}
+
+func (mig *BackfillUniqueConstraintOwners) Check(lastRun map[string]interface{}) bool {
+	if lastRun == nil {
+		lastRun = map[string]interface{}{}
+	}
+	mig.lastVersion, _ = lastRun["version"].(string)
+	mig.lastFinalized, _ = lastRun["finalized"].(bool)
+
+	versionChanged := mig.lastVersion != mig.Version
+	if mig.lastVersion == "" {
+		return true
+	}
+	return versionChanged || (mig.ForceFinalize && !mig.lastFinalized)
+}
+
+func (mig *BackfillUniqueConstraintOwners) finalizeAfterSuccess() {
+	mig.Finalized = mig.lastFinalized || mig.ForceFinalize || (mig.lastVersion != "" && mig.lastVersion != mig.Version)
+}
+
+func (mig *BackfillUniqueConstraintOwners) Execute(ctx context.Context, _ eventstore.Event) error {
+	statements, err := backfillUniqueConstraintOwnersStatements()
+	if err != nil {
+		return err
+	}
+	for _, stmt := range statements {
+		logging.Info(ctx, "backfill unique constraint owners", "file", stmt.file, "migration", mig.String())
+		if _, err := mig.dbClient.ExecContext(ctx, stmt.query); err != nil {
+			if isUndefinedTable(err) {
+				logging.Info(ctx, "skip unique constraint owners backfill, relation missing", "file", stmt.file, "migration", mig.String())
+				continue
+			}
+			return err
+		}
+	}
+
+	var unmatched int64
+	err = mig.dbClient.QueryRowContext(ctx, func(row *sql.Row) error {
+		return row.Scan(&unmatched)
+	}, `SELECT COUNT(*) FROM eventstore.unique_constraints WHERE unique_type = ANY($1) AND owners = '{}'`,
+		database.TextArray[string](eventstore.UniqueTypesWithOwners))
+	if err != nil {
+		return err
+	}
+	mig.finalizeAfterSuccess()
+	logging.Info(ctx, "unique constraint owners backfill complete", "unmatched", unmatched, "finalized", mig.Finalized, "migration", mig.String())
+	return nil
+}
+
+func (mig *BackfillUniqueConstraintOwners) String() string {
+	return eventstore.UniqueConstraintOwnersBackfillStep
+}
+
+func isUndefinedTable(err error) bool {
+	var pgErr *pgconn.PgError
+	return errors.As(err, &pgErr) && pgErr.Code == "42P01"
+}
+
+func interpolateBackfillSQL(query string) string {
+	return strings.NewReplacer(
+		"{{.users}}", projection.UserTable,
+		"{{.organization_settings}}", projection.OrganizationSettingsTable,
+		"{{.domain_policies}}", projection.DomainPolicyTable,
+		"{{.idp_user_links}}", projection.IDPUserLinkTable,
+		"{{.orgs}}", projection.OrgProjectionTable,
+		"{{.org_domains}}", projection.OrgDomainTable,
+		"{{.projects}}", projection.ProjectProjectionTable,
+		"{{.apps}}", projection.AppProjectionTable,
+		"{{.apps_saml}}", projection.AppSAMLTable,
+		"{{.project_roles}}", projection.ProjectRoleProjectionTable,
+		"{{.project_grants}}", projection.ProjectGrantProjectionTable,
+		"{{.project_grant_members}}", projection.ProjectGrantMemberProjectionTable,
+		"{{.user_grants}}", projection.UserGrantProjectionTable,
+		"{{.org_members}}", projection.OrgMemberProjectionTable,
+		"{{.project_members}}", projection.ProjectMemberProjectionTable,
+		"{{.instance_members}}", projection.InstanceMemberProjectionTable,
+		"{{.groups}}", projection.GroupProjectionTable,
+		"{{.actions}}", projection.ActionTable,
+		"{{.idps}}", projection.IDPTable,
+		"{{.idp_templates}}", projection.IDPTemplateTable,
+		"{{.message_texts}}", projection.MessageTextTable,
+	).Replace(query)
+}
+
+func backfillUniqueConstraintOwnersStatements() ([]statement, error) {
+	statements, err := readStatements(backfillUniqueConstraintOwnersFS, "79")
+	if err != nil {
+		return nil, err
+	}
+	for i := range statements {
+		statements[i].query = interpolateBackfillSQL(statements[i].query)
+	}
+	return statements, nil
+}
+
+// BackfillUniqueConstraintOwnersQuery returns the interpolated SQL for a setup 79 file.
+func BackfillUniqueConstraintOwnersQuery(file string) (string, error) {
+	statements, err := backfillUniqueConstraintOwnersStatements()
+	if err != nil {
+		return "", err
+	}
+	for _, stmt := range statements {
+		if stmt.file == file {
+			return stmt.query, nil
+		}
+	}
+	return "", errors.New("unknown backfill statement: " + file)
+}

cmd/setup/79/01_usernames_org_scoped.sql

@@ -0,0 +1,8 @@
+-- unique_field: username+resourceOwner (user.NewAddUsernameUniqueConstraint, org-scoped)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || u.resource_owner, 'user:' || u.id]
+FROM {{.users}} u
+WHERE uc.unique_type = 'usernames'
+	AND uc.instance_id = u.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (u.username || u.resource_owner, lower(u.username || u.resource_owner));

cmd/setup/79/02_usernames.sql

@@ -0,0 +1,16 @@
+-- unique_field: username (user.NewAddUsernameUniqueConstraint, instance-scoped)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || u.resource_owner, 'user:' || u.id]
+FROM {{.users}} u
+LEFT JOIN {{.organization_settings}} s
+	ON s.instance_id = u.instance_id AND s.id = u.resource_owner
+LEFT JOIN {{.domain_policies}} org_pol
+	ON org_pol.instance_id = u.instance_id AND org_pol.id = u.resource_owner
+LEFT JOIN {{.domain_policies}} inst_pol
+	ON inst_pol.instance_id = u.instance_id AND inst_pol.is_default
+WHERE uc.unique_type = 'usernames'
+	AND uc.instance_id = u.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (u.username, lower(u.username))
+	AND NOT COALESCE(s.organization_scoped_usernames, false)
+	AND NOT COALESCE(org_pol.user_login_must_be_domain, inst_pol.user_login_must_be_domain, false);

cmd/setup/79/03_external_idps.sql

@@ -0,0 +1,8 @@
+-- unique_field: idpConfigID+externalUserID (user.NewAddUserIDPLinkUniqueConstraint)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || l.resource_owner, 'idp:' || l.idp_id, 'user:' || l.user_id]
+FROM {{.idp_user_links}} l
+WHERE uc.unique_type = 'external_idps'
+	AND uc.instance_id = l.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (l.idp_id || l.external_user_id, lower(l.idp_id || l.external_user_id));

cmd/setup/79/04_org_name.sql

@@ -0,0 +1,8 @@
+-- unique_field: org name (org.NewAddOrgNameUniqueConstraint)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || o.id]
+FROM {{.orgs}} o
+WHERE uc.unique_type = 'org_name'
+	AND uc.instance_id = o.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (o.name, lower(o.name));

cmd/setup/79/05_org_domain.sql

@@ -0,0 +1,9 @@
+-- unique_field: domain (org.NewAddOrgDomainUniqueConstraint)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || d.org_id]
+FROM {{.org_domains}} d
+WHERE uc.unique_type = 'org_domain'
+	AND uc.instance_id = d.instance_id
+	AND uc.owners = '{}'
+	AND d.is_verified
+	AND uc.unique_field IN (d.domain, lower(d.domain));

cmd/setup/79/06_project_names.sql

@@ -0,0 +1,8 @@
+-- unique_field: name+resourceOwner (project.NewAddProjectNameUniqueConstraint)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || p.resource_owner, 'project:' || p.id]
+FROM {{.projects}} p
+WHERE uc.unique_type = 'project_names'
+	AND uc.instance_id = p.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (p.name || p.resource_owner, lower(p.name || p.resource_owner));

cmd/setup/79/07_appname.sql

@@ -0,0 +1,8 @@
+-- unique_field: name:projectID (project.NewAddApplicationUniqueConstraint)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || a.resource_owner, 'project:' || a.project_id]
+FROM {{.apps}} a
+WHERE uc.unique_type = 'appname'
+	AND uc.instance_id = a.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (a.name || ':' || a.project_id, lower(a.name || ':' || a.project_id));

cmd/setup/79/08_project_role.sql

@@ -0,0 +1,8 @@
+-- unique_field: roleKey:projectID (project.NewAddProjectRoleUniqueConstraint)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || r.resource_owner, 'project:' || r.project_id]
+FROM {{.project_roles}} r
+WHERE uc.unique_type = 'project_role'
+	AND uc.instance_id = r.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (r.role_key || ':' || r.project_id, lower(r.role_key || ':' || r.project_id));

cmd/setup/79/09_entity_ids.sql

@@ -0,0 +1,9 @@
+-- unique_field: entityID (project.NewAddSAMLConfigEntityIDUniqueConstraint)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || a.resource_owner, 'project:' || a.project_id]
+FROM {{.apps}} a
+JOIN {{.apps_saml}} s ON s.app_id = a.id AND s.instance_id = a.instance_id
+WHERE uc.unique_type = 'entity_ids'
+	AND uc.instance_id = a.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (s.entity_id, lower(s.entity_id));

cmd/setup/79/10_project_grant.sql

@@ -0,0 +1,9 @@
+-- unique_field: grantedOrgID:projectID (project.NewAddProjectGrantUniqueConstraint)
+-- resource_owner is the project org and granted_org_id is the granted org; they differ
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || g.resource_owner, 'org:' || g.granted_org_id, 'project:' || g.project_id, 'grant:' || g.grant_id]
+FROM {{.project_grants}} g
+WHERE uc.unique_type = 'project_grant'
+	AND uc.instance_id = g.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (g.granted_org_id || ':' || g.project_id, lower(g.granted_org_id || ':' || g.project_id));

cmd/setup/79/11_project_grant_member.sql

@@ -0,0 +1,11 @@
+-- unique_field: projectID:userID:grantID (project.NewAddProjectGrantMemberUniqueConstraint)
+-- resource_owner is the project org; also stamp the user's home org and the granted org
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || m.resource_owner, 'org:' || m.user_resource_owner, 'user:' || m.user_id, 'project:' || m.project_id]
+	|| CASE WHEN COALESCE(m.granted_org, '') <> '' THEN ARRAY['org:' || m.granted_org] ELSE ARRAY[]::text[] END
+	|| CASE WHEN COALESCE(m.grant_id, '') <> '' THEN ARRAY['grant:' || m.grant_id] ELSE ARRAY[]::text[] END
+FROM {{.project_grant_members}} m
+WHERE uc.unique_type = 'project_grant_member'
+	AND uc.instance_id = m.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (m.project_id || ':' || m.user_id || ':' || m.grant_id, lower(m.project_id || ':' || m.user_id || ':' || m.grant_id));

cmd/setup/79/12_user_grant.sql

@@ -0,0 +1,11 @@
+-- unique_field: resourceOwner:userID:projectID:grantID (usergrant.NewAddUserGrantUniqueConstraint)
+-- grant resource_owner is not always the user's org; stamp user, project, and granted orgs
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || g.resource_owner, 'org:' || g.resource_owner_user, 'org:' || g.resource_owner_project, 'user:' || g.user_id, 'project:' || g.project_id]
+	|| CASE WHEN COALESCE(g.granted_org, '') <> '' THEN ARRAY['org:' || g.granted_org] ELSE ARRAY[]::text[] END
+	|| CASE WHEN COALESCE(g.grant_id, '') <> '' THEN ARRAY['grant:' || g.grant_id] ELSE ARRAY[]::text[] END
+FROM {{.user_grants}} g
+WHERE uc.unique_type = 'user_grant'
+	AND uc.instance_id = g.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (g.resource_owner || ':' || g.user_id || ':' || g.project_id || ':' || g.grant_id, lower(g.resource_owner || ':' || g.user_id || ':' || g.project_id || ':' || g.grant_id));

cmd/setup/79/13_member_org.sql

@@ -0,0 +1,9 @@
+-- unique_field: orgID:userID (member.NewAddMemberUniqueConstraint, org aggregate)
+-- stamp both the membership org and the user's home org (they differ for cross-org members)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || m.org_id, 'org:' || m.user_resource_owner, 'user:' || m.user_id]
+FROM {{.org_members}} m
+WHERE uc.unique_type = 'member'
+	AND uc.instance_id = m.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (m.org_id || ':' || m.user_id, lower(m.org_id || ':' || m.user_id));

cmd/setup/79/14_member_project.sql

@@ -0,0 +1,9 @@
+-- unique_field: projectID:userID (member.NewAddMemberUniqueConstraint, project aggregate)
+-- stamp the project org and the user's home org (they differ for cross-org members)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || m.resource_owner, 'org:' || m.user_resource_owner, 'user:' || m.user_id, 'project:' || m.project_id]
+FROM {{.project_members}} m
+WHERE uc.unique_type = 'member'
+	AND uc.instance_id = m.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (m.project_id || ':' || m.user_id, lower(m.project_id || ':' || m.user_id));

cmd/setup/79/15_member_instance.sql

@@ -0,0 +1,9 @@
+-- unique_field: instanceID:userID (member.NewAddMemberUniqueConstraint, instance aggregate)
+-- instance members live on the instance aggregate, so also stamp the user's home org
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['user:' || m.user_id, 'org:' || m.user_resource_owner]
+FROM {{.instance_members}} m
+WHERE uc.unique_type = 'member'
+	AND uc.instance_id = m.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (m.id || ':' || m.user_id, lower(m.id || ':' || m.user_id));

cmd/setup/79/16_group_name.sql

@@ -0,0 +1,8 @@
+-- unique_field: name:resourceOwner (group.NewAddGroupNameUniqueConstraint)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || g.resource_owner]
+FROM {{.groups}} g
+WHERE uc.unique_type = 'group_name'
+	AND uc.instance_id = g.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (g.name || ':' || g.resource_owner, lower(g.name || ':' || g.resource_owner));

cmd/setup/79/17_action_names.sql

@@ -0,0 +1,8 @@
+-- unique_field: name:resourceOwner (action.NewAddActionNameUniqueConstraint)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || a.resource_owner]
+FROM {{.actions}} a
+WHERE uc.unique_type = 'action_names'
+	AND uc.instance_id = a.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (a.name || ':' || a.resource_owner, lower(a.name || ':' || a.resource_owner));

cmd/setup/79/18_idp_config_names.sql

@@ -0,0 +1,8 @@
+-- unique_field: name+resourceOwner (idpconfig.NewAddIDPConfigNameUniqueConstraint)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || i.resource_owner, 'idp:' || i.id]
+FROM {{.idps}} i
+WHERE uc.unique_type = 'idp_config_names'
+	AND uc.instance_id = i.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (i.name || i.resource_owner, lower(i.name || i.resource_owner));

cmd/setup/79/19_idp_config_names_templates.sql

@@ -0,0 +1,8 @@
+-- unique_field: name+resourceOwner (idpconfig.NewAddIDPConfigNameUniqueConstraint, templates)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || t.resource_owner, 'idp:' || t.id]
+FROM {{.idp_templates}} t
+WHERE uc.unique_type = 'idp_config_names'
+	AND uc.instance_id = t.instance_id
+	AND uc.owners = '{}'
+	AND uc.unique_field IN (t.name || t.resource_owner, lower(t.name || t.resource_owner));

cmd/setup/79/20_mail_text.sql

@@ -0,0 +1,9 @@
+-- unique_field: aggregateID:mailTextType:language (policy.NewAddMailTextUniqueConstraint)
+UPDATE eventstore.unique_constraints uc
+SET owners = ARRAY['org:' || t.aggregate_id]
+FROM {{.message_texts}} t
+WHERE uc.unique_type = 'mail_text'
+	AND uc.instance_id = t.instance_id
+	AND uc.owners = '{}'
+	AND t.aggregate_id <> t.instance_id
+	AND uc.unique_field IN (t.aggregate_id || ':' || t.type || ':' || t.language, lower(t.aggregate_id || ':' || t.type || ':' || t.language));

cmd/setup/79_test.go

@@ -0,0 +1,213 @@
+package setup
+
+import (
+	"encoding/json"
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+
+	"github.com/zitadel/zitadel/internal/eventstore"
+	"github.com/zitadel/zitadel/internal/migration"
+	"github.com/zitadel/zitadel/internal/query/projection"
+)
+
+var _ migration.RepeatableMigration = (*BackfillUniqueConstraintOwners)(nil)
+
+func TestBackfillUniqueConstraintOwnersStmts(t *testing.T) {
+	statements, err := backfillUniqueConstraintOwnersStatements()
+	require.NoError(t, err)
+
+	want := []struct {
+		file       string
+		uniqueType string
+		field      string
+		table      string
+		extra      []string
+	}{
+		{"01_usernames_org_scoped.sql", "usernames", "u.username || u.resource_owner", projection.UserTable, []string{"'org:' || u.resource_owner"}},
+		{"02_usernames.sql", "usernames", "u.username, lower(u.username)", projection.UserTable, []string{"'org:' || u.resource_owner", "organization_scoped_usernames", "user_login_must_be_domain"}},
+		{"03_external_idps.sql", "external_idps", "l.idp_id || l.external_user_id", projection.IDPUserLinkTable, []string{"'idp:' || l.idp_id"}},
+		{"04_org_name.sql", "org_name", "o.name", projection.OrgProjectionTable, nil},
+		{"05_org_domain.sql", "org_domain", "d.domain", projection.OrgDomainTable, []string{"d.is_verified"}},
+		{"06_project_names.sql", "project_names", "p.name || p.resource_owner", projection.ProjectProjectionTable, nil},
+		{"07_appname.sql", "appname", "a.name || ':' || a.project_id", projection.AppProjectionTable, nil},
+		{"08_project_role.sql", "project_role", "r.role_key || ':' || r.project_id", projection.ProjectRoleProjectionTable, nil},
+		{"09_entity_ids.sql", "entity_ids", "s.entity_id", projection.AppProjectionTable, []string{projection.AppSAMLTable}},
+		{"10_project_grant.sql", "project_grant", "g.granted_org_id || ':' || g.project_id", projection.ProjectGrantProjectionTable, []string{"'org:' || g.granted_org_id", "'grant:' || g.grant_id"}},
+		{"11_project_grant_member.sql", "project_grant_member", "m.project_id || ':' || m.user_id || ':' || m.grant_id", projection.ProjectGrantMemberProjectionTable, []string{"'grant:' || m.grant_id", "m.user_resource_owner", "m.granted_org"}},
+		{"12_user_grant.sql", "user_grant", "g.resource_owner || ':' || g.user_id || ':' || g.project_id || ':' || g.grant_id", projection.UserGrantProjectionTable, []string{"'grant:' || g.grant_id", "g.resource_owner_user", "g.granted_org"}},
+		{"13_member_org.sql", "member", "m.org_id || ':' || m.user_id", projection.OrgMemberProjectionTable, []string{"m.user_resource_owner"}},
+		{"14_member_project.sql", "member", "m.project_id || ':' || m.user_id", projection.ProjectMemberProjectionTable, []string{"m.user_resource_owner"}},
+		{"15_member_instance.sql", "member", "m.id || ':' || m.user_id", projection.InstanceMemberProjectionTable, []string{"m.user_resource_owner"}},
+		{"16_group_name.sql", "group_name", "g.name || ':' || g.resource_owner", projection.GroupProjectionTable, nil},
+		{"17_action_names.sql", "action_names", "a.name || ':' || a.resource_owner", projection.ActionTable, nil},
+		{"18_idp_config_names.sql", "idp_config_names", "i.name || i.resource_owner", projection.IDPTable, []string{"'idp:' || i.id"}},
+		{"19_idp_config_names_templates.sql", "idp_config_names", "t.name || t.resource_owner", projection.IDPTemplateTable, []string{"'idp:' || t.id"}},
+		{"20_mail_text.sql", "mail_text", "t.aggregate_id || ':' || t.type || ':' || t.language", projection.MessageTextTable, []string{"t.aggregate_id <> t.instance_id"}},
+	}
+
+	require.Len(t, statements, len(want))
+	for i, stmt := range statements {
+		assert.Equal(t, want[i].file, stmt.file)
+		assert.Contains(t, stmt.query, "unique_type = '"+want[i].uniqueType+"'")
+		assert.Contains(t, stmt.query, want[i].field)
+		assert.Contains(t, stmt.query, want[i].table)
+		assert.Contains(t, stmt.query, "uc.unique_field IN (")
+		assert.NotContains(t, stmt.query, "lower(uc.unique_field)")
+		assert.NotContains(t, stmt.query, "{{.")
+		for _, extra := range want[i].extra {
+			assert.Contains(t, stmt.query, extra)
+		}
+	}
+}
+
+func TestBackfillUniqueConstraintOwners_CheckAndFinalize(t *testing.T) {
+	tests := []struct {
+		name          string
+		forceFinalize bool
+		lastRun       map[string]interface{}
+		wantRun       bool
+		wantFinalized bool
+	}{
+		{
+			name:          "missing lastRun runs without finalizing",
+			lastRun:       nil,
+			wantRun:       true,
+			wantFinalized: false,
+		},
+		{
+			name:          "empty lastRun runs without finalizing",
+			lastRun:       map[string]interface{}{},
+			wantRun:       true,
+			wantFinalized: false,
+		},
+		{
+			name:          "missing lastRun with ForceFinalize finalizes",
+			forceFinalize: true,
+			lastRun:       nil,
+			wantRun:       true,
+			wantFinalized: true,
+		},
+		{
+			name:          "same version not finalized skips",
+			lastRun:       map[string]interface{}{"version": "v2.0.0", "finalized": false},
+			wantRun:       false,
+			wantFinalized: false,
+		},
+		{
+			name:          "same version missing finalized skips",
+			lastRun:       map[string]interface{}{"version": "v2.0.0"},
+			wantRun:       false,
+			wantFinalized: false,
+		},
+		{
+			name:          "same version not finalized with ForceFinalize runs and finalizes",
+			forceFinalize: true,
+			lastRun:       map[string]interface{}{"version": "v2.0.0", "finalized": false},
+			wantRun:       true,
+			wantFinalized: true,
+		},
+		{
+			name:          "same version already finalized skips",
+			lastRun:       map[string]interface{}{"version": "v2.0.0", "finalized": true},
+			wantRun:       false,
+			wantFinalized: true,
+		},
+		{
+			name:          "same version already finalized with ForceFinalize skips",
+			forceFinalize: true,
+			lastRun:       map[string]interface{}{"version": "v2.0.0", "finalized": true},
+			wantRun:       false,
+			wantFinalized: true,
+		},
+		{
+			name:          "version changed runs and finalizes",
+			lastRun:       map[string]interface{}{"version": "v1.0.0", "finalized": false},
+			wantRun:       true,
+			wantFinalized: true,
+		},
+		{
+			name:          "version changed keeps finalized true",
+			lastRun:       map[string]interface{}{"version": "v1.0.0", "finalized": true},
+			wantRun:       true,
+			wantFinalized: true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			mig := &BackfillUniqueConstraintOwners{Version: "v2.0.0", ForceFinalize: tt.forceFinalize}
+			assert.Equal(t, tt.wantRun, mig.Check(tt.lastRun))
+			assert.False(t, mig.Finalized, "Check must not set finalized before Execute")
+			mig.finalizeAfterSuccess()
+			assert.Equal(t, tt.wantFinalized, mig.Finalized)
+		})
+	}
+}
+
+func TestBackfillUniqueConstraintOwners_SequentialVersionChanges(t *testing.T) {
+	mig := &BackfillUniqueConstraintOwners{Version: "v1.0.0"}
+	require.True(t, mig.Check(nil))
+	mig.finalizeAfterSuccess()
+	assert.False(t, mig.Finalized)
+
+	lastRun := map[string]interface{}{"version": mig.Version, "finalized": mig.Finalized}
+	mig = &BackfillUniqueConstraintOwners{Version: "v2.0.0"}
+	require.True(t, mig.Check(lastRun))
+	mig.finalizeAfterSuccess()
+	assert.True(t, mig.Finalized)
+
+	lastRun = map[string]interface{}{"version": mig.Version, "finalized": mig.Finalized}
+	mig = &BackfillUniqueConstraintOwners{Version: "v3.0.0"}
+	require.True(t, mig.Check(lastRun))
+	mig.finalizeAfterSuccess()
+	assert.True(t, mig.Finalized)
+}
+
+func TestBackfillUniqueConstraintOwners_ForceFinalizeRestart(t *testing.T) {
+	mig := &BackfillUniqueConstraintOwners{Version: "v1.0.0"}
+	require.True(t, mig.Check(nil))
+	mig.finalizeAfterSuccess()
+	require.False(t, mig.Finalized)
+
+	lastRun := map[string]interface{}{"version": "v1.0.0", "finalized": false}
+	mig = &BackfillUniqueConstraintOwners{Version: "v1.0.0", ForceFinalize: true}
+	require.True(t, mig.Check(lastRun), "restart with ForceFinalize should re-run")
+	assert.False(t, mig.Finalized, "Check must not set finalized before Execute")
+	mig.finalizeAfterSuccess()
+	assert.True(t, mig.Finalized)
+
+	lastRun = map[string]interface{}{"version": "v1.0.0", "finalized": true}
+	mig = &BackfillUniqueConstraintOwners{Version: "v1.0.0", ForceFinalize: true}
+	assert.False(t, mig.Check(lastRun), "later restart with ForceFinalize should skip")
+}
+
+func TestUniqueTypesWithOwnersOmitsMailText(t *testing.T) {
+	assert.NotContains(t, eventstore.UniqueTypesWithOwners, "mail_text")
+	assert.Contains(t, eventstore.UniqueTypesWithOwners, "usernames")
+	assert.Contains(t, eventstore.UniqueTypesWithOwners, "idp_config_names")
+}
+
+func TestBackfillUniqueConstraintOwnersJSONOmitsForceFinalize(t *testing.T) {
+	mig := &BackfillUniqueConstraintOwners{
+		Version:       "v2.0.0",
+		Finalized:     true,
+		ForceFinalize: true,
+		lastVersion:   "v1.0.0",
+		lastFinalized: true,
+	}
+	data, err := json.Marshal(mig)
+	require.NoError(t, err)
+
+	var payload map[string]any
+	require.NoError(t, json.Unmarshal(data, &payload))
+	assert.Equal(t, "v2.0.0", payload["version"])
+	assert.Equal(t, true, payload["finalized"])
+	assert.NotContains(t, payload, "ForceFinalize")
+	assert.NotContains(t, payload, "forceFinalize")
+	assert.NotContains(t, payload, "lastVersion")
+	assert.NotContains(t, payload, "lastFinalized")
+	assert.False(t, strings.Contains(string(data), "v1.0.0"))
+}

cmd/setup/config.go

@@ -197,6 +197,8 @@ type Steps struct {
 	s75Apps7OIDCConfigsAddAppLinkConfig     *Apps7OIDCConfigsAddAppLinkConfig
 	s76Users14LoginEqualityIndexes          *Users14LoginEqualityIndexes
 	s77StampEventPositionAtInsert           *StampEventPositionAtInsert
+	s78UniqueConstraintOwners               *UniqueConstraintOwners
+	BackfillUniqueConstraintOwners          *BackfillUniqueConstraintOwners
 	RelationalTables                        *TransactionalTables
 }
 

cmd/setup/integration_test/unique_constraint_owners_test.go

@@ -0,0 +1,107 @@
+//go:build integration
+
+package setup_test
+
+import (
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+
+	"github.com/zitadel/zitadel/cmd/setup"
+	"github.com/zitadel/zitadel/internal/query/projection"
+)
+
+func TestBackfillUniqueConstraintOwners_OrgDomainVerifiedOnly(t *testing.T) {
+	instanceID := "uc-owners-domain-" + time.Now().Format("150405.000")
+	defer cleanupInstance(t, instanceID)
+
+	_, err := dbPool.Exec(CTX, `
+		INSERT INTO eventstore.unique_constraints (instance_id, unique_type, unique_field, owners)
+		VALUES ($1, 'org_domain', 'example.com', '{}')`, instanceID)
+	require.NoError(t, err)
+
+	now := time.Now()
+	_, err = dbPool.Exec(CTX, `
+		INSERT INTO `+projection.OrgDomainTable+` (
+			creation_date, change_date, sequence, domain, org_id, instance_id, is_verified, is_primary, validation_type
+		) VALUES
+			($2, $2, 1, 'example.com', 'org-a', $1, true, true, 0),
+			($2, $2, 1, 'example.com', 'org-b', $1, false, false, 0)`, instanceID, now)
+	require.NoError(t, err)
+
+	query, err := setup.BackfillUniqueConstraintOwnersQuery("05_org_domain.sql")
+	require.NoError(t, err)
+	_, err = dbPool.Exec(CTX, query)
+	require.NoError(t, err)
+
+	var owners []string
+	err = dbPool.QueryRow(CTX, `
+		SELECT owners FROM eventstore.unique_constraints
+		WHERE instance_id = $1 AND unique_type = 'org_domain' AND unique_field = 'example.com'`, instanceID).Scan(&owners)
+	require.NoError(t, err)
+	assert.Equal(t, []string{"org:org-a"}, owners)
+}
+
+func TestBackfillUniqueConstraintOwners_UsernameInstanceScopedOrg(t *testing.T) {
+	instanceID := "uc-owners-user-" + time.Now().Format("150405.000")
+	defer cleanupInstance(t, instanceID)
+
+	now := time.Now()
+	_, err := dbPool.Exec(CTX, `
+		INSERT INTO eventstore.unique_constraints (instance_id, unique_type, unique_field, owners)
+		VALUES
+			($1, 'usernames', 'bob', '{}'),
+			($1, 'usernames', 'boborg-y', '{}')`, instanceID)
+	require.NoError(t, err)
+
+	_, err = dbPool.Exec(CTX, `
+		INSERT INTO `+projection.UserTable+` (
+			id, creation_date, change_date, resource_owner, instance_id, state, sequence, username, type
+		) VALUES
+			('user-x', $2, $2, 'org-x', $1, 1, 1, 'bob', 1),
+			('user-y', $2, $2, 'org-y', $1, 1, 1, 'bob', 1)`, instanceID, now)
+	require.NoError(t, err)
+
+	_, err = dbPool.Exec(CTX, `
+		INSERT INTO `+projection.DomainPolicyTable+` (
+			creation_date, change_date, sequence, id, state, user_login_must_be_domain,
+			validate_org_domains, smtp_sender_address_matches_instance_domain, is_default, resource_owner, instance_id
+		) VALUES ($2, $2, 1, $1, 1, false, false, false, true, $1, $1)`, instanceID, now)
+	require.NoError(t, err)
+
+	_, err = dbPool.Exec(CTX, `
+		INSERT INTO `+projection.OrganizationSettingsTable+` (
+			id, creation_date, change_date, resource_owner, instance_id, sequence, organization_scoped_usernames
+		) VALUES ('org-y', $2, $2, 'org-y', $1, 1, true)`, instanceID, now)
+	require.NoError(t, err)
+
+	query, err := setup.BackfillUniqueConstraintOwnersQuery("02_usernames.sql")
+	require.NoError(t, err)
+	_, err = dbPool.Exec(CTX, query)
+	require.NoError(t, err)
+
+	var instanceOwners []string
+	err = dbPool.QueryRow(CTX, `
+		SELECT owners FROM eventstore.unique_constraints
+		WHERE instance_id = $1 AND unique_type = 'usernames' AND unique_field = 'bob'`, instanceID).Scan(&instanceOwners)
+	require.NoError(t, err)
+	assert.ElementsMatch(t, []string{"org:org-x", "user:user-x"}, instanceOwners)
+
+	var orgScopedOwners []string
+	err = dbPool.QueryRow(CTX, `
+		SELECT owners FROM eventstore.unique_constraints
+		WHERE instance_id = $1 AND unique_type = 'usernames' AND unique_field = 'boborg-y'`, instanceID).Scan(&orgScopedOwners)
+	require.NoError(t, err)
+	assert.Empty(t, orgScopedOwners)
+}
+
+func cleanupInstance(t *testing.T, instanceID string) {
+	t.Helper()
+	_, _ = dbPool.Exec(CTX, `DELETE FROM eventstore.unique_constraints WHERE instance_id = $1`, instanceID)
+	_, _ = dbPool.Exec(CTX, `DELETE FROM `+projection.OrgDomainTable+` WHERE instance_id = $1`, instanceID)
+	_, _ = dbPool.Exec(CTX, `DELETE FROM `+projection.UserTable+` WHERE instance_id = $1`, instanceID)
+	_, _ = dbPool.Exec(CTX, `DELETE FROM `+projection.DomainPolicyTable+` WHERE instance_id = $1`, instanceID)
+	_, _ = dbPool.Exec(CTX, `DELETE FROM `+projection.OrganizationSettingsTable+` WHERE instance_id = $1`, instanceID)
+}

cmd/setup/setup.go

@@ -260,6 +260,12 @@ func Setup(ctx context.Context, config *Config, steps *Steps, masterKey string)
 	steps.s75Apps7OIDCConfigsAddAppLinkConfig = &Apps7OIDCConfigsAddAppLinkConfig{dbClient: dbClient}
 	steps.s76Users14LoginEqualityIndexes = &Users14LoginEqualityIndexes{dbClient: dbClient}
 	steps.s77StampEventPositionAtInsert = &StampEventPositionAtInsert{dbClient: dbClient}
+	steps.s78UniqueConstraintOwners = &UniqueConstraintOwners{dbClient: dbClient}
+	if steps.BackfillUniqueConstraintOwners == nil {
+		steps.BackfillUniqueConstraintOwners = &BackfillUniqueConstraintOwners{}
+	}
+	steps.BackfillUniqueConstraintOwners.dbClient = dbClient
+	steps.BackfillUniqueConstraintOwners.Version = build.Version()
 
 	err = projection.Create(ctx, dbClient, eventstoreClient, config.Projections, nil, nil, nil)
 	if err != nil {
@@ -320,6 +326,8 @@ func Setup(ctx context.Context, config *Config, steps *Steps, masterKey string)
 		steps.s70AddEventStoreCommandEnforceOwner,
 		steps.s76Users14LoginEqualityIndexes,
 		steps.s77StampEventPositionAtInsert,
+		steps.s78UniqueConstraintOwners,
+		steps.BackfillUniqueConstraintOwners,
 	} {
 		setupErr = executeMigration(ctx, eventstoreClient, step, "migration failed")
 		if setupErr != nil {

cmd/setup/steps.yaml

@@ -184,4 +184,10 @@ FillFields:
 
 RelationalTables:
   # This is used for development DO NOT USE IN PRODUCTION
-  ShouldRecreateSchema: false # ZITADEL_RELATIONALTABLES_SHOULDRECREATESCHEMA
+  ShouldRecreateSchema: false # ZITADEL_RELATIONALTABLES_SHOULDRECREATESCHEMA
+
+BackfillUniqueConstraintOwners:
+  # After a rolling update on this same version, set true and re-run setup to
+  # backfill leftover empty owners and enable owner-only unique-constraint delete.
+  # Default false: owner-delete waits until the next release runs 79 again.
+  ForceFinalize: false # ZITADEL_BACKFILLUNIQUECONSTRAINTOWNERS_FORCEFINALIZE