mirror of
https://github.com/andwati/zitadel.git
synced 2026-10-18 05:52:51 +00:00
main
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
   
|
3ff4fbc7e4 |
fix(eventstore): add owners to unique constraints (#12740)
<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
# Which Problems Are Solved
Bulk unique-constraint deletes (org, IDP with many links, user, project)
reconstruct every uniqueness key in Go and emit N `DELETE`s. That can
OOM on large tenants.
A one-shot owner backfill is also not enough: during a rolling deploy,
older binaries still insert `owners = '{}'`, so a later owner-only
delete would miss those rows.
Owner-only org/user/grant delete only matches rows whose `owners` bag
contains that tag. Several grant and member types stamped the aggregate
resource owner, which is not always the user’s home org or the granted
org. Username-scope rewrites also dropped `user:` tags, so owner-only
user delete could miss rewritten username rows.
Command must not switch to owner-only delete on the first 79 of a
version, because leftover `'{}'` rows from rolling-update overlap are
only backfilled on a later 79. A failed 79 must not turn owner-delete
on, and backfill joins must not stamp the wrong org (unverified domains,
org-scoped usernames colliding with instance-scoped ones).
# How the Problems Are Solved
Uniqueness stays the existing primary key `(instance_id, unique_type,
unique_field)`. `owners` is only a lifecycle bag of tags (`org:<id>`,
`user:<id>`, `idp:<id>`, `project:<id>`, `grant:<id>`).
- Add `owners TEXT[] NOT NULL DEFAULT '{}'` on
`eventstore.unique_constraints` (init SQL for greenfield, setup 78 for
existing clusters).
- Create `unique_constraints_owners_gin` with `CREATE INDEX
CONCURRENTLY` **outside a transaction** (same pattern as step 76).
Built-in `GIN (owners)` only; no `btree_gin`.
- Stamp `WithOwners` on **add** events. Field-level removes ignore
owners. `RemoveByOwner` stores the tag in `Owners` and both v1/v3 and v2
pushers run `DELETE … WHERE instance_id = $1 AND owners @>
ARRAY[$tag]::text[]`.
- Setup 79 backfills `owners` from projections via numbered
`cmd/setup/79/*.sql` files. Table names are interpolated from
`internal/query/projection` constants. Matches use `unique_field IN (x,
lower(x))` so the primary key can be used. Missing projection tables
(`42P01`) are skipped.
- Org domains are tagged only when `is_verified`. Instance-scoped
usernames join only users that are not org-scoped (organization settings
or domain policy `user_login_must_be_domain`). Every username row still
gets `org:` + `user:` of that user.
- 79 is a **repeatable** migration. `Check` re-runs when
`lastRun["version"]` changed or when setup-only `ForceFinalize` is set
and `lastRun["finalized"]` is still false. `Finalized` is written only
after Execute succeeds.
- Failed setup events keep the last successful `lastRun`. Command reads
`finalized` only from `system.migration.repeatable.done`.
- Org / user / project remove skip N-key event replay **when that
`finalized` bit is true**. Otherwise they keep today’s N-key lists
**and** still emit `RemoveByOwner`. The “ready” result is cached in
memory once true.
Setup writes `finalized` on the 79 repeatable.done payload:
- First 79 of a version: backfill only, `finalized=false` (unless
`ForceFinalize` is already set).
- Later 79: version change or
`BackfillUniqueConstraintOwners.ForceFinalize` /
`ZITADEL_BACKFILLUNIQUECONSTRAINTOWNERS_FORCEFINALIZE` after the rolling
update is done. That run backfills leftovers and writes
`finalized=true`.
Member and grant owner bags now include every org that should delete the
row:
- Instance, org, and project members also stamp
`org:<user_resource_owner>`.
- User grants stamp grant, user, project, and granted orgs.
- Project grant members stamp the project org, user org, and granted
org.
- Project grants stamp both orgs plus `grant:<id>`. Grant remove emits
`RemoveByOwner(grant)`.
- Username-scope rewrites keep `user:<id>` as well as `org:`.
- User/grant orgs are carried as unexported add-event fields (not
persisted JSON). `WithOwners` skips empty and duplicate tags.
# Additional Changes
- Table-driven owner-tag tests, including project grant (two `org:` tags
plus `grant:`), cross-org members, user grants, and username-scope
rewrite `user:` tags.
- Table-driven 79 `Check` + `finalizeAfterSuccess` tests for version
change, sequential version bumps, ForceFinalize restart, and
already-finalized skip.
- Command tests: a failed 79 event with `finalized=true` does not enable
owner-delete.
- Integration fixtures (verified vs unverified org domain;
instance-scoped `bob` vs org-scoped `bob`).
- Instance-only and global unique types stay untagged (`'{}'`). Instance
wipe still uses `UniqueConstraintInstanceRemove`.
- Pusher owner cases live in `eventstore_pusher_owners_test.go` so
`eventstore_pusher_test.go` stays under 1k lines.
# Additional Context
A later cleanup can delete the N-key replay helpers once a release that
ran 79 a second time (or an operator who set `ForceFinalize`) is in
production. That is not required for correctness.
<!-- CURSOR_AGENT_PR_BODY_END -->
<div><a
href="https://cursor.com/agents/bc-44f9bc9b-9a93-4cd9-8dd1-663a7b9aad2a?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a
href="https://cursor.com/background-agent?bcId=bc-44f9bc9b-9a93-4cd9-8dd1-663a7b9aad2a&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div>
---------
Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: Silvan <[email protected]>
|
||
   
|
f69a6ed4f3 |
chore: rehaul DevX (#10571)
# Which Problems Are Solved Replaces Turbo by Nx and lays the foundation for the next CI improvements. It enables using Nx Cloud to speed the up the pipelines that affect any node package. It streamlines the dev experience for frontend and backend developers by providing the following commands: | Task | Command | Notes | |------|---------|--------| | **Production** | `nx run PROJECT:prod` | Production server | | **Develop** | `nx run PROJECT:dev` | Hot reloading development server | | **Test** | `nx run PROJECT:test` | Run all tests | | **Lint** | `nx run PROJECT:lint` | Check code style | | **Lint Fix** | `nx run PROJECT:lint-fix` | Auto-fix style issues | The following values can be used for PROJECT: - @zitadel/zitadel (root commands) - @zitadel/api, - @zitadel/login, - @zitadel/console, - @zitadel/docs, - @zitadel/client - @zitadel/proto The project names and folders are streamlined: | Old Folder | New Folder | | --- | --- | | ./e2e | ./tests/functional-ui | | ./load-test | ./benchmark | | ./build/zitadel | ./apps/api | | ./console | ./apps/console (postponed so the PR is reviewable) | Also, all references to the TypeScript repo are removed so we can archive it. # How the Problems Are Solved - Ran `npx nx@latest init` - Replaced all turbo.json by project.json and fixed the target configs - Removed Turbo dependency - All JavaScript related code affected by a PRs changes is quality-checked using the `nx affected` command - We move PR checks that are runnable using Nx into the `check` workflow. For workflows where we don't use Nx, yet, we restore previously built dependency artifacts from Nx. - We only use a single and easy to understand dev container - The CONTRIBUTING.md is streamlined - The setup with a generated client pat is orchestrated with Nx - Everything related to the TypeScript repo is updated or removed. A **Deploy with Vercel** button is added to the docs and the CONTRIBUTING.md. # Additional Changes - NPM package names have a consistent pattern. - Docker bake is removed. The login container is built and released like the core container. - The integration tests build the login container before running, so they don't rely on the login container action anymore. This fixes consistently failing checks on PRs from forks. - The docs build in GitHub actions is removed, as we already build on Vercel. # Additional Context - Internal discussion: https://zitadel.slack.com/archives/C087ADF8LRX/p1756277884928169 - Workflow dispatch test: https://github.com/zitadel/zitadel/actions/runs/17760122959 --------- Co-authored-by: Florian Forster <[email protected]> Co-authored-by: Tim Möhlmann <[email protected]> Co-authored-by: Copilot <[email protected]> |
||
|
Tim MöhlmannandGitHub
|
658ca3606b |
feat(permissions): project member permission filter (#9757)
# Which Problems Are Solved Add the possibility to filter project resources based on project member roles. # How the Problems Are Solved Extend and refactor existing Pl/PgSQL functions to implement the following: - Solve O(n) complexity in returned resources IDs by returning a boolean filter for instance level permissions. - Individually permitted orgs are returned only if there was no instance permission - Individually permitted projects are returned only if there was no instance permission - Because of the multiple filter terms, use `INNER JOIN`s instead of `WHERE` clauses. # Additional Changes - system permission function no longer query the organization view and therefore can be `immutable`, giving big performance benefits for frequently reused system users. (like our hosted login in Zitadel cloud) - The permitted org and project functions are now defined as `stable` because the don't modify on-disk data. This might give a small performance gain - The Pl/PgSQL functions are now tested using Go unit tests. # Additional Context - Depends on https://github.com/zitadel/zitadel/pull/9677 - Part of https://github.com/zitadel/zitadel/issues/9188 - Closes https://github.com/zitadel/zitadel/issues/9190 |
