21 Commits
Author SHA1 Message Date
684de2e691 perf(query): use UNION of index seeks for ListUsers login equality (#12701)
<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
# Which Problems Are Solved

Login V2 `ListUsers` lookups of the form username OR email (and email OR
phone) hash-join the full `users14` / `users14_humans` tables, then
filter. That cost grows with instance size (~40ms at 10k users, ~650ms
at 200k).

Top-level login-name equals already used an indexed join, but the same
login-name filter inside an `OrQuery` fell back to the slow view-based
exists query.

# How the Problems Are Solved

**TL;DR:** equality lookups on username, email, phone, or login name
become `INNER JOIN (UNION of indexed ID seeks) AS matches`. Mixed /
contains / name searches keep the old WHERE shape.

### Reviewer map

| Read first | Why |
| --- | --- |
| `internal/query/user_login_equality.go` | Typed filters + extractor +
UNION join |
| `internal/query/user.go` (`prepareUsersQuery`, constructors) | Wiring;
username/email/phone equals return `lowerEqualsFilter` |
| `cmd/setup/76.go` + `cmd/setup/76/*.sql` | Concurrent indexes for
existing deployments; drop invalid leftovers |
| `internal/query/projection/user.go` | Same indexes in `Init()` for new
databases |
| `internal/query/user_login_name_exists_test.go` | SQL-shape coverage |
| `internal/api/grpc/user/v2/integration_test/query_test.go` | Nested
username AND/OR user results |

### Query rewrite

- Extract one equals leaf, a flat `OrQuery` of those leaves, or the same
nested under `AndQuery`. Remaining AND conjuncts stay as WHERE. Equality
nested under `OrQuery` / `NotQuery` is left as WHERE.
- Login-name equals is a UNION arm that reuses `login_names3_users` SQL,
so it works inside OR as well as at the top level.
- EQUALS seeks use `LOWER(col)` for the index, then recheck `col =
value` so case-sensitive searches do not match extra rows.
EQUALS_IGNORE_CASE is unchanged.
- Empty OR arms are not skipped (that would change OR meaning); mixed
empty-OR falls back to WHERE. Login UI already omits empty arms.

### Indexes

- Setup 76: `CREATE INDEX CONCURRENTLY IF NOT EXISTS` on
`users14_username_lower_idx` `(instance_id, LOWER(username))` and
`users14_humans_phone_lower_idx` `(instance_id, LOWER(phone))`. Not
wrapped in a transaction.
- Before create, drop any leftover `pg_index.indisvalid = false` indexes
of those names (failed/cancelled `CREATE INDEX CONCURRENTLY`). `IF NOT
EXISTS` would otherwise skip rebuild.
- `Init()` creates the same indexes without `CONCURRENTLY` (empty
tables). Existing `users14_username` and email lower indexes are
unchanged.

# Additional Changes

- Unit tests assert UNION SQL for username OR email, email OR phone,
login name OR email, username OR username, nested `AndQuery(loginName,
org)`, and nested `AndQuery(Or(...), org)`; that contains / display-name
/ mixed OR / nested `OrQuery(AndQuery(...), ...)` do not UNION; that
email EQUALS rechecks the raw column; and that empty-OR is not
extracted.
- Setup 76 sqlmock coverage: missing `users14` skips work; valid/absent
indexes only create; invalid leftover username index is dropped then
both creates run.
- `TestServer_ListUsers_UsernameAndOr` checks live results: org AND
(alice OR bob) returns both users, sibling username AND is empty, nested
`AndQuery(Or(...), org)` returns both.

# Additional Context

- Base branch: `main`
- Extends the login-name equals rewrite from #12460 to
username/email/phone OR trees used by Login V2 search.

<!-- CURSOR_AGENT_PR_BODY_END -->

<div><a
href="https://cursor.com/agents/bc-1f043b1c-6168-45ca-9e9e-c18451994207?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a>&nbsp;<a
href="https://cursor.com/background-agent?bcId=bc-1f043b1c-6168-45ca-9e9e-c18451994207&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a>&nbsp;</div>

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: Silvan <[email protected]>
2026-09-08 13:11:16 +00:00
Marco A.andGitHub 382aec2d61 chore: Service Account Naming Consistency (#11557)
# Which Problems Are Solved

Inconsistent naming of service account, found in the following
variations:

  - Machine User
  - machine user
  - Service User
  - Machine Account
  - Technical Account
  - User: Type Machine

# How the Problems Are Solved

Attentive search and replace.

Localizations have been translated using Copilot

# Additional Changes

Some unused methods have been removed from the Go code.

# Additional Context

- Closes #11285
2026-02-13 12:31:43 +01:00
1f0d54a978 chore(docs): User (Human) naming consistency (#11512)
# Which Problems Are Solved

Naming inconsistencies - User (Human)

# How the Problems Are Solved

Most of the occurrences have not been changed. `User (Human)` was mostly
changed when talking about code objects and where I felt it was
necessary to distinguish them from machine users.

When both human and machine user occurrences were found, the machine
user has been changed to service account (see
https://github.com/zitadel/zitadel/issues/11285)

# Additional Context

- Closes #11284

---------

Co-authored-by: Wim Van Laer <[email protected]>
2026-02-06 09:46:24 +00:00
826039c620 Merge commit from fork
* Force v2 permission checks on user listing

* Remove duplicated tests

* cleanup and fix tests

---------

Co-authored-by: Livio Spring <[email protected]>
2025-12-10 13:13:56 +01:00
a8bbac37d9 chore: fix some eventual consistent integration testing (#10752)
# Which Problems Are Solved

Flakiness in integration tests because of eventual consistentcy.

# How the Problems Are Solved

Split tests related to feature flags and other eventual consistent
resources.

# Additional Changes

None

# Additional Context

None

Co-authored-by: Marco A. <[email protected]>
2025-10-14 18:24:09 +02:00
Tim MöhlmannandGitHub f27ca69749 fix(query): distinct count in user list (#10840)
# Which Problems Are Solved

When listing / searching users, each user got multiplied by the amount
of metadata entries they have, towards the `total_results` count. In
PostgreSQL the `COUNT(*) OVER()` window function does not support
`DISTINCT`. Even tho the query did a distinct select, the count would
still include duplicates.

# How the Problems Are Solved

Wrap the original query in a sub-select, so that the `DISTINCT` gets
handled before the count window function is executed in the outer
function. Filters, permission and solting is applied to the inner query.
Offset, limit and count are applied to the outer query.

# Additional Changes

- none

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/10825
- Backport to 4v
2025-10-14 09:43:59 +00:00
SilvanandGitHub 4900ac477a fix(test): increase retry tick duration in tests (#10787)
Adjust the retry tick duration in various tests to one minute to improve
reliability and reduce the frequency of retries.
2025-09-25 08:47:29 +02:00
Stefan BenzandGitHub 492f1826ee chore: move gofakeit integration testing calls (#10684)
# Which Problems Are Solved

Flakiness and conflicts in value from gofakeit.

# How the Problems Are Solved

Move Gofakeit calls to the integration package, to guarantee proper
usage and values for integration testing.

# Additional Changes

None

# Additional Context

None
2025-09-10 09:00:31 +03:00
8df402fb4f feat: List users by metadata (#10415)
# Which Problems Are Solved

Some users have reported the need of retrieving users given a metadata
key, metadata value or both. This change introduces metadata search
filter on the `ListUsers()` endpoint to allow Zitadel users to search
for user records by metadata.

The changes affect only v2 APIs.

# How the Problems Are Solved

- Add new search filter to `ListUserRequest`: `MetaKey` and `MetaValue`
  - Add SQL indices on metadata key and metadata value
  - Update query to left join `user_metadata` table

# Additional Context

  - Closes #9053 
  - Depends on https://github.com/zitadel/zitadel/pull/10567

---------

Co-authored-by: Silvan <[email protected]>
Co-authored-by: Stefan Benz <[email protected]>
2025-09-01 16:12:36 +00:00
6e8aa9a7d1 chore: fix flakiness in gofakeit usage with additional random string (#10385)
# Which Problems Are Solved

Flakiness in integration tests regarding gofakeit functions, which
provided the same names on 2 different occasions.

# How the Problems Are Solved

Attach a random string to the provided names, so that they are not
dependent on the gofakeit code.

# Additional Changes

None

# Additional Context

None

---------

Co-authored-by: Marco A. <[email protected]>
2025-08-07 13:27:01 +00:00
+6 07ce3b6905 chore!: Introduce ZITADEL v3 (#9645)
This PR summarizes multiple changes specifically only available with
ZITADEL v3:

- feat: Web Keys management
(https://github.com/zitadel/zitadel/pull/9526)
- fix(cmd): ensure proper working of mirror
(https://github.com/zitadel/zitadel/pull/9509)
- feat(Authz): system user support for permission check v2
(https://github.com/zitadel/zitadel/pull/9640)
- chore(license): change from Apache to AGPL
(https://github.com/zitadel/zitadel/pull/9597)
- feat(console): list v2 sessions
(https://github.com/zitadel/zitadel/pull/9539)
- fix(console): add loginV2 feature flag
(https://github.com/zitadel/zitadel/pull/9682)
- fix(feature flags): allow reading "own" flags
(https://github.com/zitadel/zitadel/pull/9649)
- feat(console): add Actions V2 UI
(https://github.com/zitadel/zitadel/pull/9591)

BREAKING CHANGE
- feat(webkey): migrate to v2beta API
(https://github.com/zitadel/zitadel/pull/9445)
- chore!: remove CockroachDB Support
(https://github.com/zitadel/zitadel/pull/9444)
- feat(actions): migrate to v2beta API
(https://github.com/zitadel/zitadel/pull/9489)

---------

Co-authored-by: Livio Spring <[email protected]>
Co-authored-by: Stefan Benz <[email protected]>
Co-authored-by: Silvan <[email protected]>
Co-authored-by: Ramon <[email protected]>
Co-authored-by: Elio Bischof <[email protected]>
Co-authored-by: Kenta Yamaguchi <[email protected]>
Co-authored-by: Harsha Reddy <[email protected]>
Co-authored-by: Livio Spring <[email protected]>
Co-authored-by: Max Peintner <[email protected]>
Co-authored-by: Iraq <[email protected]>
Co-authored-by: Florian Forster <[email protected]>
Co-authored-by: Tim Möhlmann <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Max Peintner <[email protected]>
2025-04-02 16:53:06 +02:00
70bddceda8 fix(user fields): missing creationDate in details (#9250)
# Which Problems Are Solved

The `creationDate` property on user search V2 endpoint was missing

# How the Problems Are Solved

Added property in v2 `object.proto` and in the function creating the
details on each call

# Additional Changes
- none
# Additional Context
closes #8552

---------

Co-authored-by: Stefan Benz <[email protected]>
2025-02-26 13:00:04 +00:00
f2e82d57ac fix: adding code to test ListUsers with and without permission_check_v2 flag set (#9383)
# Which Problems Are Solved

Enhancing `v2/ListUsers()` tests by adding code to run all test with and
without `permission_check_v2` flag set

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/9356

---------

Co-authored-by: Iraq Jaber <[email protected]>
2025-02-24 16:29:51 +00:00
9aad207ee4 fix(permissions): return current user when calling ListUsers() when user does not have permissions (#9374)
# Which Problems Are Solved

When running `ListUsers()` with no permissions, the calling user shoud
be returned

# How the Problems Are Solved

Added additional clause to SQL search statement 

# Additional Changes

n/a

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/9355

---------

Co-authored-by: Iraq Jaber <[email protected]>
2025-02-20 15:39:48 +00:00
e1f0d46393 fix(listUsers): Add Search User By Phone to User Service V2 (#9052)
# Which Problems Are Solved

Added search by phone to user Service V2.

```
curl --request POST \
  --url https://<zitadel_domain>/v2/users \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <Token>' \
  --header 'Content-Type: application/json' \
  --header 'content-type: application/json' \
  --data '{
  "query": {
    "offset": "0",
    "limit": 100,
    "asc": true
  },
  "sortingColumn": "USER_FIELD_NAME_UNSPECIFIED",
  "queries": [
    {
      "phoneQuery": {
        "number": "+12011223313",
        "method": "TEXT_QUERY_METHOD_EQUALS"
      }
    }
  ]
}'
```

Why?
Searching for a user by phone was missing from User Service V2 and V2
beta.

# How the Problems Are Solved

* Added to the SearchQuery proto 
* Added code to filter users by phone

# Additional Changes

N/A

# Additional Context

Search by phone is present in V3 User Service

---------

Co-authored-by: Stefan Benz <[email protected]>
2025-01-02 11:14:15 +01:00
Tim MöhlmannandGitHub 4413efd82c chore: remove parallel running in integration tests (#8904)
# Which Problems Are Solved

Integration tests are flaky due to eventual consistency.

# How the Problems Are Solved

Remove t.Parallel so that less concurrent requests on multiple instance
happen. This allows the projections to catch up more easily.

# Additional Changes

- none

# Additional Context

- none
2024-11-27 15:32:13 +01:00
32d958ea43 chore: add await for project to oidc integration tests (#8809)
# Which Problems Are Solved

In integration tests there is waiting for the application, but the
project is also included if the token can be created.

# How the Problems Are Solved

Wait for project not only for the application in the integration tests.

# Additional Changes

Some more corrections in integration tests.

# Additional Context

None

---------

Co-authored-by: Livio Spring <[email protected]>
2024-10-23 07:36:50 +00:00
Stefan BenzandGitHub fca6b28a97 chore: correct require usage to assert for eventual consistency (#8795)
# Which Problems Are Solved

Eventual consistency is handled wrongly in the newly improved
integration tests.

# How the Problems Are Solved

Correct the usage of the require package with the assert package where
necessary, to remove the panics where the EventuallyWithT functions can
rerun.

# Additional Changes

Modify the timeout values for some EventuallyWithT which can vary when a
instance is freshly setup.

# Additional Context

None
2024-10-21 19:15:02 +00:00
Stefan BenzandGitHub 8d97363642 chore: improve integration tests (#8727)
Improve integration tests:
- spliting the tests in TokenExchange to isolated instances and in
parallel
- corrected some test structure so that the check for Details is no done
anymore if the test already failed
- replace required-calls with assert-calls to not stop the testing
- add gofakeit for application, project and usernames(emails)
- add eventually checks for testing in actions v2, so the request only
get called when the execution is defined
- check for length of results in list/search endpoints to avoid index
errors
2024-10-17 21:20:57 +00:00
4d593dace2 fix: add domain as attribute to list user auth methods (#8718)
# Which Problems Are Solved

There is no option to only query auth methods related to specific
domains.

# How the Problems Are Solved

Add domain as attribute to the ListAuthenticationMethodTypes request.

# Additional Changes

OwnerRemoved column removed from the projection.

# Additional Context

Closes #8615

---------

Co-authored-by: Livio Spring <[email protected]>
2024-10-10 16:50:53 +00:00
d2e0ac07f1 chore(tests): use a coverage server binary (#8407)
# Which Problems Are Solved

Use a single server instance for API integration tests. This optimizes
the time taken for the integration test pipeline,
because it allows running tests on multiple packages in parallel. Also,
it saves time by not start and stopping a zitadel server for every
package.

# How the Problems Are Solved

- Build a binary with `go build -race -cover ....`
- Integration tests only construct clients. The server remains running
in the background.
- The integration package and tested packages now fully utilize the API.
No more direct database access trough `query` and `command` packages.
- Use Makefile recipes to setup, start and stop the server in the
background.
- The binary has the race detector enabled
- Init and setup jobs are configured to halt immediately on race
condition
- Because the server runs in the background, races are only logged. When
the server is stopped and race logs exist, the Makefile recipe will
throw an error and print the logs.
- Makefile recipes include logic to print logs and convert coverage
reports after the server is stopped.
- Some tests need a downstream HTTP server to make requests, like quota
and milestones. A new `integration/sink` package creates an HTTP server
and uses websockets to forward HTTP request back to the test packages.
The package API uses Go channels for abstraction and easy usage.

# Additional Changes

- Integration test files already used the `//go:build integration`
directive. In order to properly split integration from unit tests,
integration test files need to be in a `integration_test` subdirectory
of their package.
- `UseIsolatedInstance` used to overwrite the `Tester.Client` for each
instance. Now a `Instance` object is returned with a gRPC client that is
connected to the isolated instance's hostname.
- The `Tester` type is now `Instance`. The object is created for the
first instance, used by default in any test. Isolated instances are also
`Instance` objects and therefore benefit from the same methods and
values. The first instance and any other us capable of creating an
isolated instance over the system API.
- All test packages run in an Isolated instance by calling
`NewInstance()`
- Individual tests that use an isolated instance use `t.Parallel()`

# Additional Context

- Closes #6684
- https://go.dev/doc/articles/race_detector
- https://go.dev/doc/build-cover

---------

Co-authored-by: Stefan Benz <[email protected]>
2024-09-06 14:47:57 +02:00