Commit Graph
87 Commits
Author SHA1 Message Date
01fe34a526 fix(oidc): use authenticated encryption for opaque tokens (#12017)
# Which Problems Are Solved

Opaque tokens now use authenticated encryption.

# How the Problems Are Solved

- Upgrade zitadel/oidc to v3.47
- Copy crypto implementation for refresh and session tokens (internal to
zitadel)
- Added config that allows validating old tokens for gradual roll-out

# Additional Changes

- Set NX cache for `integration-test-build` to `false`, working on a
seperate fix.

# Additional Context

- closes #11315

---------

Co-authored-by: copilot-swe-agent[bot] <[email protected]>
Co-authored-by: muhlemmer <[email protected]>
2026-04-13 10:59:50 +00:00
b558a1f79f feat: allow ECDSA and ED25519 public keys (#11819)
# Which Problems Are Solved
Currently ZITADEL only allows the use of RSA public keys for machine
user authentication (jwt-bearer grant), including for system API users.
Attempting to use ECDSA (e.g. P-256) or ED25519 keys results in
`Errors.Internal` because `BytesToPublicKey` performs an
`ifc.(*rsa.PublicKey)` type assertion that returns `(nil, nil)` for
non-RSA keys, which then causes a nil key panic in go-jose during JWT
verification.

This is the same fix as #8433 (by @livio-a), rebased onto current
`main`.

# How the Problems Are Solved
- `BytesToPublicKey` now returns `crypto.PublicKey` (the standard
library interface) instead of `*rsa.PublicKey`
- A type switch validates the parsed key is one of `*rsa.PublicKey`,
`*ecdsa.PublicKey`, or `ed25519.PublicKey`
- A new `ErrNoPublicKey` sentinel error is returned for unsupported key
types instead of silently returning nil
- Callers in `system_token.go` and `query/key.go` are updated to use the
generic `crypto.PublicKey` interface

# Additional Changes
None

# Additional Context
Duplicate of #8433 which has been open since August 2024. We hit this
bug while implementing OIDC bootstrap for an SGX enclave that generates
ECDSA P-256 keys at runtime -- the `AddKey` API accepts the ECDSA SPKI
PEM fine, but the subsequent `jwt-bearer` token exchange fails with
`Errors.Internal` due to the nil key.

---------

Co-authored-by: Copilot <[email protected]>
Co-authored-by: Wim Van Laer <[email protected]>
2026-04-09 11:30:23 +02:00
70adaff793 feat: add option to use x.509 certificate system-api-user tokens (#11876)
# Which Problems Are Solved

System API users currently authenticate using raw RSA public keys
configured via Path or KeyData. This approach doesn't integrate well
with Kubernetes tooling.

# How the Problems Are Solved

Allow for the `path`/`keyData` to be an X.509 certificate. 

The `NotBefore` and `NotAfter` fields of the certificate are beeing
respected when validating the JWT.

# Additional Changes

# Additional Context

- Closes #11442

---------

Co-authored-by: Livio Spring <[email protected]>
2026-03-23 13:32:27 +00:00
ca4bba3992 feat: add delete session on relation tables (#11334)
# Which Problems Are Solved

As part of moving the session API to the relational database, this PR
adds the functionality to delete sessions through the session API v2 and
the OIDC end_session endpoint.

# How the Problems Are Solved

- added `SessionDeleteCommand` in domain package
- added possibility to pass the session token verifier to the domain
package
- use feature flag to switch API and OIDC usage from CQRS to relation
table
- added a `permissionCheck` condition to let permissions be checked
directly in the repository / sql
- part of it is still a placeholder until actual permission tables are
done
- added a `session_deleted` table to store (recently) deleted sessions
to handle necessary checks / cases like delete my own session, which
needs to be idempotent

# Additional Changes

Fixed `Matches` function of `existsCondition` (as missing an
implementation)

# Additional Context

- closes #11037

---------

Co-authored-by: Stefan Benz <[email protected]>
2026-03-19 08:35:53 +01:00
997aa607c4 feat(telemetry): unify request details (#11509)
# Which Problems Are Solved

In the "new" structured logging, request details were added in different
middlewares, such as request, instance and user IDs. This meant the the
upstream request logging middleware did not have access to metadata that
got added later, resulting in incomplete logs. Furthermore it was not
possible to correlate an API error response to log output.

# How the Problems Are Solved

A mutable request details object is added to the context early on. When
the api authz function run, the instance and user IDs are added to this
object as they become available. Every logline emitted after this
(time-wise) will then all contain these details under the `request` log
group.

<details>

<summary>example output in JSON</summary>

```json
{
  "time": "2026-03-13T19:21:20.890428806Z",
  "level": "INFO",
  "source": {
    "function": "github.com/zitadel/zitadel/internal/api/grpc/server/connect_middleware.LogHandler.func1.1",
    "file": "/workspaces/zitadel/internal/api/grpc/server/connect_middleware/log_interceptor.go",
    "line": 34
  },
  "msg": "request served",
  "request": {
    "id": "d6q67c04vtjmi77cbbpg",
    "instance_host": "localhost:8080",
    "instance_id": "362349751439458307",
    "user_id": "362349751440048131"
  },
  "TraceID": "a9e0fee3522224f3583bbdcda737f4b4",
  "SpanID": "a563056eee36920a",
  "stream": "request",
  "version": "2026-03-13T19:20:59Z",
  "protocol": "connect",
  "service": "zitadel.user.v2.UserService",
  "http_method": "POST",
  "path": "/zitadel.user.v2.UserService/ListUsers",
  "code": "code_0",
  "duration": 12254350
}
```

</details>

Request IDs are now also returned with a response header or metadata.
Depending on the protocol:

- HTTP calls always return the request ID as header, regardless of
status
- gRPC calls always return the request ID as header, even if there was
an error
- connect RPC calls returns the request ID as header on success, trailer
in case of error. This is because header must be set on the response
object, which is nil in case of error. When there is an error, metadata
can be added which are then sent as trailers.

# Additional Changes

- Use the existing call duration middleware for both request ID and
logging for a consistent request start timestamp in all layers.
- Upgrade sloggcp for some fixes (notably TraceID)
- Modify the NoCache middleware so it uses `SetHeaders` instead of
`SendHeaders`. The latter prevented any other handler from setting
headers, including the new request ID middleware.

# Additional Context

Follow up on demo of:
  - https://github.com/zitadel/zitadel/pull/11159
  - https://github.com/zitadel/zitadel/pull/11435
  - backport to v4

---------

Co-authored-by: Copilot <[email protected]>
Co-authored-by: muhlemmer <[email protected]>
Co-authored-by: Copilot Autofix powered by AI <[email protected]>
Co-authored-by: Marco A. <[email protected]>
2026-03-18 14:07:33 +01:00
Marco A.andGitHub 382aec2d61 chore: Service Account Naming Consistency (#11557)
# Which Problems Are Solved

Inconsistent naming of service account, found in the following
variations:

  - Machine User
  - machine user
  - Service User
  - Machine Account
  - Technical Account
  - User: Type Machine

# How the Problems Are Solved

Attentive search and replace.

Localizations have been translated using Copilot

# Additional Changes

Some unused methods have been removed from the Go code.

# Additional Context

- Closes #11285
2026-02-13 12:31:43 +01:00
b0609aa861 feat(api): return allowed_languages (#11553)
# Which Problems Are Solved

While Zitadel provides a possibility to restrict certain languages to be
used, the corresponding list could not be retrieved in the settings
service (v2). This blocked login v2 implementations from respecting the
list and they would always use all available languages.

# How the Problems Are Solved

- Retrieve the list when checking the instance and pass it into the
context.
- Return it as part of the existing `GetGeneralSettingsResponse`
- This allows us to remove an additional query in some other cases /
endpoints.

# Additional Changes

none

# Additional Context

- required for https://github.com/zitadel/zitadel/pull/11372
- backport to v4.x

---------

Co-authored-by: Copilot <[email protected]>
Co-authored-by: Marco A. <[email protected]>
2026-02-12 08:44:34 +00:00
11dbb1b277 feat(logging): add streams (#11435)
# Which Problems Are Solved

Streams allow differentiating logs produced by different components of
Zitadel.

# How the Problems Are Solved

The `backend/v3/instrumentation/logging` package now exposes convenience
function for setting and getting a logger from the context. As well as
high-level functions to emit log records at various levels. When
constructing a new logger a "stream" needs to be specified:

- **runtime**: General runtime logs, such as startup and shutdown
messages. Default for logs that do not belong to the other categories.
- **request**: Logs for incoming API and HTTP requests.
- **event_handler**: Logs for event handling in projections.
- **queue**: Logs for the job queue processing.
- **event_pusher**: Logs for event pushing to the database. Disabled by
default, contains sensitive information.

Each line from the returned logger contains a `stream` field as well as
a `version` field with the current Zitadel version.

## Runtime config

Streams can be enabled by passing an array of stream names in the
runtime config. Because some log streams may contain sensitive data
(especially events), it is now also possible to mask values by their
key.

# Additional Changes

- Wrap `slogctx` in the `logging` package. (Except API error converter
packages, because of import cycle)
- Add some docs to `logging` package so other devs understand how to add
logging to Zitadel
- Add `logging.OnError` and `logging.WithError` helper functions with
`Panic()` and `Fatal()` methods, to preserve current calls in the `cmd`
packages.
- Add instance context extractor.
- Only output request details in the request info log. Request ID
remains propagated through context.
- Moved middleware functionality into protocol specific packages. 
- Removed setting of URI to context in metric middleware. There were ony
setters and no getters. (Unused value)
- Reuse a single statusWriter in the middleware package for middlewares
that need to know the response status.

# Additional Context

- Closes #11333
- Closes #11331 
- Partly #11330

---------

Co-authored-by: Silvan <[email protected]>
2026-02-04 11:51:43 +01:00
Marco A.andGitHub 96bbd6a65c chore: management console naming inconsistency (#11390)
# Which Problems Are Solved

The following terms have all been renamed to management console:

- Customer Portal (when used to mean the console)
- Console
- Admin Console

# How the Problems Are Solved

- Search & Replace smartly
- Use Copilot for translation files

Changes done to: backend + frontend codebase, docs, translations and
protobufs (descriptions only)

# Additional Context

- Partially Closes #11279
2026-01-26 14:52:15 +00:00
Livio SpringandGitHub 72a5c33e6a Merge commit from fork
* fix: sanitize host headers before use

* add additional test
2025-10-29 10:05:37 +01:00
f04d873270 feat(domain): Organization APIs with relational tables (#10704)
# Which Problems Are Solved

This PR implements the endpoints listed in #10445 using relational
tables.

- [x] UpdateOrganization
- [x] ListOrganizations
- [x] DeleteOrganization
- [x] DeactivateOrganization
- [x] ActivateOrganization

# How the Problems Are Solved

- **UpdateOrganization:** Implemented logic for `Validate()`,
`Execute()` and `Events()`. On CQRS side, updating an organization emits
events related to the organization and its domains. Separate commanders
have been made for the domain events so that their logic is run
accordingly and we keep separation of concerns. Implementation of the
domains is left to its own ticket.

Because some domain-related data is changed during the organization
update, its original values are saved as pointers inside
[UpdateOrgCommand](https://github.com/zitadel/zitadel/blob/4f87cd8d0cfd75f290aa13c342e4939cecebef73/backend/v3/domain/org_update.go#L20).

These variables will be then used as input for the domain commanders
(that will run after the update organization one).
- **ListOrganizations:** Implemented logic for `Validate()`,
`Execute()`. Utility methods have been made to parse the input queries.
This commander is used both by the v2 and v2beta endpoint, so a
translation layer has been put in place to translate all v2beta requests
into v2. The commander strictly handles v2 request only.

The translation layer between v2beta and v2 APIs was made so to make it
simple to remove it once the v2beta APIs will be removed (a simple
deletion should suffice). A `converter` package [is
shipped](https://github.com/zitadel/zitadel/blob/4f87cd8d0cfd75f290aa13c342e4939cecebef73/backend/v3/api/org/v2/convert/README.md#L5)
as well with instruction for removal. TODOs have been put in place all
over the code to indicate that the related piece of code needs to be
removed once v2beta -> v2 transition is complete.
- **DeleteOrganization:** Implemented logic for `Validate()`,
`Execute()` and `Events()`. The `Events()` method is mostly incomplete
as it requires a lot of data that is currently not retrievable due to
the lack of their relational tables.
- **DeactivateOrganization** and **ActivateOrganization**: Implemented
logic for `Validate()`, `Execute()` and `Events()`. No notable remarks.

👉 All endpoints return matching errors to the CQRS counterpart: this way
allows re-using the integration tests and

👉 All endpoints have been unit-tested through DB mocking

👉 A bunch of TODOs have been put in place for future reworks that are
needed (e.g. `Update()` method on repository should also return a
timestamp)

# Additional Changes

- A test utility method to allow mocking query options have been made.
This was necessary because `database.QueryOption` is a function and the
only comparison possible with functions is `nil` check. So, to mock
this, `database.QueryOption` is converted first to `database.QueryOpts`.
See 2276742adaef1998ea4844e1f0dca2079e45ccfb

- A `BaseCommand` has been created with the idea to collect methods that
are going to be needed by other commands. For the moment, it only offers
a method to convert query text operations from gRPC to `domain` model.
See bb85456dd101282fa76d2e2ad165eed4c8ef61b9

- SQL operations have been reworked to allow for queries using
`ContainsIgnoreCase`, `EndsWithIgnoreCase` and other `IgnoreCase`
operations. This change is pending approval and might be reworked. See
https://github.com/zitadel/zitadel/pull/10704#discussion_r2392886177

# Additional Context

`UpdateOrganization` is partially working because the logic for handling
domain updates is not done (there is only a draft of the commanders to
showcase the call chain). See
https://github.com/zitadel/zitadel/pull/10704#discussion_r2356143178

Permissions are not implemented as they are missing. See
https://github.com/zitadel/zitadel/pull/10771

- Closes: #10445

---------

Co-authored-by: Silvan <[email protected]>
2025-10-21 15:27:38 +02:00
a9ebc06c77 perf(actionsv2): execution target router (#10564)
# Which Problems Are Solved

The event execution system currently uses a projection handler that
subscribes to and processes all events for all instances. This creates a
high static cost because the system over-fetches event data, handling
many events that are not needed by most instances. This inefficiency is
also reflected in high "rows returned" metrics in the database.

# How the Problems Are Solved

Eliminate the use of a project handler. Instead, events for which
"execution targets" are defined, are directly pushed to the queue by the
eventstore. A Router is populated in the Instance object in the authz
middleware.

- By joining the execution targets to the instance, no additional
queries are needed anymore.
- As part of the instance object, execution targets are now cached as
well.
- Events are queued within the same transaction, giving transactional
guarantees on delivery.
- Uses the "insert many fast` variant of River. Multiple jobs are queued
in a single round-trip to the database.
- Fix compatibility with PostgreSQL 15

# Additional Changes

- The signing key was stored as plain-text in the river job payload in
the DB. This violated our [Secrets
Storage](https://zitadel.com/docs/concepts/architecture/secrets#secrets-storage)
principle. This change removed the field and only uses the encrypted
version of the signing key.
- Fixed the target ordering from descending to ascending.
- Some minor linter warnings on the use of `io.WriteString()`.

# Additional Context

- Introduced in https://github.com/zitadel/zitadel/pull/9249
- Closes https://github.com/zitadel/zitadel/issues/10553
- Closes https://github.com/zitadel/zitadel/issues/9832
- Closes https://github.com/zitadel/zitadel/issues/10372
- Closes https://github.com/zitadel/zitadel/issues/10492

---------

Co-authored-by: Stefan Benz <[email protected]>
2025-09-01 07:21:10 +02:00
0a14c01412 fix: configure default url templates (#10416)
# Which Problems Are Solved

Emails are still send only with URLs to login v1.

# How the Problems Are Solved

Add configuration for URLs as URL templates, so that links can point at
Login v2.

# Additional Changes

None

# Additional Context

Closes #10236

---------

Co-authored-by: Marco A. <[email protected]>
2025-08-26 10:14:41 +00:00
Marco A.andGitHub 28f7218ea1 feat: Hosted login translation API (#10011)
# Which Problems Are Solved

This PR implements https://github.com/zitadel/zitadel/issues/9850

# How the Problems Are Solved

  - New protobuf definition
  - Implementation of retrieval of system translations
- Implementation of retrieval and persistence of organization and
instance level translations

# Additional Context

- Closes #9850

# TODO

- [x] Integration tests for Get and Set hosted login translation
endpoints
- [x] DB migration test
- [x] Command function tests
- [x] Command util functions tests
- [x] Query function test
- [x] Query util functions tests
2025-06-18 13:24:39 +02:00
b46c41e4bf fix(settings): fix for setting restricted languages (#9947)
# Which Problems Are Solved

Zitadel encounters a migration error when setting `restricted languages`
and fails to start.

# How the Problems Are Solved

The problem is that there is a check that checks that at least one of
the restricted languages is the same as the `default language`, however,
in the `authz instance` (where the default language is pulled form) is
never set.

I've added code to set the `default language` in the `authz instance` 

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/9787

---------

Co-authored-by: Livio Spring <[email protected]>
2025-06-02 08:40:19 +00:00
Tim MöhlmannandGitHub 658ca3606b feat(permissions): project member permission filter (#9757)
# Which Problems Are Solved

Add the possibility to filter project resources based on project member
roles.

# How the Problems Are Solved

Extend and refactor existing Pl/PgSQL functions to implement the
following:

- Solve O(n) complexity in returned resources IDs by returning a boolean
filter for instance level permissions.
- Individually permitted orgs are returned only if there was no instance
permission
- Individually permitted projects are returned only if there was no
instance permission
- Because of the multiple filter terms, use `INNER JOIN`s instead of
`WHERE` clauses.

# Additional Changes

- system permission function no longer query the organization view and
therefore can be `immutable`, giving big performance benefits for
frequently reused system users. (like our hosted login in Zitadel cloud)
- The permitted org and project functions are now defined as `stable`
because the don't modify on-disk data. This might give a small
performance gain
- The Pl/PgSQL functions are now tested using Go unit tests.

# Additional Context

- Depends on https://github.com/zitadel/zitadel/pull/9677
- Part of https://github.com/zitadel/zitadel/issues/9188
- Closes https://github.com/zitadel/zitadel/issues/9190
2025-04-22 08:42:59 +00:00
Tim MöhlmannandGitHub a2f60f2e7a perf(query): org permission function for resources (#9677)
# Which Problems Are Solved

Classic permission checks execute for every returned row on resource
based search APIs. Complete background and problem definition can be
found here: https://github.com/zitadel/zitadel/issues/9188

# How the Problems Are Solved

- PermissionClause function now support dynamic query building, so it
supports multiple cases.
- PermissionClause is applied to all list resources which support org
level permissions.
- Wrap permission logic into wrapper functions so we keep the business
logic clean.

# Additional Changes

- Handle org ID optimization in the query package, so it is reusable for
all resources, instead of extracting the filter in the API.
- Cleanup and test system user conversion in the authz package. (context
middleware)
- Fix: `core_integration_db_up` make recipe was missing the postgres
service.

# Additional Context

- Related to https://github.com/zitadel/zitadel/issues/9190
2025-04-15 18:38:25 +02:00
+6 07ce3b6905 chore!: Introduce ZITADEL v3 (#9645)
This PR summarizes multiple changes specifically only available with
ZITADEL v3:

- feat: Web Keys management
(https://github.com/zitadel/zitadel/pull/9526)
- fix(cmd): ensure proper working of mirror
(https://github.com/zitadel/zitadel/pull/9509)
- feat(Authz): system user support for permission check v2
(https://github.com/zitadel/zitadel/pull/9640)
- chore(license): change from Apache to AGPL
(https://github.com/zitadel/zitadel/pull/9597)
- feat(console): list v2 sessions
(https://github.com/zitadel/zitadel/pull/9539)
- fix(console): add loginV2 feature flag
(https://github.com/zitadel/zitadel/pull/9682)
- fix(feature flags): allow reading "own" flags
(https://github.com/zitadel/zitadel/pull/9649)
- feat(console): add Actions V2 UI
(https://github.com/zitadel/zitadel/pull/9591)

BREAKING CHANGE
- feat(webkey): migrate to v2beta API
(https://github.com/zitadel/zitadel/pull/9445)
- chore!: remove CockroachDB Support
(https://github.com/zitadel/zitadel/pull/9444)
- feat(actions): migrate to v2beta API
(https://github.com/zitadel/zitadel/pull/9489)

---------

Co-authored-by: Livio Spring <[email protected]>
Co-authored-by: Stefan Benz <[email protected]>
Co-authored-by: Silvan <[email protected]>
Co-authored-by: Ramon <[email protected]>
Co-authored-by: Elio Bischof <[email protected]>
Co-authored-by: Kenta Yamaguchi <[email protected]>
Co-authored-by: Harsha Reddy <[email protected]>
Co-authored-by: Livio Spring <[email protected]>
Co-authored-by: Max Peintner <[email protected]>
Co-authored-by: Iraq <[email protected]>
Co-authored-by: Florian Forster <[email protected]>
Co-authored-by: Tim Möhlmann <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Max Peintner <[email protected]>
2025-04-02 16:53:06 +02:00
84997ffe1a fix(session v2): allow searching for own sessions or user agent (fingerprintID) (#9110)
# Which Problems Are Solved

ListSessions only works to list the sessions that you are the creator
of.

# How the Problems Are Solved

Add options to search for sessions created by other users, sessions
belonging to the same useragent and sessions belonging to your user.
Possible through additional search parameters which as default use the
information contained in your session token but can also be filled with
specific IDs.

# Additional Changes

Remodel integration tests, to separate the Create and Get of sessions
correctly.

# Additional Context

Closes #8301

---------

Co-authored-by: Livio Spring <[email protected]>
2025-01-14 14:15:59 +01:00
Tim MöhlmannandGitHub 32bad3feb3 perf(milestones): refactor (#8788)
# Which Problems Are Solved

Milestones used existing events from a number of aggregates. OIDC
session is one of them. We noticed in load-tests that the reduction of
the oidc_session.added event into the milestone projection is a costly
business with payload based conditionals. A milestone is reached once,
but even then we remain subscribed to the OIDC events. This requires the
projections.current_states to be updated continuously.


# How the Problems Are Solved

The milestone creation is refactored to use dedicated events instead.
The command side decides when a milestone is reached and creates the
reached event once for each milestone when required.

# Additional Changes

In order to prevent reached milestones being created twice, a migration
script is provided. When the old `projections.milestones` table exist,
the state is read from there and `v2` milestone aggregate events are
created, with the original reached and pushed dates.

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/8800
2024-10-28 08:29:34 +00:00
Livio SpringandGitHub dc7330f251 fix(API): select org context by domain (#8706)
# Which Problems Are Solved

V2 and V3 APIs allow setting the organization context by providing the
organization domain in the request. Users currently experience the
following error: "rpc error: code = Unauthenticated desc = context
missing (AUTH-rKLWEH)"

# How the Problems Are Solved

Correctly check the org domain when set.

# Additional Changes

None

# Additional Context

- support request
2024-10-02 06:38:54 +00:00
Livio SpringandGitHub c347e75485 fix: ignore projectID and origin check for service accounts (#8704)
# Which Problems Are Solved

Calls with tokens issued through JWT Profile or Client Credentials
Grants were no longer possible and threw a "could not read projectid by
clientid (AUTH-GHpw2)" error.
ZITADEL checks the allowed origins of an application and load its
projectID into the context on any API call.
Tokens from service accounts did not contain any clientID and therefore
never did that check.
But due to a change in https://github.com/zitadel/zitadel/pull/8580,
were the service user id was set as client_id in the OIDC session to fix
the introspection response
(https://github.com/zitadel/zitadel/issues/8590).

# How the Problems Are Solved

- Check if the project and origin were retrieved and only then check the
origins

# Additional Changes

None.

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/8676
- relates to https://github.com/zitadel/zitadel/pull/8580 (released on
2.62.0)
- relates to https://github.com/zitadel/zitadel/issues/8590
2024-10-01 16:38:28 +02:00
Livio SpringandGitHub 14383cfe8c fix: race condition in system jwt signature check (#8618) 2024-09-13 17:18:37 +02:00
042c438813 feat(v3alpha): read actions (#8357)
# Which Problems Are Solved

The current v3alpha actions APIs don't exactly adhere to the [new
resources API
design](https://zitadel.com/docs/apis/v3#standard-resources).

# How the Problems Are Solved

- **Improved ID access**: The aggregate ID is added to the resource
details object, so accessing resource IDs and constructing proto
messages for resources is easier
- **Explicit Instances**: Optionally, the instance can be explicitly
given in each request
- **Pagination**: A default search limit and a max search limit are
added to the defaults.yaml. They apply to the new v3 APIs (currently
only actions). The search query defaults are changed to ascending by
creation date, because this makes the pagination results the most
deterministic. The creation date is also added to the object details.
The bug with updated creation dates is fixed for executions and targets.
- **Removed Sequences**: Removed Sequence from object details and
ProcessedSequence from search details

# Additional Changes

Object details IDs are checked in unit test only if an empty ID is
expected. Centralizing the details check also makes this internal object
more flexible for future evolutions.

# Additional Context

- Closes #8169 
- Depends on https://github.com/zitadel/zitadel/pull/8225

---------

Co-authored-by: Silvan <[email protected]>
Co-authored-by: Stefan Benz <[email protected]>
2024-08-12 22:32:01 +02:00
Livio SpringandGitHub 3d071fc505 feat: trusted (instance) domains (#8369)
# Which Problems Are Solved

ZITADEL currently selects the instance context based on a HTTP header
(see https://github.com/zitadel/zitadel/issues/8279#issue-2399959845 and
checks it against the list of instance domains. Let's call it instance
or API domain.
For any context based URL (e.g. OAuth, OIDC, SAML endpoints, links in
emails, ...) the requested domain (instance domain) will be used. Let's
call it the public domain.
In cases of proxied setups, all exposed domains (public domains) require
the domain to be managed as instance domain.
This can either be done using the "ExternalDomain" in the runtime config
or via system API, which requires a validation through CustomerPortal on
zitadel.cloud.

# How the Problems Are Solved

- Two new headers / header list are added:
- `InstanceHostHeaders`: an ordered list (first sent wins), which will
be used to match the instance.
(For backward compatibility: the `HTTP1HostHeader`, `HTTP2HostHeader`
and `forwarded`, `x-forwarded-for`, `x-forwarded-host` are checked
afterwards as well)
- `PublicHostHeaders`: an ordered list (first sent wins), which will be
used as public host / domain. This will be checked against a list of
trusted domains on the instance.
- The middleware intercepts all requests to the API and passes a
`DomainCtx` object with the hosts and protocol into the context
(previously only a computed `origin` was passed)
- HTTP / GRPC server do not longer try to match the headers to instances
themself, but use the passed `http.DomainContext` in their interceptors.
- The `RequestedHost` and `RequestedDomain` from authz.Instance are
removed in favor of the `http.DomainContext`
- When authenticating to or signing out from Console UI, the current
`http.DomainContext(ctx).Origin` (already checked by instance
interceptor for validity) is used to compute and dynamically add a
`redirect_uri` and `post_logout_redirect_uri`.
- Gateway passes all configured host headers (previously only did
`x-zitadel-*`)
- Admin API allows to manage trusted domain

# Additional Changes

None

# Additional Context

- part of #8279 
- open topics: 
  - "single-instance" mode
  - Console UI
2024-07-31 18:00:38 +03:00
Tim MöhlmannandGitHub fe9bb49caa chore(deps): update all go deps (#7773)
This change updates all go modules, including oidc, a major version of go-jose and the go 1.22 release.
2024-04-15 09:17:36 +00:00
Tim MöhlmannandGitHub c8e0b30e17 fix(oidc): return bad request for base64 errors (#7730)
* fix(oidc): return bad request for base64 errors

We've recently noticed an increased amount of 500: internal server error status returns on zitadel cloud.
The source of these errors appear to be erroneous input in fields that are supposed to be bas64 formatted.

```
time=2024-04-08T14:05:47.600Z level=ERROR msg="request error" oidc_error.parent="ID=OIDC-AhX2u Message=Errors.Internal Parent=(illegal base64 data at input byte 8)" oidc_error.description=Errors.Internal oidc_error.type=server_error status_code=500
```

Within the possible code paths of the token endpoint there are a couple of uses of base64.Encoding.DecodeString of which a returned error was not properly wrapped, but returned as-is.
This causes the oidc error handler to return a 500 with the `OIDC-AhX2u` ID.
We were not able to pinpoint the exact errors that are happening to any one call of `DecodeString`.

This fix wraps all errors from `DecodeString` so that proper 400: bad request is returned with information about the error. Each wrapper now has an unique error ID, so that logs will contain the source of the error as well.

This bug was reported internally by the ops team.

* catch op.ErrInvalidRefreshToken
2024-04-09 08:42:59 +02:00
Tim MöhlmannandGitHub 6398349c24 feat(oidc): token exchange impersonation (#7516)
* add token exchange feature flag

* allow setting reason and actor to access tokens

* impersonation

* set token types and scopes in response

* upgrade oidc to working draft state

* fix tests

* audience and scope validation

* id toke and jwt as input

* return id tokens

* add grant type  token exchange to app config

* add integration tests

* check and deny actors in api calls

* fix instance setting tests by triggering projection on write and cleanup

* insert sleep statements again

* solve linting issues

* add translations

* pin oidc v3.15.0

* resolve comments, add event translation

* fix refreshtoken test

* use ValidateAuthReqScopes from oidc

* apparently the linter can't make up its mind

* persist actor thru refresh tokens and check in tests

* remove unneeded triggers
2024-03-20 10:18:46 +00:00
062d153cfe feat: impersonation roles (#7442)
* partial work done

* test IAM membership roles

* org membership tests

* console :(, translations and docs

* fix integration test

* fix tests

* add EnableImpersonation to security policy API

* fix integration test timestamp checking

* add security policy tests and fix projections

* add impersonation setting in console

* add security settings to the settings v2 API

* fix typo

* move impersonation to instance

---------

Co-authored-by: Livio Spring <[email protected]>
2024-02-28 10:21:11 +00:00
26d1563643 feat(api): feature flags (#7356)
* feat(api): feature API proto definitions

* update proto based on discussion with @livio-a

* cleanup old feature flag stuff

* authz instance queries

* align defaults

* projection definitions

* define commands and event reducers

* implement system and instance setter APIs

* api getter implementation

* unit test repository package

* command unit tests

* unit test Get queries

* grpc converter unit tests

* migrate the V1 features

* migrate oidc to dynamic features

* projection unit test

* fix instance by host

* fix instance by id data type in sql

* fix linting errors

* add system projection test

* fix behavior inversion

* resolve proto file comments

* rename SystemDefaultLoginInstanceEventType to SystemLoginDefaultOrgEventType so it's consistent with the instance level event

* use write models and conditional set events

* system features integration tests

* instance features integration tests

* error on empty request

* documentation entry

* typo in feature.proto

* fix start unit tests

* solve linting error on key case switch

* remove system defaults after discussion with @eliobischof

* fix system feature projection

* resolve comments in defaults.yaml

---------

Co-authored-by: Livio Spring <[email protected]>
2024-02-28 10:55:54 +02:00
ed0bc39ea4 feat: block instances (#7129)
* docs: fix init description typos

* feat: block instances using limits

* translate

* unit tests

* fix translations

* redirect /ui/login

* fix http interceptor

* cleanup

* fix http interceptor

* fix: delete cookies on gateway 200

* add integration tests

* add command test

* docs

* fix integration tests

* add bulk api and integration test

* optimize bulk set limits

* unit test bulk limits

* fix broken link

* fix assets middleware

* fix broken link

* validate instance id format

* Update internal/eventstore/search_query.go

Co-authored-by: Livio Spring <[email protected]>

* remove support for owner bulk limit commands

* project limits to instances

* migrate instances projection

* Revert "migrate instances projection"

This reverts commit 214218732a56e6df823beac1972adfcf8beeded5.

* join limits, remove owner

* remove todo

* use optional bool

* normally validate instance ids

* use 302

* cleanup

* cleanup

* Update internal/api/grpc/system/limits_converter.go

Co-authored-by: Livio Spring <[email protected]>

* remove owner

* remove owner from reset

---------

Co-authored-by: Livio Spring <[email protected]>
2024-01-17 10:16:48 +00:00
f680dd934d refactor: rename package errors to zerrors (#7039)
* chore: rename package errors to zerrors

* rename package errors to gerrors

* fix error related linting issues

* fix zitadel error assertion

* fix gosimple linting issues

* fix deprecated linting issues

* resolve gci linting issues

* fix import structure

---------

Co-authored-by: Elio Bischof <[email protected]>
2023-12-08 15:30:55 +01:00
24b05dc88c fix(authz): add logging to access token verification errors (#6976)
* fix(authz): add logging to access token verification errors

Related to #6949

* use logging fields

---------

Co-authored-by: Livio Spring <[email protected]>
2023-11-27 15:35:08 +00:00
a4626f9bdb fix: return unauthenticated code (#6819)
* fix: return unauthenticated code

* remove cfg.yaml

---------

Co-authored-by: Livio Spring <[email protected]>
2023-10-27 06:09:07 +00:00
4980cd6a0c feat: add SYSTEM_OWNER role (#6765)
* define roles and permissions

* support system user memberships

* don't limit system users

* cleanup permissions

* restrict memberships to aggregates

* default to SYSTEM_OWNER

* update unit tests

* test: system user token test (#6778)

* update unit tests

* refactor: make authz testable

* move session constants

* cleanup

* comment

* comment

* decode member type string to enum (#6780)

* decode member type string to enum

* handle all membership types

* decode enums where necessary

* decode member type in steps config

* update system api docs

* add technical advisory

* tweak docs a bit

* comment in comment

* lint

* extract token from Bearer header prefix

* review changes

* fix tests

* fix: add fix for activityhandler

* add isSystemUser

* remove IsSystemUser from activity info

* fix: add fix for activityhandler

---------

Co-authored-by: Stefan Benz <[email protected]>
2023-10-25 15:10:45 +00:00
adlerhurst bd23a7a56f merge main into next 2023-10-19 12:34:00 +02:00
SilvanandGitHub b5564572bc feat(eventstore): increase parallel write capabilities (#5940)
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and  [06](https://zitadel.com/docs/support/advisory/a10006).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
2023-10-19 12:19:10 +02:00
8f6cb47567 fix: use triggering origin for notification links (#6628)
* take baseurl if saved on event

* refactor: make es mocks reusable

* Revert "refactor: make es mocks reusable"

This reverts commit 434ce12a6acf639514308bc231e76ebb8676b643.

* make messages testable

* test asset url

* fmt

* fmt

* simplify notification.Start

* test url combinations

* support init code added

* support password changed

* support reset pw

* support user domain claimed

* support add pwless login

* support verify phone

* Revert "support verify phone"

This reverts commit e40503303e2fdda0c85985b3fe3160ce96d43cca.

* save trigger origin from ctx

* add ready for review check

* camel

* test email otp

* fix variable naming

* fix DefaultOTPEmailURLV2

* Revert "fix DefaultOTPEmailURLV2"

This reverts commit fa34d4d2a83fbfd8353759c9148af9165a9dd44c.

* fix email otp challenged test

* fix email otp challenged test

* pass origin in login and gateway requests

* take origin from header

* take x-forwarded if present

* Update internal/notification/handlers/queries.go

Co-authored-by: Tim Möhlmann <[email protected]>

* Update internal/notification/handlers/commands.go

Co-authored-by: Tim Möhlmann <[email protected]>

* move origin header to ctx if available

* generate

* cleanup

* use forwarded header

* support X-Forwarded-* headers

* standardize context handling

* fix linting

---------

Co-authored-by: Tim Möhlmann <[email protected]>
2023-10-10 13:20:53 +00:00
Livio SpringandGitHub 9696fde676 fix: reduce origin check to tokens issued through code and implicit flow (#6681)
fix: only check origin for tokens issued to users through apps (code / implicit flow)
2023-10-09 10:08:18 +00:00
Livio SpringandGitHub a1942ecdaa feat(api): add and remove OTP (SMS and email) (#6295)
* refactor: rename otp to totp

* feat: add otp sms and email

* implement tests
2023-08-02 18:57:53 +02:00
Livio SpringandGitHub 80961125a7 feat(API): support V2 token and session token usage (#6180)
This PR adds support for userinfo and introspection of V2 tokens. Further V2 access tokens and session tokens can be used for authentication on the ZITADEL API (like the current access tokens).
2023-07-14 11:16:16 +00:00
14b8cf4894 feat(api): add OIDC session service (#6157)
This PR starts the OIDC implementation for the API V2 including the Implicit and Code Flow.


Co-authored-by: Livio Spring <[email protected]>
Co-authored-by: Tim Möhlmann <[email protected]>
Co-authored-by: Stefan Benz <[email protected]>
2023-07-10 13:27:00 +00:00
Tim MöhlmannandGitHub a301c40f9f feat: implement register Passkey user API v2 (#5873)
* command/crypto: DRY the code

- reuse the the algorithm switch to create a secret generator
- add a verifyCryptoCode function

* command: crypto code tests

* migrate webauthn package

* finish integration tests with webauthn mock client
2023-05-24 10:22:00 +00:00
c2cb84cd24 feat(api): new session service (#5801)
* backup new protoc plugin

* backup

* session

* backup

* initial implementation

* change to specific events

* implement tests

* cleanup

* refactor: use new protoc plugin for api v2

* change package

* simplify code

* cleanup

* cleanup

* fix merge

* start queries

* fix tests

* improve returned values

* add token to projection

* tests

* test db map

* update query

* permission checks

* fix tests and linting

* rework token creation

* i18n

* refactor token check and fix tests

* session to PB test

* request to query tests

* cleanup proto

* test user check

* add comment

* simplify database map type

* Update docs/docs/guides/integrate/access-zitadel-system-api.md

Co-authored-by: Tim Möhlmann <[email protected]>

* fix test

* cleanup

* docs

---------

Co-authored-by: Tim Möhlmann <[email protected]>
2023-05-05 15:34:53 +00:00
095ec21678 feat: user v2alpha email API (#5708)
* chore(proto): update versions

* change protoc plugin

* some cleanups

* define api for setting emails in new api

* implement user.SetEmail

* move SetEmail buisiness logic into command

* resuse newCryptoCode

* command: add ChangeEmail unit tests

Not complete, was not able to mock the generator.

* Revert "resuse newCryptoCode"

This reverts commit c89e90ae35ae924a3f706a0a7394f933910c2e65.

* undo change to crypto code generators

* command: use a generator so we can test properly

* command: reorganise ChangeEmail

improve test coverage

* implement VerifyEmail

including unit tests

* add URL template tests

* proto: change context to object

* remove old auth option

* remove old auth option

* fix linting errors

run gci on modified files

* add permission checks and fix some errors

* comments

* comments

---------

Co-authored-by: Livio Spring <[email protected]>
Co-authored-by: Tim Möhlmann <[email protected]>
2023-04-25 09:02:29 +02:00
25c3c17986 chore: upgrade to oidc v2 release (#5437)
* chore: upgrade to oidc v2 release

* fix tests

* fix build errors after rebase

* pin oidc v2.1.0

* pin oidc v2.1.1 (include bugfix)

* pin oidc v2.1.2 (include bugfix)

* pin oidc v2.2.1 (bugfix)

include fix zitadel/oidc#349

* fix: refresh token handling

* simplify cognitive complexity

* fix: handle error

---------

Co-authored-by: Livio Spring <[email protected]>
2023-03-28 11:28:56 +00:00
SilvanandGitHub e38abdcdf3 perf: query data AS OF SYSTEM TIME (#5231)
Queries the data in the storage layser at the timestamp when the call hit the API layer
2023-02-27 22:36:43 +01:00
Elio BischofandGitHub 681541f41b feat: add quotas (#4779)
adds possibilities to cap authenticated requests and execution seconds of actions on a defined intervall
2023-02-15 02:52:11 +01:00
632639ae7f feat: enable iframe use (#4766)
* feat: enable iframe use

* cleanup

* fix mocks

* fix linting

* docs: add iframe usage to solution scenarios configurations

* improve api

* feat(console): security policy

* description

* remove unnecessary line

* disable input button and urls when not enabled

* add image to docs

Co-authored-by: Max Peintner <[email protected]>
Co-authored-by: Fabi <[email protected]>
2022-12-14 07:17:36 +01:00
Livio AmstutzandGitHub 0baaaf8a05 fix: handle default org id (#3769) 2022-06-03 14:30:39 +02:00