# Which Problems Are Solved
This PR introduces the possibility of using the relational DB when
calling the ListSession gRPC endpoint.
# How the Problems Are Solved
- Implement the Querier interface for ListSession:
- Create a `domain` model for the Session object to be able to map the
gRPC session into it.
- Match the permission filters on ES side to be sure the same sessions
are returned
- Implement converter methods to go from `gRPC` to `domain` models (both
ways)
- Fix the session sql relational migration: rework the trigger to update
the `expiration` field to perform the update only when the input
`lifetime` has a non-zero value.
- Link the Querier implementation with the gRPC endpoint
- Run the same integration tests against the relational DB, making sure
the outcome is the same. ⚠️ Due to missing permission checks, some
divergence is expected. Hence, specific code to circumvent the
differences has been put in place
# Additional Context
- Closes#11039
# Which Problems Are Solved
As part of #11035 , this PR implements the TOTP check logic for session
validation
# How the Problems Are Solved
- Implement TOTP check logic and tests
- The tarpit function has been moved to a common file so that it can be
used by both TOTP and password checks
- Manual transaction management to avoid stalling the DB while verifier
function is executed
- Update `database.Change` `Matches()` function to allow comparison of
`time.Time` values
- A converter package from GRPC to Domain model has been added
# Additional Context
This is a cherry-picked PR + changes, coming from
https://github.com/zitadel/zitadel/pull/11164
- Relates to #11035
# Which Problems Are Solved
In https://github.com/zitadel/zitadel/pull/11858#discussion_r2993742914
we discussed on how to translate the ES passkey state to a relational
one. That is used to filter out passkeys during convertion to webauthn
credentials.
It was decided to consider passkeys in ready state those that have a
verified at `time.Time` set.
The change required fixing the reducer for the eventstore event.
# How the Problems Are Solved
- Set `VerifiedAt` once `HumanWebAuthNVerifiedEvent` is received
- Also set the verification to `nil`
# Additional Context
- Discussion #11858
- Follow-up for PR #11858
# Which Problems Are Solved
As part of #11035 , this PR implements the Passkey check logic for
session validation
# How the Problems Are Solved
- Refactor webauth FinishLogin to support new domain model
- Add webauth config to defaults
- Implement passkey check logic and tests
- Manual transaction management to avoid stalling the DB while
FinishLogin callback is executed
- Update passkey Type condition to allow passing a text operation
(equal, contains, etc..)
# Additional Context
This is a cherry-picked PR + minor changes, coming from
https://github.com/zitadel/zitadel/pull/11164
- Relates to #11035
---------
Co-authored-by: Fabienne Bühler <[email protected]>
Co-authored-by: Gayathri Vijayan <[email protected]>
# Which Problems Are Solved
Paragraph reorganization of `SECURITY.md` and some minor changes
# How the Problems Are Solved
- Move out of scope section under `Scope`
- Update `Supported Versions` with an up to date link
- Extend `Out of Scope`
---------
Co-authored-by: Copilot <[email protected]>
# Which Problems Are Solved
2 tests were failing on the new session checks:
- Password check (events) fails due to a race condition
- IDP intent check (events) (also password test) fails due to a flaky
time comparison
# How the Problems Are Solved
- Race condition: remove setting the password hasher in the test. It's
already globally set in `domain_test.go`
- Time comparison: use `assert.WithinRange` instead of `assert.InDelta`
and use 2 second range to make sure you don't get resolution issues
# Additional Context
- Closes#11878
# Which Problems Are Solved
As part of #11035 , this PR implements the IDP intent check logic for
session validation
# How the Problems Are Solved
- Inject encryption algorithm for IDP intent tokens at DB initialization
- Add the IDP intent repository as an invoke option
- Rework crypto mocks using gomock
- Implement IDP intent check logic
- minor - DB: expand IDP Intent deletion testing table with a test case
where IDP intent is not found
# Noteworthy
I did not implement `domain.Transactional` given the doubts that
@adlerhurst expressed in https://github.com/zitadel/zitadel/pull/11804 .
So, there is no transaction AT ALL in here because the `Execute()`
method contains one single DB call (Delete) and `Validate()` just does
Get calls.
# Additional Context
This is a cherry-picked PR + minor changes, coming from
https://github.com/zitadel/zitadel/pull/11164
- Relates to #11035
- Depends on https://github.com/zitadel/zitadel/pull/11804
# Which Problems Are Solved
As part of #11035 , this PR implements the password check logic for
session validation
# How the Problems Are Solved
- Add system config to default configuration of `domain` package for
easy initialization. Intialize the system settings when Zitadel starts
up
- Add password hasher verify logic to the default configuration of
`domain` package. Initialize it when Zitadel starts up.
- Add settings repositories with their mocks
- Implement the logic for doing a password check
# Additional Context
This is a cherry-picked PR + minor changes, coming from
https://github.com/zitadel/zitadel/pull/11164
- Relates to #11035
- Depends on https://github.com/zitadel/zitadel/pull/11777
# Which Problems Are Solved
As part of #11035 , this PR implements the user check logic for session
validation
# How the Problems Are Solved
- Add user and session repositories as invoke options, as well as their
mocks to prepare the ground for testing
- Implement the logic for the UserCheck command, following what has been
done on eventstore side
- Add a session permission chek interface to be used during the
`Validate()` part of the Commander
# Additional Context
This is a cherry-picked PR + minor changes, coming from
https://github.com/zitadel/zitadel/pull/11164
- Relates to #11035
---------
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
This is a small refactoring for tests in `backend/v3/domain` package to
use `assert.ErrorIs` instead of `assert.Equal` for error assertion.
# How the Problems Are Solved
- Add custom `Is()` function to `MultipleObjectsUpdatedError` error
- Substitute `assert.Equal` with `assert.ErrorIs` in instance and
organization tests
# Additional Context
This was part of #11035
# Which Problems Are Solved
Renaming of:
- Resource Owner
- OrgID
- OrganizationID
- Organization Id
# How the Problems Are Solved
Find & Replace, evalue entries, is resource owner referring to an
organization ? If so, do change
# Additional Context
- Closes#11305
# Which Problems Are Solved
This PR provides an example for testing DB reduction to relational
tables without exploiting integration tests.
# How the Problems Are Solved
- Set up a disposable embedded Postgres DB used to store the results of
the calls to the reducers and verify such results
- Add utility method to return transactions from the existing DB pool.
These transactions can be used by the single test cases to insert and
read data from DB
- Add a test to verify that the handler contains the expected reducers
(checking also that they are on the right aggregate type)
- Add tests to verify the single reduce functions. A parent test creates
the required objects to satisfy the foreign key constraints.
# Additional Changes
- Typo fixed in `postgres.DecodeConfig()`
- Discard unneeded `projectionName` parameter for `Executor` function
# Additional Context
- Closes#11430
# Which Problems Are Solved
Inconsistent naming of service account, found in the following
variations:
- Machine User
- machine user
- Service User
- Machine Account
- Technical Account
- User: Type Machine
# How the Problems Are Solved
Attentive search and replace.
Localizations have been translated using Copilot
# Additional Changes
Some unused methods have been removed from the Go code.
# Additional Context
- Closes#11285
# Which Problems Are Solved
Naming inconsistencies - User (Human)
# How the Problems Are Solved
Most of the occurrences have not been changed. `User (Human)` was mostly
changed when talking about code objects and where I felt it was
necessary to distinguish them from machine users.
When both human and machine user occurrences were found, the machine
user has been changed to service account (see
https://github.com/zitadel/zitadel/issues/11285)
# Additional Context
- Closes#11284
---------
Co-authored-by: Wim Van Laer <[email protected]>
# Which Problems Are Solved
In the context of moving our datastore to relational tables, Identity
Provider Intents have no definition of a relational table nor any DB API
that can be used to store the IDP intents as relational objects.
# How the Problems Are Solved
The PR follows the guidelines left in #11190
- A table `zitadel.identity_provider_intents` is created: it defines its
relational constraints (foreign keys) with the other tables, as well as
an ENUM to represent the state of the intent.
- An IDP Intent repository has been created, which offers CRUD
operations implemented as `Create`, `Get`, `Update` and `Delete` methods
for the IDP intent.
- The repository also defines all columns and conditions useful for
performing DB operations on the IDP intents.
- DB mocks have been added as well to ease testing on the `domain`
package.
- Reduction functions have been implemented for the existing IDP events
so to keep the relational table in sync with the event store.
# Additional Changes
To allow testing LDAP intent flow, the `Session.FetchUser()` method has
been extracted to a separate file. A corresponding mock has been created
as well in `fetch_user_mock.go`. The files are mutually excluded and
added at build time through go build tags: the mock is added when the
`integration` tag is specified, while the real implementation is added
when the same tag IS NOT added to the build tags.
# Additional Context
- Closes#11190
---------
Co-authored-by: Livio Spring <[email protected]>
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
The following terms have all been renamed to management console:
- Customer Portal (when used to mean the console)
- Console
- Admin Console
# How the Problems Are Solved
- Search & Replace smartly
- Use Copilot for translation files
Changes done to: backend + frontend codebase, docs, translations and
protobufs (descriptions only)
# Additional Context
- Partially Closes#11279
# Which Problems Are Solved
This PR implements the endpoints listed in
https://github.com/zitadel/zitadel/issues/10444 .
# How the Problems Are Solved
The implementation follows the same pattern as the Instance one. The
`Add` and `Remove` are implementing the `Commander` interface, while
`List` Domains endpoints are implementing the `Querier` interace.
The PR adds also some forgotten calls to the relational logic for the
Instance API. The commits are identifiable by the label `instance:`
prepended to them. This is the complete list: bafce01b0 , 6fe7257ab ,
e27bc9c34 , 8133ede73 , fde667953 , a3e1135ed , ee1a6f9c1
# Additional Context
- Closes#10444
---------
Co-authored-by: Elio Bischof <[email protected]>
Co-authored-by: Lebogang Phoshoko <[email protected]>
Co-authored-by: Wim Van Laer <[email protected]>
Co-authored-by: Max Peintner <[email protected]>
Co-authored-by: Gayathri Vijayan <[email protected]>
* Force v2 permission checks on user listing
* Remove duplicated tests
* cleanup and fix tests
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
This PR implements the endpoints listed in
https://github.com/zitadel/zitadel/issues/10443 .
# How the Problems Are Solved
The implementation follows the same pattern as the Organization one.
There are no peculiarities to this PR. The `Update`, `Delete` are
implementing the `Commander` interface, while `Get` and `List`
instance(s) endpoints are implementing the `Querier` interace.
# Additional Context
- Closes#10443
- Depends on #10445
# Which Problems Are Solved
This PR implements the endpoints listed in #10445 using relational
tables.
- [x] UpdateOrganization
- [x] ListOrganizations
- [x] DeleteOrganization
- [x] DeactivateOrganization
- [x] ActivateOrganization
# How the Problems Are Solved
- **UpdateOrganization:** Implemented logic for `Validate()`,
`Execute()` and `Events()`. On CQRS side, updating an organization emits
events related to the organization and its domains. Separate commanders
have been made for the domain events so that their logic is run
accordingly and we keep separation of concerns. Implementation of the
domains is left to its own ticket.
Because some domain-related data is changed during the organization
update, its original values are saved as pointers inside
[UpdateOrgCommand](https://github.com/zitadel/zitadel/blob/4f87cd8d0cfd75f290aa13c342e4939cecebef73/backend/v3/domain/org_update.go#L20).
These variables will be then used as input for the domain commanders
(that will run after the update organization one).
- **ListOrganizations:** Implemented logic for `Validate()`,
`Execute()`. Utility methods have been made to parse the input queries.
This commander is used both by the v2 and v2beta endpoint, so a
translation layer has been put in place to translate all v2beta requests
into v2. The commander strictly handles v2 request only.
The translation layer between v2beta and v2 APIs was made so to make it
simple to remove it once the v2beta APIs will be removed (a simple
deletion should suffice). A `converter` package [is
shipped](https://github.com/zitadel/zitadel/blob/4f87cd8d0cfd75f290aa13c342e4939cecebef73/backend/v3/api/org/v2/convert/README.md#L5)
as well with instruction for removal. TODOs have been put in place all
over the code to indicate that the related piece of code needs to be
removed once v2beta -> v2 transition is complete.
- **DeleteOrganization:** Implemented logic for `Validate()`,
`Execute()` and `Events()`. The `Events()` method is mostly incomplete
as it requires a lot of data that is currently not retrievable due to
the lack of their relational tables.
- **DeactivateOrganization** and **ActivateOrganization**: Implemented
logic for `Validate()`, `Execute()` and `Events()`. No notable remarks.
👉 All endpoints return matching errors to the CQRS counterpart: this way
allows re-using the integration tests and
👉 All endpoints have been unit-tested through DB mocking
👉 A bunch of TODOs have been put in place for future reworks that are
needed (e.g. `Update()` method on repository should also return a
timestamp)
# Additional Changes
- A test utility method to allow mocking query options have been made.
This was necessary because `database.QueryOption` is a function and the
only comparison possible with functions is `nil` check. So, to mock
this, `database.QueryOption` is converted first to `database.QueryOpts`.
See 2276742adaef1998ea4844e1f0dca2079e45ccfb
- A `BaseCommand` has been created with the idea to collect methods that
are going to be needed by other commands. For the moment, it only offers
a method to convert query text operations from gRPC to `domain` model.
See bb85456dd101282fa76d2e2ad165eed4c8ef61b9
- SQL operations have been reworked to allow for queries using
`ContainsIgnoreCase`, `EndsWithIgnoreCase` and other `IgnoreCase`
operations. This change is pending approval and might be reworked. See
https://github.com/zitadel/zitadel/pull/10704#discussion_r2392886177
# Additional Context
`UpdateOrganization` is partially working because the logic for handling
domain updates is not done (there is only a draft of the commanders to
showcase the call chain). See
https://github.com/zitadel/zitadel/pull/10704#discussion_r2356143178
Permissions are not implemented as they are missing. See
https://github.com/zitadel/zitadel/pull/10771
- Closes: #10445
---------
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
This PR introduces a new feature flag `EnableRelationalTables` that will
be used in following implementations to decide whether Zitadel should
use the relational model or the event sourcing one.
# TODO
- [x] Implement flag at system level
- [x] Display the flag on console:
https://github.com/zitadel/zitadel/pull/10615
# How the Problems Are Solved
- Implement loading the flag from config
- Add persistence of the flag through gRPC endpoint
(SetInstanceFeatures)
- Implement reading of the flag through gRPC endpoint
(GetInstanceFeatures)
# Additional Changes
Some minor refactoring to remove un-needed generics annotations
# Additional Context
- Closes#10574
---------
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
Some users have reported the need of retrieving users given a metadata
key, metadata value or both. This change introduces metadata search
filter on the `ListUsers()` endpoint to allow Zitadel users to search
for user records by metadata.
The changes affect only v2 APIs.
# How the Problems Are Solved
- Add new search filter to `ListUserRequest`: `MetaKey` and `MetaValue`
- Add SQL indices on metadata key and metadata value
- Update query to left join `user_metadata` table
# Additional Context
- Closes#9053
- Depends on https://github.com/zitadel/zitadel/pull/10567
---------
Co-authored-by: Silvan <[email protected]>
Co-authored-by: Stefan Benz <[email protected]>
# Which Problems Are Solved
Some events that are now unused are clogging the event queue from time
to time.
# How the Problems Are Solved
Remove the events described in #10458
# Additional Changes
- Updated `stringer` and `enumer` in Makefile target `core_generate_all`
to resolve generated files compilation issues
# Notes
It looks like there are a lot of changes, but most of it is fixing
translation files. I suggest doing a review per-commit
# Additional Context
- Closes#10458
- Depends on https://github.com/zitadel/zitadel/pull/10513
# Which Problems Are Solved
The roadmap page is not up to date with the latest changes that have
been released with Zitadel v4.
# How the Problems Are Solved
Update the doc according to #10309
# Dependencies
- https://github.com/zitadel/zitadel/pull/10249 -> Update the API docs
with deprecated endpoints
- https://github.com/zitadel/zitadel/pull/10364 -> Actions v2 beta to v2
# Additional Context
- Closes#10309
# Which Problems Are Solved
This PR *partially* addresses #9450 . Specifically, it implements the
resource based API for app keys.
This PR, together with https://github.com/zitadel/zitadel/pull/10077
completes #9450 .
# How the Problems Are Solved
- Implementation of the following endpoints: `CreateApplicationKey`,
`DeleteApplicationKey`, `GetApplicationKey`, `ListApplicationKeys`
- `ListApplicationKeys` can filter by project, app or organization ID.
Sorting is also possible according to some criteria.
- All endpoints use permissions V2
# TODO
- [x] Deprecate old endpoints
# Additional Context
Closes#9450
# Which Problems Are Solved
This PR *partially* addresses #9450 . Specifically, it implements the
resource based API for the apps. APIs for app keys ARE not part of this
PR.
# How the Problems Are Solved
- `CreateApplication`, `PatchApplication` (update) and
`RegenerateClientSecret` endpoints are now unique for all app types:
API, SAML and OIDC apps.
- All new endpoints have integration tests
- All new endpoints are using permission checks V2
# Additional Changes
- The `ListApplications` endpoint allows to do sorting (see protobuf for
details) and filtering by app type (see protobuf).
- SAML and OIDC update endpoint can now receive requests for partial
updates
# Additional Context
Partially addresses #9450
# Which Problems Are Solved
This PR implements https://github.com/zitadel/zitadel/issues/9850
# How the Problems Are Solved
- New protobuf definition
- Implementation of retrieval of system translations
- Implementation of retrieval and persistence of organization and
instance level translations
# Additional Context
- Closes#9850
# TODO
- [x] Integration tests for Get and Set hosted login translation
endpoints
- [x] DB migration test
- [x] Command function tests
- [x] Command util functions tests
- [x] Query function test
- [x] Query util functions tests
