# Which Problems Are Solved
This PR implements the endpoints listed in #10445 using relational
tables.
- [x] UpdateOrganization
- [x] ListOrganizations
- [x] DeleteOrganization
- [x] DeactivateOrganization
- [x] ActivateOrganization
# How the Problems Are Solved
- **UpdateOrganization:** Implemented logic for `Validate()`,
`Execute()` and `Events()`. On CQRS side, updating an organization emits
events related to the organization and its domains. Separate commanders
have been made for the domain events so that their logic is run
accordingly and we keep separation of concerns. Implementation of the
domains is left to its own ticket.
Because some domain-related data is changed during the organization
update, its original values are saved as pointers inside
[UpdateOrgCommand](https://github.com/zitadel/zitadel/blob/4f87cd8d0cfd75f290aa13c342e4939cecebef73/backend/v3/domain/org_update.go#L20).
These variables will be then used as input for the domain commanders
(that will run after the update organization one).
- **ListOrganizations:** Implemented logic for `Validate()`,
`Execute()`. Utility methods have been made to parse the input queries.
This commander is used both by the v2 and v2beta endpoint, so a
translation layer has been put in place to translate all v2beta requests
into v2. The commander strictly handles v2 request only.
The translation layer between v2beta and v2 APIs was made so to make it
simple to remove it once the v2beta APIs will be removed (a simple
deletion should suffice). A `converter` package [is
shipped](https://github.com/zitadel/zitadel/blob/4f87cd8d0cfd75f290aa13c342e4939cecebef73/backend/v3/api/org/v2/convert/README.md#L5)
as well with instruction for removal. TODOs have been put in place all
over the code to indicate that the related piece of code needs to be
removed once v2beta -> v2 transition is complete.
- **DeleteOrganization:** Implemented logic for `Validate()`,
`Execute()` and `Events()`. The `Events()` method is mostly incomplete
as it requires a lot of data that is currently not retrievable due to
the lack of their relational tables.
- **DeactivateOrganization** and **ActivateOrganization**: Implemented
logic for `Validate()`, `Execute()` and `Events()`. No notable remarks.
👉 All endpoints return matching errors to the CQRS counterpart: this way
allows re-using the integration tests and
👉 All endpoints have been unit-tested through DB mocking
👉 A bunch of TODOs have been put in place for future reworks that are
needed (e.g. `Update()` method on repository should also return a
timestamp)
# Additional Changes
- A test utility method to allow mocking query options have been made.
This was necessary because `database.QueryOption` is a function and the
only comparison possible with functions is `nil` check. So, to mock
this, `database.QueryOption` is converted first to `database.QueryOpts`.
See 2276742adaef1998ea4844e1f0dca2079e45ccfb
- A `BaseCommand` has been created with the idea to collect methods that
are going to be needed by other commands. For the moment, it only offers
a method to convert query text operations from gRPC to `domain` model.
See bb85456dd101282fa76d2e2ad165eed4c8ef61b9
- SQL operations have been reworked to allow for queries using
`ContainsIgnoreCase`, `EndsWithIgnoreCase` and other `IgnoreCase`
operations. This change is pending approval and might be reworked. See
https://github.com/zitadel/zitadel/pull/10704#discussion_r2392886177
# Additional Context
`UpdateOrganization` is partially working because the logic for handling
domain updates is not done (there is only a draft of the commanders to
showcase the call chain). See
https://github.com/zitadel/zitadel/pull/10704#discussion_r2356143178
Permissions are not implemented as they are missing. See
https://github.com/zitadel/zitadel/pull/10771
- Closes: #10445
---------
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
Eventual consistency issues.
# How the Problems Are Solved
Correctly handle timeouts and change queries to domains instead of using
the organization name.
# Additional Changes
None
# Additional Context
None
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Flakiness and conflicts in value from gofakeit.
# How the Problems Are Solved
Move Gofakeit calls to the integration package, to guarantee proper
usage and values for integration testing.
# Additional Changes
None
# Additional Context
None
# Which Problems Are Solved
Flakiness in integration tests for organization v2beta service.
# How the Problems Are Solved
Fix eventual consistent handling of integration tests.
# Additional Changes
None
# Additional Context
None
---------
Co-authored-by: Marco A. <[email protected]>
This pull request fixes an issue where the repository would fail to scan
organization or instance structs if the `domains` column was `NULL`.
## Which problems are solved
If the `domains` column of `orgs` or `instances` was `NULL`, the
repository failed scanning into the structs. This happened because the
scanning mechanism did not correctly handle `NULL` JSONB columns.
## How the problems are solved
A new generic type `JSONArray[T]` is introduced, which implements the
`sql.Scanner` interface. This type can correctly scan JSON arrays from
the database, including handling `NULL` values gracefully.
The repositories for instances and organizations have been updated to
use this new type for the domains field. The SQL queries have also been
improved to use `FILTER` with `jsonb_agg` for better readability and
performance when aggregating domains.
## Additional changes
* An unnecessary cleanup step in the organization domain tests for
already removed domains has been removed.
* The `pgxscan` library has been replaced with `sqlscan` for scanning
`database/sql`.Rows.
* Minor cleanups in integration tests.
# Which Problems Are Solved
Integration test failed sometimes with `organization already
exists`-errors.
# How the Problems Are Solved
Use a consistent function to generate name used for organization
creation.
# Additional Changes
Correct a eventual consistent test for username around organization
domain changes with eventual consistent loop.
# Additional Context
None
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Flakiness in integration tests regarding gofakeit functions, which
provided the same names on 2 different occasions.
# How the Problems Are Solved
Attach a random string to the provided names, so that they are not
dependent on the gofakeit code.
# Additional Changes
None
# Additional Context
None
---------
Co-authored-by: Marco A. <[email protected]>
# Which Problems Are Solved
When adding 2 orgs with the same ID, you get a positive response from
the API, later when the org is projected, it errors due to the id
already in use
# How the Problems Are Solved
Check org with orgID specified does not already exist before adding
events
# Additional Changes
Added additional test case for adding same org with same name twice
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/10127
---------
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
Enhancing integration test for creating org; currently the test does not
check if the created org has the assigned custom id, this will resolve
this issue.
# Which Problems Are Solved
When an organization domain is verified, e.g. also when creating a new
organization (incl. generated domain), existing usernames are checked if
the domain has been claimed.
The query was not optimized for instances with many users and
organizations.
# How the Problems Are Solved
- Replace the query, which was searching over the users projection with
(computed loginnames) with a dedicated query checking the loginnames
projection directly.
- All occurrences have been updated to use the new query.
# Additional Changes
None
# Additional Context
- reported through support
- requires backport to v3.x
# Which Problems Are Solved
The current maintained gRPC server in combination with a REST (grpc)
gateway is getting harder and harder to maintain. Additionally, there
have been and still are issues with supporting / displaying `oneOf`s
correctly.
We therefore decided to exchange the server implementation to
connectRPC, which apart from supporting connect as protocol, also also
"standard" gRCP clients as well as HTTP/1.1 / rest like clients, e.g.
curl directly call the server without any additional gateway.
# How the Problems Are Solved
- All v2 services are moved to connectRPC implementation. (v1 services
are still served as pure grpc servers)
- All gRPC server interceptors were migrated / copied to a corresponding
connectRPC interceptor.
- API.ListGrpcServices and API. ListGrpcMethods were changed to include
the connect services and endpoints.
- gRPC server reflection was changed to a `StaticReflector` using the
`ListGrpcServices` list.
- The `grpc.Server` interfaces was split into different combinations to
be able to handle the different cases (grpc server and prefixed gateway,
connect server with grpc gateway, connect server only, ...)
- Docs of services serving connectRPC only with no additional gateway
(instance, webkey, project, app, org v2 beta) are changed to expose that
- since the plugin is not yet available on buf, we download it using
`postinstall` hook of the docs
# Additional Changes
- WebKey service is added as v2 service (in addition to the current
v2beta)
# Additional Context
closes#9483
---------
Co-authored-by: Elio Bischof <[email protected]>
# Which Problems Are Solved
- It is not possible to specify a custom organization ID when creating
an organization. According to
https://github.com/zitadel/zitadel/discussions/9202#discussioncomment-11929464
this is "an inconsistency in the V2 API".
# How the Problems Are Solved
- Adds the `org_id` as an optional parameter to the
`AddOrganizationRequest` in the `v2beta` API.
# Additional Changes
None.
# Additional Context
- Discussion
[#9202](https://github.com/zitadel/zitadel/discussions/9202)
- I was mostly interested in how much work it'd be to add this field.
Then after completing this, I thought I'd submit this PR. I won't be
angry if you just close this PR with the reasoning "we didn't ask for
it". 😄
- Even though I don't think this is a breaking change, I didn't add this
to the `v2` API yet (don't know what the process for this is TBH). The
changes should be analogous, so if you want me to, just request it.
---------
Co-authored-by: Stefan Benz <[email protected]>
Improve integration tests:
- spliting the tests in TokenExchange to isolated instances and in
parallel
- corrected some test structure so that the check for Details is no done
anymore if the test already failed
- replace required-calls with assert-calls to not stop the testing
- add gofakeit for application, project and usernames(emails)
- add eventually checks for testing in actions v2, so the request only
get called when the execution is defined
- check for length of results in list/search endpoints to avoid index
errors
# Which Problems Are Solved
Use a single server instance for API integration tests. This optimizes
the time taken for the integration test pipeline,
because it allows running tests on multiple packages in parallel. Also,
it saves time by not start and stopping a zitadel server for every
package.
# How the Problems Are Solved
- Build a binary with `go build -race -cover ....`
- Integration tests only construct clients. The server remains running
in the background.
- The integration package and tested packages now fully utilize the API.
No more direct database access trough `query` and `command` packages.
- Use Makefile recipes to setup, start and stop the server in the
background.
- The binary has the race detector enabled
- Init and setup jobs are configured to halt immediately on race
condition
- Because the server runs in the background, races are only logged. When
the server is stopped and race logs exist, the Makefile recipe will
throw an error and print the logs.
- Makefile recipes include logic to print logs and convert coverage
reports after the server is stopped.
- Some tests need a downstream HTTP server to make requests, like quota
and milestones. A new `integration/sink` package creates an HTTP server
and uses websockets to forward HTTP request back to the test packages.
The package API uses Go channels for abstraction and easy usage.
# Additional Changes
- Integration test files already used the `//go:build integration`
directive. In order to properly split integration from unit tests,
integration test files need to be in a `integration_test` subdirectory
of their package.
- `UseIsolatedInstance` used to overwrite the `Tester.Client` for each
instance. Now a `Instance` object is returned with a gRPC client that is
connected to the isolated instance's hostname.
- The `Tester` type is now `Instance`. The object is created for the
first instance, used by default in any test. Isolated instances are also
`Instance` objects and therefore benefit from the same methods and
values. The first instance and any other us capable of creating an
isolated instance over the system API.
- All test packages run in an Isolated instance by calling
`NewInstance()`
- Individual tests that use an isolated instance use `t.Parallel()`
# Additional Context
- Closes#6684
- https://go.dev/doc/articles/race_detector
- https://go.dev/doc/build-cover
---------
Co-authored-by: Stefan Benz <[email protected]>
# Which Problems Are Solved
The v2beta services are stable but not GA.
# How the Problems Are Solved
The v2beta services are copied to v2. The corresponding v1 and v2beta
services are deprecated.
# Additional Context
Closes#7236
---------
Co-authored-by: Elio Bischof <[email protected]>
