# Which Problems Are Solved
There are cases where not all statements of multiExec are succeed. This
leads to inconsistent states. One example is [LDAP
IDPs](https://github.com/zitadel/zitadel/issues/7959).
If statements get executed only partially this can lead to inconsistent
states or even break projections for objects which might not were
correctly created in a sub table.
This behaviour is possible because we use
[`SAVEPOINTS`](https://www.postgresql.org/docs/current/sql-savepoint.html)
during each statement of a multiExec.
# How the Problems Are Solved
SAVEPOINTS are only created at the beginning of an exec function not
during every execution like before. Additionally `RELEASE` or `ROLLBACK`
of `SAVEPOINTS` are only used when needed.
# Additional Changes
- refactor some unused parameters
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/7959
# Which Problems Are Solved
Float64 which was used for the event.Position field is [not precise in
go and gets rounded](https://github.com/golang/go/issues/47300). This
can lead to unprecies position tracking of events and therefore
projections especially on cockcoachdb as the position used there is a
big number.
example of a unprecies position:
exact: 1725257931223002628
float64: 1725257931223002624.000000
# How the Problems Are Solved
The float64 was replaced by
[github.com/jackc/pgx-shopspring-decimal](https://github.com/jackc/pgx-shopspring-decimal).
# Additional Changes
Correct behaviour of makefile for load tests.
Rename `latestSequence`-queries to `latestPosition`
# Which Problems Are Solved
Currently there was no load test present for machine jwt profile grant.
This test is now added
# How the Problems Are Solved
K6 test implemented.
# Additional Context
- part of https://github.com/zitadel/zitadel/issues/8352
# Which Problems Are Solved
The mirror command used the wrong position to filter for events if
different database technologies for source and destination were used.
# How the Problems Are Solved
The statements which diverge are stored on the client so that different
technologies can use different statements.
# Additional Context
- https://discord.com/channels/927474939156643850/1256396896243552347
# Which Problems Are Solved
If the processing time of serializable transactions in the fields
handler take too long, the next iteration can fail.
# How the Problems Are Solved
Changed the isolation level of the current states query to Read Commited
# Which Problems Are Solved
The e2e tests fail because the organization selection is too fast.
# How the Problems Are Solved
Wait until console has loaded properly.
# Additional Context
- The tests still use the wrong browser, #8404 describes the problem
- closes https://github.com/zitadel/zitadel/issues/8378
# Which Problems Are Solved
During triggering of the fields table WriteTooOld errors can occure when
using cockroachdb.
# How the Problems Are Solved
The statements exclusively lock the projection before they start to
insert data by using `FOR UPDATE`.
# Which Problems Are Solved
The mirror command read the configurations in the wrong order
# How the Problems Are Solved
The Pre execution run of `mirror` reads the default config first and
then applies the custom configs
# Which Problems Are Solved
The connection pool of go uses a high amount of database connections.
# How the Problems Are Solved
The standard lib connection pool was replaced by `pgxpool.Pool`
# Additional Changes
The `db.BeginTx`-spans are removed because they cause to much noise in
the traces.
# Additional Context
- part of https://github.com/zitadel/zitadel/issues/7639
# Which Problems Are Solved
Bigger systems need to process many events during the initialisation
phase of the `eventstore.fields`-table. During setup these calls can
time out.
# How the Problems Are Solved
Changed the default behaviour of these projections to not time out and
increased the bulk limit.
# Which Problems Are Solved
ZITADEL returned a 404 Unimplemented error if the client sent
'application/grpc+proto' or 'application/grpc+json' which are both valid
content types.
# How the Problems Are Solved
changed the header matcher to regexp
# Additional Context
Problem occured in
https://github.com/zitadel/typescript/tree/grpc-transport
# Which Problems Are Solved
Extends load tests by testing session creation.
# How the Problems Are Solved
The test creates a session including a check for user id.
# Additional Context
- part of https://github.com/zitadel/zitadel/issues/7639
# Which Problems Are Solved
During performance testing of the `eventstore.fields` table we found
some long running queries which searched for the aggregate id.
# How the Problems Are Solved
A new index was added to the `eventstore.fields`-table called
`f_aggregate_object_type_idx`.
# Additional Changes
None
# Additional Context
- Table was added in https://github.com/zitadel/zitadel/pull/8191
- Part of https://github.com/zitadel/zitadel/issues/7639
# Which Problems Are Solved
Fixes a panic which can occur if there are no events to reduce in the fields handler
# How the Problems Are Solved
Check if there are any events to reduce
# Additional Context
- Panic was added in https://github.com/zitadel/zitadel/pull/8191
# Which Problems Are Solved
To improve performance a new table and method is implemented on
eventstore. The goal of this table is to index searchable fields on
command side to use it on command and query side.
The table allows to store one primitive value (numeric, text) per row.
The eventstore framework is extended by the `Search`-method which allows
to search for objects.
The `Command`-interface is extended by the `SearchOperations()`-method
which does manipulate the the `search`-table.
# How the Problems Are Solved
This PR adds the capability of improving performance for command and
query side by using the `Search`-method of the eventstore instead of
using one of the `Filter`-methods.
# Open Tasks
- [x] Add feature flag
- [x] Unit tests
- [ ] ~~Benchmarks if needed~~
- [x] Ensure no behavior change
- [x] Add setup step to fill table with current data
- [x] Add projection which ensures data added between setup and start of
the new version are also added to the table
# Additional Changes
The `Search`-method is currently used by `ProjectGrant`-command side.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/8094
# Which Problems Are Solved
Describes how to develop ZITADEL using dev containers.
# Additional Changes
Sets default env variables for using postgres as database in the dev
container.
# Which Problems Are Solved
Postgres versions < 16 require an alias for subqueries. The query
executed in the new eventstore didn't add this alias.
# How the Problems Are Solved
Added the alias to the subquery
# Which Problems Are Solved
Corrects reading of default configuration, despite reading all default
configs only required defaults are read.
# How the Problems Are Solved
Reading the defualt config of the `mirror`-command was moved to a
pre-run step of the command instead of doing it during initialization of
the executable.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/8059
- https://discord.com/channels/927474939156643850/1248594307267559535
# Which Problems Are Solved
Adds the possibility to mirror an existing database to a new one.
For that a new command was added `zitadel mirror`. Including it's
subcommands for a more fine grained mirror of the data.
Sub commands:
* `zitadel mirror eventstore`: copies only events and their unique
constraints
* `zitadel mirror system`: mirrors the data of the `system`-schema
* `zitadel mirror projections`: runs all projections
* `zitadel mirror auth`: copies auth requests
* `zitadel mirror verify`: counts the amount of rows in the source and
destination database and prints the diff.
The command requires one of the following flags:
* `--system`: copies all instances of the system
* `--instance <instance-id>`, `--instance <comma separated list of
instance ids>`: copies only the defined instances
The command is save to execute multiple times by adding the
`--replace`-flag. This replaces currently existing data except of the
`events`-table
# Additional Changes
A `--for-mirror`-flag was added to `zitadel setup` to prepare the new
database. The flag skips the creation of the first instances and initial
run of projections.
It is now possible to skip the creation of the first instance during
setup by setting `FirstInstance.Skip` to true in the steps
configuration.
# Additional info
It is currently not possible to merge multiple databases. See
https://github.com/zitadel/zitadel/issues/7964 for more details.
It is currently not possible to use files. See
https://github.com/zitadel/zitadel/issues/7966 for more information.
closes https://github.com/zitadel/zitadel/issues/7586
closes https://github.com/zitadel/zitadel/issues/7486
### Definition of Ready
- [x] I am happy with the code
- [x] Short description of the feature/issue is added in the pr
description
- [x] PR is linked to the corresponding user story
- [x] Acceptance criteria are met
- [x] All open todos and follow ups are defined in a new ticket and
justified
- [x] Deviations from the acceptance criteria and design are agreed with
the PO and documented.
- [x] No debug or dead code
- [x] My code has no repetitions
- [x] Critical parts are tested automatically
- [ ] Where possible E2E tests are implemented
- [x] Documentation/examples are up-to-date
- [x] All non-functional requirements are met
- [x] Functionality of the acceptance criteria is checked manually on
the dev system.
---------
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Queriying events by an aggregate id can produce high loads on the
database if the aggregate id contains many events (count > 1000000).
# How the Problems Are Solved
Instead of using the postion and in_tx_order columns we use the sequence
column which guarantees correct ordering in a single aggregate and uses
more optimised indexes.
# Additional Context
Closes https://github.com/zitadel/DevOps/issues/50
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
Curretly loading targets fails on cockraochdb because the order of
`with`-statements is wrong.
# How the Problems Are Solved
Changed the order of queries in the statements.
If the feature is enabled the new packages are used to query org by id
Part of: https://github.com/zitadel/zitadel/issues/7639
### Definition of Ready
- [x] I am happy with the code
- [x] Short description of the feature/issue is added in the pr
description
- [x] PR is linked to the corresponding user story
- [ ] Acceptance criteria are met
- [ ] All open todos and follow ups are defined in a new ticket and
justified
- [ ] Deviations from the acceptance criteria and design are agreed with
the PO and documented.
- [x] No debug or dead code
- [x] My code has no repetitions
- [ ] Critical parts are tested automatically
- [ ] Where possible E2E tests are implemented
- [ ] Documentation/examples are up-to-date
- [ ] All non-functional requirements are met
- [x] Functionality of the acceptance criteria is checked manually on
the dev system.
* chore: use pgx v5
* chore: update go version
* remove direct pq dependency
* remove unnecessary type
* scan test
* map scanner
* converter
* uint8 number array
* duration
* most unit tests work
* unit tests work
* chore: coverage
* go 1.21
* linting
* int64 gopfertammi
* retry go 1.22
* retry go 1.22
* revert to go v1.21.5
* update go toolchain to 1.21.8
* go 1.21.8
* remove test flag
* go 1.21.5
* linting
* update toolchain
* use correct array
* use correct array
* add byte array
* correct value
* correct error message
* go 1.21 compatible
* docs(api): describe which flow and trigger types word together
* docs(actions): describe which flow and trigger types work together
* Update management.proto
---------
Co-authored-by: Livio Spring <[email protected]>
Even though this is a feature it's released as fix so that we can back port to earlier revisions.
As reported by multiple users startup of ZITADEL after leaded to downtime and worst case rollbacks to the previously deployed version.
The problem starts rising when there are too many events to process after the start of ZITADEL. The root cause are changes on projections (database tables) which must be recomputed. This PR solves this problem by adding a new step to the setup phase which prefills the projections. The step can be enabled by adding the `--init-projections`-flag to `setup`, `start-from-init` and `start-from-setup`. Setting this flag results in potentially longer duration of the setup phase but reduces the risk of the problems mentioned in the paragraph above.
* fix(setup): unmarshal of failed step
* fix(cleanup): cleanup all stuck states
* use lastRun for repeatable steps
* typo
---------
Co-authored-by: Livio Spring <[email protected]>
* start user by id
* ignore debug bin
* use new user by id
* new sql
* fix(sql): replace STRING with text for psql compatabilit
* some changes
* fix: correct user queries
* fix tests
* unify sql statements
* use specific get user methods
* search login name case insensitive
* refactor: optimise user statements
* add index
* fix queries
* fix: correct domain segregation
* return all login names
* fix queries
* improve readability
* query should be correct now
* cleanup statements
* fix username / loginname handling
* fix: psql doesn't support create view if not exists
* fix: create pre-release
* ignore release comments
* add lower fields
* fix: always to lower
* update to latest projection
---------
Co-authored-by: Livio Spring <[email protected]>
* fix(eventstore): differentiate unique constraint error format
* docs: add comment to eventstore vars
* fix(eventstore): return correct error type if unique constraint already exists
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and [06](https://zitadel.com/docs/support/advisory/a10006).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
SilvanandGitHub
