# Which Problems Are Solved
When a user changes their password, Zitadel needs to terminate all of
that user's active sessions. This query can take many seconds on
deployments with large session and user tables. This happens as part of
session projection handling, so doesn't directly impact user experience,
but potentially bogs down the projection handler which isn't great. In
the future, this index could be used to power a "see all of my current
sessions" feature in Zitadel.
# How the Problems Are Solved
Adds new index on `user_id` column on `projections.sessions8` table.
Alternatively, we can index on `(instance_id, user_id)` instead but
opted for keeping the index smaller as we already index on `instance_id`
separately.
# Additional Changes
None
# Additional Context
None
---------
Co-authored-by: Silvan <[email protected]>
# Which Problems Are Solved
With current provided telemetry it's difficult to predict when a
projection handler is under increased load until it's too late and
causes downstream issues. Importantly, projection updating is in the
critical path for many login flows and increased latency there can
result in system downtime for users.
# How the Problems Are Solved
This PR adds three new prometheus-style metrics:
1. **projection_events_processed** (_labels: projection, success_) -
This metric gives us a counter of the number of events processed per
projection update run and whether they we're processed without error. A
high number of events being processed can let us know how busy a
particular projection handler is.
2. **projection_handle_timer** _(labels: projection)_ - This is the time
it takes to process a projection update given a batch of events - time
to take the current_states lock, query for new events, reduce,
update_the projection, and update current_states.
3. **projection_state_latency** _(labels: projection)_ - This is the
time from the last event processed in the current_states table for a
given projection. It tells us how old was the last event you processed?
Or, how far behind are you running for this projection? Higher latencies
could mean high load or stalled projection handling.
# Additional Changes
I also had to initialize the global otel metrics provider (`metrics.M`)
in the `setup` step additionally to `start` since projection handlers
are initialized at setup. The initialization checks if a metrics
provider is already set (in case of `start-from-setup` or
`start-from-init` to prevent overwriting, which causes the otel metrics
provider to stop working.
# Additional Context
## Example Dashboards
---------
Co-authored-by: Silvan <[email protected]>
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
The subquery of the notification requested and retry requested is
missing the aggregate_type filter that would allow it to utilize the
`es_projection` or `active_instances_events` on the eventstore.events2
table.
# How the Problems Are Solved
Add additional filter on subquery. Final query:
```sql
SELECT <all the fields omitted> FROM eventstore.events2
WHERE
instance_id = $1
AND aggregate_type = $2
AND event_type = $3
AND created_at > $4
AND aggregate_id NOT IN (
SELECT aggregate_id
FROM eventstore.events2
WHERE
aggregate_type = $5 <-- NB: previously missing
AND event_type = ANY ($6)
AND instance_id = $7
AND created_at > $8
)
ORDER BY "position", in_tx_order
LIMIT $9
FOR UPDATE SKIP LOCKED
```
# Additional Changes
# Additional Context
Co-authored-by: Livio Spring <[email protected]>
# Which Problems Are Solved
- ImportHuman was not checking for a `UserStateDeleted` state on import,
resulting in "already existing" errors when attempting to delete and
re-import a user with the same id
# How the Problems Are Solved
Use the `Exists` helper method to check for both `UserStateUnspecified`
and `UserStateDeleted` states on import
# Additional Changes
N/A
# Additional Context
N/A
Co-authored-by: Livio Spring <[email protected]>
