perf(actionsv2): execution target router (#10564)

# Which Problems Are Solved

The event execution system currently uses a projection handler that
subscribes to and processes all events for all instances. This creates a
high static cost because the system over-fetches event data, handling
many events that are not needed by most instances. This inefficiency is
also reflected in high "rows returned" metrics in the database.

# How the Problems Are Solved

Eliminate the use of a project handler. Instead, events for which
"execution targets" are defined, are directly pushed to the queue by the
eventstore. A Router is populated in the Instance object in the authz
middleware.

- By joining the execution targets to the instance, no additional
queries are needed anymore.
- As part of the instance object, execution targets are now cached as
well.
- Events are queued within the same transaction, giving transactional
guarantees on delivery.
- Uses the "insert many fast` variant of River. Multiple jobs are queued
in a single round-trip to the database.
- Fix compatibility with PostgreSQL 15

# Additional Changes

- The signing key was stored as plain-text in the river job payload in
the DB. This violated our [Secrets
Storage](https://zitadel.com/docs/concepts/architecture/secrets#secrets-storage)
principle. This change removed the field and only uses the encrypted
version of the signing key.
- Fixed the target ordering from descending to ascending.
- Some minor linter warnings on the use of `io.WriteString()`.

# Additional Context

- Introduced in https://github.com/zitadel/zitadel/pull/9249
- Closes https://github.com/zitadel/zitadel/issues/10553
- Closes https://github.com/zitadel/zitadel/issues/9832
- Closes https://github.com/zitadel/zitadel/issues/10372
- Closes https://github.com/zitadel/zitadel/issues/10492

---------

Co-authored-by: Stefan Benz <[email protected]>
This commit is contained in:
Tim Möhlmann
2025-09-01 07:21:10 +02:00
committed by GitHub
co-authored by Stefan Benz
parent 5721b63bcb
commit a9ebc06c77
76 changed files with 1316 additions and 1815 deletions
+10 -10
View File
@@ -65,9 +65,10 @@ require (
github.com/pquerna/otp v1.5.0
github.com/rakyll/statik v0.1.7
github.com/redis/go-redis/v9 v9.8.0
github.com/riverqueue/river v0.22.0
github.com/riverqueue/river/riverdriver v0.22.0
github.com/riverqueue/river/rivertype v0.22.0
github.com/riverqueue/river v0.24.0
github.com/riverqueue/river/riverdriver v0.24.0
github.com/riverqueue/river/riverdriver/riverdatabasesql v0.24.0
github.com/riverqueue/river/rivertype v0.24.0
github.com/riverqueue/rivercontrib/otelriver v0.5.0
github.com/robfig/cron/v3 v3.0.1
github.com/rs/cors v1.11.1
@@ -77,7 +78,7 @@ require (
github.com/sony/sonyflake v1.2.1
github.com/spf13/cobra v1.9.1
github.com/spf13/viper v1.20.1
github.com/stretchr/testify v1.10.0
github.com/stretchr/testify v1.11.1
github.com/ttacon/libphonenumber v1.2.1
github.com/twilio/twilio-go v1.26.1
github.com/zitadel/exifremove v0.1.0
@@ -97,12 +98,12 @@ require (
go.opentelemetry.io/otel/sdk/metric v1.35.0
go.opentelemetry.io/otel/trace v1.35.0
go.uber.org/mock v0.5.2
golang.org/x/crypto v0.38.0
golang.org/x/crypto v0.41.0
golang.org/x/exp v0.0.0-20250506013437-ce4c2cf36ca6
golang.org/x/net v0.40.0
golang.org/x/net v0.42.0
golang.org/x/oauth2 v0.30.0
golang.org/x/sync v0.16.0
golang.org/x/text v0.27.0
golang.org/x/text v0.28.0
google.golang.org/api v0.233.0
google.golang.org/genproto/googleapis/api v0.0.0-20250512202823-5a2f75b736a9
google.golang.org/grpc v1.72.1
@@ -150,7 +151,7 @@ require (
github.com/pelletier/go-toml/v2 v2.2.3 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
github.com/riverqueue/river/rivershared v0.22.0 // indirect
github.com/riverqueue/river/rivershared v0.24.0 // indirect
github.com/sagikazarmark/locafero v0.7.0 // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/spiffe/go-spiffe/v2 v2.5.0 // indirect
@@ -225,7 +226,6 @@ require (
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.62.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/riverqueue/river/riverdriver/riverpgxv5 v0.22.0
github.com/rs/xid v1.6.0 // indirect
github.com/russellhaering/goxmldsig v1.4.0 // indirect
github.com/sirupsen/logrus v1.9.3
@@ -238,7 +238,7 @@ require (
github.com/xrash/smetrics v0.0.0-20240312152122-5f08fbb34913 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.35.0 // indirect
go.opentelemetry.io/proto/otlp v1.5.0 // indirect
golang.org/x/sys v0.33.0
golang.org/x/sys v0.35.0
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
nhooyr.io/websocket v1.8.11 // indirect