mirror of
https://github.com/andwati/zitadel.git
synced 2026-10-28 18:12:16 +00:00
docs: introduce versioned docs and migrate to fuma (#11166)
## Todos for release - [x] Configure Env in docs project on vercel - [x] Configure Root Path in the docs project on vercel - [ ] Remove old CSP https://github.com/zitadel/website/pull/1592 ## What we did This pull request migrates the project documentation from the old `docs/` directory to the new `apps/docs/` directory, introduces a new documentation system built with Next.js and Fumadocs, and updates all relevant references, configuration files, and documentation to reflect this change. It also adds new configuration and ignore files for the new documentation app, updates CI and linting to exclude the new docs from certain checks, and revises the contributing guidelines accordingly. **Documentation System Migration and New Docs App** * Migrated all documentation from `docs/` to `apps/docs/`, and updated all references in `README.md`, `CONTRIBUTING.md`, and other files to point to the new location. [[1]](diffhunk://#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055L585-L640) [[2]](diffhunk://#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055L660-R607) [[3]](diffhunk://#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055L740-R687) [[4]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L2-R3) [[5]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L30-R30) * Added a new Next.js/Fumadocs-based documentation app under `apps/docs/`, including core app files, layouts, routing, search API, and a comprehensive `README.md` with development and contribution instructions. [[1]](diffhunk://#diff-5a1b07344a2c1b4d3f37b23ff1388b62cd9f57dea3c1cd23d8a0412b7602b132R1-R74) [[2]](diffhunk://#diff-462b9ad1eabbb7d1c29bb9c36e4931eb180fd7190900e6d2babf8f4d66ad1c28R1-R39) [[3]](diffhunk://#diff-e16ae25660ded787b10ac35dea96d5ecaacf895dae0afc8a9bd4382dc79a8c87R1-R7) [apps/docs/app/[[...slug]]/layout.tsxR1-R81](diffhunk://#diff-59e08acde4e805b7aeccef1dcf98f1d71dfc550777e6b9402085cee0e9fa4e0aR1-R81), [apps/docs/app/[[...slug]]/page.tsxR1-R79](diffhunk://#diff-e5df3f80d0fa01e12d63d81f29c57a9d14e846c78fd3beabb2ef768e38fd9580R1-R79), [[4]](diffhunk://#diff-389b34918e040cacaa87cd7201ffa462cc2b0b716736f537e3d3c660ac69353bR1-R7) [[5]](diffhunk://#diff-d3b03416d1c457b19f1c27b26f2db741412df841b875c26ccadc36e4522247f4R1-R29) [[6]](diffhunk://#diff-c8fb8339570a5305809be7c618e14705fd86390dc278e6ce8ba224a7bc8b0c3cR1-R25) **Configuration and Tooling Updates** * Updated `.github/workflows/codeql.yml`, `.golangci.yaml`, and `.github/dependabot.yml` to properly handle the new docs app: excluded `apps/docs` from certain checks, added npm dependency updates for the docs app, and excluded generated content. [[1]](diffhunk://#diff-12783128521e452af0cfac94b99b8d250413c516ec71fe6d97dbea666ff7ba27L8-R14) [[2]](diffhunk://#diff-9917ddc9f1c3304218f7269265b746d997c5c0615478177b5fceecd33ef47cb5R5-R6) [[3]](diffhunk://#diff-9917ddc9f1c3304218f7269265b746d997c5c0615478177b5fceecd33ef47cb5R126-R129) [[4]](diffhunk://#diff-dd4fbda47e51f1e35defb9275a9cd9c212ecde0b870cba89ddaaae65c5f3cd28R89-R106) * Updated `.devcontainer/devcontainer.json` to use the latest Go 1.25.3 version for consistency. **Licensing and Miscellaneous** * Added `apps/docs/` to the list of licensed directories in `LICENSING.md`. These changes ensure the documentation is now maintained in a modern, scalable system and all project tooling is updated to support the new structure. --------- Co-authored-by: Federico Coppede <[email protected]>
This commit is contained in:
Florian Forster
co-authored by
Federico Coppede
Federico Coppede
parent
7b6395c348
commit
d8ea041c56
@@ -0,0 +1,43 @@
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
TODO: describe the outcome of the test?
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|-:-------------------------------------|-:-----|
|
||||
| Baseline | none |
|
||||
| Purpose | |
|
||||
| Test start | UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | |
|
||||
| k6 version | |
|
||||
| VUs | |
|
||||
| Client location | |
|
||||
| Client machine specification | vCPU: <br/> memory: Gb |
|
||||
| ZITADEL location | |
|
||||
| ZITADEL container specification | vCPU: <br/> Memory: Gb <br/>Container count: |
|
||||
| ZITADEL Version | |
|
||||
| ZITADEL Configuration | |
|
||||
| ZITADEL feature flags | |
|
||||
| Database | type: psql<br />version: |
|
||||
| Database location | |
|
||||
| Database specification | vCPU: <br/> memory: Gb |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | |
|
||||
| Top 3 most expensive database queries | |
|
||||
| Database metrics during test | |
|
||||
| k6 Iterations per second | |
|
||||
| k6 overview | |
|
||||
| k6 output | |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
TODO: add summary of k6
|
||||
```
|
||||
@@ -0,0 +1,107 @@
|
||||
---
|
||||
title: Benchmarks
|
||||
sidebar_label: Benchmarks
|
||||
---
|
||||
|
||||
import DocCardList from '@/components/docusaurus/doc-card-list';
|
||||
|
||||
Benchmarks are crucial to understand if ZITADEL fulfills your expected workload and what resources it needs to do so.
|
||||
|
||||
This document explains the process and goals of load-testing zitadel in a cloud environment.
|
||||
|
||||
The results can be found on sub pages.
|
||||
|
||||
## Goals
|
||||
|
||||
The primary goal is to assess if ZITADEL can scale to required proportion. The goals might change over time and maturity of ZITADEL. At the moment the goal is to assess how the application’s performance scales. There are some concrete goals we have to meet:
|
||||
|
||||
1. [https://github.com/zitadel/zitadel/issues/8352](https://github.com/zitadel/zitadel/issues/8352) defines 1000 JWT profile auth/sec
|
||||
2. [https://github.com/zitadel/zitadel/issues/4424](https://github.com/zitadel/zitadel/issues/4424) defines 1200 logins / sec.
|
||||
|
||||
## Procedure
|
||||
|
||||
First we determine the “target” of our load-test. The target is expressed as a make recipe in the load-test [Makefile](https://github.com/zitadel/zitadel/blob/main/load-test/Makefile). See also the load-test [readme](https://github.com/zitadel/zitadel/blob/main/load-test/README.md) on how to configure and run load-tests.
|
||||
A target should be tested for longer periods of time, as it might take time for certain metrics to show up. For example, cloud SQL samples query insights. A runtime of at least **30 minutes** is advised at the moment.
|
||||
|
||||
After each iteration of load-test, we should consult the [After test procedure](#after-test-procedure) to conclude an outcome:
|
||||
|
||||
1. Scale
|
||||
2. Log potential issuer and scale
|
||||
3. Terminate testing and resolve issues
|
||||
|
||||
## Methodology
|
||||
|
||||
### Benchmark definition
|
||||
|
||||
Tests are implemented in the ecosystem of [k6](https://k6.io). The tests are publicly available in the [zitadel repository](https://github.com/zitadel/zitadel/tree/main/load-test). Custom extensions of k6 are implemented in the [xk6-modules repository](https://github.com/zitadel/xk6-modules).
|
||||
The tests must at least measure the request duration for each API call. This gives an indication on how zitadel behaves over the duration of the load test.
|
||||
|
||||
### Metrics
|
||||
|
||||
The following metrics must be collected for each test iteration. The metrics are used to follow the decision path of the [After test procedure](https://drive.google.com/open?id=1WVr7aA8dGgV1zd2jUg1y1h_o37mkZF2O6M5Mhafn_NM):
|
||||
|
||||
| Metric | Type | Description | Unit |
|
||||
| :---- | :---- | :---- | :---- |
|
||||
| Baseline | Comparison | Defines the baseline the test is compared against. If not specified the baseline defined in this document is used. | Link to test result |
|
||||
| Purpose | Description | Description what should been proved with this test run | text
|
||||
| Test start | Setup | Timestamp when the test started. This is useful for gathering additional data like metrics or logs later | Date |
|
||||
| Test duration | Setup | Duration of the test | Duration |
|
||||
| Executed test | Setup | Name of the make recipe executed. Further information about specific test cases can be found [here](?tab=t.0#heading=h.xav4f3s5r2f3). | Name of the make recipe |
|
||||
| k6 version | Setup | Version of the test client (k6) used | semantic version |
|
||||
| VUs | Setup | Virtual Users which execute the test scenario in parallel | Number |
|
||||
| Client location | Setup | Region or location of the machine which executed the test client. If not further specified the hoster is Google Cloud | Location / Region |
|
||||
| Client machine specification | Setup | Definition of the client machine the test client ran on. The resources of the machine could be maxed out during tests therefore we collect this metric as well. The description must at least clarify the following metrics: vCPU Memory egress bandwidth | **vCPU**: Amount of threads ([additional info](https://cloud.google.com/compute/docs/cpu-platforms)) **memory**: GB **egress bandwidth**:Gbps |
|
||||
| ZITADEL location | Setup | Region or location of the deployment of zitadel. If not further specified the hoster is Google Cloud | Location / Region |
|
||||
| ZITADEL container specification | Setup | As ZITADEL is mainly run in cloud environments it should also be run as a container during the load tests. The description must at least clarify the following metrics: vCPU Memory egress bandwidth Scale | **vCPU**: Amount of threads ([additional info](https://cloud.google.com/compute/docs/cpu-platforms)) **memory**: GB **egress bandwidth**:Gbps **scale**: The amount of containers running during the test. The amount must not vary during the tests |
|
||||
| ZITADEL Version | Setup | The version of zitadel deployed | Semantic version or commit |
|
||||
| ZITADEL Configuration | Setup | Configuration of zitadel which deviates from the defaults and is not secret | yaml |
|
||||
| ZITADEL feature flags | Setup | Changed feature flags | yaml |
|
||||
| Database | Setup | Database type and version | **type**: psql **version**: semantic version |
|
||||
| Database location | Setup | Region or location of the deployment of the database. If not further specified the hoster is Google Cloud SQL | Location / Region |
|
||||
| Database specification | Setup | The description must at least clarify the following metrics: vCPU, Memory and egress bandwidth (Scale) | **vCPU**: Amount of threads ([additional info](https://cloud.google.com/compute/docs/cpu-platforms)) **memory**: GB **egress bandwidth**:Gbps |
|
||||
| ZITADEL metrics during test | Result | This metric helps understanding the bottlenecks of the executed test. At least the following metrics must be provided: CPU usage Memory usage | **CPU usage** in percent **Memory usage** in percent |
|
||||
| Observed errors | Result | Errors worth mentioning, mostly unexpected errors | description |
|
||||
| Top 3 most expensive database queries | Result | The execution plan of the top 3 most expensive database queries during the test execution | database execution plan |
|
||||
| Database metrics during test | Result | This metric helps understanding the bottlenecks of the executed test. At least the following metrics must be provided: CPU usage Memory usage | **CPU usage** in percent **Memory usage** in percent |
|
||||
| k6 Iterations per second | Result | How many test iterations were done per second | Number |
|
||||
| k6 overview | Result | Shows some basic metrics aggregated over the test run At least the following metrics must be included: duration per request (min, max, avg, p50, p95, p99) VUS For simplicity just add the whole test result printed to the terminal | terminal output |
|
||||
| k6 output | Result | Trends and metrics generated during the test, this contains detailed information for each step executed during each iteration | csv |
|
||||
|
||||
### Test setup
|
||||
|
||||
#### Make recipes
|
||||
|
||||
Details about the tests implemented can be found in [this readme](https://github.com/zitadel/zitadel/blob/main/load-test/README.md#test).
|
||||
|
||||
### Test conclusion
|
||||
|
||||
After each iteration of load-test, we should consult the [Flowchart](#after-test-procedure) to conclude an outcome:
|
||||
|
||||
1. [Scale](#scale)
|
||||
2. [Log potential issue and scale](#potential-issues)
|
||||
3. [Terminate testing](#termination) and resolve issues
|
||||
|
||||
#### Scale
|
||||
An outcome of scale means that the service hit some kind of resource limit, like CPU or RAM which can be increased. In such cases we increase the suggested parameter and rerun the load-test for the same target. On the next test we should analyze if the increase in scale resulted in a performance improvement proportional to the scale parameter. For example if we scale from 1 to 2 containers, it might be reasonable to expect a doubling of iterations / sec. If such an increase is not noticed, there might be another bottleneck or underlying issue, such as locking.
|
||||
|
||||
#### Potential issues
|
||||
A potential issue has an impact on performance, but does not prevent us to scale. Such issues must be logged in GH issues and load-testing can continue. The issue can be resolved at a later time and the load-tests repeated when it is. This is primarily for issues which require big changes to ZITADEL.
|
||||
|
||||
#### Termination
|
||||
Scaling no longer improves iterations / second, or some kind of critical error or bug is experienced. The root cause of the issue must be resolved before we can continue with increasing scale.
|
||||
|
||||
### After test procedure
|
||||
|
||||
This flowchart shows the procedure after running a test.
|
||||
|
||||
|
||||
|
||||
## Baseline
|
||||
|
||||
Will be established as soon as the goal described above is reached.
|
||||
|
||||
## Test results
|
||||
|
||||
This chapter provides a table linking to the detailed test results.
|
||||
|
||||
<DocCardList />
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
title: machine jwt profile grant benchmark of zitadel v2.65.0
|
||||
sidebar_label: machine jwt profile grant
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Tests are halted after this test run because of too many [client read events](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/wait-event.clientread.html) on the database.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
| :---- | :---- |
|
||||
| Baseline | none |
|
||||
| Test start | 22-10-2024 16:20 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | machine\_jwt\_profile\_grant |
|
||||
| k6 version | v0.54.0 |
|
||||
| VUs | 50 |
|
||||
| Client location | US1 |
|
||||
| Client machine specification | e2-high-cpu-4 |
|
||||
| Zitadel location | US1 |
|
||||
| Zitadel container specification | vCPUs: 2<br/>Memory: 512 MiB<br/>Container count: 2 |
|
||||
| Zitadel feature flags | none |
|
||||
| Database | postgres v15 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPUs: 4<br/>Memory: 16 GiB |
|
||||
| Zitadel metrics during test | |
|
||||
| Observed errors | Many client read events during push |
|
||||
| Top 3 most expensive database queries | 1: Query events `instance_id = $1 AND aggregate_type = $2 AND aggregate_id = $3 AND event_type = ANY($4)`<br/>2: latest sequence query during push events<br/>3: writing events during push (caused lock wait events) |
|
||||
| k6 iterations per second | 193 |
|
||||
| k6 overview | [output](#k6-output) |
|
||||
| flowchart outcome | Halt tests, must resolve an issue |
|
||||
|
||||
## /token endpoint latencies
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
checks...............................: 100.00% ✓ 695739 ✗ 0
|
||||
data_received........................: 479 MB 265 kB/s
|
||||
data_sent............................: 276 MB 153 kB/s
|
||||
http_req_blocked.....................: min=178ns avg=5µs max=119.8ms p(50)=460ns p(95)=702ns p(99)=921ns
|
||||
http_req_connecting..................: min=0s avg=1.24µs max=43.45ms p(50)=0s p(95)=0s p(99)=0s
|
||||
http_req_duration....................: min=18ms avg=255.3ms max=1.22s p(50)=241.56ms p(95)=479.19ms p(99)=600.92ms
|
||||
{ expected_response:true }.........: min=18ms avg=255.3ms max=1.22s p(50)=241.56ms p(95)=479.19ms p(99)=600.92ms
|
||||
http_req_failed......................: 0.00% ✓ 0 ✗ 347998
|
||||
http_req_receiving...................: min=25.92µs avg=536.96µs max=401.94ms p(50)=89.44µs p(95)=2.39ms p(99)=11.12ms
|
||||
http_req_sending.....................: min=24.01µs avg=63.86µs max=4.48ms p(50)=60.97µs p(95)=88.69µs p(99)=141.74µs
|
||||
http_req_tls_handshaking.............: min=0s avg=2.8µs max=51.05ms p(50)=0s p(95)=0s p(99)=0s
|
||||
http_req_waiting.....................: min=17.65ms avg=254.7ms max=1.22s p(50)=240.88ms p(95)=478.6ms p(99)=600.6ms
|
||||
http_reqs............................: 347998 192.80552/s
|
||||
iteration_duration...................: min=33.86ms avg=258.77ms max=1.22s p(50)=245ms p(95)=482.61ms p(99)=604.32ms
|
||||
iterations...........................: 347788 192.689171/s
|
||||
login_ui_enter_login_name_duration...: min=218.61ms avg=218.61ms max=218.61ms p(50)=218.61ms p(95)=218.61ms p(99)=218.61ms
|
||||
login_ui_enter_password_duration.....: min=18ms avg=18ms max=18ms p(50)=18ms p(95)=18ms p(99)=18ms
|
||||
login_ui_init_login_duration.........: min=90.96ms avg=90.96ms max=90.96ms p(50)=90.96ms p(95)=90.96ms p(99)=90.96ms
|
||||
login_ui_token_duration..............: min=140.02ms avg=140.02ms max=140.02ms p(50)=140.02ms p(95)=140.02ms p(99)=140.02ms
|
||||
oidc_token_duration..................: min=29.85ms avg=255.38ms max=1.22s p(50)=241.61ms p(95)=479.23ms p(99)=600.95ms
|
||||
org_create_org_duration..............: min=64.51ms avg=64.51ms max=64.51ms p(50)=64.51ms p(95)=64.51ms p(99)=64.51ms
|
||||
user_add_machine_key_duration........: min=44.93ms avg=87.89ms max=159.52ms p(50)=84.43ms p(95)=144.59ms p(99)=155.54ms
|
||||
user_create_machine_duration.........: min=65.75ms avg=266.53ms max=421.58ms p(50)=276.59ms p(95)=380.84ms p(99)=414.43ms
|
||||
vus..................................: 0 min=0 max=50
|
||||
vus_max..............................: 50 min=50 max=50
|
||||
|
||||
running (30m04.9s), 00/50 VUs, 347788 complete and 0 interrupted iterations
|
||||
default ✓ [======================================] 50 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,98 @@
|
||||
---
|
||||
title: machine jwt profile grant benchmark of zitadel v2.66.0
|
||||
sidebar_label: machine jwt profile grant
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
The tests showed heavy database load by time by the first two database queries. These queries need to be analyzed further.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 15:39 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | machine\_jwt\_profile\_grant |
|
||||
| k6 version | v0.54.0 |
|
||||
| VUs | 150 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 2<br/> Memory: 512 Mib <br/>Container count: 5 |
|
||||
| ZITADEL Version | v2.66.0 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v15.8 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | |
|
||||
| Top 3 most expensive database queries | 1: Write events using the newly added eventstore.push function<br/>2: Query events by instance\_id, aggregate\_type, aggregate\_id, event\_types<br/>3: Query user<br/> |
|
||||
| k6 Iterations per second | 439 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | Scale out |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
✓ openid configuration
|
||||
✗ token status ok
|
||||
↳ 99% — ✓ 790655 / ✗ 5
|
||||
✗ access token returned
|
||||
↳ 99% — ✓ 790655 / ✗ 5
|
||||
|
||||
█ setup
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✓ generate machine key status ok
|
||||
|
||||
█ teardown
|
||||
|
||||
✓ org removed
|
||||
|
||||
checks...............................: 99.99% ✓ 1581773 ✗ 10
|
||||
data_received........................: 1.1 GB 623 kB/s
|
||||
data_sent............................: 628 MB 347 kB/s
|
||||
http_req_blocked.....................: min=167ns avg=20.68µs max=493.59ms p(50)=468ns p(95)=717ns p(99)=928ns
|
||||
http_req_connecting..................: min=0s avg=10.06µs max=388.27ms p(50)=0s p(95)=0s p(99)=0s
|
||||
http_req_duration....................: min=17.71ms avg=337.34ms max=16.27s p(50)=249.03ms p(95)=888.75ms p(99)=1.4s
|
||||
{ expected_response:true }.........: min=17.71ms avg=337.29ms max=3.56s p(50)=249.03ms p(95)=888.7ms p(99)=1.4s
|
||||
http_req_failed......................: 0.00% ✓ 5 ✗ 791265
|
||||
http_req_receiving...................: min=25.49µs avg=1.58ms max=539.43ms p(50)=89.69µs p(95)=7.55ms p(99)=23.46ms
|
||||
http_req_sending.....................: min=22.7µs avg=69.14µs max=480.23ms p(50)=59.16µs p(95)=85.15µs p(99)=129.88µs
|
||||
http_req_tls_handshaking.............: min=0s avg=9.38µs max=98.15ms p(50)=0s p(95)=0s p(99)=0s
|
||||
http_req_waiting.....................: min=15.11ms avg=335.69ms max=16.26s p(50)=246.91ms p(95)=888.27ms p(99)=1.4s
|
||||
http_reqs............................: 791270 437.256468/s
|
||||
iteration_duration...................: min=32.28ms avg=341.46ms max=16.27s p(50)=253ms p(95)=892.49ms p(99)=1.41s
|
||||
iterations...........................: 790660 436.919382/s
|
||||
login_ui_enter_login_name_duration...: min=179.27ms avg=179.27ms max=179.27ms p(50)=179.27ms p(95)=179.27ms p(99)=179.27ms
|
||||
login_ui_enter_password_duration.....: min=17.71ms avg=17.71ms max=17.71ms p(50)=17.71ms p(95)=17.71ms p(99)=17.71ms
|
||||
login_ui_init_login_duration.........: min=77.66ms avg=77.66ms max=77.66ms p(50)=77.66ms p(95)=77.66ms p(99)=77.66ms
|
||||
login_ui_token_duration..............: min=86.79ms avg=86.79ms max=86.79ms p(50)=86.79ms p(95)=86.79ms p(99)=86.79ms
|
||||
oidc_token_duration..................: min=28.38ms avg=337.54ms max=16.27s p(50)=249.17ms p(95)=889.01ms p(99)=1.4s
|
||||
org_create_org_duration..............: min=44.94ms avg=44.94ms max=44.94ms p(50)=44.94ms p(95)=44.94ms p(99)=44.94ms
|
||||
user_add_machine_key_duration........: min=38.11ms avg=66.64ms max=160.59ms p(50)=60.28ms p(95)=104.99ms p(99)=112.5ms
|
||||
user_create_machine_duration.........: min=37.12ms avg=122.76ms max=1.03s p(50)=78.25ms p(95)=266.95ms p(99)=306.94ms
|
||||
vus..................................: 150 min=0 max=150
|
||||
vus_max..............................: 150 min=150 max=150
|
||||
|
||||
running (30m09.6s), 000/150 VUs, 790660 complete and 0 interrupted iterations
|
||||
default ✓ [======================================] 150 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,98 @@
|
||||
---
|
||||
title: machine jwt profile grant benchmark of zitadel v2.70.0
|
||||
sidebar_label: machine jwt profile grant
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
The performance goals of [this issue](https://github.com/zitadel/zitadel/issues/8352) are reached. Next we will test linear scalability.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 09:48 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | machine\_jwt\_profile\_grant |
|
||||
| k6 version | v0.57.0 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v2.70.0 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.2 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | |
|
||||
| Top 3 most expensive database queries | 1: Write events using eventstore.push function<br/>2: Query user<br/>3: Query events by instance id, aggregate type, aggregate id, event types, owner<br/> |
|
||||
| k6 Iterations per second | 1806 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | Scale out |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
✓ openid configuration
|
||||
✗ token status ok
|
||||
↳ 99% — ✓ 3289559 / ✗ 5
|
||||
✗ access token returned
|
||||
↳ 99% — ✓ 3289559 / ✗ 5
|
||||
|
||||
█ setup
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✓ generate machine key status ok
|
||||
|
||||
█ teardown
|
||||
|
||||
✓ org removed
|
||||
|
||||
checks...............................: 99.99% 6580931 out of 6580941
|
||||
data_received........................: 4.8 GB 2.6 MB/s
|
||||
data_sent............................: 2.8 GB 1.5 MB/s
|
||||
http_req_blocked.....................: min=110ns avg=53.66µs max=937.62ms p(50)=420ns p(95)=660ns p(99)=989ns
|
||||
http_req_connecting..................: min=0s avg=22.26µs max=532.7ms p(50)=0s p(95)=0s p(99)=0s
|
||||
http_req_duration....................: min=16.42ms avg=323.71ms max=3.4s p(50)=209.85ms p(95)=903.86ms p(99)=1.01s
|
||||
{ expected_response:true }.........: min=16.42ms avg=323.71ms max=3.4s p(50)=209.85ms p(95)=903.85ms p(99)=1.01s
|
||||
http_req_failed......................: 0.00% 5 out of 3291974
|
||||
http_req_receiving...................: min=18.23µs avg=1.73ms max=913.21ms p(50)=73.15µs p(95)=8.73ms p(99)=29.35ms
|
||||
http_req_sending.....................: min=17.54µs avg=53.13µs max=501.66ms p(50)=46.07µs p(95)=75.92µs p(99)=120.24µs
|
||||
http_req_tls_handshaking.............: min=0s avg=29.45µs max=657.29ms p(50)=0s p(95)=0s p(99)=0s
|
||||
http_req_waiting.....................: min=3.84ms avg=321.92ms max=3.4s p(50)=206.45ms p(95)=901.97ms p(99)=1.01s
|
||||
http_reqs............................: 3291974 1807.469453/s
|
||||
iteration_duration...................: min=18.62ms avg=328.21ms max=3.41s p(50)=215.77ms p(95)=907.48ms p(99)=1.01s
|
||||
iterations...........................: 3289564 1806.146234/s
|
||||
login_ui_enter_login_name_duration...: min=131.65ms avg=131.65ms max=131.65ms p(50)=131.65ms p(95)=131.65ms p(99)=131.65ms
|
||||
login_ui_enter_password_duration.....: min=18.55ms avg=18.55ms max=18.55ms p(50)=18.55ms p(95)=18.55ms p(99)=18.55ms
|
||||
login_ui_init_login_duration.........: min=68.72ms avg=68.72ms max=68.72ms p(50)=68.72ms p(95)=68.72ms p(99)=68.72ms
|
||||
login_ui_token_duration..............: min=77.56ms avg=77.56ms max=77.56ms p(50)=77.56ms p(95)=77.56ms p(99)=77.56ms
|
||||
oidc_token_duration..................: min=16.42ms avg=323.79ms max=3.4s p(50)=209.87ms p(95)=903.91ms p(99)=1.01s
|
||||
org_create_org_duration..............: min=38.42ms avg=38.42ms max=38.42ms p(50)=38.42ms p(95)=38.42ms p(99)=38.42ms
|
||||
user_add_machine_key_duration........: min=23.47ms avg=280.32ms max=851.94ms p(50)=304.18ms p(95)=437.53ms p(99)=443.37ms
|
||||
user_create_machine_duration.........: min=63.91ms avg=406.02ms max=663.96ms p(50)=437.61ms p(95)=518.85ms p(99)=521.88ms
|
||||
vus..................................: 425 min=0 max=600
|
||||
vus_max..............................: 600 min=600 max=600
|
||||
|
||||
running (30m21.3s), 000/600 VUs, 3289564 complete and 0 interrupted iterations
|
||||
default ✓ [======================================] 600 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,106 @@
|
||||
---
|
||||
title: oidc session benchmark of Zitadel v2.70.0
|
||||
sidebar_label: oidc session
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
The test implements [Support for (OIDC) Standard in a Custom Login UI flow](https://zitadel.com/docs/guides/integrate/login-ui/oidc-standard).
|
||||
|
||||
The tests showed that querying the user takes too much time because Zitadel ensures the projection is up to date. This performance bottleneck must be resolved.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 14:24 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | oidc\_session |
|
||||
| k6 version | v0.57.0 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v2.70.0 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.2 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | |
|
||||
| Top 3 most expensive database queries | 1: lock current\_states table<br/>2: write events<br/>3: get events for projection<br/> |
|
||||
| k6 Iterations per second | 153 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | Resolve locking issue |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
✓ authorize status ok
|
||||
✓ auth request id returned
|
||||
✓ add Session status ok
|
||||
✓ finalize auth request status ok
|
||||
|
||||
█ setup
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✓ generate machine key status ok
|
||||
✓ member added successful
|
||||
✓ openid configuration
|
||||
✓ access token returned
|
||||
|
||||
█ teardown
|
||||
|
||||
✓ org removed
|
||||
|
||||
checks...............................: 100.00% 1097103 out of 1097103
|
||||
data_received........................: 482 MB 267 kB/s
|
||||
data_sent............................: 206 MB 114 kB/s
|
||||
http_req_blocked.....................: min=150ns avg=185.63µs max=639.06ms p(50)=360ns p(95)=790ns p(99)=1.11µs
|
||||
http_req_connecting..................: min=0s avg=76.84µs max=394.03ms p(50)=0s p(95)=0s p(99)=0s
|
||||
http_req_duration....................: min=2.27ms avg=1.31s max=6.57s p(50)=326.44ms p(95)=3.94s p(99)=4.28s
|
||||
{ expected_response:true }.........: min=2.27ms avg=1.31s max=6.57s p(50)=326.44ms p(95)=3.94s p(99)=4.28s
|
||||
http_req_failed......................: 0.00% 0 out of 823429
|
||||
http_req_receiving...................: min=22.92µs avg=143.73µs max=245.98ms p(50)=105.17µs p(95)=188.26µs p(99)=260.56µs
|
||||
http_req_sending.....................: min=22.37µs avg=67.8µs max=41.57ms p(50)=63.65µs p(95)=104.8µs p(99)=138.46µs
|
||||
http_req_tls_handshaking.............: min=0s avg=106.12µs max=580.5ms p(50)=0s p(95)=0s p(99)=0s
|
||||
http_req_waiting.....................: min=2.11ms avg=1.31s max=6.57s p(50)=326.17ms p(95)=3.94s p(99)=4.28s
|
||||
http_reqs............................: 823429 456.440453/s
|
||||
iteration_duration...................: min=713.37ms avg=3.94s max=8.94s p(50)=3.92s p(95)=4.98s p(99)=5.44s
|
||||
iterations...........................: 274271 152.032998/s
|
||||
login_ui_enter_login_name_duration...: min=113.75ms avg=113.75ms max=113.75ms p(50)=113.75ms p(95)=113.75ms p(99)=113.75ms
|
||||
login_ui_enter_password_duration.....: min=2.27ms avg=2.27ms max=2.27ms p(50)=2.27ms p(95)=2.27ms p(99)=2.27ms
|
||||
login_ui_init_login_duration.........: min=20.48ms avg=156.67ms max=6.57s p(50)=126.64ms p(95)=280.16ms p(99)=675.36ms
|
||||
login_ui_token_duration..............: min=68.53ms avg=68.53ms max=68.53ms p(50)=68.53ms p(95)=68.53ms p(99)=68.53ms
|
||||
membership_iam_member................: min=34.16ms avg=34.16ms max=34.16ms p(50)=34.16ms p(95)=34.16ms p(99)=34.16ms
|
||||
oidc_auth_requst_by_id_duration......: min=20.59ms avg=370.56ms max=2.87s p(50)=294.12ms p(95)=911.42ms p(99)=1.08s
|
||||
oidc_session_duration................: min=713ms avg=3.94s max=8.94s p(50)=3.92s p(95)=4.98s p(99)=5.44s
|
||||
oidc_token_duration..................: min=40.67ms avg=40.67ms max=40.67ms p(50)=40.67ms p(95)=40.67ms p(99)=40.67ms
|
||||
org_create_org_duration..............: min=48.92ms avg=48.92ms max=48.92ms p(50)=48.92ms p(95)=48.92ms p(99)=48.92ms
|
||||
session_add_session_duration.........: min=92.04ms avg=3.4s max=6.16s p(50)=3.46s p(95)=4.18s p(99)=4.48s
|
||||
user_add_machine_key_duration........: min=32.08ms avg=32.08ms max=32.08ms p(50)=32.08ms p(95)=32.08ms p(99)=32.08ms
|
||||
user_create_machine_duration.........: min=91.73ms avg=91.73ms max=91.73ms p(50)=91.73ms p(95)=91.73ms p(99)=91.73ms
|
||||
vus..................................: 82 min=0 max=600
|
||||
vus_max..............................: 600 min=600 max=600
|
||||
|
||||
running (30m04.0s), 000/600 VUs, 274271 complete and 0 interrupted iterations
|
||||
default ✓ [======================================] 600 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,97 @@
|
||||
---
|
||||
title: create session benchmark of zitadel v4
|
||||
sidebar_label: create session
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Benchmark results of v4 release of Zitadel.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 11:31 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | add\_session |
|
||||
| k6 version | v1.0.0-rc1 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v4.0.0-rc2 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.4 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | |
|
||||
| Top 3 most expensive database queries | 1: Query events by instance_id, aggregate_types, event_types, position<br/>2: write events<br/>3: list members<br/> |
|
||||
| k6 Iterations per second | 607 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
Until :37 were containers starting because it was a cold start test.
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
█ TOTAL RESULTS
|
||||
|
||||
checks_total.......................: 2244565 1213.769815/s
|
||||
checks_succeeded...................: 99.99% 2244563 out of 2244565
|
||||
checks_failed......................: 0.00% 2 out of 2244565
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✗ add Session status ok
|
||||
↳ 99% — ✓ 1121975 / ✗ 2
|
||||
✓ add session is status ok
|
||||
✓ org removed
|
||||
|
||||
CUSTOM
|
||||
add_session_duration....................................................: min=236ms avg=962.51ms max=3.66s p(50)=951ms p(95)=1.33s p(99)=1.55s
|
||||
login_ui_enter_login_name_duration......................................: min=184.04ms avg=184.04ms max=184.04ms p(50)=184.04ms p(95)=184.04ms p(99)=184.04ms
|
||||
login_ui_enter_password_duration........................................: min=15.68ms avg=15.68ms max=15.68ms p(50)=15.68ms p(95)=15.68ms p(99)=15.68ms
|
||||
login_ui_init_login_duration............................................: min=128.36ms avg=128.36ms max=128.36ms p(50)=128.36ms p(95)=128.36ms p(99)=128.36ms
|
||||
login_ui_token_duration.................................................: min=87.2ms avg=87.2ms max=87.2ms p(50)=87.2ms p(95)=87.2ms p(99)=87.2ms
|
||||
org_create_org_duration.................................................: min=76.73ms avg=76.73ms max=76.73ms p(50)=76.73ms p(95)=76.73ms p(99)=76.73ms
|
||||
session_add_session_duration............................................: min=0s avg=961.43ms max=3.66s p(50)=950.06ms p(95)=1.33s p(99)=1.55s
|
||||
user_create_human_duration..............................................: min=548.74ms avg=10.69s max=16.39s p(50)=13.39s p(95)=15.97s p(99)=16.28s
|
||||
|
||||
HTTP
|
||||
http_req_duration.......................................................: min=0s avg=966.15ms max=18.87s p(50)=950.04ms p(95)=1.33s p(99)=1.55s
|
||||
{ expected_response:true }............................................: min=15.68ms avg=966.15ms max=18.87s p(50)=950.04ms p(95)=1.33s p(99)=1.55s
|
||||
http_req_failed.........................................................: 0.00% 2 out of 1123187
|
||||
http_reqs...............................................................: 1123187 607.374025/s
|
||||
|
||||
EXECUTION
|
||||
iteration_duration......................................................: min=1.63ms avg=962.61ms max=3.66s p(50)=951.17ms p(95)=1.33s p(99)=1.55s
|
||||
iterations..............................................................: 1121977 606.719706/s
|
||||
vus.....................................................................: 0 min=0 max=600
|
||||
vus_max.................................................................: 600 min=600 max=600
|
||||
|
||||
NETWORK
|
||||
data_received...........................................................: 617 MB 334 kB/s
|
||||
data_sent...............................................................: 184 MB 100 kB/s
|
||||
|
||||
running (30m49.3s), 000/600 VUs, 1121977 complete and 0 interrupted iterations
|
||||
default ✓ [======================================] 600 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,104 @@
|
||||
---
|
||||
title: human password login benchmark of zitadel v4
|
||||
sidebar_label: human password login
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Benchmark results of v4 release of Zitadel.
|
||||
|
||||
This benchmark tests the flow of a user signing in to Zitadel using username and password which is a highly CPU intensive task for checking the password.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 06:43 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | human\_password\_login |
|
||||
| k6 version | v1.0.0-rc1 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v4.0.0-rc2 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.4 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | |
|
||||
| Top 3 most expensive database queries | 1: Locking projections for reduce<br/>2: Query events by aggregate type and event type<br/>3: Query events by instance id, aggregate types, event types, position<br/> |
|
||||
| k6 Iterations per second | 39 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
Until :45 were containers starting because it was a cold start test.
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
█ TOTAL RESULTS
|
||||
|
||||
checks_total.......................: 867495 464.541713/s
|
||||
checks_succeeded...................: 99.88% 866490 out of 867495
|
||||
checks_failed......................: 0.11% 1005 out of 867495
|
||||
|
||||
✓ user defined
|
||||
✗ authorize status ok
|
||||
↳ 99% — ✓ 72470 / ✗ 435
|
||||
✗ login name status ok
|
||||
↳ 99% — ✓ 72331 / ✗ 139
|
||||
✓ login shows password page
|
||||
✗ password status ok
|
||||
↳ 99% — ✓ 72032 / ✗ 155
|
||||
✗ password callback
|
||||
↳ 99% — ✓ 72000 / ✗ 32
|
||||
✓ code set
|
||||
✗ token status ok
|
||||
↳ 99% — ✓ 71866 / ✗ 134
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✗ openid configuration
|
||||
↳ 98% — ✓ 600 / ✗ 11
|
||||
✗ userinfo status ok
|
||||
↳ 99% — ✓ 71755 / ✗ 99
|
||||
✓ org removed
|
||||
|
||||
CUSTOM
|
||||
human_password_login_duration...........................................: min=984ms avg=14.73s max=58.88s p(50)=12.41s p(95)=36.8s p(99)=40.55s
|
||||
login_ui_enter_login_name_duration......................................: min=98.28ms avg=10.57s max=38.72s p(50)=5.99s p(95)=32.4s p(99)=36.16s
|
||||
login_ui_enter_password_duration........................................: min=1.12ms avg=119.12ms max=1m0s p(50)=2.15ms p(95)=3.96ms p(99)=3.8s
|
||||
login_ui_init_login_duration............................................: min=41.73ms avg=600.61ms max=22.51s p(50)=292.69ms p(95)=1.84s p(99)=2.82s
|
||||
login_ui_token_duration.................................................: min=53.03ms avg=712.39ms max=21.76s p(50)=339.01ms p(95)=2.11s p(99)=3.32s
|
||||
oidc_user_info_duration.................................................: min=14.54ms avg=187.13ms max=21.51s p(50)=74.47ms p(95)=398.78ms p(99)=619.27ms
|
||||
org_create_org_duration.................................................: min=50.09ms avg=50.09ms max=50.09ms p(50)=50.09ms p(95)=50.09ms p(99)=50.09ms
|
||||
user_create_human_duration..............................................: min=425.26ms avg=11.83s max=16.89s p(50)=13.67s p(95)=16.07s p(99)=16.78s
|
||||
|
||||
HTTP
|
||||
http_req_duration.......................................................: min=1.12ms avg=1.87s max=1m0s p(50)=257.41ms p(95)=12.28s p(99)=31.18s
|
||||
{ expected_response:true }............................................: min=1.12ms avg=1.84s max=38.72s p(50)=256.14ms p(95)=11.83s p(99)=31.19s
|
||||
http_req_failed.........................................................: 0.19% 1117 out of 581385
|
||||
http_reqs...............................................................: 581385 311.330421/s
|
||||
|
||||
EXECUTION
|
||||
iteration_duration......................................................: min=983.33ms avg=14.88s max=1m11s p(50)=12.64s p(95)=36.85s p(99)=40.62s
|
||||
iterations..............................................................: 72904 39.039936/s
|
||||
vus.....................................................................: 18 min=0 max=600
|
||||
vus_max.................................................................: 600 min=600 max=600
|
||||
|
||||
NETWORK
|
||||
data_received...........................................................: 1.2 GB 658 kB/s
|
||||
data_sent...............................................................: 235 MB 126 kB/s
|
||||
|
||||
running (31m07.4s), 000/600 VUs, 72904 complete and 0 interrupted iterations
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,98 @@
|
||||
---
|
||||
title: introspection benchmark of zitadel v4
|
||||
sidebar_label: introspection
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Benchmark results of v4 release of Zitadel.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 13:33 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | introspect |
|
||||
| k6 version | v1.0.0-rc1 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v4.0.0-rc2 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.4 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | A lot of over fetching on the database |
|
||||
| Top 3 most expensive database queries | 1: get app<br/>2: Query events by instance id, aggregate types, event types, position<br/>3: Query executions<br/> |
|
||||
| k6 Iterations per second | 18 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
█ TOTAL RESULTS
|
||||
|
||||
checks_total.......................: 36207 19.664102/s
|
||||
checks_succeeded...................: 99.99% 36205 out of 36207
|
||||
checks_failed......................: 0.00% 2 out of 36207
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ add project status ok
|
||||
✓ add api status ok
|
||||
✓ add app key status ok
|
||||
✓ openid configuration
|
||||
✗ introspect status ok
|
||||
↳ 99% — ✓ 33792 / ✗ 2
|
||||
✓ org removed
|
||||
|
||||
CUSTOM
|
||||
app_add_app_duration....................................................: min=32.73ms avg=779.27ms max=1.11s p(50)=771.76ms p(95)=1.08s p(99)=1.1s
|
||||
app_add_app_key_duration................................................: min=340.78ms avg=12.14s max=17.19s p(50)=13.51s p(95)=16.78s p(99)=16.93s
|
||||
login_ui_enter_login_name_duration......................................: min=212.91ms avg=212.91ms max=212.91ms p(50)=212.91ms p(95)=212.91ms p(99)=212.91ms
|
||||
login_ui_enter_password_duration........................................: min=29.53ms avg=29.53ms max=29.53ms p(50)=29.53ms p(95)=29.53ms p(99)=29.53ms
|
||||
login_ui_init_login_duration............................................: min=65.24ms avg=65.24ms max=65.24ms p(50)=65.24ms p(95)=65.24ms p(99)=65.24ms
|
||||
login_ui_token_duration.................................................: min=98.54ms avg=98.54ms max=98.54ms p(50)=98.54ms p(95)=98.54ms p(99)=98.54ms
|
||||
oidc_introspect_duration................................................: min=731.31µs avg=32.04s max=50.94s p(50)=28.25s p(95)=47.83s p(99)=49.39s
|
||||
org_create_org_duration.................................................: min=51.76ms avg=51.76ms max=51.76ms p(50)=51.76ms p(95)=51.76ms p(99)=51.76ms
|
||||
project_add_project_duration............................................: min=52.11ms avg=754.58ms max=1.2s p(50)=778.66ms p(95)=1.14s p(99)=1.19s
|
||||
|
||||
HTTP
|
||||
http_req_duration.......................................................: min=731.31µs avg=30.22s max=50.94s p(50)=26.12s p(95)=47.74s p(99)=49.35s
|
||||
{ expected_response:true }............................................: min=13.11ms avg=30.22s max=50.94s p(50)=26.12s p(95)=47.74s p(99)=49.35s
|
||||
http_req_failed.........................................................: 0.00% 2 out of 36204
|
||||
http_reqs...............................................................: 36204 19.662473/s
|
||||
|
||||
EXECUTION
|
||||
iteration_duration......................................................: min=1.53ms avg=32.14s max=50.94s p(50)=28.25s p(95)=47.83s p(99)=49.39s
|
||||
iterations..............................................................: 33794 18.353597/s
|
||||
vus.....................................................................: 0 min=0 max=600
|
||||
vus_max.................................................................: 600 min=600 max=600
|
||||
|
||||
NETWORK
|
||||
data_received...........................................................: 25 MB 14 kB/s
|
||||
data_sent...............................................................: 33 MB 18 kB/s
|
||||
|
||||
running (30m41.3s), 000/600 VUs, 33794 complete and 0 interrupted iterations
|
||||
default ✓ [======================================] 600 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,103 @@
|
||||
---
|
||||
title: machine client credentials login benchmark of zitadel v4
|
||||
sidebar_label: machine client credentials login
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Benchmark results of v4 release of Zitadel.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 08:28 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | machine\_client\_credentials\_login |
|
||||
| k6 version | v1.0.0-rc1 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v4.0.0-rc2 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.4 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | |
|
||||
| Top 3 most expensive database queries | 1: Locking projections for reduce<br/>2: Query events by aggregate type and event type<br/>3: user by id query<br/> |
|
||||
| k6 Iterations per second | 424 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
Until :37 were containers starting because it was a cold start test.
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
|
||||
█ TOTAL RESULTS
|
||||
|
||||
checks_total.......................: 2336254 1274.250394/s
|
||||
checks_succeeded...................: 99.99% 2336250 out of 2336254
|
||||
checks_failed......................: 0.00% 4 out of 2336254
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✓ generate machine secret status ok
|
||||
✓ openid configuration
|
||||
✗ client credentials status ok
|
||||
↳ 99% — ✓ 778146 / ✗ 3
|
||||
✓ client credentials token ok
|
||||
✗ userinfo status ok
|
||||
↳ 99% — ✓ 778145 / ✗ 1
|
||||
✓ org removed
|
||||
|
||||
CUSTOM
|
||||
login_ui_enter_login_name_duration......................................: min=191.97ms avg=191.97ms max=191.97ms p(50)=191.97ms p(95)=191.97ms p(99)=191.97ms
|
||||
login_ui_enter_password_duration........................................: min=14.65ms avg=14.65ms max=14.65ms p(50)=14.65ms p(95)=14.65ms p(99)=14.65ms
|
||||
login_ui_init_login_duration............................................: min=91.36ms avg=91.36ms max=91.36ms p(50)=91.36ms p(95)=91.36ms p(99)=91.36ms
|
||||
login_ui_token_duration.................................................: min=93.72ms avg=93.72ms max=93.72ms p(50)=93.72ms p(95)=93.72ms p(99)=93.72ms
|
||||
oidc_client_credentials_duration........................................: min=244.96µs avg=794.35ms max=2.22s p(50)=783.9ms p(95)=1.1s p(99)=1.23s
|
||||
oidc_user_info_duration.................................................: min=578.67µs avg=592.86ms max=1.98s p(50)=583.33ms p(95)=840.94ms p(99)=939.62ms
|
||||
org_create_org_duration.................................................: min=60.95ms avg=60.95ms max=60.95ms p(50)=60.95ms p(95)=60.95ms p(99)=60.95ms
|
||||
user_add_machine_secret_duration........................................: min=54.63ms avg=2.62s max=4.92s p(50)=2.67s p(95)=4.69s p(99)=4.87s
|
||||
user_create_machine_duration............................................: min=159.68ms avg=681.87ms max=868.85ms p(50)=746.38ms p(95)=840.65ms p(99)=859.33ms
|
||||
|
||||
HTTP
|
||||
http_req_duration.......................................................: min=244.96µs avg=693.86ms max=4.92s p(50)=671.37ms p(95)=1.05s p(99)=1.18s
|
||||
{ expected_response:true }............................................: min=14.27ms avg=693.86ms max=4.92s p(50)=671.37ms p(95)=1.05s p(99)=1.18s
|
||||
http_req_failed.........................................................: 0.00% 4 out of 1558705
|
||||
http_reqs...............................................................: 1558705 850.156045/s
|
||||
|
||||
EXECUTION
|
||||
iteration_duration......................................................: min=656.54µs avg=1.38s max=3.43s p(50)=1.38s p(95)=1.8s p(99)=2.01s
|
||||
iterations..............................................................: 778149 424.421604/s
|
||||
vus.....................................................................: 292 min=0 max=600
|
||||
vus_max.................................................................: 600 min=600 max=600
|
||||
|
||||
NETWORK
|
||||
data_received...........................................................: 1.6 GB 898 kB/s
|
||||
data_sent...............................................................: 528 MB 288 kB/s
|
||||
|
||||
running (30m33.4s), 000/600 VUs, 778149 complete and 0 interrupted iterations
|
||||
default ✓ [======================================] 600 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,95 @@
|
||||
---
|
||||
title: machine jwt profile grant benchmark of zitadel v4
|
||||
sidebar_label: machine jwt profile grant
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Benchmark results of v4 release of Zitadel.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 18:33 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | machine\_jwt\_profile\_grant |
|
||||
| k6 version | v1.0.0-rc1 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v4.0.0-rc2 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.4 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | Zitadel containers did not scale but database would have headroom |
|
||||
| Top 3 most expensive database queries | 1: Query events by instance id, aggregate types, event types, position<br/>2: write events<br/>3: get user by id<br/> |
|
||||
| k6 Iterations per second | 851 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
█ TOTAL RESULTS
|
||||
|
||||
checks_total.......................: 3128039 1702.561961/s
|
||||
checks_succeeded...................: 100.00% 3128039 out of 3128039
|
||||
checks_failed......................: 0.00% 0 out of 3128039
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✓ generate machine key status ok
|
||||
✓ openid configuration
|
||||
✓ access token returned
|
||||
✓ org removed
|
||||
|
||||
CUSTOM
|
||||
login_ui_enter_login_name_duration......................................: min=185.23ms avg=185.23ms max=185.23ms p(50)=185.23ms p(95)=185.23ms p(99)=185.23ms
|
||||
login_ui_enter_password_duration........................................: min=19.29ms avg=19.29ms max=19.29ms p(50)=19.29ms p(95)=19.29ms p(99)=19.29ms
|
||||
login_ui_init_login_duration............................................: min=103.07ms avg=103.07ms max=103.07ms p(50)=103.07ms p(95)=103.07ms p(99)=103.07ms
|
||||
login_ui_token_duration.................................................: min=147.4ms avg=147.4ms max=147.4ms p(50)=147.4ms p(95)=147.4ms p(99)=147.4ms
|
||||
oidc_token_duration.....................................................: min=58.6ms avg=688.05ms max=3.95s p(50)=470.3ms p(95)=1.16s p(99)=1.24s
|
||||
org_create_org_duration.................................................: min=88.83ms avg=88.83ms max=88.83ms p(50)=88.83ms p(95)=88.83ms p(99)=88.83ms
|
||||
user_add_machine_key_duration...........................................: min=29.55ms avg=481.01ms max=742.31ms p(50)=487.76ms p(95)=728.51ms p(99)=739.94ms
|
||||
user_create_machine_duration............................................: min=88.42ms avg=748.16ms max=4.46s p(50)=732.58ms p(95)=1.01s p(99)=1.03s
|
||||
|
||||
HTTP
|
||||
http_req_duration.......................................................: min=19.29ms avg=687.53ms max=4.46s p(50)=470.17ms p(95)=1.16s p(99)=1.24s
|
||||
{ expected_response:true }............................................: min=19.29ms avg=687.53ms max=4.46s p(50)=470.17ms p(95)=1.16s p(99)=1.24s
|
||||
http_req_failed.........................................................: 0.00% 0 out of 1565523
|
||||
http_reqs...............................................................: 1565523 852.099321/s
|
||||
|
||||
EXECUTION
|
||||
iteration_duration......................................................: min=61.7ms avg=690.96ms max=3.96s p(50)=473.07ms p(95)=1.16s p(99)=1.25s
|
||||
iterations..............................................................: 1563113 850.787581/s
|
||||
vus.....................................................................: 1 min=0 max=600
|
||||
vus_max.................................................................: 600 min=600 max=600
|
||||
|
||||
NETWORK
|
||||
data_received...........................................................: 2.3 GB 1.2 MB/s
|
||||
data_sent...............................................................: 1.3 GB 725 kB/s
|
||||
|
||||
running (30m37.3s), 000/600 VUs, 1563113 complete and 0 interrupted iterations
|
||||
default ✓ [======================================] 600 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,102 @@
|
||||
---
|
||||
title: machine pat login benchmark of zitadel v4
|
||||
sidebar_label: machine pat login
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Benchmark results of v4 release of Zitadel.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 07:34 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | machine\_pat\_login |
|
||||
| k6 version | v1.0.0-rc1 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v4.0.0-rc2 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.4 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | |
|
||||
| Top 3 most expensive database queries | 1: Locking projections for reduce<br/>2: Query events by aggregate type and event type<br/>3: Query events by instance id, aggregate types, event types, position<br/> |
|
||||
| k6 Iterations per second | 1906 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
Until :37 were containers starting because it was a cold start test.
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
█ TOTAL RESULTS
|
||||
|
||||
checks_total.......................: 867495 464.541713/s
|
||||
checks_succeeded...................: 99.88% 866490 out of 867495
|
||||
checks_failed......................: 0.11% 1005 out of 867495
|
||||
|
||||
✓ user defined
|
||||
✗ authorize status ok
|
||||
↳ 99% — ✓ 72470 / ✗ 435
|
||||
✗ login name status ok
|
||||
↳ 99% — ✓ 72331 / ✗ 139
|
||||
✓ login shows password page
|
||||
✗ password status ok
|
||||
↳ 99% — ✓ 72032 / ✗ 155
|
||||
✗ password callback
|
||||
↳ 99% — ✓ 72000 / ✗ 32
|
||||
✓ code set
|
||||
✗ token status ok
|
||||
↳ 99% — ✓ 71866 / ✗ 134
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✗ openid configuration
|
||||
↳ 98% — ✓ 600 / ✗ 11
|
||||
✗ userinfo status ok
|
||||
↳ 99% — ✓ 71755 / ✗ 99
|
||||
✓ org removed
|
||||
|
||||
CUSTOM
|
||||
human_password_login_duration...........................................: min=984ms avg=14.73s max=58.88s p(50)=12.41s p(95)=36.8s p(99)=40.55s
|
||||
login_ui_enter_login_name_duration......................................: min=98.28ms avg=10.57s max=38.72s p(50)=5.99s p(95)=32.4s p(99)=36.16s
|
||||
login_ui_enter_password_duration........................................: min=1.12ms avg=119.12ms max=1m0s p(50)=2.15ms p(95)=3.96ms p(99)=3.8s
|
||||
login_ui_init_login_duration............................................: min=41.73ms avg=600.61ms max=22.51s p(50)=292.69ms p(95)=1.84s p(99)=2.82s
|
||||
login_ui_token_duration.................................................: min=53.03ms avg=712.39ms max=21.76s p(50)=339.01ms p(95)=2.11s p(99)=3.32s
|
||||
oidc_user_info_duration.................................................: min=14.54ms avg=187.13ms max=21.51s p(50)=74.47ms p(95)=398.78ms p(99)=619.27ms
|
||||
org_create_org_duration.................................................: min=50.09ms avg=50.09ms max=50.09ms p(50)=50.09ms p(95)=50.09ms p(99)=50.09ms
|
||||
user_create_human_duration..............................................: min=425.26ms avg=11.83s max=16.89s p(50)=13.67s p(95)=16.07s p(99)=16.78s
|
||||
|
||||
HTTP
|
||||
http_req_duration.......................................................: min=1.12ms avg=1.87s max=1m0s p(50)=257.41ms p(95)=12.28s p(99)=31.18s
|
||||
{ expected_response:true }............................................: min=1.12ms avg=1.84s max=38.72s p(50)=256.14ms p(95)=11.83s p(99)=31.19s
|
||||
http_req_failed.........................................................: 0.19% 1117 out of 581385
|
||||
http_reqs...............................................................: 581385 311.330421/s
|
||||
|
||||
EXECUTION
|
||||
iteration_duration......................................................: min=983.33ms avg=14.88s max=1m11s p(50)=12.64s p(95)=36.85s p(99)=40.62s
|
||||
iterations..............................................................: 72904 39.039936/s
|
||||
vus.....................................................................: 18 min=0 max=600
|
||||
vus_max.................................................................: 600 min=600 max=600
|
||||
|
||||
NETWORK
|
||||
data_received...........................................................: 1.2 GB 658 kB/s
|
||||
data_sent...............................................................: 235 MB 126 kB/s
|
||||
|
||||
running (31m07.4s), 000/600 VUs, 72904 complete and 0 interrupted iterations
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,102 @@
|
||||
---
|
||||
title: manipulate user benchmark of zitadel v4
|
||||
sidebar_label: manipulate user
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Benchmark results of v4 release of Zitadel.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 15:07 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | manipulate\_user |
|
||||
| k6 version | v1.0.0-rc1 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v4.0.0-rc2 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.4 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | |
|
||||
| Top 3 most expensive database queries | 1: query events by instance id, aggregate types, event types, position<br/>2: write events<br/>3: query user by id<br/> |
|
||||
| k6 Iterations per second | 16 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
Each iteration creates, updates and deletes a user.
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
█ TOTAL RESULTS
|
||||
|
||||
checks_total.......................: 199261 108.759178/s
|
||||
checks_succeeded...................: 57.15% 113882 out of 199261
|
||||
checks_failed......................: 42.84% 85379 out of 199261
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✗ create user is status ok
|
||||
↳ 99% — ✓ 28528 / ✗ 4
|
||||
✗ update user is status ok
|
||||
↳ 25% — ✓ 28469 / ✗ 85359
|
||||
✗ lock user is status ok
|
||||
↳ 99% — ✓ 28460 / ✗ 8
|
||||
✗ delete user is status ok
|
||||
↳ 99% — ✓ 28412 / ✗ 8
|
||||
✓ org removed
|
||||
|
||||
CUSTOM
|
||||
delete_user_duration....................................................: min=109.24ms avg=3.48s max=1m0s p(50)=2s p(95)=7.65s p(99)=9.78s
|
||||
lock_user_duration......................................................: min=77.95ms avg=2.7s max=1m0s p(50)=1.1s p(95)=6.36s p(99)=7.86s
|
||||
login_ui_enter_login_name_duration......................................: min=175.39ms avg=175.39ms max=175.39ms p(50)=175.39ms p(95)=175.39ms p(99)=175.39ms
|
||||
login_ui_enter_password_duration........................................: min=34.12ms avg=34.12ms max=34.12ms p(50)=34.12ms p(95)=34.12ms p(99)=34.12ms
|
||||
login_ui_init_login_duration............................................: min=100.5ms avg=100.5ms max=100.5ms p(50)=100.5ms p(95)=100.5ms p(99)=100.5ms
|
||||
login_ui_token_duration.................................................: min=154.05ms avg=154.05ms max=154.05ms p(50)=154.05ms p(95)=154.05ms p(99)=154.05ms
|
||||
org_create_org_duration.................................................: min=71.03ms avg=71.03ms max=71.03ms p(50)=71.03ms p(95)=71.03ms p(99)=71.03ms
|
||||
update_human_duration...................................................: min=70.77ms avg=2.43s max=1m0s p(50)=807.73ms p(95)=5.99s p(99)=7.36s
|
||||
user_create_human_duration..............................................: min=503.01ms avg=3.69s max=1m0s p(50)=2.07s p(95)=7.66s p(99)=13.98s
|
||||
|
||||
HTTP
|
||||
http_req_duration.......................................................: min=34.12ms avg=7.64s max=1m0s p(50)=4.07s p(95)=32.29s p(99)=41.1s
|
||||
{ expected_response:true }............................................: min=34.12ms avg=7.63s max=54.32s p(50)=4.07s p(95)=32.26s p(99)=41.03s
|
||||
http_req_failed.........................................................: 0.02% 31 out of 142384
|
||||
http_reqs...............................................................: 142384 77.714991/s
|
||||
|
||||
EXECUTION
|
||||
iteration_duration......................................................: min=17.1s avg=38.22s max=1m30s p(50)=36.76s p(95)=58.22s p(99)=1m7s
|
||||
iterations..............................................................: 28434 15.519637/s
|
||||
vus.....................................................................: 0 min=0 max=600
|
||||
vus_max.................................................................: 600 min=600 max=600
|
||||
|
||||
NETWORK
|
||||
data_received...........................................................: 82 MB 45 kB/s
|
||||
data_sent...............................................................: 26 MB 14 kB/s
|
||||
|
||||
running (30m32.1s), 000/600 VUs, 28434 complete and 98 interrupted iterations
|
||||
default ✓ [======================================] 600 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,104 @@
|
||||
---
|
||||
title: oidc session benchmark of zitadel v4
|
||||
sidebar_label: oidc session
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Benchmark results of v4 release of Zitadel.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 12:33 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | oidc_session |
|
||||
| k6 version | v1.0.0-rc1 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v4.0.0-rc2 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.4 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | Database CPU usage was 60% |
|
||||
| Top 3 most expensive database queries | 1: query events by instance id, aggregate types, event types, position<br/>2: writing events<br/>3: query memberships<br/> |
|
||||
| k6 Iterations per second | 105 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
█ TOTAL RESULTS
|
||||
|
||||
checks_total.......................: 1061560 420.116279/s
|
||||
checks_succeeded...................: 99.99% 1061559 out of 1061560
|
||||
checks_failed......................: 0.00% 1 out of 1061560
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✓ generate machine key status ok
|
||||
✓ member added successful
|
||||
✓ openid configuration
|
||||
✓ access token returned
|
||||
✓ auth request id returned
|
||||
✗ add Session status ok
|
||||
↳ 99% — ✓ 265394 / ✗ 1
|
||||
✓ finalize auth request status ok
|
||||
✓ org removed
|
||||
|
||||
CUSTOM
|
||||
login_ui_enter_login_name_duration......................................: min=100.93ms avg=100.93ms max=100.93ms p(50)=100.93ms p(95)=100.93ms p(99)=100.93ms
|
||||
login_ui_enter_password_duration........................................: min=4.86ms avg=4.86ms max=4.86ms p(50)=4.86ms p(95)=4.86ms p(99)=4.86ms
|
||||
login_ui_init_login_duration............................................: min=71.62ms avg=497.81ms max=19m45s p(50)=442.5ms p(95)=557.71ms p(99)=642.49ms
|
||||
login_ui_token_duration.................................................: min=80.88ms avg=80.88ms max=80.88ms p(50)=80.88ms p(95)=80.88ms p(99)=80.88ms
|
||||
membership_iam_member...................................................: min=65.45ms avg=65.45ms max=65.45ms p(50)=65.45ms p(95)=65.45ms p(99)=65.45ms
|
||||
oidc_auth_request_finalize..............................................: min=89.04ms avg=1.32s max=2m8s p(50)=1.26s p(95)=1.62s p(99)=1.78s
|
||||
oidc_session_duration...................................................: min=1.73s avg=2.88s max=21m54s p(50)=2.73s p(95)=3.26s p(99)=3.54s
|
||||
oidc_token_duration.....................................................: min=34.16ms avg=34.16ms max=34.16ms p(50)=34.16ms p(95)=34.16ms p(99)=34.16ms
|
||||
org_create_org_duration.................................................: min=41.58ms avg=41.58ms max=41.58ms p(50)=41.58ms p(95)=41.58ms p(99)=41.58ms
|
||||
session_add_session_duration............................................: min=105.71ms avg=1.06s max=15m9s p(50)=1s p(95)=1.34s p(99)=1.5s
|
||||
user_add_machine_key_duration...........................................: min=50.3ms avg=50.3ms max=50.3ms p(50)=50.3ms p(95)=50.3ms p(99)=50.3ms
|
||||
user_create_machine_duration............................................: min=32.72ms avg=32.72ms max=32.72ms p(50)=32.72ms p(95)=32.72ms p(99)=32.72ms
|
||||
|
||||
HTTP
|
||||
http_req_duration.......................................................: min=4.86ms avg=1.04s max=19m45s p(50)=966.65ms p(95)=1.5s p(99)=1.68s
|
||||
{ expected_response:true }............................................: min=4.86ms avg=987.51ms max=19m45s p(50)=966.55ms p(95)=1.5s p(99)=1.68s
|
||||
http_req_failed.........................................................: 0.01% 144 out of 796955
|
||||
http_reqs...............................................................: 796955 315.397876/s
|
||||
|
||||
EXECUTION
|
||||
iteration_duration......................................................: min=1.73s avg=2.88s max=21m55s p(50)=2.73s p(95)=3.26s p(99)=3.54s
|
||||
iterations..............................................................: 265117 104.921029/s
|
||||
vus.....................................................................: 0 min=0 max=600
|
||||
vus_max.................................................................: 600 min=600 max=600
|
||||
|
||||
NETWORK
|
||||
data_received...........................................................: 475 MB 188 kB/s
|
||||
data_sent...............................................................: 200 MB 79 kB/s
|
||||
|
||||
running (42m06.8s), 000/600 VUs, 265104 complete and 600 interrupted iterations
|
||||
default ✓ [======================================] 600 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,98 @@
|
||||
---
|
||||
title: otp session benchmark of zitadel v4
|
||||
sidebar_label: otp session
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Benchmark results of v4 release of Zitadel.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 14:56 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | otp_session |
|
||||
| k6 version | v1.0.0-rc1 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v4.0.0-rc2 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.4 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | database usage was 80% |
|
||||
| Top 3 most expensive database queries | 1: query events by instance id, aggregate types, event types, position<br/>2: lock projection<br/>3: write events<br/> |
|
||||
| k6 Iterations per second | 60 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
█ TOTAL RESULTS
|
||||
|
||||
checks_total.......................: 338059 181.945817/s
|
||||
checks_succeeded...................: 99.99% 338058 out of 338059
|
||||
checks_failed......................: 0.00% 1 out of 338059
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✓ set email otp status ok
|
||||
✓ add Session status ok
|
||||
✗ set Session status ok
|
||||
↳ 99% — ✓ 224563 / ✗ 1
|
||||
✓ org removed
|
||||
|
||||
CUSTOM
|
||||
login_ui_enter_login_name_duration......................................: min=179.7ms avg=179.7ms max=179.7ms p(50)=179.7ms p(95)=179.7ms p(99)=179.7ms
|
||||
login_ui_enter_password_duration........................................: min=14.73ms avg=14.73ms max=14.73ms p(50)=14.73ms p(95)=14.73ms p(99)=14.73ms
|
||||
login_ui_init_login_duration............................................: min=76.13ms avg=76.13ms max=76.13ms p(50)=76.13ms p(95)=76.13ms p(99)=76.13ms
|
||||
login_ui_token_duration.................................................: min=152.52ms avg=152.52ms max=152.52ms p(50)=152.52ms p(95)=152.52ms p(99)=152.52ms
|
||||
org_create_org_duration.................................................: min=59.87ms avg=59.87ms max=59.87ms p(50)=59.87ms p(95)=59.87ms p(99)=59.87ms
|
||||
otp_session_duration....................................................: min=2.2s avg=9.63s max=15.61s p(50)=9.59s p(95)=12.09s p(99)=13.33s
|
||||
session_add_session_duration............................................: min=154ms avg=8.31s max=12.63s p(50)=8.42s p(95)=10.1s p(99)=10.52s
|
||||
session_set_session_duration............................................: min=0s avg=659.94ms max=4.45s p(50)=513.63ms p(95)=1.62s p(99)=1.9s
|
||||
set_human_email_otp_duration............................................: min=29.12ms avg=494.09ms max=806.37ms p(50)=581.1ms p(95)=646.91ms p(99)=680.35ms
|
||||
user_create_human_duration..............................................: min=698.95ms avg=9.83s max=17.55s p(50)=11.1s p(95)=17s p(99)=17.46s
|
||||
|
||||
HTTP
|
||||
http_req_duration.......................................................: min=0s avg=3.21s max=20.01s p(50)=1.21s p(95)=9.59s p(99)=10.3s
|
||||
{ expected_response:true }............................................: min=14.73ms avg=3.21s max=20.01s p(50)=1.21s p(95)=9.59s p(99)=10.3s
|
||||
http_req_failed.........................................................: 0.00% 1 out of 338656
|
||||
http_reqs...............................................................: 338656 182.267126/s
|
||||
|
||||
EXECUTION
|
||||
iteration_duration......................................................: min=2.2s avg=9.63s max=15.61s p(50)=9.59s p(95)=12.09s p(99)=13.33s
|
||||
iterations..............................................................: 112282 60.43099/s
|
||||
vus.....................................................................: 0 min=0 max=600
|
||||
vus_max.................................................................: 600 min=600 max=600
|
||||
|
||||
NETWORK
|
||||
data_received...........................................................: 183 MB 98 kB/s
|
||||
data_sent...............................................................: 82 MB 44 kB/s
|
||||
|
||||
running (30m58.0s), 000/600 VUs, 112282 complete and 0 interrupted iterations
|
||||
default ✓ [======================================] 600 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,99 @@
|
||||
---
|
||||
title: session with password benchmark of zitadel v4
|
||||
sidebar_label: password session
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Benchmark results of v4 release of Zitadel.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 15:14 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | password\_session |
|
||||
| k6 version | v1.0.0-rc1 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v4.0.0-rc2 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.4 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | |
|
||||
| Observed errors | Container startup latency is > 3 minutes |
|
||||
| Top 3 most expensive database queries | 1: lock projection<br/>2: query events by instance id, aggregate types, event types, position<br/>3: query events by aggregate type, event type<br/> |
|
||||
| k6 Iterations per second | 51 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
█ TOTAL RESULTS
|
||||
|
||||
checks_total.......................: 188813 101.308159/s
|
||||
checks_succeeded...................: 99.35% 187589 out of 188813
|
||||
checks_failed......................: 0.64% 1224 out of 188813
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✓ set email otp status ok
|
||||
✗ add Session status ok
|
||||
↳ 99% — ✓ 93457 / ✗ 686
|
||||
✗ set Session status ok
|
||||
↳ 99% — ✓ 92919 / ✗ 538
|
||||
✓ org removed
|
||||
|
||||
CUSTOM
|
||||
login_ui_enter_login_name_duration......................................: min=240.4ms avg=240.4ms max=240.4ms p(50)=240.4ms p(95)=240.4ms p(99)=240.4ms
|
||||
login_ui_enter_password_duration........................................: min=14.32ms avg=14.32ms max=14.32ms p(50)=14.32ms p(95)=14.32ms p(99)=14.32ms
|
||||
login_ui_init_login_duration............................................: min=84.45ms avg=84.45ms max=84.45ms p(50)=84.45ms p(95)=84.45ms p(99)=84.45ms
|
||||
login_ui_token_duration.................................................: min=148.53ms avg=148.53ms max=148.53ms p(50)=148.53ms p(95)=148.53ms p(99)=148.53ms
|
||||
org_create_org_duration.................................................: min=59.73ms avg=59.73ms max=59.73ms p(50)=59.73ms p(95)=59.73ms p(99)=59.73ms
|
||||
password_session_duration...............................................: min=371ms avg=10.97s max=1m4s p(50)=9.52s p(95)=23.81s p(99)=27.57s
|
||||
session_add_session_duration............................................: min=36.81ms avg=10.62s max=1m0s p(50)=9.13s p(95)=23.23s p(99)=29.47s
|
||||
session_set_session_duration............................................: min=94.6ms avg=881.65ms max=1m0s p(50)=365.48ms p(95)=1.88s p(99)=4.75s
|
||||
set_human_email_otp_duration............................................: min=39.12ms avg=570.13ms max=868.83ms p(50)=549.06ms p(95)=844.01ms p(99)=864.28ms
|
||||
user_create_human_duration..............................................: min=502.43ms avg=10.82s max=16.72s p(50)=13.42s p(95)=16.28s p(99)=16.56s
|
||||
|
||||
HTTP
|
||||
http_req_duration.......................................................: min=14.32ms avg=5.75s max=1m0s p(50)=583.63ms p(95)=22.01s p(99)=26.51s
|
||||
{ expected_response:true }............................................: min=14.32ms avg=5.47s max=59.87s p(50)=566.75ms p(95)=21.8s p(99)=24.55s
|
||||
http_req_failed.........................................................: 0.64% 1224 out of 189410
|
||||
http_reqs...............................................................: 189410 101.628481/s
|
||||
|
||||
EXECUTION
|
||||
iteration_duration......................................................: min=38.37ms avg=11.5s max=1m21s p(50)=9.68s p(95)=24.21s p(99)=54.53s
|
||||
iterations..............................................................: 94143 50.512698/s
|
||||
vus.....................................................................: 0 min=0 max=600
|
||||
vus_max.................................................................: 600 min=600 max=600
|
||||
|
||||
NETWORK
|
||||
data_received...........................................................: 102 MB 55 kB/s
|
||||
data_sent...............................................................: 45 MB 24 kB/s
|
||||
|
||||
running (31m03.7s), 000/600 VUs, 94143 complete and 0 interrupted iterations
|
||||
default ✓ [======================================] 600 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,96 @@
|
||||
---
|
||||
title: user info benchmark of zitadel v4
|
||||
sidebar_label: user info
|
||||
---
|
||||
|
||||
import OutputSource from "./output.json";
|
||||
import { BenchmarkChart } from '@/components/benchmark_chart';
|
||||
|
||||
Benchmark results of v4 release of Zitadel.
|
||||
|
||||
## Performance test results
|
||||
|
||||
| Metric | Value |
|
||||
|:--------------------------------------|:------|
|
||||
| Baseline | none |
|
||||
| Purpose | Test current performance |
|
||||
| Test start | 16:06 UTC |
|
||||
| Test duration | 30min |
|
||||
| Executed test | user\_info |
|
||||
| k6 version | v1.0.0-rc1 |
|
||||
| VUs | 600 |
|
||||
| Client location | US1 |
|
||||
| ZITADEL location | US1 |
|
||||
| ZITADEL container specification | vCPU: 6<br/> Memory: 6 Gi <br/>Container min scale: 2<br/>Container max scale: 7 |
|
||||
| ZITADEL Version | v4.0.0-rc2 |
|
||||
| ZITADEL feature flags | webKey: true, improvedPerformance: \[\"IMPROVED\_PERFORMANCE\_ORG\_BY\_ID\", \"IMPROVED\_PERFORMANCE\_PROJECT\", \"IMPROVED\_PERFORMANCE\_USER\_GRANT\", \"IMPROVED\_PERFORMANCE\_ORG\_DOMAIN\_VERIFIED\", \"IMPROVED\_PERFORMANCE\_PROJECT\_GRANT\"\] |
|
||||
| Database | type: psql<br />version: v17.4 |
|
||||
| Database location | US1 |
|
||||
| Database specification | vCPU: 8<br/> memory: 32Gib |
|
||||
| ZITADEL metrics during test | Container startup latency is > 3 minutes |
|
||||
| Observed errors | Container startup latency is > 3 minutes, elevated error rate during container startup |
|
||||
| Top 3 most expensive database queries | 1: events by aggregate type and event type<br/>2: events by instance id, aggregate types, event types, position<br/>3: user by id<br/> |
|
||||
| k6 Iterations per second | 2653 |
|
||||
| k6 output | [output](#k6-output) |
|
||||
| flowchart outcome | |
|
||||
|
||||
## Endpoint latencies
|
||||
|
||||
<BenchmarkChart testResults={OutputSource} />
|
||||
|
||||
## k6 output
|
||||
```bash
|
||||
█ TOTAL RESULTS
|
||||
|
||||
checks_total.......................: 4784344 2653.094897/s
|
||||
checks_succeeded...................: 99.97% 4783333 out of 4784344
|
||||
checks_failed......................: 0.02% 1011 out of 4784344
|
||||
|
||||
✓ user defined
|
||||
✓ authorize status ok
|
||||
✓ login name status ok
|
||||
✓ login shows password page
|
||||
✓ password status ok
|
||||
✓ password callback
|
||||
✓ code set
|
||||
✓ token status ok
|
||||
✓ access token created
|
||||
✓ id token created
|
||||
✓ info created
|
||||
✓ org created
|
||||
✓ create user is status ok
|
||||
✗ openid configuration
|
||||
↳ 89% — ✓ 600 / ✗ 74
|
||||
✗ userinfo status ok
|
||||
↳ 99% — ✓ 4782709 / ✗ 936
|
||||
✗ org removed
|
||||
↳ 0% — ✓ 0 / ✗ 1
|
||||
|
||||
CUSTOM
|
||||
login_ui_enter_login_name_duration......................................: min=149.1ms avg=176.4ms max=203.69ms p(50)=176.4ms p(95)=200.96ms p(99)=203.15ms
|
||||
login_ui_enter_password_duration........................................: min=34.7ms avg=248.44ms max=462.19ms p(50)=248.44ms p(95)=440.82ms p(99)=457.92ms
|
||||
login_ui_init_login_duration............................................: min=83.7ms avg=85.38ms max=87.05ms p(50)=85.38ms p(95)=86.88ms p(99)=87.01ms
|
||||
login_ui_token_duration.................................................: min=92.78ms avg=118.84ms max=144.89ms p(50)=118.84ms p(95)=142.29ms p(99)=144.37ms
|
||||
oidc_user_info_duration.................................................: min=453.34µs avg=224.58ms max=1m0s p(50)=83.13ms p(95)=792.51ms p(99)=926.8ms
|
||||
org_create_org_duration.................................................: min=53.42ms avg=53.42ms max=53.42ms p(50)=53.42ms p(95)=53.42ms p(99)=53.42ms
|
||||
user_create_human_duration..............................................: min=373.72ms avg=373.72ms max=373.72ms p(50)=373.72ms p(95)=373.72ms p(99)=373.72ms
|
||||
|
||||
HTTP
|
||||
http_req_duration.......................................................: min=453.34µs avg=225.49ms max=1m0s p(50)=83.13ms p(95)=792.51ms p(99)=927.03ms
|
||||
{ expected_response:true }............................................: min=3.29ms avg=212.92ms max=29.69s p(50)=83.1ms p(95)=792.3ms p(99)=923.17ms
|
||||
http_req_failed.........................................................: 0.02% 1011 out of 4784339
|
||||
http_reqs...............................................................: 4784339 2653.092124/s
|
||||
|
||||
EXECUTION
|
||||
iteration_duration......................................................: min=731.85µs avg=225.74ms max=1m0s p(50)=83.33ms p(95)=792.71ms p(99)=927.27ms
|
||||
iterations..............................................................: 4783719 2652.748311/s
|
||||
vus.....................................................................: 600 min=0 max=600
|
||||
vus_max.................................................................: 600 min=600 max=600
|
||||
|
||||
NETWORK
|
||||
data_received...........................................................: 3.4 GB 1.9 MB/s
|
||||
data_sent...............................................................: 378 MB 210 kB/s
|
||||
|
||||
running (30m03.3s), 000/600 VUs, 4783719 complete and 0 interrupted iterations
|
||||
default ✓ [======================================] 600 VUs 30m0s
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user