Files
zitadel/apps/docs
75bc058bce feat(eventstore): autovacuum tuning for events2 table (#12449)
# Which Problems Are Solved

In Zitadel's append-only event-sourced architecture, the
`eventstore.events2` table grows indefinitely. PostgreSQL's default
autovacuum uses a percentage-based scale factor, so as the table grows,
the number of changed rows required to trigger a `VACUUM` or `ANALYZE`
drifts towards infinity. Without regular vacuums, the table's Visibility
Map becomes stale, disabling fast Index-Only Scans and forcing expensive
heap reads. Without regular analyzes, query planner statistics become
stale, leading to suboptimal execution plans.

This causes eventstore operations to progressively degrade as `events2`
grows, even without CPU, memory, or I/O saturation. A manual `VACUUM
ANALYZE` immediately restores performance, confirming the root cause.

- If `Eventstore.Autovacuum` is left at its default, `events2` keeps
using PostgreSQL's default, percentage-based autovacuum/autoanalyze
scale factors, which become impractically infrequent on large tables.
- There was previously no supported way to apply static, table-level
autovacuum tuning to `events2` through Zitadel's own configuration/setup
process.

# How the Problems Are Solved

- Added an `Eventstore.Autovacuum` runtime configuration block to
`cmd/defaults.yaml` (disabled by default):
  ```yaml
  Eventstore:
    Autovacuum:
      Enabled: false # ZITADEL_EVENTSTORE_AUTOVACUUM_ENABLED
VacuumThreshold: 50000 # ZITADEL_EVENTSTORE_AUTOVACUUM_VACUUMTHRESHOLD
AnalyzeThreshold: 50000 # ZITADEL_EVENTSTORE_AUTOVACUUM_ANALYZETHRESHOLD
  ```
- Added a repeatable `zitadel setup` migration step
(`cmd/setup/eventstore_autovacuum.go`) that:
- When `Enabled: true`, disables the percentage-based
`autovacuum_vacuum_scale_factor`, `autovacuum_analyze_scale_factor`, and
`autovacuum_vacuum_insert_scale_factor` on `eventstore.events2`, and
applies static thresholds (`autovacuum_vacuum_insert_threshold`,
`autovacuum_vacuum_threshold`, `autovacuum_analyze_threshold`) from the
config instead.
- When `Enabled: false`, resets those storage parameters on
`eventstore.events2` back to the cluster defaults.
- Implements `Repeatable.Check()` so the step only re-runs when the
configuration actually changed since the last `zitadel setup` run.
  - Added documentation in the new "Performance tuning" page.
 
# Additional Changes

- Move the projection documentation into performance tuning page
- Expand and update the projection documentation to the latest state in
zitadel. (contained some stale information)

# Additional Context

Several other issues describe read-performance symptoms consistent with
this same root cause (stale `events2` visibility map / planner
statistics at scale, without resource saturation). Since this PR
addresses the shared root cause:

- Closes #12448
- Closes #10754
- Closes #10260
- Closes #8585
- Closes #9239


---
_Generated by [Claude
Code](https://claude.ai/code/session_01HoKvEY7niCVgLCz7CajBwW)_

---------

Co-authored-by: Claude <[email protected]>
Co-authored-by: Copilot Autofix powered by AI <[email protected]>
Co-authored-by: Silvan <[email protected]>
2026-07-16 14:42:11 +00:00
..
2026-01-30 18:11:12 +00:00

@zitadel/docs

Documentation for Zitadel, built with Fumadocs and Next.js.

Getting Started

Ensure you have followed the root quick start to set up dependencies.

Local Development

Ensure all dependencies are installed:

pnpm install

Start the development server:

pnpm nx run @zitadel/docs:dev

The site will be available at http://localhost:3000.

Scripts

Key scripts for documentation workflows:

Script Description
dev Starts the development server.
build Builds the production application.
fetch:remote-content Fetches remote tags and referenced content.
generate Runs all generation steps (fetch:remote-content, generate:proto-docs, generate:api-reference, generate:index-pages).
check:links Validates content integrity (broken links, missing frontmatter, schema errors).
check-types Validates typescript types.
test Runs all validation steps (check-types, check:links).
lint checks for code style and syntax errors (ESLint).
clean Cleans the build output and generated files.

Validation

  • Code Quality: Run pnpm nx run @zitadel/docs:lint to check for syntax and style issues in JS/TS/MDX files.
  • Content Integrity: Run pnpm nx run @zitadel/docs:check-links to validate content structure, including:
    • Broken internal links
    • Missing required front-matter (e.g., title)
    • Image references

Troubleshooting

Issue: Nx configuration not found or "nx" command not found

If you see errors like:

 NX   Cannot find configuration for task @zitadel/zitadel:@zitadel/docs:generate

Pass --verbose to see the stacktrace.

or

pnpm nx run @zitadel/docs:dev
 ERR_PNPM_RECURSIVE_EXEC_FIRST_FAIL  Command "nx" not found

Did you mean "pnpm nx"?

Root cause: Your dependency cache or node_modules is out of sync.

Solution: Run these commands from the repository root in order:

  1. Clear node_modules: rm -rf node_modules — removes the installed dependency folder
  2. Clear pnpm cache: pnpm cache delete — clears pnpm's internal package cache
  3. Reinstall dependencies: pnpm install — reinstalls all dependencies fresh
  4. Test the setup: pnpm nx run @zitadel/docs:generate — verifies everything works

This resolves 99% of setup-related issues.

Contributing

Build Process

The docs build process automatically handles the following steps via generate:

  1. Downloads required protoc plugins.
  2. Generates gRPC documentation from proto files.
  3. Generates API documentation from OpenAPI specs.
  4. Generates index files for directory structures.

Style Guide

  • Variables: Use environment variables in code snippets where possible.
  • Embedded Content: Use _filename.mdx for content embedded in other pages (not indexed individually).
  • Code Embedding: Use the file property in code blocks to embed code from the repo.
  • Voice: Use active voice and sentence case for titles.

Refer to the Google Developer Style Guide for general guidelines.

Adding Content

All documentation content is located in the content directory. Note that the system strictly accepts only .mdx files.

To add a new page:

  1. Create a .mdx file in the appropriate subdirectory of content.
  2. Register the new page in the sidebar settings at lib/sidebar-data.ts to make it accessible in the navigation.

Pull Requests

Use docs(<scope>): <short summary> for PR titles. Pass quality checks before submitting:

pnpm nx run @zitadel/docs:build