mirror of
https://github.com/andwati/zitadel.git
synced 2026-11-07 14:52:15 +00:00
docs: introduce versioned docs and migrate to fuma (#11166)
## Todos for release - [x] Configure Env in docs project on vercel - [x] Configure Root Path in the docs project on vercel - [ ] Remove old CSP https://github.com/zitadel/website/pull/1592 ## What we did This pull request migrates the project documentation from the old `docs/` directory to the new `apps/docs/` directory, introduces a new documentation system built with Next.js and Fumadocs, and updates all relevant references, configuration files, and documentation to reflect this change. It also adds new configuration and ignore files for the new documentation app, updates CI and linting to exclude the new docs from certain checks, and revises the contributing guidelines accordingly. **Documentation System Migration and New Docs App** * Migrated all documentation from `docs/` to `apps/docs/`, and updated all references in `README.md`, `CONTRIBUTING.md`, and other files to point to the new location. [[1]](diffhunk://#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055L585-L640) [[2]](diffhunk://#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055L660-R607) [[3]](diffhunk://#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055L740-R687) [[4]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L2-R3) [[5]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L30-R30) * Added a new Next.js/Fumadocs-based documentation app under `apps/docs/`, including core app files, layouts, routing, search API, and a comprehensive `README.md` with development and contribution instructions. [[1]](diffhunk://#diff-5a1b07344a2c1b4d3f37b23ff1388b62cd9f57dea3c1cd23d8a0412b7602b132R1-R74) [[2]](diffhunk://#diff-462b9ad1eabbb7d1c29bb9c36e4931eb180fd7190900e6d2babf8f4d66ad1c28R1-R39) [[3]](diffhunk://#diff-e16ae25660ded787b10ac35dea96d5ecaacf895dae0afc8a9bd4382dc79a8c87R1-R7) [apps/docs/app/[[...slug]]/layout.tsxR1-R81](diffhunk://#diff-59e08acde4e805b7aeccef1dcf98f1d71dfc550777e6b9402085cee0e9fa4e0aR1-R81), [apps/docs/app/[[...slug]]/page.tsxR1-R79](diffhunk://#diff-e5df3f80d0fa01e12d63d81f29c57a9d14e846c78fd3beabb2ef768e38fd9580R1-R79), [[4]](diffhunk://#diff-389b34918e040cacaa87cd7201ffa462cc2b0b716736f537e3d3c660ac69353bR1-R7) [[5]](diffhunk://#diff-d3b03416d1c457b19f1c27b26f2db741412df841b875c26ccadc36e4522247f4R1-R29) [[6]](diffhunk://#diff-c8fb8339570a5305809be7c618e14705fd86390dc278e6ce8ba224a7bc8b0c3cR1-R25) **Configuration and Tooling Updates** * Updated `.github/workflows/codeql.yml`, `.golangci.yaml`, and `.github/dependabot.yml` to properly handle the new docs app: excluded `apps/docs` from certain checks, added npm dependency updates for the docs app, and excluded generated content. [[1]](diffhunk://#diff-12783128521e452af0cfac94b99b8d250413c516ec71fe6d97dbea666ff7ba27L8-R14) [[2]](diffhunk://#diff-9917ddc9f1c3304218f7269265b746d997c5c0615478177b5fceecd33ef47cb5R5-R6) [[3]](diffhunk://#diff-9917ddc9f1c3304218f7269265b746d997c5c0615478177b5fceecd33ef47cb5R126-R129) [[4]](diffhunk://#diff-dd4fbda47e51f1e35defb9275a9cd9c212ecde0b870cba89ddaaae65c5f3cd28R89-R106) * Updated `.devcontainer/devcontainer.json` to use the latest Go 1.25.3 version for consistency. **Licensing and Miscellaneous** * Added `apps/docs/` to the list of licensed directories in `LICENSING.md`. These changes ensure the documentation is now maintained in a modern, scalable system and all project tooling is updated to support the new structure. --------- Co-authored-by: Federico Coppede <[email protected]>
This commit is contained in:
Florian Forster
co-authored by
Federico Coppede
Federico Coppede
parent
7b6395c348
commit
d8ea041c56
@@ -0,0 +1,137 @@
|
||||
---
|
||||
title: How ZITADEL Processes and Stores Secrets
|
||||
sidebar_label: Secrets
|
||||
---
|
||||
|
||||
In this chapter you can find information of how ZITADEL processes and stores secrets and credentials in a secure fashion.
|
||||
|
||||
<Callout>
|
||||
We use the terms secret and credentials interchangeable to keep this guide lean.
|
||||
</Callout>
|
||||
|
||||
## Secrets Principles
|
||||
|
||||
ZITADEL uses the following principles when handling Secrets across their lifecycle:
|
||||
|
||||
- Automate rotation
|
||||
- Limit lifetime
|
||||
- Show only once
|
||||
- Prefer public / private key concepts (FIDO2, U2F, JWT Profile, ...)
|
||||
- Irreversible hash secrets / passwords
|
||||
- Encrypt secrets storage
|
||||
- When sent across unsecure channels (eMail, SMS, ...)
|
||||
- Forced changed through receiver
|
||||
- Verify that the secret can only be used once
|
||||
|
||||
## Secrets Storage
|
||||
|
||||
By default ZITADEL stores secrets from its users, clients as well as its generated secrets like signing keys in the database.
|
||||
To protect the secrets against extraction from database as well as database dumps they are encrypted with AES256.
|
||||
|
||||
<Callout>
|
||||
The key used to encrypt and decrypt the secrets in the ZITADEL database is called `masterkey` and needs to be exactly 32 bytes long.
|
||||
The only secrets stored outside of the Secrets Storage are the masterkey, the TLS Keys, the initial Admin User (including the password)
|
||||
</Callout>
|
||||
|
||||
## Secrets stored in the Secrets Storage
|
||||
|
||||
### Public Keys
|
||||
|
||||
ZITADEL does handle many different public keys. These include:
|
||||
|
||||
- FIDO2
|
||||
- U2F
|
||||
- JWT Profile
|
||||
- Signing Keys
|
||||
|
||||
<Callout>
|
||||
Due to the inherent nature of a public key being public we safeguard them against malicious key changes with our unique [eventstore concept](../eventstore/overview).
|
||||
</Callout>
|
||||
|
||||
### Hashed Secrets
|
||||
|
||||
ZITADEL does handle many different passwords and secrets. These include:
|
||||
|
||||
- User Authentication
|
||||
- Password
|
||||
- Client / Machine Authentication
|
||||
- Client Secrets
|
||||
|
||||
<Callout>
|
||||
ZITADEL hashes all Passwords and Client Secrets in an non reversible way to further reduce the risk of a Secrets Storage breach.
|
||||
</Callout>
|
||||
|
||||
Passwords and secrets are always hashed with a random salt and stored as an encoded string that contains the Algorithm, its Parameters, Salt and Hash.
|
||||
The storage encoding used by ZITADEL is Modular Crypt Format and a full reference can be found in our [Passwap library](https://github.com/zitadel/passwap#encoding).
|
||||
|
||||
The following hash algorithms are supported:
|
||||
|
||||
- argon2i / id[^1]
|
||||
- bcrypt (Default)
|
||||
- md5: implementation of md5Crypt with salt and password shuffling [^2]
|
||||
- md5plain: md5 digest of a password without salt [^2]
|
||||
- md5salted: md5 digest of a salted password [^2]
|
||||
- phpass: md5 digest with PHPass algorithm (used in WordPress) [^2]
|
||||
- drupal7: Custom hashing format used in Drupal 7 [^2]
|
||||
- sha2: implementation of crypt(3) SHA-256 & SHA-512
|
||||
- scrypt
|
||||
- pbkdf2
|
||||
|
||||
[^1]: argon2 algorithms are currently disabled on ZITADEL Cloud due to its steep memory requirements.
|
||||
[^2]: md5 and drupal7 are insecure and can only be used to import and verify users, not hash new passwords.
|
||||
|
||||
<Callout>
|
||||
ZITADEL updates stored hashes when the configured algorithm or its parameters are updated,
|
||||
the first time verification succeeds.
|
||||
This allows to increase cost along with growing computing power.
|
||||
ZITADEL allows to import user passwords from systems that use any of the above hashing algorithms.
|
||||
|
||||
Note however that by default, only `bcrypt` is enabled.
|
||||
Further `Verifiers` must be enabled in the [configuration](/self-hosting/manage/configure) by the system administrator.
|
||||
</Callout>
|
||||
|
||||
### Encrypted Secrets
|
||||
|
||||
Some secrets cannot be hashed because they need to be used in their raw form. These include:
|
||||
|
||||
- Federation
|
||||
- Client Secrets of Identity Providers (IdPs)
|
||||
- Multi-factor Authentication
|
||||
- TOTP Seed Values
|
||||
- Validation Secrets
|
||||
- Verifying contact information like eMail, Phonenumbers
|
||||
- Verifying proof of ownership over domain names (DNS)
|
||||
- Resting accounts of users (password, MFA reset, ...)
|
||||
- Private Keys
|
||||
- Token Signing (JWT, ...)
|
||||
- Token Encryption (Opaque Bearer Tokens)
|
||||
- Useragent Cookies (Session Cookies) Encryption
|
||||
- CSRF Cookie Encryption
|
||||
- Mail Provider
|
||||
- SMTP Passwords
|
||||
- SMS Provider
|
||||
- Twilio API Keys
|
||||
|
||||
<Callout>
|
||||
By default ZITADEL uses `RSA256` for signing purposes and `AES256` for encryption
|
||||
</Callout>
|
||||
|
||||
## Secrets stored outside the Secrets Storage
|
||||
|
||||
### Masterkey
|
||||
|
||||
Since the Masterkey is used as means of protecting the Secrets Storage it cannot be stored in the storage.
|
||||
You find [here the many ways how ZITADEL can consume the Masterkey](/self-hosting/manage/configure).
|
||||
|
||||
### TLS Material
|
||||
|
||||
ZITADEL does support end to end TLS as such it can consume TLS Key Material.
|
||||
Please check our [TLS Modes documentation](/self-hosting/manage/tls_modes) for more details.
|
||||
|
||||
### Admin User
|
||||
|
||||
The initial Admin User of ZITADEL can be configured through [ZITADELs config options](/self-hosting/manage/configure).
|
||||
|
||||
<Callout>
|
||||
To prevent elevated breaches ZITADEL forces the Admin Users password to be changed during the first login.
|
||||
</Callout>
|
||||
@@ -0,0 +1,151 @@
|
||||
---
|
||||
title: Zitadel's Software Architecture
|
||||
sidebar_label: Software Architecture
|
||||
---
|
||||
|
||||
Zitadel is built with two essential patterns. Event Sourcing (ES) and Command and Query Responsibility Segregation (CQRS).
|
||||
Due to the nature of Event Sourcing Zitadel provides the unique capability to generate a strong audit trail of ALL the things that happen to its resources, without compromising on storage cost or audit trail length.
|
||||
|
||||
The combination of ES and CQRS makes Zitadel eventual consistent which, from our perspective, is a great benefit in many ways.
|
||||
It allows us to build a Source of Records (SOR) which is the one single point of truth for all computed states.
|
||||
The SOR needs to be transaction safe to make sure all operations are in order.
|
||||
You can read more about this in our [ES documentation](../eventstore/overview).
|
||||
|
||||
Each Zitadel binary contains all components necessary to serve traffic
|
||||
From serving the API, rendering GUI's, background processing of events and task.
|
||||
This All in One (AiO) approach makes operating Zitadel simple.
|
||||
|
||||
## The Architecture
|
||||
|
||||
Zitadels software architecture is built around multiple components at different levels.
|
||||
This chapter should give you an idea of the components as well as the different layers.
|
||||
|
||||
|
||||
|
||||
### Service Layer
|
||||
|
||||
The service layer includes all components who are potentially exposed to consumers of Zitadel.
|
||||
|
||||
#### HTTP Server
|
||||
|
||||
The http server is responsible for the following functions:
|
||||
|
||||
- serving the management GUI called Zitadel Management Console
|
||||
- serving the static assets
|
||||
- rendering server side html (login, password-reset, verification, ...)
|
||||
|
||||
#### API Server
|
||||
|
||||
The API layer consist of the multiple APIs provided by Zitadel. Each serves a dedicated purpose.
|
||||
All APIs of Zitadel are always available as gRCP, gRPC-web and REST service.
|
||||
The only exception is the [OpenID Connect & OAuth](/apis/openidoauth/endpoints) and [Asset API](/apis/introduction#assets) due their unique nature.
|
||||
|
||||
- [OpenID Connect & OAuth](/apis/openidoauth/endpoints) - allows to request authentication and authorization of Zitadel
|
||||
- [SAML](/apis/saml/endpoints) - allows to request authentication and authorization of Zitadel through the SAML standard
|
||||
- [Authentication API](/apis/introduction#authentication) - allow a user to do operation in its own context
|
||||
- [Management API](/apis/introduction#management) - allows an admin or machine to manage the Zitadel resources on an organization level
|
||||
- [Administration API](/apis/introduction#administration) - allows an admin or machine to manage the Zitadel resources on an instance level
|
||||
- [System API](/apis/introduction#system) - allows to create and change new Zitadel instances
|
||||
- [Asset API](/apis/introduction#assets) - is used to upload and download static assets
|
||||
|
||||
### Core Layer
|
||||
|
||||
#### Commands
|
||||
|
||||
The Command Side has some unique requirements, these include:
|
||||
|
||||
- Transaction safety is a MUST
|
||||
- Availability MUST be high
|
||||
|
||||
> When we classify this with the CAP theorem we would choose Consistent and Available but leave Partition Tolerance aside.
|
||||
|
||||
##### Command Handler
|
||||
|
||||
The command handler receives all operations who alter a resource managed by Zitadel.
|
||||
For example if a user changes his name. The API Layer will pass the instruction received through the API call to the command handler for further processing.
|
||||
The command handler is then responsible of creating the necessary commands.
|
||||
After creating the commands the command hand them down to the command validation.
|
||||
|
||||
##### Command Validation
|
||||
|
||||
With the received commands the command validation will execute the business logic to verify if a certain action can take place.
|
||||
For example if the user really can change his name is verified in the command validation.
|
||||
If this succeeds the command validation will create the events that reflect the changes.
|
||||
These events now are being handed down to the storage layer for storage.
|
||||
|
||||
#### Events
|
||||
|
||||
Zitadel handles events in two ways.
|
||||
Events that should be processed in near real time are processed by a in memory pub sub system.
|
||||
Some events can be handled asynchronously using the spooler.
|
||||
|
||||
##### Pub Sub
|
||||
|
||||
The pub sub system job is it to keep a query view up-to-date by feeding a constant stream of events to the projections.
|
||||
Our pub sub system built into Zitadel works by placing events into an in memory queue for its subscribers.
|
||||
There is no need for specific guarantees from the pub sub system. Since the SOR is the ES everything can be retried without loss of data.
|
||||
In case of an error an event can be reapplied in two ways:
|
||||
|
||||
- The next event might trigger the projection to apply the whole difference
|
||||
- The spooler takes care of background cleanups in a scheduled fashion
|
||||
|
||||
> The decision to incorporate an internal pub sub system with no need for specific guarantees is a deliberate choice.
|
||||
> We believe that the toll of operating an additional external service like a MQ system negatively affects the ease of use of Zitadel as well as its availability guarantees.
|
||||
> One of the authors of Zitadel did his thesis to test this approach against established MQ systems.
|
||||
|
||||
##### Spooler
|
||||
|
||||
The spoolers job is it to keep a query view up-to-date or at least look that it does not have a too big lag behind the Event Store.
|
||||
Each query view has its own spooler which is responsible to look for the events who are relevant to generate the query view. It does this by triggering the relevant projection.
|
||||
Spoolers are especially necessary where someone can query datasets instead of single ids.
|
||||
|
||||
> Each view can have exactly one spooler, but spoolers are dynamically leader elected, so even if a spooler crashes it will be replaced in a short amount of time.
|
||||
|
||||
#### Projections
|
||||
|
||||
Projections are responsible of normalizing data for the query side or for analytical purpose.
|
||||
They generally work by being invoked either through a scheduled spooler or the pub sub subscription.
|
||||
|
||||
When they receive events they will create their normalized object and then store this into the query view and its storage layer.
|
||||
|
||||
#### Queries
|
||||
|
||||
The query side is responsible for answering read requests on data.
|
||||
It has some unique requirements, which include:
|
||||
|
||||
- It needs to be easy to query
|
||||
- Short response times are a MUST (80%of queries below 100ms on the api server)
|
||||
- Availability MUST be high, even during high loads
|
||||
- The query view MUST be able to be persisted for most request
|
||||
|
||||
> When we classify this with the CAP theorem we would choose **Available** and **Performance** but leave **Consistent** aside
|
||||
|
||||
##### Query Handler
|
||||
|
||||
The query handler receives all read relevant operations. These can either be query or simple `getById` calls.
|
||||
When receiving a query it will proceed by passing this to the repository which will call the database and return the dataset.
|
||||
If a request calls for a specific id the call will, most of the times, be revalidated against the Event Store.
|
||||
This is achieved by triggering the projection to make sure that the last sequence of a id is loaded into the query view.
|
||||
|
||||
> The query side has the option to dynamically check the Event Store for newer events on a certain id to make sure for consistent responses without delay.
|
||||
|
||||
##### Query View
|
||||
|
||||
The query view is responsible to query the storage layer with the request from the command handler.
|
||||
It is also responsible to execute authorization checks. To check if a request is valid and can be answered.
|
||||
|
||||
### Storage Layer
|
||||
|
||||
As Zitadel itself is built completely stateless only the storage layer is needed to persist states.
|
||||
The storage layer of Zitadel is responsible for multiple tasks. For example:
|
||||
|
||||
- Guarantee strong consistency for the command side
|
||||
- Guarantee good query performance for the query side
|
||||
- Backup and restore operation for disaster recovery purpose
|
||||
|
||||
Zitadel currently supports PostgreSQL.
|
||||
Make sure to read our [Production Guide](/self-hosting/manage/production#prefer-postgre-sql) before you decide on using one of them.
|
||||
|
||||
<Callout>
|
||||
Zitadel v2 supported CockroachDB and PostgreSQL. Zitadel v3 only supports PostgreSQL. Please refer to [the mirror guide](/self-hosting/manage/cli/mirror) to migrate to PostgreSQL.
|
||||
</Callout>
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
title: Zitadel's Deployment Architecture
|
||||
sidebar_label: Deployment Architecture
|
||||
---
|
||||
|
||||
## High Availability
|
||||
|
||||
Zitadel can be run as high available system with ease.
|
||||
Since the storage layer takes the heavy lifting of making sure that data in synched across, server, data centers or regions.
|
||||
|
||||
Depending on your projects needs our general recommendation is to run Zitadel across multiple availability zones in the same region or across multiple regions.
|
||||
Make sure to read our [Production Guide](/self-hosting/manage/production#prefer-postgre-sql) before you decide to use it.
|
||||
Consult the [Postgres documentation](https://www.postgresql.org/docs/) for more details.
|
||||
|
||||
## Scalability
|
||||
|
||||
Zitadel can be scaled in a linear fashion in multiple dimensions.
|
||||
|
||||
- Vertical on your compute infrastructure
|
||||
- Horizontal in a region
|
||||
- Horizontal in multiple regions
|
||||
|
||||
Our customers can reuse the same already known binary or container and scale it across multiple server, data center and regions.
|
||||
To distribute traffic an already existing proxy infrastructure can be reused.
|
||||
Simply steer traffic by path, hostname, IP address or any other metadata to the Zitadel of your choice.
|
||||
|
||||
> To improve your service quality we recommend steering traffic by path to different Zitadel deployments
|
||||
> Feel free to [contact us](https://zitadel.com/contact/) for details
|
||||
|
||||
## Example Deployment Architecture
|
||||
|
||||
### Single Cluster / Region
|
||||
|
||||
A Zitadel Cluster is a highly available IAM system with each component critical for serving traffic laid out at least three times.
|
||||
Our storage layer (Postgres) is built for single region deployments.
|
||||
Hence our reference design for Kubernetes is to have three application nodes and one storage node.
|
||||
|
||||
> If you are using a serverless offering like Google Cloud Run you can scale Zitadel from 0 to 1000 Pods without the need of deploying the node across multiple availability zones.
|
||||
|
||||
|
||||
|
||||
### Multi Cluster / Region
|
||||
|
||||
To scale Zitadel across regions it is recommend to create at least three clusters.
|
||||
Each cluster is a fully independent ZITADEL setup.
|
||||
To keep the data in sync across all clusters, we recommend using Postgres with read-only replicas as a storage layer.
|
||||
Make sure to read our [Production Guide](/self-hosting/manage/production#prefer-postgre-sql) before you decide to use it.
|
||||
Consult the [Postgres documentation](https://www.postgresql.org/docs/current/high-availability.html) for more details.
|
||||
|
||||
|
||||
|
||||
## Zero Downtime Updates
|
||||
|
||||
Since an Identity system tends to be a critical piece of infrastructure, the "in place zero downtime update" is a well needed feature.
|
||||
Zitadel is built in a way that upgrades can be executed without downtime by just updating to a more recent version.
|
||||
|
||||
The common update involves the following steps and do not need manual intervention of the operator:
|
||||
|
||||
- Keep the old version running
|
||||
- Deploy the version in parallel to the old version
|
||||
- The new version will start ...
|
||||
- by updating databases schemas if needed
|
||||
- participate in the leader election for background jobs
|
||||
- As soon as the new version is ready to accept traffic it will signal this on the readiness endpoint `/debug/ready`
|
||||
- At this point your network infrastructure can send traffic to the new version
|
||||
|
||||
Users who use [Kubernetes/Helm](/self-hosting/deploy/kubernetes) or serverless container services like Google Cloud Run can benefit from the fact the above process is automated.
|
||||
|
||||
<Callout>
|
||||
As a good practice we recommend creating Database Backups prior to an update.
|
||||
It is also recommend to read the release notes on GitHub before upgrading.
|
||||
Since Zitadel utilizes Semantic Versioning Breaking Changes of any kind will always increase the major version (e.g Version 2 would become Version 3).
|
||||
</Callout>
|
||||
Reference in New Issue
Block a user