chore: rehaul DevX (#10571)

# Which Problems Are Solved

Replaces Turbo by Nx and lays the foundation for the next CI
improvements. It enables using Nx Cloud to speed the up the pipelines
that affect any node package.
It streamlines the dev experience for frontend and backend developers by
providing the following commands:

| Task | Command | Notes |
|------|---------|--------|
| **Production** | `nx run PROJECT:prod` | Production server |
| **Develop** | `nx run PROJECT:dev` | Hot reloading development server
|
| **Test** | `nx run PROJECT:test` | Run all tests |
| **Lint** | `nx run PROJECT:lint` | Check code style |
| **Lint Fix** | `nx run PROJECT:lint-fix` | Auto-fix style issues |

The following values can be used for PROJECT:

- @zitadel/zitadel (root commands)
- @zitadel/api,
- @zitadel/login,
- @zitadel/console,
- @zitadel/docs,
- @zitadel/client
- @zitadel/proto

The project names and folders are streamlined:

| Old Folder | New Folder |
| --- | --- |
| ./e2e | ./tests/functional-ui |
| ./load-test | ./benchmark |
| ./build/zitadel | ./apps/api |
| ./console | ./apps/console (postponed so the PR is reviewable) |  

Also, all references to the TypeScript repo are removed so we can
archive it.

# How the Problems Are Solved

- Ran `npx nx@latest init`
- Replaced all turbo.json by project.json and fixed the target configs
- Removed Turbo dependency
- All JavaScript related code affected by a PRs changes is
quality-checked using the `nx affected` command
- We move PR checks that are runnable using Nx into the `check`
workflow. For workflows where we don't use Nx, yet, we restore
previously built dependency artifacts from Nx.
- We only use a single and easy to understand dev container
- The CONTRIBUTING.md is streamlined
- The setup with a generated client pat is orchestrated with Nx
- Everything related to the TypeScript repo is updated or removed. A
**Deploy with Vercel** button is added to the docs and the
CONTRIBUTING.md.

# Additional Changes

- NPM package names have a consistent pattern.
- Docker bake is removed. The login container is built and released like
the core container.
- The integration tests build the login container before running, so
they don't rely on the login container action anymore. This fixes
consistently failing checks on PRs from forks.
- The docs build in GitHub actions is removed, as we already build on
Vercel.

# Additional Context

- Internal discussion:
https://zitadel.slack.com/archives/C087ADF8LRX/p1756277884928169
- Workflow dispatch test:
https://github.com/zitadel/zitadel/actions/runs/17760122959

---------

Co-authored-by: Florian Forster <[email protected]>
Co-authored-by: Tim Möhlmann <[email protected]>
Co-authored-by: Copilot <[email protected]>
This commit is contained in:
Elio Bischof
2025-10-08 10:27:02 +02:00
committed by GitHub
co-authored by Florian Forster Tim Möhlmann Copilot
parent e47acaeef2
commit f69a6ed4f3
236 changed files with 5708 additions and 6549 deletions
+3
View File
@@ -0,0 +1,3 @@
PGUSER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_HOST_AUTH_METHOD=trust
-20
View File
@@ -1,20 +0,0 @@
FROM mcr.microsoft.com/devcontainers/typescript-node:20-bookworm
ENV SHELL=/bin/bash \
DEBIAN_FRONTEND=noninteractive \
LANG=C.UTF-8 \
LC_ALL=C.UTF-8 \
CI=1 \
PNPM_HOME=/home/node/.local/share/pnpm \
PATH=/home/node/.local/share/pnpm:$PATH
RUN apt-get update && \
apt-get --no-install-recommends install -y \
# Cypress dependencies
libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2 libxtst6 xauth xvfb && \
apt-get clean && \
corepack enable && COREPACK_ENABLE_DOWNLOAD_PROMPT=0 corepack prepare [email protected] --activate
COPY --chown=node:node commands /commands
USER node
@@ -1,2 +0,0 @@
*
!commands
@@ -1,39 +0,0 @@
#!/bin/bash
if [ "$FAIL_COMMANDS_ON_ERRORS" == "true" ]; then
set -e
fi
echo
echo
echo
echo -e "THANKS FOR CONTRIBUTING TO ZITADEL 🚀"
echo
echo "Your dev container is configured for fixing login integration tests."
echo "The login is running in a separate container with the same configuration."
echo "It calls the mock-zitadel container which provides a mocked Zitadel gRPC API."
echo
echo "Also the test suite is configured correctly."
echo "For example, run a single test file:"
echo "pnpm cypress run --spec integration/integration/login.cy.ts"
echo
echo "You can also run the test interactively."
echo "However, this is only possible from outside the dev container."
echo "On your host machine, run:"
echo "cd apps/login"
echo "pnpm cypress open"
echo
echo "If you want to change the login code, you can replace the login container by a hot reloading dev server."
echo "docker stop login-integration"
echo "pnpm turbo dev"
echo "Navigate to the page you want to fix, for example:"
echo "http://localhost:3001/ui/v2/login/verify?userId=221394658884845598&code=abc"
echo "Change some code and reload the page for instant feedback."
echo
echo "When you are done, make sure all integration tests pass:"
echo "pnpm cypress run"
echo
if [ "$FAIL_COMMANDS_ON_ERRORS" != "true" ]; then
exit 0
fi
@@ -1,18 +0,0 @@
#!/bin/bash
if [ "$FAIL_COMMANDS_ON_ERRORS" == "true" ]; then
echo "Running in fail-on-errors mode"
set -e
fi
pnpm install --frozen-lockfile \
--filter @zitadel/login \
--filter @zitadel/client \
--filter @zitadel/proto \
--filter zitadel-monorepo
pnpm cypress install
pnpm test:integration:login
if [ "$FAIL_COMMANDS_ON_ERRORS" != "true" ]; then
exit 0
fi
@@ -1,30 +0,0 @@
#!/bin/bash
if [ "$FAIL_COMMANDS_ON_ERRORS" == "true" ]; then
set -e
fi
echo
echo
echo
echo -e "THANKS FOR CONTRIBUTING TO ZITADEL 🚀"
echo
echo "Your dev container is configured for fixing linting and unit tests."
echo "No other services are running alongside this container."
echo
echo "To fix all auto-fixable linting errors, run:"
echo "pnpm turbo lint:fix"
echo
echo "To watch console linting errors, run:"
echo "pnpm turbo watch lint --filter console"
echo
echo "To watch @zitadel/client unit test failures, run:"
echo "pnpm turbo watch test:unit --filter @zitadel/client"
echo
echo "To watch @zitadel/login relevant unit tests and linting failures, run:"
echo "pnpm turbo watch lint test:unit --filter @zitadel/login..."
echo
if [ "$FAIL_COMMANDS_ON_ERRORS" != "true" ]; then
exit 0
fi
@@ -1,12 +0,0 @@
#!/bin/bash
if [ "$FAIL_COMMANDS_ON_ERRORS" == "true" ]; then
set -e
fi
pnpm install --frozen-lockfile --recursive
pnpm turbo lint test:unit
if [ "$FAIL_COMMANDS_ON_ERRORS" != "true" ]; then
exit 0
fi
-29
View File
@@ -1,29 +0,0 @@
{
"$schema": "https://raw.githubusercontent.com/devcontainers/spec/refs/heads/main/schemas/devContainer.schema.json",
"name": "Base: Build and Run the Components you need",
"dockerComposeFile": "docker-compose.yaml",
"service": "devcontainer",
"runServices": [
"devContainer",
"db"
],
"workspaceFolder": "/workspaces",
"remoteEnv": {
"DISPLAY": ""
},
"forwardPorts": [
3000,
3001,
4200,
8080
],
"onCreateCommand": "pnpm install --frozen-lockfile --recursive --prefer-offline",
"features": {
"ghcr.io/devcontainers/features/go:1": {
"version": "1.24"
},
"ghcr.io/guiyomh/features/golangci-lint:0": {},
"ghcr.io/jungaretti/features/make:1": {},
"ghcr.io/devcontainers/features/docker-outside-of-docker": {}
}
}
-198
View File
@@ -1,198 +0,0 @@
services:
devcontainer:
container_name: devcontainer
build:
context: ../base
volumes:
- ../../:/workspaces:cached
command: sleep infinity
working_dir: /workspaces
environment:
ZITADEL_DATABASE_POSTGRES_HOST: db
ZITADEL_EXTERNALSECURE: false
db:
container_name: db
image: postgres:17.0-alpine3.19
restart: unless-stopped
volumes:
- postgres-data:/var/lib/postgresql/data
environment:
PGUSER: postgres
POSTGRES_PASSWORD: postgres
healthcheck:
test: [ "CMD-SHELL", "pg_isready" ]
interval: "10s"
timeout: "30s"
retries: 5
start_period: "20s"
ports:
- "5432:5432"
zitadel:
container_name: zitadel
image: "${ZITADEL_TAG:-ghcr.io/zitadel/zitadel:latest}"
command: 'start-from-init --masterkey "MasterkeyNeedsToHave32Characters" --config /zitadel.yaml --steps /zitadel.yaml'
volumes:
- ../../apps/login/acceptance/pat:/pat:delegated
- ../../apps/login/acceptance/zitadel.yaml:/zitadel.yaml:cached
network_mode: service:devcontainer
healthcheck:
test:
- CMD
- /app/zitadel
- ready
- --config
- /zitadel.yaml
depends_on:
db:
condition: "service_healthy"
configure-login:
container_name: configure-login
restart: no
build:
context: ../../apps/login/acceptance/setup
dockerfile: ../go-command.Dockerfile
entrypoint: "./setup.sh"
network_mode: service:devcontainer
environment:
PAT_FILE: /pat/zitadel-admin-sa.pat
ZITADEL_API_URL: http://localhost:8080
WRITE_ENVIRONMENT_FILE: /login-env/.env.test.local
SINK_EMAIL_INTERNAL_URL: http://sink:3333/email
SINK_SMS_INTERNAL_URL: http://sink:3333/sms
SINK_NOTIFICATION_URL: http://sink:3333/notification
LOGIN_BASE_URL: http://localhost:3000/ui/v2/login/
ZITADEL_API_DOMAIN: localhost
ZITADEL_ADMIN_USER: [email protected]
volumes:
- ../../apps/login/acceptance/pat:/pat:cached # Read the PAT file from zitadels setup
- ../../apps/login:/login-env:delegated # Write the environment variables file for the login
depends_on:
zitadel:
condition: "service_healthy"
login-acceptance:
container_name: login
image: "${LOGIN_TAG:-ghcr.io/zitadel/zitadel-login:latest}"
network_mode: service:devcontainer
volumes:
- ../../apps/login/.env.test.local:/env-files/.env:cached
depends_on:
configure-login:
condition: service_completed_successfully
mock-notifications:
container_name: mock-notifications
build:
context: ../../apps/login/acceptance/sink
dockerfile: ../go-command.Dockerfile
args:
- LOGIN_TEST_ACCEPTANCE_GOLANG_TAG=${LOGIN_TEST_ACCEPTANCE_GOLANG_TAG:-golang:1.24-alpine}
environment:
PORT: '3333'
command:
- -port
- '3333'
- -email
- '/email'
- -sms
- '/sms'
- -notification
- '/notification'
ports:
- "3333:3333"
depends_on:
configure-login:
condition: "service_completed_successfully"
mock-oidcrp:
container_name: mock-oidcrp
build:
context: ../../apps/login/acceptance/oidcrp
dockerfile: ../go-command.Dockerfile
args:
- LOGIN_TEST_ACCEPTANCE_GOLANG_TAG=${LOGIN_TEST_ACCEPTANCE_GOLANG_TAG:-golang:1.24-alpine}
network_mode: service:devcontainer
environment:
API_URL: 'http://localhost:8080'
API_DOMAIN: 'localhost'
PAT_FILE: '/pat/zitadel-admin-sa.pat'
LOGIN_URL: 'http://localhost:3000/ui/v2/login'
ISSUER: 'http://localhost:8000'
HOST: 'localhost'
PORT: '8000'
SCOPES: 'openid profile email'
volumes:
- ../../apps/login/acceptance/pat:/pat:cached
depends_on:
configure-login:
condition: "service_completed_successfully"
# mock-oidcop:
# container_name: mock-oidcop
# build:
# context: ../../apps/login/acceptance/idp/oidc
# dockerfile: ../../go-command.Dockerfile
# args:
# - LOGIN_TEST_ACCEPTANCE_GOLANG_TAG=${LOGIN_TEST_ACCEPTANCE_GOLANG_TAG:-golang:1.24-alpine}
# network_mode: service:devcontainer
# environment:
# API_URL: 'http://localhost:8080'
# API_DOMAIN: 'localhost'
# PAT_FILE: '/pat/zitadel-admin-sa.pat'
# SCHEMA: 'http'
# HOST: 'localhost'
# PORT: "8004"
# volumes:
# - "../apps/login/packages/acceptance/pat:/pat:cached"
# depends_on:
# configure-login:
# condition: "service_completed_successfully"
mock-samlsp:
container_name: mock-samlsp
build:
context: ../../apps/login/acceptance/samlsp
dockerfile: ../go-command.Dockerfile
args:
- LOGIN_TEST_ACCEPTANCE_GOLANG_TAG=${LOGIN_TEST_ACCEPTANCE_GOLANG_TAG:-golang:1.24-alpine}
network_mode: service:devcontainer
environment:
API_URL: 'http://localhost:8080'
API_DOMAIN: 'localhost'
PAT_FILE: '/pat/zitadel-admin-sa.pat'
LOGIN_URL: 'http://localhost:3000/ui/v2/login'
IDP_URL: 'http://localhost:8080/saml/v2/metadata'
HOST: 'http://localhost:8001'
PORT: '8001'
volumes:
- "../apps/login/packages/acceptance/pat:/pat:cached"
depends_on:
configure-login:
condition: "service_completed_successfully"
# mock-samlidp:
# container_name: mock-samlidp
# build:
# context: ../../apps/login/acceptance/idp/saml
# dockerfile: ../../go-command.Dockerfile
# args:
# - LOGIN_TEST_ACCEPTANCE_GOLANG_TAG=${LOGIN_TEST_ACCEPTANCE_GOLANG_TAG:-golang:1.24-alpine}
# network_mode: service:devcontainer
# environment:
# API_URL: 'http://localhost:8080'
# API_DOMAIN: 'localhost'
# PAT_FILE: '/pat/zitadel-admin-sa.pat'
# SCHEMA: 'http'
# HOST: 'localhost'
# PORT: "8003"
# volumes:
# - "../apps/login/packages/acceptance/pat:/pat"
# depends_on:
# configure-login:
# condition: "service_completed_successfully"
volumes:
postgres-data:
+6
View File
@@ -0,0 +1,6 @@
#!/bin/sh
# This script is used to run Docker Compose commands with the appropriate compose project set.
# This makes sure the commands don't break on different workspace_root folder names.
export DEVCONTAINER_LOCAL_WORKSPACE_ROOT=${DEVCONTAINER_LOCAL_WORKSPACE_ROOT:-${NX_WORKSPACE_ROOT:-zitadel}}
docker compose -p ${DEVCONTAINER_LOCAL_WORKSPACE_ROOT##*/}_devcontainer "$@"
+57
View File
@@ -0,0 +1,57 @@
{
"$schema": "https://raw.githubusercontent.com/devcontainers/spec/refs/heads/main/schemas/devContainer.schema.json",
"name": "Zitadel Devcontainer",
"dockerComposeFile": "docker-compose.yaml",
"service": "devcontainer",
"runServices": [
"devcontainer"
],
"workspaceFolder": "/workspaces/zitadel",
"forwardPorts": [
3000,
4200,
8080,
3100
],
"portsAttributes": {
"3000": {
"label": "Login",
"onAutoForward": "notify"
},
"4200": {
"label": "Console",
"onAutoForward": "notify"
},
"8080": {
"label": "API",
"onAutoForward": "notify"
},
"3100": {
"label": "Docs",
"onAutoForward": "notify"
}
},
"features": {
"ghcr.io/devcontainers/features/go:1": {
"version": "1.24"
},
"ghcr.io/devcontainers/features/docker-outside-of-docker": {},
"ghcr.io/schlich/devcontainer-features/cypress:1": {
"version": "14"
}
},
"containerEnv": {
"DEVCONTAINER_LOCAL_WORKSPACE_ROOT": "${localWorkspaceFolder}"
},
"customizations": {
"vscode": {
"extensions": [
"golang.Go",
"Angular.ng-template",
"dbaeumer.vscode-eslint",
"esbenp.prettier-vscode",
"nrwl.angular-console"
]
}
}
}
+88
View File
@@ -0,0 +1,88 @@
services:
devcontainer:
image: mcr.microsoft.com/devcontainers/typescript-node:22-bookworm
volumes:
- ../:/workspaces/zitadel:cached
command: sleep infinity
working_dir: /workspaces/zitadel
environment:
SHELL: /bin/bash
DEBIAN_FRONTEND: noninteractive
LANG: C.UTF-8
LC_ALL: C.UTF-8
COREPACK_ENABLE_DOWNLOAD_PROMPT: 0
DEVCONTAINER_DB_HOST: db
DEVCONTAINER_API_HOST: api
DEVCONTAINER_LOGIN_HOST: login
DEVCONTAINER_LOGIN_API_MOCK_HOST: login-api-mock
DEVCONTAINER_DB_API_INTEGRATION_HOST: db-api-integration
DEVCONTAINER_DB_FUNCTIONAL_UI_HOST: db-functional-ui
ports:
- "8080:8080"
- "3000:3000"
- "4200:4200"
db:
image: postgres:17
restart: unless-stopped
env_file:
- ./.env.db
volumes:
- db-data-dev:/var/lib/postgresql/data
healthcheck:
test: [ "CMD-SHELL", "pg_isready", "-U", "postgres" ]
interval: "10s"
timeout: "30s"
retries: 5
start_period: "20s"
ports:
- 5432:5432
db-api-integration:
image: 'postgres:17'
env_file:
- ./.env.db
environment:
PGPORT: 5433
command: postgres -c shared_preload_libraries=pg_stat_statements -c pg_stat_statements.track=all -c shared_buffers=1GB -c work_mem=16MB -c effective_io_concurrency=100 -c wal_level=minimal -c archive_mode=off -c max_wal_senders=0
healthcheck:
test: [ "CMD-SHELL", "pg_isready", "-U", "postgres" ]
interval: '10s'
timeout: '30s'
retries: 5
start_period: '20s'
ports:
- 5433:5433
cache-api-integration:
image: 'redis:8'
ports:
- 6379:6379
db-functional-ui:
image: 'postgres:17'
env_file:
- ./.env.db
environment:
PGPORT: 5434
healthcheck:
test: [ "CMD-SHELL", "pg_isready", "-U", "postgres" ]
interval: '10s'
timeout: '30s'
retries: 5
start_period: '20s'
ports:
- "5434:5434"
login-api-mock:
build:
context: ../apps/login/integration/api-mock
additional_contexts:
zitadel-protos: ../proto
ports:
- 22220:22220
- 22222:22222
volumes:
db-data-dev:
@@ -1,26 +0,0 @@
{
"$schema": "https://raw.githubusercontent.com/devcontainers/spec/refs/heads/main/schemas/devContainer.schema.json",
"name": "Login Integration",
"dockerComposeFile": [
"./docker-compose.yaml"
],
"service": "login-integration-dev",
"runServices": [
"login-integration"
],
"workspaceFolder": "/workspaces/apps/login",
"forwardPorts": [
22220,
22222,
3001
],
"remoteEnv": {
"FAIL_COMMANDS_ON_ERRORS": "${localEnv:FAIL_COMMANDS_ON_ERRORS}",
"DISPLAY": ""
},
"updateContentCommand": "/commands/login-integration.update-content.sh",
"postAttachCommand": "/commands/login-integration.post-attach.sh",
"features": {
"ghcr.io/devcontainers/features/docker-outside-of-docker": {}
}
}
@@ -1,35 +0,0 @@
services:
login-integration-dev:
extends:
file: ../base/docker-compose.yaml
service: devcontainer
container_name: login-integration-dev
env_file: ../../apps/login/.env.test
environment:
CORE_MOCK_STUBS_URL: http://localhost:22220/v1/stubs
LOGIN_BASE_URL: http://localhost:3001/ui/v2/login
CYPRESS_CACHE_FOLDER: /workspaces/.artifacts/cypress
network_mode: service:mock-zitadel
depends_on:
login-integration:
condition: service_healthy
login-integration:
container_name: login-integration
image: "${LOGIN_TAG:-ghcr.io/zitadel/zitadel-login:latest}"
build:
context: ../..
dockerfile: build/login/Dockerfile
env_file: ../../apps/login/.env.test
network_mode: service:mock-zitadel
mock-zitadel:
container_name: mock-zitadel
build:
context: ../../apps/login/integration/core-mock
additional_contexts:
- zitadel-protos=../../proto
ports:
- 22220:22220
- 22222:22222
- 3001:3001
@@ -1,22 +0,0 @@
{
"$schema": "https://raw.githubusercontent.com/devcontainers/spec/refs/heads/main/schemas/devContainer.base.schema.json",
"image": "mcr.microsoft.com/devcontainers/typescript-node:20-bookworm",
"name": "Login Subtree Container - Use the Login As If You Would Have Forked the Mirror Repo",
"workspaceFolder": "/login",
"workspaceMount": "source=${localWorkspaceFolder}/apps/login,target=/login,type=bind,consistency=cached",
"mounts": [],
"forwardPorts": [
22220,
22222,
3000,
3001
],
"features": {
"ghcr.io/devcontainers/features/go:1": {
"version": "1.24"
},
"ghcr.io/guiyomh/features/golangci-lint:0": {},
"ghcr.io/jungaretti/features/make:1": {},
"ghcr.io/devcontainers/features/docker-outside-of-docker": {}
}
}
+14
View File
@@ -0,0 +1,14 @@
{
"$schema": "../node_modules/nx/schemas/project-schema.json",
"name": "@zitadel/devcontainer",
"targets": {
"compose": {
"description": "Runs arbitrary Docker Compose commands with dev container support. All running services are manageable and addressable from in- and outside the dev container. The services are defined in .devcontainer/docker-compose.yaml.",
"executor": "nx:run-commands",
"options": {
"cwd": "{projectRoot}",
"command": "./compose.sh"
}
}
}
}
@@ -1,20 +0,0 @@
{
"$schema": "https://raw.githubusercontent.com/devcontainers/spec/refs/heads/main/schemas/devContainer.schema.json",
"name": "Turbo Lint and Unit Tests",
"dockerComposeFile": [
"../base/docker-compose.yaml"
],
"service": "devcontainer",
"runServices": [
"devcontainer"
],
"workspaceFolder": "/workspaces",
"forwardPorts": [
3001
],
"remoteEnv": {
"FAIL_COMMANDS_ON_ERRORS": "${localEnv:FAIL_COMMANDS_ON_ERRORS}"
},
"updateContentCommand": "/commands/turbo-lint-unit.update-content.sh",
"postAttachCommand": "/commands/turbo-lint-unit.post-attach.sh"
}