--- title: Observability description: "Collect traces, metrics, and logs from ZITADEL and Login containers using OpenTelemetry and Prometheus for observability monitoring" sidebar_label: Observability --- This guide covers how to collect traces, metrics, and logs from the Zitadel and Login containers. ## Zitadel Container ### Traces Zitadel can push traces to an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) using the [OTLP protocol](https://opentelemetry.io/docs/specs/otlp/) over gRPC: ```yaml env: - name: ZITADEL_TRACING_TYPE value: "otel" - name: ZITADEL_TRACING_ENDPOINT value: "otelcol-opentelemetry-collector.monitoring.svc.cluster.local:4317" - name: ZITADEL_TRACING_SERVICENAME value: "zitadel" ``` The `ZITADEL_TRACING_ENDPOINT` should point to your OpenTelemetry Collector's gRPC receiver, typically listening on port 4317. The collector can then forward traces to your preferred backend such as [Jaeger](https://www.jaegertracing.io/), [Grafana Tempo](https://grafana.com/oss/tempo/), or [OpenObserve](https://openobserve.ai/). You can deploy an OpenTelemetry Collector in your cluster using the [OpenTelemetry Operator](https://github.com/open-telemetry/opentelemetry-operator) or the [OpenTelemetry Collector Helm chart](https://github.com/open-telemetry/opentelemetry-helm-charts/tree/main/charts/opentelemetry-collector). ### Metrics Zitadel exposes [Prometheus](https://prometheus.io/) metrics at `/debug/metrics`. Metrics must be scraped by Prometheus. Pushing metrics is not supported. Enable the metrics endpoint: ```yaml metrics: enabled: true serviceMonitor: enabled: false ``` Set `serviceMonitor.enabled` to `true` if you are using the [Prometheus Operator](https://prometheus-operator.dev/). This creates a [ServiceMonitor](https://prometheus-operator.dev/docs/operator/api/#monitoring.coreos.com/v1.ServiceMonitor) resource that configures Prometheus to scrape the Zitadel pods automatically. For standard Prometheus without the Operator, add scrape annotations to the pods: ```yaml podAnnotations: prometheus.io/scrape: "true" prometheus.io/port: "8080" prometheus.io/path: "/debug/metrics" ``` Your Prometheus configuration must include a [scrape job](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) that discovers pods with these annotations. If you are using [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack), pod annotation discovery is typically enabled by default. ### Logs Zitadel writes structured JSON logs to stdout. Logs must be collected by a log collector running in your cluster. Pushing logs is not supported. Common log collectors include: - [Fluent Bit](https://fluentbit.io/) — Lightweight log processor and forwarder - [Vector](https://vector.dev/) — High-performance observability data pipeline - [Fluentd](https://www.fluentd.org/) — Unified logging layer - [Promtail](https://grafana.com/docs/loki/latest/clients/promtail/) — Log collector for Grafana Loki Configure your log collector to tail container logs from the Zitadel pods and forward them to your preferred backend such as [Grafana Loki](https://grafana.com/oss/loki/), [Elasticsearch](https://www.elastic.co/elasticsearch/), or [OpenObserve](https://openobserve.ai/). ## Login Container ### Traces The login container does not support tracing. Traces cannot be collected from the login container. ### Metrics The login container does not expose metrics. Prometheus metrics are not available for the login container. ### Logs The login container writes logs to stdout. Logs must be collected by a log collector running in your cluster. Pushing logs is not supported. Configure your log collector to tail container logs from the login pods. The login container runs as a separate deployment, so ensure your log collector is configured to collect from all pods in the namespace or specifically from pods with the login container labels.