Files
zitadel/internal/api/http/middleware/metrics_interceptor.go
T
83854e80e0 fix(telemetry): record route patterns instead of request paths on HTTP metrics (#12557)
# Which Problems Are Solved

- The `uri` label of `http.server.request_count` and
`http.server.return_code_counter` holds the concrete request path, so
every object ID served over a REST path becomes its own metric series.
On our v4.16.2 instance `/debug/metrics` exposes 8,000 series, 4,467 of
which (56%) belong to `http_server_return_code_counter_total` alone,
spread over 4,390 distinct `uri` values such as
`/v2/sessions/385063742120926058`. The set only grows with the number of
objects the installation has served.
- Paths matching no route are recorded verbatim, so unauthenticated
scanner traffic (`/wp-json/`, `/login.php`, …) keeps adding series.
- The grpc-gateway answers `405 Method Not Allowed` without going
through the configured `errorHandler`, so those responses never reported
a route pattern, not even before the regression.

This is a regression, first shipped in v4.11.0; v4.10.1 is unaffected.
#9286 introduced the mechanism, #9523 extended it to unknown paths, and
#11435 removed it while reorganising the middleware packages:

> Removed setting of URI to context in metric middleware. There were
only setters and no getters. (Unused value)

The getter was `*recorder.RequestURI` in `RegisterRequestCounter` /
`RegisterRequestCodeCounter`, which the same PR replaced with
`baseURI(r)`. Both ends went at once, so nothing failed.

# How the Problems Are Solved

- `metrics.WithRequestURIPattern` / `metrics.SetRequestURIPattern` are
back, and `metrics.RequestURI` is now the single place deciding what to
label with: the pattern a router reported, or the requested path if none
did. `UnknownPath` moved to the metrics package so every surface shares
one constant.
- The HTTP metrics middleware prepares the context before passing the
request on, which is the half that got dropped.
- `setRequestURIPattern` in the grpc-gateway reports the pattern to
metrics again, not only to tracing, and is now also called on the `405`
branch.
- Requests routed by chi (the OIDC endpoints) fall back to the pattern
chi matched. That covers the RFC 7592 client configuration routes added
in #12315 (`/oauth/v2/register/{client_id}`), which are templated on the
client ID, and collapses unknown paths below the OIDC prefixes to
`UNKNOWN_PATH` as well.

Against the metrics dump from our instance,
`http_server_return_code_counter_total` drops from 4,467 series to
roughly 35, and the endpoint as a whole from 8,000 to about 3,570.

# Additional Changes

- Regression tests, which were missing and are the reason the revert
went unnoticed:
- `metrics`: resolution of the `uri` label, including that a pattern set
on a derived context reaches the middleware.
- `gateway`: every way the gateway can answer — success, error,
unroutable path, wrong method.
- `middleware`: both routing styles, i.e. a router reporting its own
pattern and a chi routed request.
- A doc comment on `SetRequestURIPattern` explaining that it is written
by routers and read back by the middleware through the context, so it
does not read as an unused setter again.

# Additional Context

- Closes #12556
- Restores #9286 and #9523
- Regressed in #11435

Co-authored-by: Cursor <[email protected]>
2026-08-07 17:23:56 +02:00

86 lines
2.5 KiB
Go

package middleware
import (
"net/http"
"github.com/go-chi/chi/v5"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"github.com/zitadel/zitadel/backend/v3/instrumentation"
"github.com/zitadel/zitadel/backend/v3/instrumentation/metrics"
)
func MetricsHandler(metricTypes []metrics.MetricType, ignoredMethods ...string) func(http.Handler) http.Handler {
return func(handler http.Handler) http.Handler {
return &Handler{
handler: handler,
methods: metricTypes,
filter: instrumentation.RequestFilter(ignoredMethods...),
}
}
}
type Handler struct {
handler http.Handler
methods []metrics.MetricType
filter otelhttp.Filter
}
// ServeHTTP implements [http.Handler]
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if len(h.methods) == 0 {
h.handler.ServeHTTP(w, r)
return
}
if !h.filter(r) {
// Simply pass through to the handler if a filter rejects the request
h.handler.ServeHTTP(w, r)
return
}
recorder := newStatusWriter(w)
// Requested paths contain object IDs and are therefore unbounded, so the router below
// gets the opportunity to report the route pattern it served the request with instead.
ctx := metrics.WithRequestURIPattern(r.Context())
r = r.WithContext(ctx)
h.handler.ServeHTTP(recorder, r)
if pattern := chiRoutePattern(r); pattern != "" {
metrics.SetRequestURIPattern(ctx, pattern)
}
if h.containsMetricsMethod(metrics.MetricTypeRequestCount) {
metrics.RegisterRequestCounter(recorder, r)
}
if h.containsMetricsMethod(metrics.MetricTypeTotalCount) {
metrics.RegisterTotalRequestCounter(r)
}
if h.containsMetricsMethod(metrics.MetricTypeStatusCode) {
metrics.RegisterRequestCodeCounter(recorder, r)
}
}
// chiRoutePattern returns the route pattern chi matched the request against, e.g.
// "/oauth/v2/register/{client_id}", or [metrics.UnknownPath] if chi routed the request but
// found no route for it. Routers that are not chi based report their pattern themselves
// through [metrics.SetRequestURIPattern]; for those an empty string is returned.
//
// chi only knows the pattern once it routed the request, so this must be called after the
// wrapped handler returned.
func chiRoutePattern(r *http.Request) string {
routeCtx := chi.RouteContext(r.Context())
if routeCtx == nil {
return ""
}
if pattern := routeCtx.RoutePattern(); pattern != "" {
return pattern
}
return metrics.UnknownPath
}
func (h *Handler) containsMetricsMethod(method metrics.MetricType) bool {
for _, m := range h.methods {
if m == method {
return true
}
}
return false
}