The alert rule most teams write first is some variant of: "if metric X exceeds threshold Y for duration Z, fire." This is rational. It is clear, it is easy to reason about, and it covers the most obvious failure modes. The problem is that modern distributed systems fail in ways that don't look like single metrics crossing thresholds. They fail as correlated degradation across multiple dimensions simultaneously, and univariate alerting, by design, cannot see that pattern.
This is not a theoretical gap. When we look at incident postmortems involving non-trivial detection latency, the pattern is consistent: the individual metrics all stayed below their individual thresholds for long enough that no alert fired, while the composite picture was clearly anomalous to anyone who happened to be watching all the metrics at once. The incident was visible in the data. It was invisible to the alerting layer.
What a real incident looks like in metric space
Take a concrete scenario. A background job processing service starts experiencing memory pressure due to a gradual object accumulation bug introduced in a recent deploy. At 2 AM, the process starts exhibiting the following pattern over a 25-minute window:
- Heap memory usage climbs from 62% to 78% of container limit (threshold: 90%)
- GC pause duration increases from 12ms average to 47ms average (no alert rule)
- Job processing throughput drops from 340 jobs/min to 190 jobs/min (threshold: 100 jobs/min)
- Queue depth increases from 420 to 1,800 items (threshold: 5,000 items)
- Pod restart count: 0 (threshold: 1)
No individual alert fires. The job processing throughput is down by nearly half. The queue is growing. The process is on a trajectory toward OOM that will cause a pod restart. But each metric in isolation looks like "trending in a concerning direction but not yet at threshold."
A human looking at these five metrics together at 2:07 AM would recognize this as a developing incident with perhaps 30-40 minutes until the pod crashes and job processing stops entirely. The univariate alerting layer sees nothing unusual.
Why statistical correlation catches what thresholds miss
Multivariate anomaly detection works by evaluating a set of related metrics jointly, looking for patterns of co-movement that are statistically unusual given historical behavior. The key insight is that many "normal" events cause individual metrics to move in isolation: CPU spikes during a batch job, memory climbs during cache warm-up, queue depth increases during traffic bursts. These are expected. What is not expected is all of them moving in the same direction simultaneously outside of known-normal operational patterns.
The statistical approach we use at ObsrvHQ is based on building a joint distribution model over related metric sets. For a given service, we learn the typical correlation structure across its key signals over a rolling baseline window. An incident typically manifests as a deviation from that correlation structure, not just a deviation in any single metric's magnitude.
Concretely: if memory and CPU typically have low correlation for a given service (they move somewhat independently under normal load), and suddenly they start moving in lockstep in an unusual direction, that correlation shift is itself a signal. We can detect it even when neither metric has crossed an absolute threshold.
The baseline problem with multivariate approaches
Multivariate detection is not free. The main operational challenge is baseline stability. Joint distribution models for correlated metrics require longer observation windows to converge reliably than single-metric baselines. A single-metric ARIMA or exponential smoothing baseline can produce reasonable estimates after a few days of data. A joint covariance model over 8-12 metrics for a service needs at least 2-3 weeks to capture weekly periodicity and handle the typical variance in correlation structure across deployment events.
This creates a cold-start problem. New services have no baseline history. Services with recent significant architectural changes have baselines that may not reflect current behavior. Services with strong weekly or monthly seasonality need longer windows to capture that variation accurately.
In practice, we handle this by maintaining per-service baseline maturity flags. A service's baseline is marked as "provisional" for the first 21 days, and during that period we reduce the detection sensitivity to avoid false positives from incomplete correlation learning. We're not saying provisional detection is useless. It catches magnitude anomalies even without good correlation history. But the full correlation-aware detection quality doesn't kick in until the baseline is mature.
Choosing the right metric groupings
The value of multivariate detection depends significantly on how you group metrics. Not all metric combinations are meaningful. CPU and error rate for a stateless web service are causally connected in ways that CPU and GC pause duration might not be, depending on service architecture.
The most useful groupings correspond to causal pathways. For a request-serving service, the typical meaningful group is: request rate, error rate, latency percentiles, and pod-level resource utilization. These are connected because they all reflect the same underlying service health. An anomaly that moves all of them simultaneously almost always corresponds to a real user-impacting event.
For a data pipeline service, the useful group looks different: input queue depth, processing throughput, output queue depth, error rate, and process memory. These form the input-processing-output chain, and anomalies propagate forward through the chain with a time lag that is itself diagnostic.
We auto-generate suggested metric groupings from service topology metadata when it is available, typically from Kubernetes labels, service mesh telemetry, or explicit tagging in the ObsrvHQ dashboard. When topology metadata isn't available, we fall back to correlation-based grouping: metrics that have historically shown correlation are grouped together for joint evaluation.
When single-metric alerting is still the right tool
This is worth being clear about: we are not saying single-metric alerting should be replaced by multivariate detection. There are failure modes where a single metric crossing a threshold is unambiguous and requires immediate action regardless of any other context. A pod crash loop. Certificate expiry below 7 days. A disk filling past 95%. These are binary conditions where threshold alerting is exactly the right tool.
The gap we are addressing is the large class of developing incidents that don't manifest as clean threshold crossings. Performance degradation, resource pressure trajectories, cascading slowdowns from upstream dependencies. These are pattern anomalies that look normal metric-by-metric until they're too advanced to ignore. Multivariate detection catches them earlier, in the period when a platform team can intervene without end-user impact.
Integration with existing alert infrastructure
One practical consideration for teams evaluating this approach: multivariate detection doesn't require you to replace Prometheus and Alertmanager. We position ObsrvHQ as a parallel signal path. Your existing threshold-based rules continue to fire through Alertmanager as before. ObsrvHQ evaluates the multivariate correlation model independently and generates its own alert events when joint anomalies are detected. Both paths can route to PagerDuty or your on-call tooling.
The advantage of this parallel architecture is that you don't have to migrate or refactor existing alert rules during onboarding. The multivariate layer adds coverage for the incidents your threshold rules miss, while the threshold rules continue to provide fast, unambiguous notification for binary failure conditions.
We've found that teams typically run the two systems in parallel for 4-8 weeks, comparing what each catches, before deciding how much weight to shift toward correlation-based routing. Some teams end up with the multivariate detection as primary and thresholds as a safety backstop. Others keep thresholds as primary and use multivariate detection as early warning. The right balance depends on your service topology and on-call culture.