The most common structural problem we see in teams' alerting setups is not bad thresholds or missing alerts. It's the disconnect between how they've defined service level objectives and how their alert rules are actually written. The two exist in parallel, each maintained by different people, with no mechanical connection between them.
You can have a carefully crafted SLO for your API's availability and a well-tuned error rate alert rule, and those two things can contradict each other entirely. The SLO might allow for a 0.1% error rate over 30 days, meaning a sustained 0.05% error rate for 3 hours is completely within budget. The alert rule might fire on a 60-second window of 0.03% errors because that's the threshold someone chose two years ago. The alert fires. The SLO has not been violated. An engineer is paged for something that, by the explicit reliability commitment your team has made, is not an incident.
This gap is fundamental, and closing it produces more signal-consistent alerting than any amount of threshold tuning.
What a good SLO definition actually contains
Before connecting SLOs to alerting, you need SLOs that are actionable. A lot of SLOs are written at a level of abstraction that makes them hard to use as alert inputs.
"99.9% availability" is not an actionable SLO unless you've defined availability precisely: is it measured as successful HTTP requests by count? By percentage of 1-minute windows where the service returned at least one successful response? Over what time period? For which set of endpoints? These definitional choices are not academic; they determine what measurement you'd use in an alert rule.
A more actionable form: "HTTP 5xx rate for endpoints matching /api/v2/* must not exceed 0.1% measured as a 5-minute average, maintained over a 30-day rolling window, with an error budget of 43.2 minutes total downtime per month." Now you have a measurement, a window, and a budget. These can become alert inputs.
The error budget framing is particularly important for connecting SLOs to alerting. Your alert rules don't need to fire every time a metric changes. They need to fire when the rate of change threatens your error budget. A 15-minute window of elevated errors that consumes 0.1% of your monthly budget is different from a 15-minute window that consumes 4% of your monthly budget, even if the raw error rate looks the same.
Burn rate alerts: connecting the two systems
Burn rate alerting is the mechanism that connects SLO definitions to alert rules. The core idea: instead of alerting on a metric threshold directly, you alert when the error budget is being consumed faster than a sustainable rate.
A 30-day error budget is sustainable if you consume it at rate 1x. If you're consuming it at rate 2x, you'll exhaust the budget in 15 days. At rate 14x, you'll exhaust the budget in about 2 days. The burn rate thresholds that warrant a page are typically in the higher range, around 14x over a short window (1 hour), which indicates a serious ongoing incident, or a lower rate like 3x-5x over a longer window (6 hours), which indicates sustained degradation that won't cause immediate SLO failure but needs attention soon.
This structure produces a very different alert firing pattern than raw threshold rules. A spike that's 15x burn rate for 20 minutes and then self-resolves will not consume enough budget to trigger the 6-hour window alert, and will only briefly touch the 1-hour window alert. The on-call engineer may get a brief notification that gets auto-resolved, or may not get paged at all if the spike duration is below the minimum sustained window. That's often the correct behavior: a 20-minute spike that didn't actually threaten the SLO is not an incident, even if it crossed a threshold rule's line.
Where suppression connects to SLO context
ObsrvHQ can ingest your SLO definitions and use burn rate as a suppression factor. When a metric-level alert rule fires, we check the current error budget state for that service. If the service is consuming budget at a rate that's comfortably within sustainable bounds, the alert is a candidate for suppression. If the service is burning budget at an elevated rate, suppression is released and the page escalates normally.
This is not a substitute for proper burn rate alerts. We'd recommend having proper burn rate alerts as your primary SLO-impact signal, with the metric-level rules as a faster-response secondary layer. What the SLO context integration adds is a sanity check: don't wake an engineer for a metric fluctuation when the SLO impact of that fluctuation is demonstrably negligible.
In practice, this catches a category of alert that burn rate alerts don't: rules that fire on absolute metric values rather than rates, like "queue depth above N" or "active connections above N." These rules don't have a natural SLO translation, but the SLO context lets us ask: is this service currently outside its acceptable operating envelope according to its own stated reliability targets? If not, the page is suspicious.
The SLO review problem
One friction point teams encounter is that SLOs age out of accuracy the same way alert rules do. An availability target that was appropriate when the service was handling 50 requests/second may be the wrong target at 500 requests/second. A latency SLO written for a synchronous request path may no longer reflect what customers actually experience after the team migrated to async processing.
We're not saying SLOs are hard to maintain and therefore not worth having. They're still worth having, and they're not that hard to maintain if you build review into your regular cadence. A quarterly SLO review, timed with your on-call rotation retrospective, is enough for most services. The question to ask at each review: does this SLO still reflect what a customer would consider a bad experience? If the service has changed significantly since the SLO was written, update the SLO before it becomes the wrong input to your alerting system.
The teams that get the most value from SLO-connected alerting treat their SLOs as living documents rather than compliance artifacts. They update them when services change. They retire SLOs for decommissioned features. They add SLOs for new critical paths rather than waiting until the service causes an incident before writing down what "working correctly" means.
A path from disconnected to connected
If you're starting from a state where your SLOs and your alert rules have no connection, the path to connecting them doesn't require replacing everything at once. Start with one service. Write a proper SLO definition for it. Add burn rate recording rules for that SLO in Prometheus. Add burn rate alert rules. Check how often those new burn rate alerts fire compared to the existing threshold rules for the same service. You'll almost certainly find the burn rate alerts are much quieter, and that the threshold rules were firing on things that didn't matter.
Once you have that comparison for one service, you have a data point you can bring to the team: "Our threshold rules for the payment service fired 43 times last month. Our burn rate alerts fired 3 times, and all 3 were real incidents." That conversation is much easier to have when it's grounded in actual numbers from your own environment rather than abstract principles.
The goal is not to replace all threshold rules with burn rate rules. Some metrics don't have natural SLO translations. Some threshold rules serve purposes other than customer-impact detection, like capacity planning signals or dependency health checks. The goal is to make sure that the threshold rules that are waking people up at night are connected to outcomes that actually matter, and that the ones that aren't are suppressed or removed.