The unit economics of a false positive

Abstract visualization representing the cost and weight of false positive alerts

Most engineering organizations track on-call incidents. Fewer track false positives as a distinct category. Fewer still have ever put a dollar figure on what a false positive actually costs. This is understandable. The cost feels diffuse and hard to attribute. But the arithmetic is not complicated, and when you do it, the numbers are uncomfortable enough to take seriously.

Let's build the model from first principles. An on-call SRE at a mid-size tech company earns somewhere around $175,000-210,000 in base salary, depending on geography and experience. Add employer costs (FICA, benefits, overhead, management allocation) and you are looking at roughly $300,000-360,000 in total annual cost. Divide by 2,080 working hours and you get approximately $144-173 per working hour.

But on-call pages don't happen only during working hours. A page at 2:30 AM doesn't cost you $150. It costs you considerably more, because the engineer was not working, they were sleeping, and the disruption to sleep has follow-on productivity effects through the next day. Industry estimates for the loaded cost of an after-hours wake-up, accounting for the page triage time, sleep disruption, and next-day productivity impact, typically range from $75-150 per incident. Let's use $90 as a conservative midpoint for the calculation below.

Building the annual false positive cost estimate

Take a team with 6 SREs on a weekly rotation. In a reasonably noisy monitoring environment, each engineer experiences roughly 8-15 false positive pages per week during their on-call rotation. Using 11 per week as a middle estimate:

  • 11 false positives per on-call week
  • 6 engineers, each on-call for roughly 8.7 weeks per year (52 weeks / 6)
  • Total team false positives per year: approximately 572
  • At $90 per false positive: $51,480 per year in direct on-call disruption cost

That figure covers only after-hours pages. It doesn't count daytime false positives, which interrupt deep work at a cost of 15-25 minutes per interruption in lost focus time. It doesn't count the follow-on effects on code velocity during on-call weeks. And it doesn't count any of the harder-to-quantify costs we'll discuss below.

We are not presenting this number as precise. The point is directional: a small team with average alert noise is absorbing tens of thousands of dollars annually in false positive cost, and it doesn't appear anywhere in budget discussions because it doesn't have a line item.

The second-order costs that don't appear in the model

Direct disruption cost is the easy part to calculate. The harder costs are structural.

On-call rotation quality is a real factor in engineering retention. A rotation that regularly wakes engineers for noise is not a neutral experience. It is a sustained stressor that accrues over months. The engineers most likely to leave because of it are often the most experienced ones, because they have the most market options. When an SRE with 3 years of deep system knowledge leaves, the replacement and ramp cost is typically in the range of $50,000-100,000 in recruiting fees and 6-12 months of reduced productivity while the replacement learns the system. False positive load is rarely cited as the primary reason for departure, but it frequently appears as a contributing factor in exit interviews at organizations that bother to ask.

Alert trust degradation is the other major second-order cost. When engineers are conditioned by experience to expect that most alerts are noise, their response to real alerts is slower. Not by much on any individual incident, but the pattern compounds. A 2-minute delay in starting investigation on a real P1, caused by the engineer's trained skepticism about whether this alert is "the real thing," can add 15-20 minutes to total incident duration if it delays the identification of root cause. At $10,000-50,000 per minute of downtime for user-impacting services (industry-realistic range for revenue-generating products), the cost math of a degraded SLO from delayed response far exceeds the direct false positive cost.

Why false positives are under-counted in practice

The reason organizations don't track false positive costs accurately is partly definitional and partly instrumentation. A false positive requires a judgment call: was this alert genuinely unfounded, or was it technically correct but representing a transient condition that self-resolved? Engineers making that call in a PagerDuty incident note are often inconsistent. Some mark self-resolving events as "no action required." Others create a brief investigation note. Others close the incident without comment.

Without consistent categorization, the data to build the cost model above doesn't exist. The false positive rate is unknown. Organizations know they have alert noise. They don't know how much, in what categories, at what frequency, or at what cost. The problem is felt but unmeasured, which makes it resistant to prioritization.

Instrumenting it properly requires two things: a consistent taxonomy for incident resolution notes (at minimum: "action taken," "no action required," "self-resolved"), and a periodic report that surfaces false positive rates by alert rule and service. Neither is technically difficult. Both require organizational discipline to maintain.

Where suppression fits in the cost structure

Suppression doesn't eliminate false positives. It prevents some of them from reaching the engineer before the cost is incurred. The distinction matters: an alert that fires internally but gets suppressed before PagerDuty is not a false positive in the cost sense. No engineer was woken. No $90 was spent. No focus was interrupted. The alert exists in ObsrvHQ's review log, where a platform engineer can evaluate it during working hours without urgency.

This is the economic logic behind detection-layer suppression. The false positive event still happened at the monitoring layer. The cost was intercepted before it transferred to the human layer. Over a year of operation across a team like the one described above, suppressing a meaningful fraction of the 572 annual false positive wake-ups changes the math substantially. We're not saying suppression eliminates the cost entirely. Some suppression errors will happen, and a genuine alert will occasionally be held back. The calibration of the suppression sensitivity is exactly the operational variable you tune during onboarding.

Making the case internally

If you are trying to make a business case for investing in alert quality, the unit economics approach is more effective than describing alert fatigue in qualitative terms. Engineering managers and finance partners respond to numbers. The calculation above is defensible and conservative, and it excludes the second-order costs, which means the real number is likely higher.

The conversation worth having: how many false positive pages did we generate last quarter? What was the loaded on-call cost? What would we do with the engineering time that is currently absorbed by noise triage? Framed this way, alert quality stops being an infrastructure hygiene concern and becomes a resource allocation question with a measurable answer.