Skip to content
Richard Cooper
Go back

Effect or cause: who owns the alert?

Every dashboard is green. CPU is fine, error rates are flat, the queues are empty, no alert has fired in a week. And customers can’t buy anything.

This is not a hypothetical, and it’s not a tooling failure. It’s the most common monitoring failure there is, and it happens because of a sentence that sounds reasonable right up until you take it apart: “that’s technical, so it’s IT’s to watch.”

The split that decides everything

Here’s the distinction I keep coming back to, and it’s the whole post in two lines:

Both are legitimate. Both need watching. They are not the same alert, and the mistake almost every organisation makes is assuming that because the cause is technical, the effect must be technical too — and therefore someone in IT should be the one woken up about it.

That inference is wrong, and it’s worth saying plainly: a technical root cause does not imply technical ownership. Most business-insight alerts have a technical cause somewhere underneath. Almost all of them do. Payment failures have a cause. Login failures have a cause. That tells you who fixes it. It tells you nothing at all about who should be told.

Why one alert can’t do both jobs

The tempting shortcut is to wire a single alert and route it to the technical team, on the theory that they’ll pass on anything the business needs to know. It doesn’t work, for a reason that only shows up in the shape of the failure.

Say a third-party lookup sits in the middle of a customer journey, and the business has decided the journey is unacceptable if that lookup takes longer than fifteen seconds. Two different things can go wrong:

The inverse fails just as badly. A system can be visibly on its way down — p99 climbing week on week, a step change after a release — and if the only alert is a business threshold, the technical team don’t hear about it until it has fully sunk. All the warning was in the trend, and nobody was watching the trend.

Which gives you the actual division of labour, and it’s sharper than “IT does monitoring”:

Neither substitutes for the other. Two alerts, one underlying event, two different owners — the process owner gets “the customer journey has breached its time limit”, the delivery team gets “the downstream service is responding slowly”. Same incident, two audiences, two jobs.

Not everything the business owns is an alert

There’s a second split hiding inside the first one, and I missed it for a long time. A colleague put it to me plainly: some signals are alerts — something is wrong now, and there’s an action to take in the next hour. Others are management information — a measure you read after the fact, in context, on a cadence. Both can be business-owned. They are not the same instrument.

The discriminator isn’t importance. It’s how wide the space of causes is.

A spike in failed payments is narrow: a short list of candidate causes, nearly all of them technical, and something you can do about it immediately. Page someone.

Conversion is the opposite. Conversion drops, and the honest list of reasons starts: the market’s soft, our pricing is wrong, a competitor has moved, the campaign driving the traffic ended, it’s seasonal, the mix of visitors changed — and somewhere down that list, maybe, something in the system is broken. None of that is answerable at three in the morning. There is no bounded action, because you don’t yet know whether anything is wrong at all.

So conversion is business-owned — completely — but owned as MI, not as an alert. Which changes the mechanism entirely: not a threshold and a page, but a number someone named reads on a cadence, alongside enough context to tell a system problem from a market one.

And this is where defaulting to IT does something worse than fail. Defaulting the alert leg leaves the effect unwatched, which is bad enough. Defaulting the MI leg is actively misleading, because if you hand a business measure to a technical owner they will go looking for a technical cause — and they will find one. There is always something in the logs: a latency regression, a retry storm, a slow query you could plausibly blame. So a fortnight goes into a performance investigation while the real answer, a competitor’s price cut in week two, goes unexamined by the only people equipped to spot it.

The MI leg also needs plumbing the alert leg doesn’t. An alert needs a threshold and a route. MI needs somewhere durable to live — a warehouse, a BI layer, something you can trend over a quarter and lay against a campaign calendar — because the entire job is interpretation against context, and you cannot interpret a number you can only see as of right now. That’s usually the bit nobody built: asking “is this getting worse?” means hand-building a chart as a proof of concept. Which, incidentally, is how you win the argument — show the trend line, and “we should chart this” stops being abstract.

Underneath it: nobody owns the process

If that split is so obvious, why is it so rare? Because it depends on something most organisations don’t actually have — someone who owns the end-to-end business process and will say out loud what “working” means.

I’ve written before that you can’t draw a boundary the business hasn’t decided. This is the same ownership vacuum, one phase later. That post is about design time: you can’t model a process nobody owns. This is run time: you can’t monitor a process nobody owns, because monitoring requires someone to define the healthy state, and only the process owner can do that.

When no one will, monitoring doesn’t stop. It quietly defaults to whatever IT can define without asking anyone — which is the technical layer, because that’s the layer engineers can specify unilaterally. That’s how you end up with a wall of green dashboards that are all telling the truth about a system nobody’s checking the point of. The signal isn’t missing because the tooling is bad. It’s missing because the ownership question was never answered, so the measurable stood in for the meaningful.

The question that forces the decision

So what do you do when you make the case and get a flat “it’s technical, I don’t want to be alerted”? Stop arguing about whether it’s technical. It isn’t the real question. Ask this instead:

Do you care that it’s down, or not?

Both answers are acceptable. Only the ambiguity isn’t.

That settles anything alert-shaped. It doesn’t touch conversion, because “do you care that it’s down” isn’t a question you can ask about a number that has no down. The MI leg needs its own version, and it’s three questions rather than one:

Who reads this, how often, and what happens when it moves?

Same principle, different teeth. A measure with no named reviewer and no cadence is exactly as unowned as an alert with no route — it just fails more quietly, because an unread report doesn’t wake anybody the way an unanswered page does. “It’s on the dashboard” is the MI equivalent of an alert firing into an empty room.

That’s the move on both legs: make the options legible and let the business pick one. It’s the same forcing function as the boundary post — the sin isn’t picking best-effort, it’s leaving the choice unmade, because undecided always resolves to IT by default. Nobody chooses that. It’s just where things land when nobody chooses at all.

In a regulated business you may not even have the luxury of ambiguity: if you’re expected to demonstrate you’re treating customers fairly, or delivering fair value under the FCA’s Consumer Duty, then “can the customer complete the journey” is an outcome someone accountable has to be able to evidence. Which is a rather good argument for that person owning the alert that tells them.

The uncomfortable bit

None of this is free. You’re asking business colleagues to accept being paged — or at minimum to owe a standing read of a number — for something they’d rather delegate, and you’re adding signal to an estate that probably already has more than it can afford. You’ll be told, more than once, that this is what the technical team is for.

The alternative is worse, and it has a tell. When ownership defaults to IT for long enough, people stop describing the system as theirs at all — it becomes something IT runs on their behalf, which is exactly the framing that guarantees nobody outside IT ever specifies what “working” means. As ever, what you optimise for is the real decision: a clean line of blame on the org chart, or a business that can tell when it’s broken.

You cannot alert your way out of an ownership problem. But you can make the ownership problem visible — and an alert with no owner is the clearest way I know to point at one.


Share this post:

Previous Post
Shadow mode as a default, not a luxury
Next Post
Debugging is the skill you can't outsource