These two reports were never supposed to tie

The most expensive meeting in marketing is the one where two dashboards disagree and everyone assumes somebody broke something.

You know the meeting. The quarterly review is underway, someone puts two numbers on the screen, and they do not match. A pause. Then the search for fault begins, and it always proceeds in the same order: the data must be wrong, then the system must be wrong, then the person who built the report must be wrong.

I have sat in that meeting many times. In most of them, nothing was broken.

On a recent engagement we were asked to reconcile three revenue figures that leadership had been treating as three views of one thing. They were not. One included closed-won business. One was scoped to open pipeline in specific stages. One applied a forecast-category filter the others did not. Every number was correct. Every number had been correctly produced. And no arrangement of arithmetic would ever have made them agree, because they were measurements of three different populations that happened to share a name.

The organization had spent months intermittently trying to reconcile them.

The failure is upstream of the report

This is not a rare pathology. It is close to universal, and it has a name in the data world: metric drift. The same business term gets calculated differently in different places because the definitions live scattered across dashboards, spreadsheets and one-off queries rather than in any single governed location. Definitions get copied, then edited, then inherited by someone who was not there for the edit.

The engineering response to this is a semantic or metrics layer: one place where a metric is defined, which every tool then draws from, so a change propagates everywhere instead of nowhere. That is the right long-term answer and it is a real project.

But most marketing organizations are not going to stand up a semantic layer this quarter, and they still have the meeting on Thursday. So here is the smaller discipline that captures most of the value.

Diff the filters before you diff the numbers

When two reports disagree, do not start with the numbers. Start with the filter sets. Put them side by side, field by field, and establish whether the two reports were ever describing the same population. In my experience, most of the time they were not, and the discrepancy dissolves the moment somebody writes both definitions down in the same place.

One practical warning, because it has cost me real hours. Reporting tools routinely display only some of a report’s filters, tucking the rest behind an overflow menu. The visible chips understate the actual filter set. Trust the filter count, not the chips. Two reports that do not share a filter set were never going to tie, and you can spend a week proving that the hard way.

Say the words out loud

There is a sentence that is very difficult to say to an executive team and enormously valuable once said: these two numbers are not reconcilable, and that is correct.

It is difficult because it sounds like an excuse. It is valuable because the alternative is worse. When an organization believes two irreconcilable numbers ought to reconcile, it does one of two things. It keeps paying analysts to attempt the impossible, month after month. Or, more damagingly, someone eventually forces them into agreement by quietly loosening a filter, and from that moment the organization is running on a number that is wrong in a way nobody has documented.

The second outcome is the one that should frighten you, because it is invisible. A wrong number that agrees with another wrong number looks like corroboration. Internal consistency is not evidence of accuracy. It is frequently the opposite.

Why healthcare gets this worse

If you market into health systems or payers, you carry a structural aggravation: your revenue is recognized on a different clock than your marketing is measured on. A contract signs in one quarter, goes live in another, and produces recognized revenue across several more. Pipeline is often tracked against contract value while finance tracks something closer to annualized recognized revenue, and go-live dates behave differently from close dates in almost every reporting tool.

Two teams can therefore be scrupulously honest, use the same source system, and produce numbers that will never meet. The fix is not a better dashboard. It is a written definition of what each measure includes, agreed by marketing and finance together, before anyone builds anything.

A modest proposal

Take your five most-quoted numbers. For each, write one paragraph: what population it counts, what filters apply, whose definition governs, and what it must never be compared against. One page in total. Circulate it.

That page will do more for your credibility with your CFO than any dashboard you commission this year, because it demonstrates the thing executives actually want from marketing and rarely get, which is not better numbers but a defensible account of what the numbers mean.

The next time two reports disagree, the useful question is not who made the mistake.

It is whether these two things were ever measuring the same population — and if not, who decided they were, and when.

Michael Carlson is the founder of Pertinacity, a growth agency that builds the AI and the systems underneath the marketing it runs.

Sources

• Atlan, Metrics layer: a single source of truth for KPI definitions — https://atlan.com/metrics-layer/
• Improvado, What is a metrics layer — https://improvado.io/blog/what-is-a-metrics-layer
• dbt Labs, Semantic Layer — https://www.getdbt.com/product/semantic-layer