metrics are tools, not truths

In the world of software, metrics are tools, not truths.

That’s an uncomfortable claim for many. Because while everyone knows numbers are proxies, placeholders, approximations—we still treat them like gospel. As if hitting the metric is the goal, rather than the thing the metric is supposed to represent.

fuzzy glass

Metrics are always fuzzy at best. More like looking through frosted glass.

They can be gamed.
They differ wildly from team to team (one team’s story point “2” is another team’s “5”).
They often create more cycles of measurement than cycles of progress.

We can’t stop measuring—but we can stop obsessing.

the usual suspects

In engineering, the same few metrics get trotted out again and again:

PR frequency
Commit-to-merge time
Revert rate
DORA metrics (speed and stability)

But what about measuring quality and impact? That’s where it gets slippery. One line of code can change the performance of an entire system. A thousand lines might move nothing forward.

Numbers rarely tell the full story.

when measuring slows the work

I notice this in my own work. I recently started at a hypergrowth startup: full days of meetings, writing docs, strategizing, talking to clients, organizing sessions. At first, I thought I’d track everything I did—have a neat daily summary, metrics on myself.

But the reality? I don’t have the energy for that. The overhead of measurement slows me down.

I suspect the same holds at the org level. Metrics are great for middle managers who need dashboards. They’re necessary for the C-suite who must justify ROI to a board or shareholders. But they often come at the cost of cycles that could have been spent shipping.

the paradox

Don’t get me wrong: measurement is critical. Without the scientific method, we’d be lost. But when science becomes scientism—when the act of measuring becomes more important than the thing itself—we get in trouble.

And yet: how do you know you’ve delivered value if you’re not measuring?

Checkmate, right?

Yes, we must measure. But we also have to weigh the cost of measurement against the value it provides. If we’re measuring only to justify ourselves, not to improve outcomes, the system is backwards.

anecdote > obsession

Sometimes the most honest data point is a trusted IC telling their manager: “Trust me, this changes everything.” The numbers will come later.

I see this with Cursor, the AI-powered IDE. I know it makes me a better coder. Features that would’ve taken me a week now take an hour. Entire projects I never would have attempted before now feel easy. I feel it in my bones.

But could I prove it? What would that even look like—building the same feature twice, once by hand and once with AI, timing both? The measurement itself would cost more than the insight.

That’s the hidden truth: measuring is expensive. We rely on proxies, we justify after the fact, and sometimes anecdotal evidence is the best evidence we’ve got.

the real risk

Ultimately, you want the business to improve. But it’s almost impossible to draw a clean line from engineering output to ROI.

Which is why we have to beware the trap: mistaking the statue for the deity. Mistaking the metric for the truth.

Metrics are necessary. They orient us. They give us a sense of direction through the frosted glass.