measurement is a tool, not a truth
Immeasurable impact
We’re obsessed with metrics. With quantification.
And don’t get me wrong — I love data as much as the next guy.
But over and over again, I see the number matter more than the action.
Just like grades in school:
Did you actually learn the material?
Or did you cram just enough to get an A?
Measurement and the Body
Even though I’m a technologist, I’m a luddite in some ways.
When I go to the gym, I don’t wear a heart-rate tracker.
I don’t log every calorie, or measure every single step.
I don’t want a part-time job as my own data analyst.
And let me tell you, I tried.
When I measured every speed, rep, and calorie burned — I stopped showing up. I got too caught up in the tracking itself.
What finally worked? Reducing the barrier to entry.
Just a stopwatch and a notebook. Simple.
That’s how I got to 30 minutes of jump rope a day.
Strength training 3–5 days a week.
Five minutes in the cold plunge.
So yes, we need measurement. I’m not saying throw it out.
But when measurement becomes data porn — when the numbers themselves become the object of obsession rather than the actions and connections behind them — that’s when we get in trouble.
Immeasurable Music
I’ve been thinking about this more broadly.
With my music: after a show, kind folks often come up to thank me.
Sometimes they say the music did something for them.
And sometimes, I get down on myself: What’s the point if I’m barely touching anyone?
But then I remember the day someone told me a performance helped them get through a really hard time. Helped them move emotions they couldn’t process alone.
That kind of impact doesn’t show up in the numbers.
Most people won’t come tell you, not because they weren’t moved, but because they’re shy, or in a rush, or who knows.
So maybe I’m naive. But I hold onto the sense that it’s still worth showing up, even when the results are invisible.
The Measurement Problem in Software
In software, businesses run on numbers. There’s no way around it — we need to track money, spending, and ROI.
But productivity? That’s slippery.
We’ve tried lines of code.
We’ve tried number of pull requests.
We’ve tried story points in agile.
And like grades in school, these numbers can all be gamed.
Worse yet, every team does it differently. One team’s “2” is another team’s “5.” When we try to standardize, it all falls apart.
Agile defenders protest: That’s the point. We only compare a team against itself, sprint over sprint.
Sure, but teams change all the time. People leave, new folks join. Velocity resets. At an org-wide level, it’s chaos.
So confusion reigns. Engineers eventually shrug, guesstimate complexity, and move on.
Enter DORA metrics and research-backed frameworks.
They approximate productivity. Useful, maybe. But still approximations.
The truth is: productivity in engineering teams is a wild animal. Not impossible to glimpse, but very hard to cage.
The CFO’s Dilemma
And I get it. At some level, all this has to roll up to a CFO who asks: Why are we spending this money? What’s the ROI?
So we track. We report.
But let’s be real — most of the time it’s less like reading a Rosetta Stone and more like watching a play through frosted glass: you see shapes, you catch outlines, but you’re probably missing half the plot.
And now, on top of all this, we’ve got AI in the mix.
How do we measure AI productivity? How do we measure changes in velocity?
The irony: we weren’t good at measuring baseline productivity in the first place. Most of our numbers were fuzzy at best.
The White Whale of Productivity
I hold this opinion somewhat lightly — open to pushback — but the pattern I see is an obsession with measuring something that is constantly shifting, deeply human, and hard to pin down.
Boardrooms love to say: Just capture this info. Just grab that data point.
But the cost of wrangling those numbers — building pipelines, forcing humans to report consistently across contexts — is enormous. And humans are the noisiest data source of all.
You’ll never really get a clear picture of productivity.
Unsatisfying, I know. Maybe it’s my white whale. Maybe one day someone will write a book about it — the way Robert Pirsig tried to define “Quality” in Zen and the Art of Motorcycle Maintenance. A whole treatise, circling the word, without ever being able to fully pin it down.
We’re moving from a deterministic world to a probabilistic one. AI outputs are more like human ones: nonlinear, hard to measure, impossible to standardize cleanly.
Don’t Worship the Numbers
So maybe the point is this:
Our metrics will never be perfect. Sometimes they’ll be flat-out bad.
But we still need them. Just don’t worship them.
Because the numbers aren’t the thing itself. They’re like statues representing a deity — imperfect symbols pointing toward something deeper.
The danger is when we mistake the statue for the god.
When we forget that the numbers are just tools to orient us toward what we can’t directly see.
So measure, yes. But keep your focus on what remains immeasurable.
