How to Actually Measure ROI of AI
Here's a thought experiment:
Imagine you're a CEO. Someone offers to sell you a pill that everyone in your company can take that will make each of them 30% smarter and 30% faster at completing tasks.
The pill is completely safe and priced reasonably. So no brainer, right? Not only are you thrilled, but your people are too. (Who doesn't want to be 30% smarter and be able to complete tasks faster?)
Everybody takes it. Day one, whole company. Done.
Six months later, your CFO walks in and says: okay, time for the ROI assessment. Did we make our money back?
Easy, you think. Look at output.
And then you sit down to actually do it, and something strange happens. What is the output of your people?
Some of it you can count - claims processed, tickets closed, roles filled. Fine.
But what's the output of the analyst whose job is to be right about the market? The manager whose job is keeping eight people pointed in the same direction? What about the strategist whose most valuable move last quarter was talking you out of a project?
The challenge begins to emerge:
Every other technology you've ever bought came with its own meter in the box.
New machine on the line: units per hour. Salesforce: sales cycle, conversion rate. You knew the metric before you signed the contract.
And your people fit themselves to the tool - everyone used it basically the same way, you trained them on it, you expected them to use it, and that was that. Clear, finite ROI. Once everyone's on it, you've captured the value.
It was measurable.
But.
When it comes to knowledge work, everyone's work is a little different, everyone's output is a little different - and some of the most valuable work your people do has never shown up in a number, ever.
That's when the real question hits you. It's not "what changed?"
It's: what did you expect to change?
If your people got 30% smarter and 30% faster - which numbers did you think would move?
Which processes? By how much, by when?
Here's the problem:
Nobody wrote that down. There was no target.
And if you never set a target, you can't know if you hit it. The pill might be working spectacularly, all around you, right now, and yet you would have no way to see it.
That's not a measurement problem - that's a leadership problem. And notice - nobody in this story did anything wrong. The pill works! The gap is at the top: nobody ever said what "better" would look like.
With past technology, the people fit to the tool. With AI, the tool fits to the person. It literally doesn't matter what that person does, what role they play in what industry. It fits to them.
Which means the gain isn't built into the box this time. It runs through your people - they're the ones who speed the work up and reinvent it, and they'll do it better than any vendor ever could, because nobody knows the work like they do.
That's why the ceiling on this is higher than anything you've ever bought, and why nothing about it is standardized or guaranteed.
So the problem was never measuring the AI. The problem is that we've never defined what our people's work looks like when it gets better - because until now, we never had to.
Which tells you exactly where to start.
First - make sure everybody actually takes the pill. Notice the thought experiment assumed that part. With AI, it's the first thing that breaks. You can't mandate it the way you mandated Salesforce, because it's not replacing an old system - there's nothing to switch off. So leaders are left encouraging people, and encouragement doesn't transform anything.
The answer is to build AI into the processes and workflows themselves - so using it isn't a choice people have to keep making. It's just how the work gets done.
Second - set the target. What do your people produce today? What would you expect to change if they got 30% smarter and faster - which numbers, by how much, by when? Write it down. That's the meter that never came in the box. This time, you have to build it yourself.
Then watch. Some processes will get faster - that's the productivity every tool promises. And some will get reinvented - that's the part no tool has ever been able to give you. Lock those in.
The pill works. The only question is whether you'll be able to see it.
AI NEWS OF THE WEEK
1. The people building it just asked for a brake pedal
More than 1,200 employees at OpenAI, Anthropic, Google DeepMind and Meta signed a letter this week asking the US government to help build tools that could deliberately slow AI development. Not pause it. Pace it. And the names on it aren’t outside critics, they’re chief scientists, co-founders and CEOs. The people with their hands on the wheel are asking someone to fit a brake.
2. The value showed up, just not where anyone wrote it down
SAP and Oxford Economics asked 2,600 directors and C-suite executives across 13 countries how AI is going. Satisfaction with ROI is up to 69%. But cost efficiency wasn’t the top benefit anyone reported. Better insights, better decisions, better customer conversations were. So a lot of companies wrote down the wrong target before they started, then went looking for it anyway.
3. Adoption jumped six points and it still isn’t the number that matters
Gallup’s Q2 data is out. Organisational AI adoption climbed from 41% to 47% in a single quarter, and 52% of US workers now use AI in their role. But the line buried underneath is the one to read twice: having AI tools at work doesn’t guarantee anyone uses them. It depends on manager support and whether the thing fits the workflow.
