PHASE 06
IF YOU CAN'T MEASURE IT,
YOU CAN'T DEFEND IT.
The honest answer to whether an automation paid for itself is usually nobody knows, because nothing was measured before it was built. That is a structural problem and it is fixable, but only in advance.
Measure before, or do not bother
The single most important thing on this page is also the least interesting. You have to capture the baseline before you change anything.
After the fact, memory is useless. Everyone will agree things feel smoother, which is a statement about mood as much as operations. Meanwhile the person who was skeptical will report that it is roughly the same, and there is no way to settle it. That is how operations investments end up defended by anecdote and eventually abandoned.
The baseline does not need to be rigorous. It needs to exist and to be written down with a date on it. Two weeks of counting something is enough to beat a memory.
One caution about what gets measured. Pick measures that cannot be improved by making the underlying work worse. Response time improves beautifully if you respond with nothing useful. Pair any speed measure with a quality measure, or you will optimize your way into faster failure.
The five things worth measuring
Time recovered
Hours per week previously spent on the specific task, by the specific people. Count it before. Count it after. This is the most direct measure and the easiest to game, so count honestly.
Response time
Elapsed time from a lead arriving to a real first response. Median, not average, because a few outliers will hide the pattern entirely.
Leak points closed
How many items fell out of the process without a decision. Leads never contacted twice. Jobs that stalled with nobody chasing. Count the leaks, not the wins.
Error rate
Rework, corrections, duplicate records, things sent to the wrong person. Errors are usually the hidden cost that made the old process expensive.
Owner hours
Hours per week the owner spends on work that is not judgment. If this has not moved, the project did not do what it was for, whatever else improved.
Cycle time
Elapsed time from start to finish of one unit of work. Almost always dominated by waiting rather than working, which is what makes it revealing.
Pick two or three. Measuring six things well is beyond most small businesses, and three measured consistently beats six measured for a month and abandoned.
For the full context behind this, see the free real estate investing playbook.
How to structure the measurement
- Name the measure before you build. Write the sentence: we expect this to change [measure] because [mechanism]. If you cannot name the mechanism, you are guessing about what the change will do.
- Capture two weeks of baseline. Count it by hand if you have to. Written down, dated, with the method noted so it can be repeated the same way.
- Note everything else that changed. Headcount, season, marketing spend, a new source. Operations changes never happen in a vacuum and pretending otherwise makes the result unreadable.
- Wait long enough. Measure again after the change has run through a full normal cycle, not the first good week.
- Compare honestly, including the cost. Subscription, setup, and the hours your team spent building and learning it. The build time is real and it is the line most often left out.
Be careful about attribution. If response time improved and you also hired someone, you do not know which one did it. Change one significant thing at a time where you can, and where you cannot, write down the confound so you are not fooled later.
What to do with the result
Three honest outcomes, and all three are useful.
It worked. Write down what the mechanism was, not just the result. The transferable knowledge is why it worked, because that is what tells you where to look next.
It did not move the measure. This is the most common outcome and it is information rather than failure. Usually one of two things: the constraint was somewhere else, or the process was not actually defined and the automation encoded the ambiguity. Go back to the map.
It made something worse. Rare, and worth taking seriously immediately. Common versions are speed improving while quality dropped, or a step being automated that turned out to require judgment. Switch it off, fix it, or revert. A system nobody is willing to switch off is a system nobody controls.
What not to measure
Some numbers feel like measurement and are not.
- Activity counts. Messages sent, tasks created, records updated. These go up whenever you add a system and tell you nothing about outcomes.
- Adoption for its own sake. Whether people log in is not whether the business improved.
- Vendor-reported metrics. The dashboard inside the tool is designed to demonstrate the tool's value. Use your own numbers.
- Feelings, collected informally. Useful as a signal that something is wrong. Useless as evidence that something worked.
- Revenue, in isolation. Too many inputs. It moves for reasons that have nothing to do with your operations work, in both directions.
And be skeptical of any figure quoted to you about what automation typically returns. Those numbers come from somebody with an interest in the answer, and they were measured in a business that is not yours. Your own baseline is the only evidence that means anything.
Frequently asked
Questions people actually ask
What if I did not capture a baseline?
Capture one now for the next change, and for the current one reconstruct what you can from records rather than memory. Historical data in your CRM or job system is imperfect and far better than recollection.
How long before I should expect to see a difference?
Long enough for the change to run through a full normal cycle of your work, which varies by business. The first week after any change is unrepresentative in both directions, because attention is elevated.
What is the single best measure for a small business?
Owner hours spent on non-judgment work, if you only track one. It is the measure that connects most directly to why this work is being done, and it is the one that quietly fails to move when a project solved the wrong problem.
Can I trust vendor ROI claims?
Treat them as marketing. They were produced by someone with an interest in the answer, in businesses unlike yours, with methods you cannot inspect. Your own before-and-after is the only evidence worth acting on.
What if the numbers say it did not work?
That is a useful finding and the cheapest lesson available. Check whether the constraint was actually somewhere else, and whether the process was genuinely defined before it was automated. Those two causes cover most of it.
Should I measure my team's individual performance with this?
Be careful. The moment process measurement becomes personal evaluation, the numbers get managed instead of the work. Measure the process. Handle performance separately and directly.
Make your next move
A year from now, what will you be glad you started today?
You don't need another promise that everything will be easy. You need something useful to learn — and a next step you're willing to take.