What I Got Wrong About Qualifying Buyers
I spent months building a way to find businesses with a measurable problem. It worked perfectly and produced almost nothing, because a measurable problem and a motivated buyer are different objects. No query would ever have told me that.
For a long stretch this year I ran an outbound system that was, technically, very good. It measured a prospect's website, found real defects, and produced a report showing exactly what was wrong and what it was costing them. The measurement was accurate. The report was honest. The problems were genuinely there.
Reply rate across 899 sends: 3.78%. Thirty-four replies.
I want to walk through why, because the answer took me months and it is not a marketing lesson. It is a reasoning lesson, and it applies to any practitioner deciding who to spend their attention on.
The failure was not in the execution
The tempting explanations are all about craft. The message was too long. The subject line was weak. The list was not tight enough. I tried variations of all three and moved the number a little.
Then a much simpler observation arrived, and it flattened everything I had built on top of it.
I was qualifying websites. I needed to qualify buyers.
Slowness is a property of a website. It can be measured precisely, at scale, for free, and I had built an excellent machine for doing exactly that. Motivation is a property of a person. It cannot be measured from the outside at all.
Those two properties are close to independent. A business can have a badly broken website and an owner who has never once thought about it, is not troubled by it, and has no intention of spending money on it this year. My system found that owner with great efficiency and then sent him a precise document about a problem that was his and not his concern.
Why no amount of data would have found this
Here is the part I actually want to record, because it is the transferable bit.
The insight required no data. It was available on day one. Nothing in any table would ever have surfaced it, because every table I had was a table about websites, and the question was about people. You cannot query your way to the observation that you are measuring the wrong noun.
What I had been doing instead was answering "what does the data say" when the question was "what is actually true". Those come apart more often than is comfortable, and the failure mode has a specific shape: measurement is cheap, immediately available, and looks exactly like diligence. Running a query feels like work. Sitting with a mechanism feels like stalling.
So the query gets run, a number comes back, and the number does not present itself as uncertainty. It presents as knowledge. Then it gets built on, and every layer above inherits the flaw without anyone seeing it.
Three separate times that month I did a version of this. Once I computed a reply rate from a table that only exists because people reply, and got 17% instead of 2.1%. Once I called an award badge broken three times when it was mid-decode. Once I verified a set of prices against a hidden DOM element and reported them clean when the rendered page said something else. Same structure every time: I checked the convenient proxy and treated it as the thing.
Structure is reasoned about. It is not measured.
What this means if you are choosing who to talk to
Translate it out of my business and into a practice.
A practitioner deciding who to reach out to will usually reach for an observable attribute. Job title. Company size. Industry. Whether they run their own business. These are the website-speed of prospecting: measurable, cheap, and describing the situation rather than the person.
What you actually need to know is whether this person is currently bothered. Whether the thing you help with is, this month, a live irritation rather than a permanent background condition. Two people with identical circumstances are completely different prospects if one has just decided something has to change.
That is not visible in a database, which is inconvenient, and it is why the best sources of it are unglamorous. Somebody said something out loud. Somebody asked a question. Somebody came to an event on the topic. Somebody replied to a thing you wrote about it. Those are all evidence of a person, not of a circumstance, and one of them is worth a hundred matching job titles.
It is also, incidentally, why the list of people who already contacted you and went quiet is the strongest list you own. Every name on it demonstrated motivation at some point. That is the property you cannot buy or measure, and you already have it recorded.
The correction I am still making
I have not finished absorbing this. The pull toward the measurable is strong, particularly when the measurable thing can be automated and the unmeasurable thing requires paying attention to individual people.
The check I use now is a question: is this number telling me about the world, or about the process that produced the number? It catches the reply-rate error. It catches the website-speed error. It would have saved me several months if I had been asking it in June.
Every figure in this post comes from my own outbound log, queried on the day of writing: 899 sends, 34 replies, 3.78%.
An earlier version of the reply-rate figure in my own notes said 17%. That number came from a table of hand-logged conversations, which only contains people who replied. The correction, and the standard I now hold numbers to, is in Measured Forward, or It Is Not Evidence.