I have sat on both sides of this table. I have written proposals and I have
evaluated them, and I have spent twenty-five years watching organisations commit to programmes
they did not fully understand.
The hardest position is the common one: you are the person who has to decide, the sum involved
is significant, and you are not a specialist in the thing being sold. Demonstrations do not help,
because a demonstration is designed to look good. Reference sites do not help much either, because
the supplier chose them.
What does help is a small number of questions that are difficult to answer well without being
honest. Here are the five I use.
1. “Tell me about an implementation of yours that went badly, and what you changed afterwards.”
This is the most useful question I know, and the reaction matters more than the answer.
A supplier with real experience has failures, because everybody does. One who has thought about
them will tell you what went wrong, what they got wrong specifically, and what they changed in
their method as a result. That answer tells you they learn.
Watch for two bad responses. The first is “we’ve never had one” — which means either no
experience or no candour, and neither is what you want. The second is a failure entirely blamed on
the client: “the customer didn’t engage.” Sometimes true, but a supplier who cannot name their own
contribution to a failure will not name it on your project either.
2. “Which of our requirements would you push back on?”
Most requirements documents contain things that are expensive, low-value, or actively
counterproductive. A supplier who has read yours properly will have spotted at least one.
If the answer is “they all look sensible”, you have learned something important: they are
selling, not advising. The proposal is priced to say yes to everything, and everything you asked
for will be built whether or not it should be.
The suppliers worth having tell you which parts of your own specification are a bad idea. It is
uncomfortable in a sales meeting and it is exactly what you are paying for.
3. “Who specifically will be on our project, and what else are they working on?”
The team presenting to you is frequently not the team that arrives. This is not always
deceptive — good people are in demand and schedules move — but you should know the answer before
you sign, not in week three.
Ask for names. Ask what proportion of their time you are getting. Ask what else those people
are committed to during your delivery window. If the answer is vague, the commitment is vague.
Then ask what happens if the named lead leaves mid-project. A supplier with a real answer to
that has thought about continuity. One who is surprised by the question has not.
4. “What happens to your involvement after go-live?”
This is where most of the value is realised and most contracts end.
Adoption — people actually using the thing, in the way that produces the benefit you paid for —
happens after launch. If the supplier’s engagement stops the day the system goes live, the hardest
part of the work is being handed to you at exactly the point your own team is most tired.
You may decide that is fine, and take on adoption yourself. But make it a decision rather than
a discovery.
5. “What’s the first measurable outcome, and when does it land?”
If the honest answer is more than ninety days away, treat that as a warning.
Long discovery phases before anything is delivered are how consultancy budgets disappear
without anything changing. It is not that discovery has no value — it is that a proposal with no
early, measurable deliverable gives you no evidence and no exit. You are committed to the full
programme before you know whether the supplier is any good.
Insist on something real and measurable early. It protects you, and a competent supplier will
be comfortable with it.
Two more, for the reference call
When you take references — and you should — ask the supplier to introduce you to a client whose
project was difficult, not the showcase one. Then ask that client two questions:
- “What would you do differently?” The useful answer is usually about their own
side: we should have appointed a data owner, we underestimated training. That tells you what the
supplier failed to insist on. - “Where did they push back on you?” If the answer is nowhere, you have
confirmation of question two.
A supplier who will not facilitate that conversation has told you something.
The thing none of this replaces
These questions will improve any procurement decision. What they cannot do is tell you whether
you should be buying at all — and in my experience that is the more expensive mistake.
A great supplier delivering a project you did not need is still a waste. Before evaluating
proposals, be able to state the benefit as a number, with a named person accountable for it. If
you cannot, you do not yet have a business case. You have a wish, and no supplier can fix that
for you.
We publish this kind of thing rather than keeping it back because the alternative — an
organisation buying badly — costs everyone, including us. If you are staring at a proposal and
would like a second opinion from someone with no interest in the outcome, that is a conversation
we are happy to have.