I have sat on both sides of this table. I have written proposals and I have evaluated them, and I have spent twenty-five years watching organisations commit to programmes they did not fully understand.
The hardest position is the common one: you are the person who has to decide, the sum involved is significant, and you are not a specialist in the thing being sold. Demonstrations do not help, because a demonstration is designed to look good. Reference sites do not help much either, because the supplier chose them.
What does help is a small number of questions that are difficult to answer well without being honest. Here are the five I use.
1. “Tell me about an implementation of yours that went badly, and what you changed afterwards.”
This is the most useful question I know, and the reaction matters more than the answer.
A supplier with real experience has failures, because everybody does. One who has thought about them will tell you what went wrong, what they got wrong specifically, and what they changed in their method as a result. That answer tells you they learn.
Watch for two bad responses. The first is “we’ve never had one” — which means either no experience or no candour, and neither is what you want. The second is a failure entirely blamed on the client: “the customer didn’t engage.” Sometimes true, but a supplier who cannot name their own contribution to a failure will not name it on your project either.
2. “Which of our requirements would you push back on?”
Most requirements documents contain things that are expensive, low-value, or actively counterproductive. A supplier who has read yours properly will have spotted at least one.
If the answer is “they all look sensible”, you have learned something important: they are selling, not advising. The proposal is priced to say yes to everything, and everything you asked for will be built whether or not it should be.
The suppliers worth having tell you which parts of your own specification are a bad idea. It is uncomfortable in a sales meeting and it is exactly what you are paying for.
3. “Who specifically will be on our project, and what else are they working on?”
The team presenting to you is frequently not the team that arrives. This is not always deceptive — good people are in demand and schedules move — but you should know the answer before you sign, not in week three.
Ask for names. Ask what proportion of their time you are getting. Ask what else those people are committed to during your delivery window. If the answer is vague, the commitment is vague.
Then ask what happens if the named lead leaves mid-project. A supplier with a real answer to that has thought about continuity. One who is surprised by the question has not.
4. “What happens to your involvement after go-live?”
This is where most of the value is realised and most contracts end.
Adoption — people actually using the thing, in the way that produces the benefit you paid for — happens after launch. If the supplier’s engagement stops the day the system goes live, the hardest part of the work is being handed to you at exactly the point your own team is most tired.
You may decide that is fine, and take on adoption yourself. But make it a decision rather than a discovery.
5. “What’s the first measurable outcome, and when does it land?”
If the honest answer is more than ninety days away, treat that as a warning.
Long discovery phases before anything is delivered are how consultancy budgets disappear without anything changing. It is not that discovery has no value — it is that a proposal with no early, measurable deliverable gives you no evidence and no exit. You are committed to the full programme before you know whether the supplier is any good.
Insist on something real and measurable early. It protects you, and a competent supplier will be comfortable with it.
Two more, for the reference call
When you take references — and you should — ask the supplier to introduce you to a client whose project was difficult, not the showcase one. Then ask that client two questions:
- “What would you do differently?” The useful answer is usually about their own side: we should have appointed a data owner, we underestimated training. That tells you what the supplier failed to insist on.
- “Where did they push back on you?” If the answer is nowhere, you have confirmation of question two.
A supplier who will not facilitate that conversation has told you something.
The thing none of this replaces
These questions will improve any procurement decision. What they cannot do is tell you whether you should be buying at all — and in my experience that is the more expensive mistake.
A great supplier delivering a project you did not need is still a waste. Before evaluating proposals, be able to state the benefit as a number, with a named person accountable for it. If you cannot, you do not yet have a business case. You have a wish, and no supplier can fix that for you.
We publish this kind of thing rather than keeping it back because the alternative — an organisation buying badly — costs everyone, including us. If you are staring at a proposal and would like a second opinion from someone with no interest in the outcome, that is a conversation we are happy to have.
Questions we get asked
What if a supplier says they have never had a project go badly?
That means either no experience or no candour, and neither is what you want. Everyone has failures; the useful answer names their own contribution to one.
How early should a proposal deliver something measurable?
Within ninety days. Long discovery phases before anything is delivered give you no evidence and no exit — you are committed to the full programme before you know whether the supplier is any good.
Who should I ask for as a reference?
Ask to speak to a client whose project was difficult rather than the showcase one, then ask what they would do differently and where the supplier pushed back on them.