RFP stands for “request for pain.”

The practice was seriously awful when I first wrote about RFPs in InfoWorld back in 1996, and the situation has, if anything, deteriorated since then.

Requests for proposal and their request for information (RFI) brethren is to select the best vendor to handle a particular business need.

The actual results are, at best, hit and miss. What goes wrong?

Overspecification: I’ve seen RFPs that list hundreds of specific, detailed requirements, presumably as an attempt to minimize puffery, blow-hardery, and general-purpose hand-waving in the responses.

On the surface this might seem admirable — a way to minimize misunderstandings and assure solutions that solve the actual problems.

Regrettably, the requirements listed in the RFP are only as good as the business analysts who wrote the list. Strike that — they’re only as good as the insights provided to those business analysts by those who will be stakeholders.

Worse, even the best lists are just headlines and not the complete stories, and as responders weren’t part of the discussions that generated the list, their responses are sadly lacking in context.

But wait. It’s even worse! By insisting responders address specific and detailed requirements the requester loses the opportunity to take advantage of responders’ creativity, expertise, and innovation. Instead they force every vendor to submit a response that’s just about identical to the responses submitted by every other vendor.

Price tyranny: Overspecification leads to undifferentiated responses. As an inevitable consequence, evaluators are left with little beyond price and intangibles for their decision.

But intangibles are what RFPs are supposed to drive out of decision-making.

As for price, imagine you’re responding to an RFP for creating a custom application. The other vendors estimate the cost of building it. You also estimate the effort that will be needed to modify or redesign business processes, and to identify and deal with sources of organizational change resistance, something not included in the RFP but vital for real success.

But your proposal costs more. What a shame.

Agility exclusion: Those issuing RFPs understandably want responders to provide a fixed price for their proposed solution. Increasingly, they also want projects that make use of Agile techniques. The contortions needed to commit to a fixed price in the face of Agile’s core strength — dynamic adjustments to the project backlog as everyone involved in the implementation “learns their way into it” — are, to be as diplomatic as I can manage, something of a challenge.

Case studies: All RFPs ask for case studies — demonstrations that a responder has done this before. This would be logical if responders were individual human beings.

Actual responders aren’t. They’re corporations, which aren’t just like individual persons only bigger. Which means that even though a responding corporation might have done this a dozen time before or more, that doesn’t mean even one member of the implementation team will have participated in even one of the case studies included in the response.

This can’t be fixed, because even with all the best of intentions, even if a responder can state that several members of the response team were, in fact, involved in one or more of the case studies, there’s no way to guarantee any of them will still be on the bench and available for the project, given the multi-month delays that are typical between the time responders submit their proposals and the issuer signs a contract with one of them.

One more problem with insisting on relevant case studies: They completely preclude innovation. In case the reason isn’t obvious and self-explanatory, innovation means coming up with something new and different, which means there can’t be a case study.

The result of which is that proposers either (1) avoid proposing anything innovative; (2) propose an innovative solution and explain, clearly, why case studies aren’t possible; or (3) propose the innovation and stretch actual case studies to fit.

Most responders will play it safe and choose option #1. But to fix what’s wrong with RFP-driven vendor selection, proposers will have to take the lead, playing it the opposite of safe. They’ll have to try something quite different.

For example, speaking as a more-than-occasional responder, I’d love to receive an RFP that says, “Here’s the business problem we’re trying to solve,” followed by a clear and compelling narrative, followed in turn by this question: “How would you go about solving it for us?”

I think all parties would find the results quite refreshing.