All Posts in Project Management

|In: Project Management|Last Updated:

Congratulations! You’re on the “FitShoes™” project. FitShoes™ will be like Fitbits®, only because the chip is built into the shoe, they won’t interpret gesticulation as jogging a few miles. Even better, they’ll include GPS. They’ll keep track of the route … and therefore will know when the wearer is walking uphill and downhill so they can calculate exertion more accurately.

The theory is that while your company will make some money on the shoes themselves, it will make much more on the intelligence to be gained by knowing customer exercise patterns, both for direct marketing and from strategic intelligence gained from analytics performed on the pooled data.

How to proceed? The first step (NPI) is, of course, a proof of concept. There’s no point pouring R&D dollars into a product like this if the concept doesn’t hold up to the sturm und drang of life here on earth.

So the company forms a cross-functional team, and, recognizing that throwing money at a problem rarely solves anything, keeps its budget lean, letting everyone know that if the initial proof of concept yields promising results, additional funds will be forthcoming.

Armed with a deep understanding of the vision plus a modest budget to play with, the team designs the pilot. They agree that this early in the game, with detailed requirements still to be learned, they’ll use Excel for the database and processing.

The other key agreement: Creating a chip/circuit board to power a prototype FitShoe™ would be far too expensive and time-consuming. Instead, volunteers will keep track of their movements using a smartphone-based spreadsheet template they email to the project team every day for a month.

And …

This “proof” of concept will prove nothing at all. But it resembles many of the so-called proofs of concept I’ve seen over the years. It tests the least-challenging dimensions of an idea, leaving the hard stuff for the actual implementation when there’s enough time and budget.

Also when it’s too late to change direction.

To avoid this trap:

  • Be clear about what concept you’re proving. Or concepts. In our FitShoe™ example, the only concept being proved is that data sent from elsewhere can be used to update a database.
  • Know when more than one concept must be proved. The FitShoe™ depends on several of them: (1) It’s possible to put that much functionality on a chip or small circuit board; (2) that when you do it will be durable enough to withstand the pounding when someone exercises in the shoes; (3) people who wear gesticulation-foolable exercise monitors want more accurate exertion data; and (4) there’s enough to be learned from those who do to warrant the investment.
  • Test concepts independently. If you try to build a scale model that incorporates every key aspect of whatever it is you’re developing, you’ll save a lot of time.

If, that is, everything works according to the initial vision. If it doesn’t, figuring out which concepts are sound and which ones aren’t is a lot harder when they’re all jumbled together.

Prove each concept separately first. Then assemble a working prototype.

  • Test the complexity. In IT, unlike product development, proofs of concept usually revolve around the applicability of some already-proven technology or other to one or more business processes or business process cases. The temptation, parallel to the FitShoe™ parable, is to start with the simplest case on the grounds that this will be the least expensive pilot project, and the one most likely to be successful.

Except that with proofs of concept, “success” means learning something important about what actual use will be like. Just building something that works by avoiding all the nastiness the final product will have to deal with means that when you’ve finished you won’t have learned anything at all.

  • Don’t create an incentive for success. The FitShoe™ project team’s incentive was a more generous budget for the next phase. Bad idea: It’s another version of defining success as building something that works rather than learning something important about what building the final product will take.

In any proof of concept there’s already too much of an incentive, in the form of political pressure. Someone’s name is on this; in too many companies that means if it doesn’t work their name will be on something that’s considered a failure.

Which means that as is the case throughout the world of business leadership, incentives and “holding people accountable,” are just about the worst choices you can make. True success requires, not these, but a culture of honest inquiry instead.

Without this, proof and “proof” will have nothing at all in common.

Comments

|In: Process Management and Improvement|Last Updated:

“Proof” is a tricky concept. “Proof of concept” is, if anything an even trickier concept.

Before we go on, a warning: If you buy into what follows and try to promote the ideas, you’ll gain a (or add to your existing) reputation as a persnickety pain in the keister. You might be better off just going with the flow, without worrying about all those nasty details that can be the difference between successful implementations and pouring money down a rabbit hole.

Still with me? Don’t say I didn’t warn you.

Mathematicians and geometricians more or less invented the notion of proof. They start with explicit assumptions, including a class of assumptions that are the rules of logic. They rigorously apply logic to their assumptions to create, one at a time, proved statements they can then rely on to take their proof to the next stage in the sequence (so long as their assumptions hold).

It’s like FORTH programming, where you combine defined verbs to create new defined verbs (and if you knew that you have, like me, joined the Geezing Geek Club).

Wouldn’t it be lovely if you could prove business concepts through pure logic? But you can’t.

Businesses are, among other things, collections of processes and practices. And business processes and practices have to handle, not only the mainstream set of inputs, but also a wide variety of exceptions. How employees handle many of those exceptions isn’t documented, because documenting them is impractical. There are too many of them and none happen often enough to be worth the time to document once an employee figures out what to do with it and moves on.

Which is just one reason proofs of concept are necessary in the first place. Add the rest (listing them is left as an exercise for the reader) and you get to the result: Logic can only get you so far. Then you need evidence.

Beyond mathematicians and geometricians is the next level of professional provers — researchers in the hard sciences. They develop hypotheses in much the same way mathematicians develop proofs, except that as scientists deal with the physical universe they have to accept that their assumptions might not always hold; also that there are almost always too many variables that might affect a phenomenon to include them all in a formal mathematical model.

Which is why they have to test their hypotheses through observation and experiments.

What scientists and philosophers of science figured out a long time ago, though, is that they can never prove anything through observation and experimentation, if for no other reason (and there are actually lots of other reasons) than that the next time they do the exact same thing, something different might happen.

So all good researchers understand that the best they can ever do is fail to disprove a proposition. Subject it to enough different tests and have it not fail any of them and, over time, they start to have confidence in it.

Which is why scientists have great confidence in Einstein’s theories of relativity and the Darwin/Mendel/Fisher theory of evolution by natural selection, and, for that matter, an increasing level of confidence in the theory of anthropogenic climate change: Each makes predictions about what scientists should find if the theory is true; so far, when scientists have looked, they’ve found what the theory predicts … or they’ve found something that doesn’t completely invalidate the theory but does call for modification or elaboration, after which the modified, elaborated theory is what’s subjected to future testing.

But they’re never certain, and good scientists never say they’ve ever proved anything. They say they’ve tested it thoroughly and it’s held up.

Think you’ll ever have the opportunity to test your business concepts this thoroughly?

Nope. At best you’ll be allowed to conduct one or two so-called proofs of concept, which are, by the above standards far far short of proof. Your average “proof” of concept is really nothing more than an attempt to disprove the simplest and easiest application of whatever the concept is to your business.

It’s a bit (but just a bit) like strapping a jet engine to the back of a cart and adding a big parachute. You might win a drag race with it, but you certainly haven’t proven the concept of jet-powered automobiles.

You’ve barely scratched the surface of testing it. If you have any more confidence than that, you’ll almost certainly find yourself in the middle of a fiery crash later on.

Comments