Metrics seems like such a good idea. If you can’t measure, after all, you can’t manage, or so Peter Drucker asserted a long time ago. He’d be horrified at how this dictum has been misapplied.
Drucker was talking about business processes, where if you can’t measure you really can’t know when something has broken down and needs attention. That’s the proper scope of Drucker’s Metrics Dictum. It applies to business processes. Too many business managers think it applies to everything.
We’ve been talking about DevOps, whose name comes for the inclusion of Ops members in Dev teams, but whose most interesting characteristic is that it replaces waterfall’s big-bang deployments, and even Agile’s large-scale releases, with a continuous stream of small deployments that are subjected to automated testing and then automatically promoted to production without so much as a rubber stamp from a Change Advisory Board.
The metrics question of the day: Do Agile and DevOps speed things up?
The answer is that there is no easy answer. To understand the challenge, answer this question instead: Which is faster, the Internet or a truck?
Figure the truck has a 4,000 cubic foot capacity. You fill it with SanDisks, each holding 64GB of data. The truck drives 1,000 miles at an average of 50 mph. If I’ve done my arithmetic right, the truck delivers more than 35 million terabits of data in 20 hours, yielding a bandwidth of just under 500 terabits per second.
Imagine your MPLS monthly bill for that kind of bandwidth. The truck wins by, you’ll pardon the expression, a mile.
Fortunately, network engineers recognize that bandwidth only tells half the speed story. The other critical network metric is latency. Here, the truck doesn’t fare so well. The first data packet doesn’t arrive until 20 hours after it’s transmitted, compared to a typical Internet ping of maybe 15 milliseconds.
So which is faster, the Internet or the truck? Answer: It depends on what you need more, high bandwidth or short latency.
Waterfall methodologies are the trucks of application development. They deliver a truckload of software features in one big delivery. The business waits for six months or more before anything goes into production, but when the wait is over it gets a lot of new functionality all at once.
Agile methodologies, and DevOps even more so, are the Internet of app dev, delivering new functionality more frequently but in smaller increments.
Where the metaphor breaks down is that with our truck vs Internet comparison we had a useful standard unit of data delivery – the bit.
Comparing app dev methodologies, we don’t. For a while we had function points, but they never really caught on, mostly because they’re too durned complicated to properly count. Also, they’re useless for Agile because function point analysis has deep connections to waterfall’s up-front specifications, which is one reason Agile replaced them with user stories and story points (estimated degree of difficulty).
So trying to compare waterfall and Agile for the speed of software delivery just isn’t going to happen in any reliable way. Even comparing Agile and DevOps is dicey. Agile delivers user stories weighted by story points. Increasingly, DevOps delivers microservices, not full user stories.
Try to apply Drucker’s Metrics Dictum to application development and you’ll find you’re trying to answer the question (to change metaphors mid-stream): Which is better for beating the other team — a great passing game, or the knuckleball?
And oh, by the way, these things have strong connections to the type of business change you need to achieve. Standard Agile practices are just the ticket when your goal is continuous improvement. Waterfall actually can work well, assuming you’re implementing a new business process you’re able to specify with precision and that will still be relevant when the multi-year initiative wraps up.
When designing a good metric turns into an intellectual quagmire, the problem is probably that we’re asking the wrong question. IT’s goal isn’t software delivery, after all. It’s supporting the achievement of intentional business change.
That being the case, what we should be looking at is whether, for a given desired business change, IT’s app dev methodology is a major source of friction.
Increasingly, business leaders care more about the organization’s ability to change direction quickly to address threats and pursue opportunities, and less about organizing and implementing large-scale strategic change.
With this change in the style of business change there’s no longer much doubt. The Agile/DevOps spectrum of methodologies is far more likely to keep IT from being sand in the company’s gears of progress.
Dude, that is the best waterfall versus agile analogy I’ve ever heard.
Loved the truck vs internet analogy!
Maybe function points (FPs) can, over the long term, answer the question of which method produces more “bandwidth”. Nearly all FP analyses I’ve seen have calculated the FPs by the “backfire” method, i.e. converting lines of code in a working application to FPs, using a conversion factor developed for the implementation language(s).
That’s not useful for predicting effort, but it might allow us to see how many FPs per month a given shop has produced by either the DevOps or the Waterfall method. Then you would have an estimate of “bandwidth” for each method.
Do you know anyone who would be willing to provide data for such analyses?
If it isn’t predictive I’m not sure how useful it would prove to be. Beyond that, (and I know you know this), lines of code is an awful metric, even when limited to a single language, because different programmers can solve the same problem with dramatically different code quantities.
I wish somebody would come up with a metric that calculates unnecessary lines of code written because management/sales didn’t bother to ask engineering if a particular feature was feasible or possible. Because in my career, this is the biggest reason for software projects failing. Dictates from above may make people feel powerful or get customers to sign on the dotted line easily but they don’t produce successful software. It doesn’t matter if you are using Agile or Waterfall. Poor leadership will tank any effort.
Hi, I think I must have been on Mars as I missed this whole DevOps. I know agile and the rest of the stuff but it sounds interesting.
Could you recommend a book to read up more on it? Of course it would be great if you were writing one as I love your writing style… (hint hint!)
Nothing in the works from yours truly on this subject. Sorry. DevOps for Dummies isn’t bad, and while I haven’t read it yet, if you like the business novel approach to learning, I’ve heard that The Phoenix Project isn’t bad either.
The only challenge with both of them (to the best of my knowledge, at least) is that they’re a bit uncritical. But then, that’s what I’m here for, I guess.
An effective estimation of the business value of the change would be the ideal measure. While it is just moving the goalposts in a sense, it does have the benefit of kicking the question into the business space it belongs to, rather than attempting to define an IT-centric measure.