All Posts in App Dev Methodologies

|In: App Dev Methodologies|Last Updated:

Not that I’m skeptical or nuthin’ …

The number being bandied about, and cited here last week, is that Amazon releases new software to production every 11.6 seconds.

It appears Amazon employs roughly 3,000 web developers, which means it’s arithmetic time. Let’s see: 86,400 seconds per day, 11.6 seconds per release, and that gets us to … carry the one … about 2.5 releases per day per web developer.

Does someone in earshot know enough about the situation to clarify exactly what each programmer is releasing 2.5 of to the wild every day at Amazon? Because never mind DevOps, and for that matter never mind some awfully fast coding. With developers this fast, who comes up with enough ideas to keep them busy, and how?

While the exact number seems suspicious enough to warrant skepticism, let’s give everyone in the DevOps public relations supply chain enough benefit of the doubt to accept that whether the actual number is 7,500, 750, or 75, Amazon releases a lot of new code to production every day.

When the new releases are going to a website or mobile app, as they are with Amazon, nobody has to worry about training or organizational change management. Amazon’s shoppers expect to figure out what they’re supposed to do next based on what they see on their computer or smartphone screens. They’re shopping, not engaged in mission-critical activities.

And what you see on Amazon, while it’s quite a bit more complicated than Google’s legendarily spare user interface, is still quite simple compared to what business users see when looking at a screen from one of the company’s enterprise applications.

Also: While there is, as has been pointed out in this space ad infinitum, not to mention ad nauseum, no such thing as an IT project (they’re all about business change or what’s the point? In case you haven’t been paying attention) when it comes to enhancing the company’s website, or mobile app, or, for that matter, software products like operating systems and office suites, for all intents and purposes the business change is done when the software hits the production servers.

That’s in contrast to any change you’d be likely to make to the company’s ERP, CRM, supply chain systems, or what-have-you. The whole point of changing or enhancing the company’s internal systems is to support a change in how we conduct business around here.

DevOps for internal applications is, that is, qualitatively different from DevOps for customer-facing applications.

Which brings up another thought: When it comes to customer-facing systems, faster is very often better. Even if it isn’t truly “time to market” in the sense of being able to sell “new and improved” before your competitors can, when interacting with Real Paying Customers, novelty has intrinsic value.

But when it comes to employee-facing applications, slip-streaming application changes mostly means confusing employees, disrupting an established business process with no clearly defined process change in view.

Internal releases aren’t just software changes except for those that do nothing but fix bugs or modify algorithms hidden from the view of the employees who see their results.

Internal releases are business change releases. Otherwise they’re just releasable builds — dots to be connected when the actual release is ready for deployment.

Releases that are business changes call for communication, retraining, sometimes changes in the metrics the company uses to gauge its effectiveness, and occasionally something even more extreme, like changes in reporting relationships.

There are situations when changes to business processes can be just as iterative and incremental as Agile makes software changes and enhancements.

But just as often, and especially when regulatory and compliance issues are involved, bundling business changes into controlled releases really is the better choice.

Business change like this isn’t necessarily better for being faster.

DevOps still might have a place here, but its place is more likely to be releasing a continuous stream of small changes to a sandbox environment than true continuous integration and delivery.

There’s little value in IT dramatically outpacing the natural pace of change in one or another part of the business. Unless, that is, the natural pace of business change is slower than it ought to be because everyone expects IT to be the bottleneck, and adjusts their velocity expectations accordingly.

Blowing up those expectations? It can do wonders for a business culture steeped in the perverse pleasures of slow.

Comments

|In: App Dev Methodologies|Last Updated:

If everyone did everything right from the beginning, it would be easy.

“It,” in this case, is automated regression testing. “Did everything right” is building a strong test plan every time IT put new software into production, and then added it to the existing regression test suite.

Easy peasy.

Easy peasy if, that is, you’re a management consultant like me and not CIO of an actual IT organization. Easier peasier than the alternatives, perhaps, but easy? There’s nothing about software quality assurance that’s easy.

But very few IT disciplines rank as high in importance when it comes to succeeding as a modern IT organization.

The IT pundit class has discovered the need for speed, and about time. Speed, more than any other single factor, is what lets businesses outmaneuver their competitors, thereby profitably selling more products to more customers.

And … in most businesses, most of the time, delivering needed information technology is what limits the pace of change. Speed up IT, speed up the business.

Not there aren’t any number of other factors waiting in the wings to keep things slow, because slow quickly becomes a habit, not a matter of critical path planning. But I digress …

For business to be faster, IT has to be faster at delivering changes to the applications portfolio.

Which in turn means DevOps is in your future, which in its turn means automated regression testing is in your future, if it isn’t an important part of your past and present.

Yes, DevOps. Call me a converted skeptic. Back when folks thought the lead DevOps story was that Dev and Ops were now collaborating, it earned a gigantic ho hum from yours truly.

But as it turns out, DevOps is the least interesting aspect of DevOps. What’s most interesting: DevOps blows up our old understanding of how to move software out of development and into production.

The Standard Model involves bundling software changes into major releases, which then are subjected, not only to the full range of test protocols (unit, integration, regression, stress, and user acceptance), but then have to run the Change Advisory Board (CAB) gauntlet.

The assumption behind this bulky approach to software implementation is that each new release carries with it the potential for blowing up production … either by sucking up server or network capacity so as to slow everything to a crawl; by corrupting one or more corporate databases; by opening up a gaping, easily exploited security hole; or by some other nasty consequence software defects can cause.

Not that these concerns are unfounded. Software defects can cause any or all of these problems, and the bigger the release, the more opportunities there are for bugs to be hiding that can cause them.

What DevOps does that’s truly interesting is stand this equation on its head: Instead of bundling changes into the major releases that create so much risk that drastic measures are called for, it puts changes into production in large numbers of small doses.

And because each release is small, and has been … and this is crucial … subjected to automated regression and stress testing, the risk of it blowing things up is so small that the whole CAB process becomes a fifty buck solution to a five buck problem, as it were.

The magic buzz-phrase is “continuous delivery,” and to give you an idea as to whether “continuous” is an exaggeration or not, way back in 2011 Amazon was making production changes every 11.6 seconds.

This incredibly rapid pace of change lets Amazon test different selling approaches, fine-tuning its merchandising to an astonishing, and, if you’re a competing retailer intimidating extent.

In your case?

Here’s where it gets interesting (or, depending on your level of interest, more interesting): As you know because you’re a regular reader here, there’s no such thing as an IT project, which means there’s no such thing as a software implementation.

This is something even the most advanced DevOps practitioners get wrong. When Amazon deploys its website changes, what’s changing is its selling approach to a large enough fraction of its online customers to provide a valid statistical comparison to its current practices.

When a DevOps team working on an internal application releases changes, for the most part they’re changes to internal business practices.

Which leads to this question: Should DevOps teams just slipstream changes into production as Amazon does on its website? If not …

* * *

Tell you what — I’m not going to do all the work on this. Post your thoughts on how IT should issue changes to internal business systems and the business processes and practices they support as Comments this week, and we’ll continue the conversation in next week’s KJR.

Comments