“When I die, I want the people I did group projects with to lower me into my grave so they can let me down one last time.” — Source unknown, provided by my friend Tom Law.
Year: 2019
Captain Obvious presents: DevOps!
When my children were young, I offered them two alternatives. They could either make the same mistakes I made growing up, or they could learn from my mistakes and make a bunch of new ones instead. I’ll let you guess. Heck, I can only guess myself.
But I don’t have to guess about DevOps, where some practitioners are making mistakes IT learned to avoid decades ago.
In particular, we learned IT shouldn’t release application changes into the wild without: (1) conducting comprehensive regression tests if the application change in any way alters system integrations; and (2) providing at least delta communication, and in many cases delta training, if the user interface changes.
Wait! Why am I wasting your time with 30+ year old wisdom?
I know it looks like I’m changing my name to Captain Obvious. But while I know you know better than to engage in these IT worst practices, that doesn’t mean your technology vendors do too.
If you’ve read anything about DevOps, you know CICD is a key element. But many vendors, having invested significant money and effort into adopting DevOps as How We Do Things Around Here, only got three of the four letters right: Continuous, Integration, and Continuous.
But they read somewhere that the “D” stands for Deployment, and, with the enthusiasm of the converted, gave the matter no further thought.
As a regular KJR reader you know the difference between a release and a releasable build, and with that the difference between Continuous Integration/Continuous Deployment … appropriate for eCommerce applications where what changes is the customer’s shopping experience … and Continuous Integration/Continuous Delivery, the model that works for applications whose purpose is to support internal processes and practices.
Just in case: The difference between delivery and deployment is simple: Delivery installs to the staging environment; deployment installs to production.
It’s past time to make sure your vendors deliver to staging and don’t make you vulnerable to vendor-DevOps-driven unmanaged deployments.
Start with your contracts. Do they prohibit your vendors from deploying changes to the UI and application interfaces without your permission? If not, start negotiating the requisite contract changes.
SaaS providers are particularly notorious for unmanaged deployments, because you can’t block changes they install on their servers with your defenses. But increasingly, providers of customer-installed COTS are adopting DevOps practices too.
This doesn’t make it okay to go back to … or, in many cases, to persist in … the outdated practice of staying on stable versions as long as possible. Staying current or nearly current is no longer one choice among many. In an age of state-sponsored and organized-crime-sponsored assaults on your information systems, staying current or nearly current is now the choice, not a choice.
So let your vendors off the hook, and accept a deadline for implementing new releases. Your vendors should give you ample time to test. You should let them retire ancient versions.
This doesn’t just apply to the IT vendors you have, either. Every time you go through a solution selection, make sure you include your requirement that the vendor doesn’t push changes into your production environment without your knowledge and consent.
Second: It’s time to automate regression testing. Yes, setting it up is painful and expensive. For that matter, maintaining the automated test plan is no picnic either.
The alternative, though? There’s an old, old rule in IT, which is that you always test. Professionals test before putting software in production. Amateurs test by putting it into production.
And we’re well past the time when just grabbing a bunch of end-users and having them bang on their keyboards for an hour or two will give IT a passing grade.
Third: Make #2 less expensive by cleaning up your interface tangle once and for all. While reliable industry statistics are hard to come by (strike that — they’re impossible to come by), anecdotal and conversational evidence suggests that the ongoing cost of maintaining an ad hoc collection of point-to-point interfaces can, over time, overwhelm IT’s ability to implement new applications and application changes.
To put a bow on it: How could DevOps proponents make such an elementary mistake as conflating delivery and deployment? It’s sadly easy to understand. As a member in good standing of the KJR community, you understand there’s no such thing as an IT project.
But we’re still in the minority. The IT pundit class thinks moving from projects to products is an exciting transformation.
If the job is done when the software runs, CI/CDeployment is fine and dandy.
If it isn’t … choose the wrong goal and wrong practices are inevitable.