It’s always more complicated than it looks.
Case in point: Houston
Your hypothetical challenge: Bring the city back on line. What does this entail?
Lots and lots of lots and lots, that’s what.
Understand, I know nothing at all about civil engineering, or emergency management, and not all that much about enterprise risk management. What I do know is that nobody else can keep the list of everything that has to come back on line in their head either, let alone the interdependencies that could lead to creating a proper Gantt chart.
Oh, by the way, a half hour of Googling failed to uncover anything resembling a disaster recovery plan for the city of Houston. An emergency management plan yes, a recovery plan no.
Which isn’t necessarily bad management. As with IT’s ancient habit of trying to create comprehensive software designs before beginning to write any code, so disaster recovery plans of metropolitan scale or larger for disasters of inordinate magnitude are probably pointless. If, that is, they do much more than list the organizations that need a recovery plan, specify what one should encompass, and insist they have them.
Even then there are limits. As everyone who’s involved in disaster recovery and business continuity knows, a plan that isn’t tested isn’t a plan.
And along came Harvey.
Case in point: New York City.
I worked with a client there several years ago, enough that a corporate apartment made more sense than staying in hotels. The result, though, was that for a couple of years I qualified as a resident, meaning I owed New York City taxes. These were (1) substantial, and (2) business expenses, not directly deductible from my Minnesota state tax bill.
This sawed me off. Until, that is, I saw a sign in the midtown Whole Foods that explained New York City creates enough garbage every day to fill the Empire State Building.
It occurred to me then that I had not the slightest idea how to go about removing that quantity of garbage every day, and that garbage removal is far from the most complicated challenge in running New York City.
As I had no idea how to run NYC I certainly had no idea how to run it at a lower cost, which meant I should put up and shut up. I benefited from services I didn’t even know were being provided. My New York City tax burden was how I paid for them.
Case in point: Any legacy retirement
Over and over again, companies make this mistake: They decide to retire the ancient mainframe batch COBOL system the whole company has been running on for forty years. And from this decision comes a logistical nightmare, because no matter how you go about it, you can’t shift the entire company from the old system to a replacement at the flip of a switch. It has to be phased and staged.
And no matter how you go about the planning it turns out many business areas will be running in a mixed environment for a year or more.
But unlike New York City or Houston, a lot of this complexity is a self-inflicted wound, the result of looking at exactly the wrong end of the horse.
The problem is the decision to retire the mainframe. Not that the company should stay on it. No the problem is that this focuses everyone’s attention on what they’re migrating from. In addition to the logistical migraines this thought process creates, it results in something even worse than the planning nightmare: When the project is done and the mainframe has been retired, the business runs pretty much the same as it ran before it invested the zillion or so direct dollars, plus sweat and opportunity costs, that were needed to make it all happen.
Much of which would have been avoided had everyone focused on the opposite question: What to migrate to.
Even better, they should be focused on how each business manager at every level wants to run his or her part of the business differently and better, leading to an applications portfolio plan that will mostly let them do so.
Taking this approach, things still aren’t simple. They are, however, simpler — a lot simpler, because (for example) moving Sales to a modern CRM system is, at a minimum, clear in what has to happen.
And moving Sales to a better sales process that’s possible because of the modern CRM system’s features has actual business benefit beyond a modest reduction in the IT operating budget.
Saw this article posted on LinkedIn about HEB grocery store chain in Texas. Certainly not the same scale and as complex as a city, but an interesting insight into how they worked right before, during, and after of Hurricane Harvey:
https://www.linkedin.com/pulse/inside-story-what-took-keep-texas-grocery-chain-running-chip-cutter
Dan, I wanted thank you for suggesting this article. For me, it simultaneously brought out the benefits of motivated synergy, as well the quality of response, when someone in authority who has faced an organizational event that affects an entire organization several times previous, has built on those experiences for the future.
Go, HEB!
I don’t see how you plan for a disaster where 30,000 random people will be displaced out of a population of 5.8 million, even if you know a year before hand, let alone a week. I don’t see a way for meaningful prior recognition of the situation that occurred.
I do see the value of an individual taking a pros and cons approach where the benefits and drawbacks are discernible, even thought processes that produce those outcomes are only partially discernible. If you trust those responsible for those outcomes. The choice to learn and to trust, is yours.
For me, your article pointed to three challenges for any successful IT department. Two are obvious, trust and recognition. Trust by the organization that IT knows what it’s doing, even if you don’t understand what they are doing. Trust that IT recognizes the scope of the change they recommend and care about the success of departments affected. The third challenge I see is the minority who oppose important things simply because they don’t have a greatly detailed understanding of those things.
While their behavior will mostly seem obstructionist and irritating, I’m beginning to think that there is level of violence underneath whose goal is the appeasement of all those whose views are different than theirs.
But, if functionality and timely accomplishments for all affected remain the ironclad priorities, everyone will know when to listen and when to act.
My interest in this article is twofold. First I work for a company that is “migrating”, “bless its heart.”
Second I used to live in Houston. Some of the wounds Houston has are self-inflected.
The Addicks and Barker dams are the solution to a problem not quite a century old. In that time no one thought to verify they 1) were still solving the original problem, which the Internet said was flooding downtown; 2) were not becoming a problem in themselves.
There are two root causes here. a) The amount of water coming into the dams has increased because of the amount of rain and because all the land that drains into the dams has gone from “pasture” to “developed.” The water gets there faster. There was ongoing monitoring of the function of the dam, but not of the solution it provided. Since this was happening to all tributaries of Buffalo Bayou, the dams needed to hold water longer, not less time.
b) One of the sources of water in Houston is subterranean sands. Also there has been oil&gas production going on in some places. The height of some suburban areas has decreased. That increased runoff times by decreasing runoff capacity of the runoff channels. Those channels have not been expanded, for one thing because there was “no room.” I lived in Houston ~1971-1986. This phenomenon was already being experienced then. The Braes Bayou flood should have been a wakeup call.
But the true root cause of this and so many problems in IT is “kicking the can down the road.” In my work I like to use the term “technical debt.” I like to say that what is not appreciated is that each and every kick makes the can heavier. At some point the kicker gets a broken foot.
Turning our eyes elsewhere, where in our society don’t we “kick the can down the road?” In our communities there is transportation debt, infrastructure debt, social debt, …. And in Washington right now they are kicking the financial can in re federal debt.
It is more fun to argue concepts than to implement positive change. Marketing triumph over engineering. And over maintenance.
The thing to remember is the Darwin award. I like to tell about the guy who hooked a jet engine to a truck. During testing in a Western desert, he was ok until there was a slight deviation from straight in the road. He imbedded himself into the scenery. To what extent is the human race to this to itself?
I am waiting for that same thing to happen at work when a “conversion” being done right now using a method I like to call “big bang” goes in over the next 4+ months. Luckily I am past retirement age, so I care less about the outcome. I need to sell my company stock next year.
When I worked for a software manufacturer, we used to say there were three principles of Tech Support: 1) Easy does it; 2) If it hurts, don’t do it; 3) Always keep a backup, or as I might say today, always have a plan B (C D …).
OUT-TMESIS-STANDING.
Thank you, Sir Bob! This approach to integrated planning IS the holy grail (if one wants to make it to the future).
Very timely article; our institution is replacing six home-grown legacy enterprise applications with a consolidated, modern ERP. The approach is–very surprisingly–exactly what you suggest: focus on where we want to be rather than replicating decades-old processes in a new platform.
The political and human challenges are large, but somehow focusing on the “new and better” rather than the “same old” seems to be making things go more smoothly. An interesting illustration of human nature, perhaps…