In the first presidential debate, the candidates vied for primacy in foreign policy expertise by demonstrating their ability to pronounce the name of Iran’s president, “Ahmandinejad.”

I call it a tie: Both successfully managed all five syllables enough times to prove it wasn’t just luck. It was almost as exciting as a spelling bee, although with fewer surprises.

Speaking of current events, last week’s column explored some of the lessons IT can apply from the current financial crisis. After these last licks we’ll take a break from breaking news for awhile:

It’s never quite that simple: Last week I said wishing doesn’t repeal arithmetic: “Aggregating a thousand loans, each with a one percent probability of failure, creates a very safe investment. Risk calculations don’t work that way, as anyone who has taken Statistics 101 knows.”

The real problem requires Statistics 201. As several readers pointed out, the risks of the thousand loans are correlated, so as mortgages started to default they crashed housing prices, causing more defaults.

Kevin O’Donnell drew the Lesson for IT: When performing a risk analysis, ask yourself, “What could make several of these things fail at the same time?” Don’t put your backup server on the same UPS (better, don’t put it in the same data center).

And ask your multiple ISPs about facilities sharing. Some Silicon Valley companies have spent painful hours out of the eMarketplace because their risk management plan arranged Internet access from two separate providers . . . who shared the same pipes.

Know your sources’ biases. In a recent story about the crisis, The Economist called the financial sector “the economy’s brain.”

Brain? Id is more like it — uncontrolled “I want what I want when I want it” behavior. As Freud might have explained, every id needs a superego to keep it in check.

The Economist has, for all its strengths, a clear regulation-is-bad/private-enterprise-solves-all-problems slant. Hence the overly flattering “brain” metaphor.

Lesson for IT: CIOs need to analyze the bias of every source of information they use (except, of course, Keep the Joint Running, which is unfailingly objective in its accounts of things).

Those with no skin in the game break the game. Deep down in the core of the financial meltdown lies a very soft bedrock of bad mortgages. They were sold by companies that didn’t care at all if they were good mortgages. They sold the paper and moved on to the next borrower.

My business partner suggests a regulation requiring mortgage originators to hold every mortgage for three years before selling it. An alternative: An outside regulator selects five percent of the mortgages sold every month at random, to be serviced by the mortgage originator for the life of the debt.

The result: No underestimated risk, no “liar loans,” no financial meltdown.

Lesson for IT: Give the rest of the business a direct stake in the health of the IT organization and the company’s technical architecture. This isn’t easily achieved, and the most common tactics — IT treating the rest of the business as its internal customer, and charging back for its services — achieve the exact opposite result.

Urgency isn’t crisis. In a true crisis, speed matters most — any decision now is better than a perfect decision too late. When the situation allows for thought and debate, thought and debate are worthwhile investments.

Treasury Secretary Paulson’s original “plan” was, at three pages, shorter than my consulting company’s standard contract, even though the immediate crisis had passed and the situation was merely urgent, not critical. No matter what happens next, I give everyone … including Paulson … credit for thinking, debating and compromising, instead of acting like the crisis is still unfolding.

Lesson for IT: Autocracy works for crisis and art, and I’m not that sure about art. When a decision is important, find ways to involve smart people in its making.

Every boom ends. Even booms that aren’t bubbles eventually come to a close. And yet, company after company … and individual after individual … rely on the money being real and the income sustainable. So when one, the other or both go away, the results are catastrophic rather than inconvenient.

Lesson for IT: Use temporary corporate affluence to invest in sustainable improvements, not in expansions of service.

Lesson for those who work in IT … and everyone else, too: When you’re making good money, put enough away that if you find yourself unemployed, or needing to walk away from your job for any reason, you can.

Fail to do so and you’ve given up control of your own life.


* In Forbidden Planet, the Krell succumbed to “Monsters from the id.” Rent the DVD. It’s old, but the special effects still hold up and the plot is remarkably sophisticated.

Microsoft could have prevented this mess.

No, not by buying Lehman Brothers, pumping cash into Fannie and Freddie, or counter-lobbying for increased regulation. An Excel model of the financial sector would have done the job — a model of the various financial instruments used to create the trillion or so dollars of sham wealth that vanished last week.

An accurate model would have flagged the whole mess as a circular reference.

I can’t prove this. I’m pretty sure, though, that a whole bunch of financial instruments owned by the failed former financial titans included each other in their portfolios — fundamentally no different from a spreadsheet that has the formula =B1+1 in cell A1 and =A1+1 in cell B1.

Their assets increased every time they pushed F9.

In addition to possible circular references, inside this crisis are lessons — not only for the future of the financial sector, but also for IT organizations that want to prevent problems instead of dealing with frequent crises. Among them:

Excessive deregulation is as bad as excessive regulation. Maybe worse. In spite of all the bleating about the awful bureaucracy of it all, I know of no company that failed because of Sarbanes/Oxley, PCI, HIPAA, or building codes.

Lesson for IT: When developing IT policies and procedures, establish a healthy balance between a free-for-all and excessively tight controls. As is so often the case, danger lives in the extremes.

Wishing doesn’t repeal arithmetic. The core premise of recent financial instruments: Aggregating a thousand loans, each with a one percent probability of failure, creates a very safe investment.

Risk calculations don’t work that way, as anyone who has taken Statistics 101 knows. The only difference is that with one item a one percent chance of failure constitutes risk. With a thousand, it’s nearly certain that ten or so will fail.

Lesson for IT: Entrepreneurial start-ups live on risk. They run thin, and hope nothing goes wrong. That only lasts so long — after awhile, hope becomes a very bad strategy for managing risk.

No matter how implausible, logic that rationalizes what we want to be true is compelling. Conversely, no matter how airtight the logic and evidence, anything that debunks what we want to be true is implausible. See “wishing doesn’t repeal arithmetic,” above. It’s one of the two reasons anyone believed this nonsense. (The other: The greater fool theory — I don’t believe it, but as long as I can sell the junk to a greater fool before it falls apart, I don’t care.)

Lesson for IT: Just because we’re technical doesn’t mean we’re immune. We’re just as vulnerable as anyone else.

The bigger we let companies get, the more unstable the economy becomes. When small companies fail, policy makers see it as capitalism in action. When AIG was about to fail, the alternative to Paulson buying it was serious risk of total economic collapse.

Maybe unfettered mergers and acquisitions weren’t such a good idea after all. Size might have improved efficiency and economies of scale, but it also decreased overall economic stability.

Lesson for IT: This is the theory behind using distributed systems instead of mainframes — each component has a lower level of performance, but the impact of one component failing is much smaller.

The desire to blame is stronger than the desire to survive. We got lucky — Paulson and Bernanke took care of business, including themselves out of the blame-fest. They might have been the only two in Washington who did so.

Lesson for IT: Same as the lesson everywhere. Blame doesn’t matter. Taking responsibility does. If the root cause of a problem happens to be a personnel problem, take care of it after you’ve dealt with the crisis.

When principles and practicality conflict, it means the principles aren’t practical. I’m not the first to point out the irony of a Republican administration effectively socializing the world’s biggest insurer and mortgage companies. Irony notwithstanding it was the best available choice.

Lesson for IT: Be as principled as is practical, and no more so.

And finally, if you think you’re smarter than your predecessors, you’re probably more foolish. To prevent a recurrence of the Great Depression, the Glass-Steagall Act kept investment and commercial banks separate, creating a firebreak between the high risks of the former and the requirement for safety of the latter.

We’re approximately the length of the Roaring ’20s after the act’s repeal. Draw your own conclusions.

Lesson for IT (and everyone else): Before you tear down what your predecessors built, be certain you don’t need it anymore.