Speaking of metrics, and their limitations, here’s a biggie: You can’t measure the results of any path you didn’t take.
The ramifications are enormous whether it’s public policy, business strategy, or your own day-to-day management decisions. While we don’t know, will never know, and in fact can’t know the outcome of the alternatives we didn’t pursue, we still have to choose between engaging in post mortem analysis and denying its value.
Those who promote post mortem analysis (I’m among them) consider it essential for personal and organizational development. Even though you can’t know with certainty what would have happened had you chosen a different path, the exercise will still make you smarter and better-able to make the next decision.
Whether the subject is:
- FDR’s New Deal, and whether it does or does not deserve credit for the GDP’s 10%-per-year growth ($600B to $860B) throughout his first term …
- Healthcare, and whether the country would be better off had “HillaryCare” been turned into law in the 1990s …
- Microsoft’s tragic decision (I’m biased) to replace its old menu-plus-button-bar user interface with The Ribbon …
- Your company’s decision to invest in a new product line or abandon an aging one …
- Your decision to outsource data-center operations …
… the question isn’t whether you should engage in post mortem analysis. It’s how you should go about it.
For guidance, look to the game of bridge.
Those who prefer to “move forward instead of looking backward” explain that retrospective analysis is a distraction from the important work of building the future (or in the case of bridge-playing, dealing the next hand).
Good bridge players, in contrast, dissect difficult hands after playing them. Both sides participate. For them, the post mortem is an intellectually honest attempt to find better ways of analyzing the situation, describing hands through changes to the bidding system, guessing the placement of hidden cards, and at times tricking opponents into making bad plays. Their goal: To understand the game better so they can bid and play future hands better.
In addition, the discussion will help them figure out which players will make good partners, and will help them understand how each player thinks, so as to partner with them better.
Some players do dissect so as to blame their partner for failures in an attempt to avoid blame for their own decisions: “My bidding was fine — you could have made the hand if you had played it better.” While annoying, even these second-raters end up learning from the discussion.
Take your guidance from the best bridge players. Your world has parallels to all of their post mortem goals, beginning with the importance of intellectual honesty, and, as promoted here from time to time, of its organizational equivalent — a “culture of honest inquiry.”
It isn’t difficult to tell who is and who isn’t intellectually honest. For example, anyone who uses both of these phrases: “the blame game”; and “we need to hold people accountable”; is intellectually dishonest. They are engaging in my-team/your-team game-playing, not honest discovery. It’s the blame game when you’re trying to determine how their team’s actions contributed to a problem, but it’s holding people accountable when they’re trying to pin the blame on a member of your team.
Likewise, anyone who tries to make you angry at someone else or some identifiable group is intellectually dishonest: Anger interferes with dispassionate analysis (by definition) which is why it’s a favored tool of the propagandist. Ignore them.
Something that should go without saying but apparently doesn’t: Those who deliberately spread falsehoods in order to “win” are also among the intellectually dishonest. Ignore them too.
The intellectually honest begin a post mortem analysis by laying out the previously accepted model or the competing models of How Things Work. They describe the actions taken for which the post mortem is being performed. They ask how the results affect the group’s shared understanding, and work to bring all stakeholders into consensus regarding an improved model … possible when everyone involved participates in a culture of honest inquiry; impossible otherwise, which is why the culture is so important.
When all stakeholders share an understanding of how the planetary climate, or national economy, or company business model, or IT organizational model works, discussions of how to address a current situation are efficient and productive.
When they don’t, the only alternatives are authoritarian decisions or political compromises.
It’s hard to say which is worse.
I was with you, Bob, right up until you included ‘holding people accountable’ in the intellectually dishonest category. I agree that many times it is simply a more politically correct way to say ‘it’s not my fault’, and thus deserves to be on your list. However, there are times when it is being used honestly, and sometimes the post mortem identifies the underlying issue as a dishonest, lazy or intentionally negligent individual, and that individual issue needs to be addressed, NOT covered up with process modification, group training, or other workarounds.
We’re in agreement then. I’ve written quite a bit about the problems that go with holding people accountable – among them, the inevitable game of “whack a mole” that ensues, scapegoating, and the impossibility of obtaining accurate information about what actually happened.
There’s a huge difference between discovering that an individual is the root cause of the problem and assuming all problems are the result of employee performance deficiencies. “Holding people accountable” is an exercise in assumption, not conclusion, which, along with the aforementioned fogging of information channels, is why I include it in my list of symptoms of intellectual dishonesty. – Bob
I am have not been in a mgmt position for several years now, so I am no longer in charge of the post mortem analysis in our IT department, but I have participated in many of them. There is a pattern to our post mortems, namely, answering these three questions:
1) What did we do right?
2) What did we do wrong/poorly/suboptimally?
3) What can we do to improve?
The astute ones will notice that question 3 only addresses question 2. There is little (typically, nothing) in the way of comparing the answer of question 1 with the plan, the norm, the project mgmt methodology, or whatever else you want to call “the way we do things around here”.
While there are some people who must think they know what that plan/norm/methodology is, it is either poorly communicated to us in the trenches, or is almost ad hoc for every project. Aside from a few standard (and sometimes perfunctory) documents that are cranked out to “define” the problem, we don’t have anything in the way of reliably repeatable processes.
(I used to be in mgmt…I tried to get upper mgmt to see the need for a little more structure…and I’m no longer in mgmt…I don’t know whether there’s a correaltion there…)
And now, the big plan is to launch off into the world of Agile development. How will we know it works? Down-trending bug reports? Getting products out the door more rapidly than has been the historical norm? Or merely a declaration of victory by someone in upper mgmt?
btw, how does one do post mortems in Agile-land?
Two small points.
When creating something or achieving a milestone, it sounds much better to call it post partum analysis. Most of us in our everyday work don’t create corpses.
And in our organization, PMA is covered under the “root cause analysis” methodology. The company is all about back office financial processing. Most things are repetitive, even the act of modifying/creating code in its various forms. The goal, then, is that if the outcome appears not optimal, to adjust the process where possible.
Bob
As always, an intriguing article. In my experience, the “hold them accountable” fails on many levels. It is paternal, it supports unneccesary hierarchy, it assumes 100%-0% culpability, and much more. The best approach is to (and it takes time) a culture of discipline where people hold themselves accountable. I get to work on time and get assignments done not because of fear or external sanctions but self image, doing lots of meaningful work, having a sense of belonging, and sensing that the organization is generally fair. These and other factors build self discipline and the sense of not letting down colleages, many of whom I respect. They, in turn, don’t want to let me down. These intrinsic factors are the powerful forces, not some boss trying to act like a “boss”.
But what I wanted to add to your various manifesto points is this: all organizations I have worked for or with have some to many managers who reward problem solvers. They praise Bob or Mary for jumping in and solving problem X before it gets worse or before it harms customers. But I rarely see managers reward or even notice the problem preventors. These people prevented a problem by engaging the staff who in turn changed a procedure that, if not changed, would have led to a problem. Or they are fair and respectful to their colleagues who in turn give you a “heads up” about a change being considered by a project leader. But because of the received information, you go and have a chat that keeps this disaster from happening. I have mentioned this phenomenon is class and meeting and typically get blank stares-managers are unaware of the myriad problems prevented by talented people but clearly see the highly visible problem solvers who, many times, created the problem with poor management skills that they then solve.
Another excellent column. And, as a good duplicate bridge player myself, I think the analogy perfect. There’s another saying we have a bridge that applies to management: Even a bad plan is better than no plan at all.
Dana Murphy
Sonoma Orchid Inn
Pingback: IS Survivor Publishing
Well said, Bob
One major problem that prevents PMA from being successful or properly undertaken, is that we are often too concerned about WHO did wrong, vs WHAT went wrong. And this speaks to your concerns about “accountability”.
Also, we need to take the time to recognize mid-project changes that were implemented to avoid issues, but where not formally a part of the methodology. These might need to be incorporated in a more formal fashion.
And I really agree with the points that Sam made in the earlier comments.
ASB (My XeeSM Profile)
Providing Competitive Advantage through Effective IT Leadership
Pingback: Lack-of-analysis paralysis | IS Survivor Publishing