The Standish Group, in its annual Chaos Study of project completion rates, has been quoting from Tom Clancy’s The Sum of All Fears: “The Roman bridges of antiquity were very inefficient structures. By modern standards, they used too much stone, and as a result, far too much labor to build. Over the years we have learned to build bridges more efficiently, using fewer materials and less labor to perform the same task.”

At the risk of quibbling with Mr. Clancy, some of those Roman bridges are still standing a millennium later, while some of our more efficient ones have tumbled into the bay [Corrected from “a half millennium later – Bob]. Adherence to budgets and schedules is our preeminent ethic. One suspects Rome held different values.

But I come, not to praise Caesar, but to bury him: An IT department’s credibility depends on its ability to get projects done. And if we believe the Standish Group, getting them done is the exception, since about half finish late and over budget and nearly a third are put out of their misery altogether.

If you’ve read anything about this subject, you’ll find an unfortunate focus on process and methodology. Supposedly, the key to success is following the right steps in the right order, whether you’re building bridges or writing software.

The experts, I’m afraid, are wrong. Or, rather, they’re insufficiently right: Following good project management practices — the right steps in the right order — is a darned good idea. It’s insufficient, though: effective project management isn’t a cookbook exercise. You can’t just follow the recipe and be sure the cookies will come out tasty.

The most important factor in project success isn’t having a repeatable process or sophisticated methodology, nor is it, for that matter, the project management software. It’s having a strong, experienced project manager to lead the project. Seem obvious? Of course.

So why do so many companies actively prevent project success by ensuring they have no experienced project managers? The career ladder is the problem: Project management is used as a bridge position – not in the Roman sense of solidity and durability, but in the American sense of being something to cross as quickly as possible.

Project management is the bridge between staff and management.

It’s a test to see whether you deserve a department of your own. Do well and you’re promoted out of project management. Faced with the additional salary and prestige of a management position, how many good project managers will say, “No thanks. I want to keep managing projects, because I like working longer hours for less money and respect.”?

Here’s a novel thought: Create a project management career path. The entry point is apprenticing with an experienced project manager, the progression moves from managing small projects through large programs, and the end-point is leading your company’s program management office (PMO). Managers of small projects should be parallel in compensation and prestige to workgroup supervisors; the head of the PMO should have as much influence and prestige as the CTO.

Want your technology projects to succeed? Stop worrying about employing project management methodologies and start worrying about employing strong project managers.

They’ll take care of the methodologies.

There’s only one long-term answer to the IT labor shortage, and that’s Tom Swift Jr. If you aren’t familiar with the series, Tom Swift was for science fiction what Nancy Drew and the Hardy Boys were for mysteries. Swift was a genius in his late teens who invented cool stuff, had lots of adventures, and always saved the day.

I grew up on the Tom Swift stories, and dreamed of doing stuff like that too.

If Isaac Asimov is to be believed (and if you can’t believe Ike, who can you believe?) there’s a very strong correlation between reading science fiction as a child and entering a technical profession as an adult. If, as a nation, we want more home-grown scientists, engineers and programmers, the long-term solution is obvious.

In the meantime, you need to hire some IT professionals, you can’t find them, and you can’t wait until the science fiction curriculum is added to the schools, kids grow up in it, and they graduate and enter the workforce. What are you supposed to do right now?

I wrote about the IT labor shortage two years ago, and to show you just how much influence we pundits have, very little has changed between then and now. The fact of the matter? There is an IT labor shortage, but it’s mostly self-inflicted.

Recruitment ads still specify computer science degrees and long lists of specific products. Why? Companies run too lean, hiring only when there’s a specific, urgent need. They can’t afford the time needed to retrain good employees or new hires. Voila! Instant shortage. When nobody is willing to train their employees, it’s inevitable.

For example, some companies are laying off their Y2K staff, who don’t have the “right” skills, while simultaneously recruiting other IT professionals who do — a process that may easily take three months and a $20K+ signing bonus.

Well gee whiz, kids, you are in a pickle, aren’t you? I wonder what would have happened if you’d spent the $20K and three months retraining the poor schmuck who worked his eyeballs out fixing your Y2K problems instead.

Are these companies really that dumb? It’s possible. According to astrophysicists, stupidity may be the “dark matter” that makes up as much as 90% of the universe’s mass. Unlike IT labor, there’s no shortage of the stuff.

But there’s another possibility.

Firing unproductive employees is difficult. It’s emotionally draining, procedurally intense, and legally risky. Sometimes, layoffs are smokescreens that allow companies to terminate employees who aren’t making the grade. They may be incompetent, hard to work with, or they may have just run afoul of company politics, but for one reason or another they’ve been labeled as “undesirable”.

Three decades ago Harold Sackman showed that the best programmers are 20 times more productive than average ones. Imagine what the other side of the bell curve produces. If you factor these folks out … and if you’re hiring, you should … you find another reason there’s a shortage.

One thing is certain. IT managers can’t find the people they’re looking for. So if you’re looking for work in IT, that puts you in the catbird seat. All you gotta do is to be that person. Which also means that if you can’t find work … and there are a lot of programmers and analysts who can’t … it’s time to take a long, hard look in the mirror.

Companies are hiring. If they aren’t hiring you, there’s a reason.