Ready for a classic horror story?

It’s a dark and stormy night (of course it is). Deep shadows are spreading. In them, a nameless evil starts to take form.

A secret society is there to name that evil and fight the shadows (sound fx: crack of thunder). But it needs your help!

The secret society is Internal Audit, the spreading shadows are shadow projects — the too-small-to-notice-but-too-important-to-let-fail projects business managers charter. The name of the nameless evil? “Shadow IT“!

Now you know why I don’t write fiction.

It is, however, a horror story because the last thing IT should be doing in most organizations is treating these as evils.

The story so far:

Before the cloud became a force, IT had mostly stomped out shadow IT by locking down desktops, limiting the availability of MS Access, and disabling VBA. That successfully eliminated the risk that non-IT staff might create value with IT innovation, while shifting intruders from buffer-overflow exploits to phishing attacks.

But then came the cloud, and specifically Salesforce.com, which represented a triple threat:

  • It catered to Sales, the least rule-conscious group in any business. Being part of revenue and all, it has more political muscle than IT, too.
  • Unlike most business applications, Salesforce.com can be implemented effectively without any integration into other business systems.
  • IT couldn’t stomp it out without instituting Internet filtering, which is a labor-intensive pain in the patootie requiring additional headcount the CFO probably wouldn’t approve were IT to try to make its case.

Face it: The cloud means shadow IT is going to happen. That ship has sailed. We can either climb on board or wait at the Greyhound station, doing what we can to keep our so-called “internal customers” from climbing onto buses they no longer want to ride anyway.

Oh, and in a classic case of turnabout being annoyingly fair play, while IT can’t stop shadow IT anymore, our partners throughout the business can easily stop us from climbing the gangplank to join them.

Then there are shadow projects. Like shadow IT, shadow projects happen outside the company’s approval processes. The managers who want them to happen simply charter and assign them, because they have enough authority without anyone else butting in to make sure they’re “done right,” whatever that might mean.

Something else the two shadows have in common: As a practical matter, their costs are low, their benefits are, in proportion, high, and the cost of stopping them would exceed the cost of just letting them happen.

One more characteristic they share is that they increase risk, because the folks who take them on often aren’t as familiar with the company’s compliance requirements as IT professionals are who have to take them into account with everything they do.

That doesn’t, however, mean IT has to become the Deputy Dawg of company compliance.

The fact of the matter is, Internal Audit gets a bad rap in most companies. It’s goal isn’t to prevent people from doing their jobs. It’s to recommend a healthy set of management controls, then to make sure everyone complies with the controls management actually adopts.

Describe a hypothetical shadow project to Internal Audit. Make it a typical one — one that’s going to build some shadow IT that won’t cost very much, and will do something useful and profitable. Ask if it should be stopped because it might lack the proper controls, and the most likely answer will be, no, don’t stop it. Just have it add the proper controls.

Internal Audit isn’t the only compliance function in the company. Inside IT (or inside its orbit), enterprise technical architecture management (ETAM) and information security wear compliance hats, and sometimes the PMO as well. Because they’re compliance functions, making sure everyone complies is almost instinctual. That is, after all, what compliance means.

And so, ETAM turns into the architecture police, the PMO turns into the methodology police, InfoSec turns into the Value Prevention Society, and IT takes the easy way out, doing its best Sergeant I-see-nuthink! Schultz impression while deploring the whole shadow enterprise.

There are alternatives. ETAM and InfoSec can collaborate to create a secure and supported end-user-computing platform. Internal Audit can provide a compliance checklist available to anyone who wants it.

IT can provide consulting services on how to design and build small applications. And if you have a PMO, it can provide training and coaching in ultra-basic project management techniques.

Preventing failure and encouraging success aren’t the same thing. The difference is the vast gap separating “No!” from “How can we can help?”

Remember the rule from the KJR Manifesto, that there’s no such thing as an IT project — they’re all business change projects that make use of information technology?

It’s just as true for the projects that result in so-called “shadow IT” — the information technology that happens without IT’s direct involvement. And because it’s shadow IT, the folks who ask for it know this. They’re looking for business improvement — that’s where their thought process starts. The linkage is automatic.

Last week’s column explained why IT should start supporting shadow IT. But that isn’t enough. We need to support shadow projects as well … the too-small-to-notice-but-too-important-to-let-fail projects business managers charter to make their shadow IT happen, and also to make all kinds of other stuff happen too.

Let’s imagine, for the sake of argument, that your company has established a PMO or EPMO ([enterprise] program management office). If it’s like most PMOs, the company’s project managers all report there, and one of the rules is that all company projects must be managed by its trained project managers. That way, the company doesn’t risk investing in projects that are managed poorly.

Sounds a lot like the arguments against shadow IT, doesn’t it? Like those arguments, the driving force is risk reduction, but the actual impact is mostly opportunity avoidance.

Limiting the number of projects a business can take on to the number of available project managers artificially limits the company’s capacity for change. And when it comes to change, any bottleneck other than the company’s ability to absorb it is inappropriately limiting — a decision to adapt and improve more slowly than necessary.

Which is why, in so many companies that have established an official PMO or EMPO, business managers charter lots of under-the-radar projects.

The shadow project situation sounds more and more like shadow IT, doesn’t it?

On the whole, shadow projects have less risk and yield higher returns than most of the official projects in the company’s portfolio, a natural consequence of their being small, short, tightly focused, and properly sponsored.

Yes, properly sponsored, something that’s more-often true of shadow projects than official ones, because shadow projects are started by business managers who personally want them to succeed. This makes them sponsors … real sponsors, by definition … and the importance of sponsorship in effective project management is well known.

Just in case: Real sponsors want their projects to succeed enough to stick their necks out and take risks when necessary to support their project-manager partners. That’s in contrast to assigned sponsors, who are thrown in front of official projects, just because the methodology says every project has to have one. Assigned sponsors don’t really care, because why would they?

So shadow projects have less risk than their formally chartered brethren. Except for one thing: They’re mostly led by employees who, while promising, have no project management training or previous experience. Their managers/sponsors, themselves usually unaware of what project management actually takes, tell them, “This will be a terrific development opportunity for you,” ManagementSpeak for “There’s a bus approaching at high speed!” followed by a shove.

The result is that right now, many shadow projects aren’t managed as projects at all, because the employees who are put in charge of them have never managed a project and have no idea where to start.

They need help.

So here’s a thought: Instead of trying to stamp out these shadow projects the way IT used to try to stamp out shadow IT, why not provide some support?

Like, for example, giving about-to-be-run-over-by-a-bus neophyte project managers some tools and training, instead of treating them like orphan stepchildren. The secret, and the challenge: Those best equipped to provide the tools and training know too much about the subject. They know, that is, the techniques needed to implement SAP, erect a skyscraper, or build a nuclear submarine.

What many of them don’t know is which of those techniques can be safely jettisoned when the task at hand is managing a team of three people for a few months — at a rough guess, 90% of their expertise. As is so often but so strangely the case, scaling something down can be harder than scaling it up.

Still, it can be done, and doing it is important. In the aggregate, shadow projects add up, even if no one of them is a big hairy deal.

If the PMO/EPMO reports inside IT, the CIO can make shadow project support part of its charter. If not, there’s no reason IT can’t provide it on its own.

Which is a nice irony: Where IT used to do its best to stamp out shadow activities, it has just become an active conspirator in them.