Integrating Point and Counterpoint

Bob here. This week we share a conversation about one of IT’s critical success factors – integration. The past couple of weeks Greg has focused on this as one of IT’s central challenges, so much so that he proposed changing the CIO acronym to stand for “Chief Integration Officer.”

Now I’ve been involved in IT integration since the ancient days of Electronic Data Interchange service bureaus. Call it 1980 or thereabouts, and even back then we ran into the problem of conflicting semantics, where the sender’s definition of a data field didn’t match the receiver’s definition. The more integration projects I’ve been involved in, the more I came to the conclusion that the technological solutions to simplifying and streamlining integration did a terrific job of making the easy aspects of integration even easier while doing little or nothing to solve the hard problems, like resolving semantic mis-matches.

What do you think, Greg? Am I overstating the challenge? Or am I just out of date and it was solved a long time ago?

Greg says:  I wouldn’t say you’re out of date, although, in my opinion at least, semantic matching has gotten easier. But Integrations are as hard as they have ever been.  The problem has just shifted, a bit like squeezing a tube of refrigerated cookie dough.

Bob says: Now we know what Greg snacks on.

Greg says: Ahem. As I was sayin’, integration is as hard as it ever was. The problems are still in familiar places – especially, volume, performance and speed.  In some ways, a lot of integration projects now feel like near-real-time Extract-Transform-Load problems. The expectations are really high, such as thinking we can make a 1 MB payload of multiple features move between systems instantaneously.

Then there’s security and trust. Can we ensure that the payload we are moving is trustworthy?  Passing bounds testing is simple, but are we sure that it is data that we want to be committed to another system?

Bob: So it’s just as hard, just differently hard. Okay. Then a follow-up: I’ve often wondered if pre-built integration might be the dominant reason IT shops find ERP solutions attractive.

Somewhere or other I read a piece of advice for deciding when to use an ERP module vs a best-of-breed application. The advice: If the ERP module provides even 50% of the functionality the business needs, choose it over integrating the best of breed. My question: do you agree? And to the extent you agree, do you think pre-built integration is a driving force behind ERP? Maybe on a 1 to 5 scale, where 5 says it’s the main driver and everything else is pretext; 1 means it’s nice, but not all that important.

Greg: Prebuilt integrations do help CIOs! Especially if you call them Chief Integration Officers (of course).

ERPs do still have a long way to go, but have made improvements.  Using your scale, I think ERP prebuilt integrations probably are at a solid 2, making it so that as a CIO, you can focus on some of the more value-oriented challenges of an implementation.

CRM systems lead the way.  I am constantly amazed at the creativity of certain CRM brands and the prebuilt integrations offered to make them more enticing.  I also notice that these same brands don’t market themselves to tech leadership – they spend their marketing money talking to Sales leaders about the ease of implementation. At the risk of being controversial, my guess is that they are giving Sales leaders talking points to deflect any concerns IT might raise about the true cost and complexity of the implementation effort.

Bob: Interesting bit (to me, at least). In the early cloud days, AWS and Azure started as Infrastructure as a Service providers (IaaS) and then moved up the stack to become Platform as a Service (PaaS) providers. But while nobody was looking, Salesforce, which started life as a Software as Service (SaaS), grew down the stack with its Force development environment (now Salesforce Platform), turning itself into a PaaS provider from the opposite direction.

Which gets me to the world of integration tools like enterprise service buses (ESB), enterprise application integration (EAI) tools, APIs, and Integration as a Service vendors. My question: Do these work better than custom interfaces and integrations? Do they, that is, succeed at simplifying inter-application integration?

Greg:

That is a great question, and like a friend of mine often says, “It depends!”

ESBs, Integration as a Service, and EAI tools make a lot of sense when we are facing-

Lots of systems that all need to talk to each other (less so when all the systems need to talk to just one system).

Older platforms need to talk to newer platform. I tip my hat to the remaining RPG and COBOL programmers who figure out integrations everyday.

A need for a “Clean” architecture. The initial efforts need to be worth the outcome of orderliness and stability (And they usually are).

The biggest drawback that we face are the initial upfront costs, as well as the ongoing costs.  Business Leaders may face a bit of sticker shock with this approach.  This conversation is likely to require a lot of explanation into the gnarly bits of Tech.

Bob: I think we’ll have to hold off on the gnarly bits. Greg, it’s been a pleasure figuring out if and where we agree and disagree. Everyone who’s read this far – add your agreements and disagreements in the Comments.