It’s called “solution selection.”
There are those who grouse that we should stop the doublespeak and call it software selection.
Not me. We engage in this sort of thing because we have a problem that needs solving, after all (or, in happier circumstances, an opportunity that needs chasing). A software product by itself is, as logicians might put it, necessary but not sufficient for solving (or chasing) it.
The fundamentals for going about solution selection are, by now, well understood: When comparing the alternatives, evaluators should dig into: (1) Features and functionality — does the solution do what you need it to do? (2) Internal construction — is it built well? (3) Terms and Conditions — what will it cost, and beyond cost is contract and licensing language acceptable? (4) Vendor characteristics — is the seller financially viable and, almost as important, easy to work with?
The fundamentals are well understood, and yet the results, more often than not, are disappointing.
What goes wrong? What follows is just a starting list. I hope you and your fellow members of the KJR community will add to it in the Comments so we can, as the saying goes, all be smarter than any of us are.
So with that in mind, here’s my shortlist of hard-to-avoid solution selection gotchas:
Wrong selection: The selection is always wrong and has to be. No matter the depth of due diligence, some warts only appear when you implement. So you’ll see the awful reality of the selected solution. Everything else exists in the glorious world of brochureware. It’s a case of the grass always being greener under someone else’s GrowLux.
Each requirement vs all requirements: I’ve seen this a few times — a supplier’s products can handle every requirement the solution selection team has identified. Sadly, no one of the supplier’s products can handle them all, and the ones you need to cover everything aren’t integrated.
Feature scaling: A vendor can usually get its software to do what you need it to do. It can make it do something else you need it to do, too. But there’s often a limit on how many different something-else-you-need-it-to-do-toos their software can do at the same time, because really, what they’ll have to do to get their software do each of those things is a one-off. Doing them all makes it a kludge.
SaaS means not having to worry about the internals: Wrong! Saying that a software product’s internal engineering doesn’t matter because it’s SaaS is a lot like saying an airplane’s engineering doesn’t matter because someone else is flying it.
There’s a limit to how much of a solution’s internal engineering a provider will share. In the case of software what you probably care about the most is the data model. Explain what you need to know about each type of stuff the software will manage, and ask where that data gets stashed.
Terms, conditions, conditions about conditions, and conditions about conditions about conditions. Some software vendors still hide terms and conditions inside linked documents, inside documents linked to the linked documents, and so on ad infinitum. I guess it makes sense that if good software design means that modules invoke modules that invoke modules, that good software license design would be similar.
Regardless, some T&Cs can place unfortunate limits on what you can do with what you’re licensing. This is why you need lawyers.
Vendor from where? In Greek mythology, Prometheus was the good guy, stealing fire from the gods for humans to use. In Christian mythology, Lucifer (light-bringer) was guilty of pretty much the same crime, assuming you’re willing to overlook that little war-on-heaven peccadillo.
Some software vendors are more Lucifer than Prometheus (you were wondering where this was going, weren’t you?).
What can you do to anticipate this? References won’t get you there. Even the worst vendor will have a few customers who like it. The best you can do is ask around and hope a few people you trust have had experience with the vendors you’re considering.
What else can go wrong? Lots else can go wrong. As I said at the beginning of this column, this list is just for starters, and it doesn’t include any of the mistakes customers make while implementing the solution they’ve so painfully selected (for example, thinking they’re implementing the software and not changing the business).
So now it’s your turn. Jump to the Comments and share your insights as to what can go wrong while choosing the right solution.
The KJR community awaits your wisdom!
Wow — lots I can think of. Let’s see:
— Shiny-object syndrome. Some feature is so attractive that the rest of the due diligence is duly ignored. (Note: This can be a problem for both business-side stakeholders and for IT-side stakeholders.)
— Forgetting about implementation, especially training. As a former manager of an IT training department, it was often up to me to tell some contract manager that the vendor has proprietary training, that only they can deliver it, that it isn’t included, and it costs about 20% of your original purchase price. Oh, you didn’t include it in your contract? Whoops.
— Forgetting about interoperability and APIs. Somehow, the more expensive the solution, the more closed it is. Without naming names, I can think of an enterprise HR solution that kept all the data (and I mean all of it) in proprietary formats inside their own custom database solution. The only way to get any of the data out was to dump it, one report at a time, to a CSV file. Everyone from IT screamed about this, but Shiny Object Syndrome prevailed.
There’s three that immediately come to mind. I’m sure I can think of others, but I’ll back out and let others take a crack at the RFP pinata.
How long to sunset? One issue I’ve run into is purchasing a solution from a vendor only to find that the solution was sunset a couple of months later. Oh, but the vendor will continue to maintain it … sure they will. It’s important to ask the vendor if a new replacement solution is imminent. There might even be an opportunity to beta-test the new solution if your organization has the resources to help out.
As you already pointed out, a software product, by itself, is necessary but not sufficient.
In my experience, a significant missing piece is “business process.” But the entire process is fallacious. It is simply not possible to create a itemization of requirements that adequately or accurately describes either the problem or the solution, partially because the objective is constantly moving but more because it’s not possible to describe a “whole” by itemizing its “parts.” This is the fundamental absurdity of the RFP process that our government(s) seem to be addicted to. It is also the reason that agile development, however you choose to do it, gets acceptable results — stepwise approximation of the solution, persistently getting closer and closer, but never completing.
I recall, some decades ago, sitting through a tedious class, trying to remain awake, when the instructor drew a graph on the board describing the “delta” problem. It was a revelation.
After great soul searching and very diligent effort, we lock down the requirements and commence development (fundamentally, we selected the vendor and they are us). As we progress, glacially, a strange tension begins to emerge. Our developers are beginning to understand the business problem and have an ever-growing clue that what we’re building isn’t what’s needed. Meanwhile, the “users” (the customers of a pusher) have an equally growing awareness that what we’re building ain’t what they want. There’s a big reluctance to issue change orders (and a bigger reluctance to accept them), particularly when the fundamental architecture is indicted. So, we conspire to “look the other way,” because the effort is already behind schedule (largely because the schedule had nothing to do with reality and was imposed by those who had no idea what was involved). Eventually, there’s a lot of fingerpointing and we embark on V2.0. Or 3.0. Or the same project with a new name.
No commercially-available solution can possibly escape this same descent into hell. Yes, the vendor will tell you how wonderful it’s going to be, and will keep insisting upon that until you finally face the one requirement they cannot meet. (And, yes, there always is one.) Your choice is to abandon the investment made to this point, unlikely, or change your business to comply with the business process the vendor’s architect originally envisioned.
Back in the “good old days,” when a new system was actually the driver for transformation (nice disguise, huh?) we sometimes lucked out and had an architect who designed the business process in parallel with the software. But those were the days when businesses were significantly less complex and one “chief programmer” could keep the whole enterprise in his/her head.
The critical requirement, one I’ve never seen documented, is that the business is changing at the speed of the Internet and the “solution” needs to be able to adapt even faster. Unfortunately, a solution sold to more than one customer can never meet that requirement, leading to the “how much are we able to compromise?” question. In the end, compromise leads to a lack of uniqueness. You give up your competitive advantage to be able to keep operating on the solution selected last week, last month, last year, last decade, or so long ago no one remembers.
If the solution is essential to your competitive success, you have no choice but to build it (or, at least, the parts of it that need to be unique). If it’s not a competitive differentiator, buy something close and adopt the work flow the vendor intended, then put your efforts into being exceptional at something else.
All that other stuff — that’s an exercise for the lawyers 🙂
When Manglement trusts their favorite vendor without checking with IT whether the proffered solution actually solves the problem at hand, you can bet that it won’t. That unchecks pretty much all the boxes you listed, but I see it happen over and over….
Being more than a little impressed by depth and variety of perceptive things to look out for above, I’ll take a different tack: strongly involve staff who were experienced developers before joining your IT department.
I had 10 years programming and development experience in my own tiny company. My boss had 10 years experience as programmer and a manager for a small software company, before he hired me. His approach was top-down, often seen natural systems analysts. My approach was bottom-up, my nature style being the strategic analyst.
Our experience and ability to communicate with each other and our users meant we double-covered the bases, because we had already internalized looking for many of the red flags above.
We weren’t smarter than any other IT professional, but we had already made many of the mistakes your article and other readers have mentioned.
It is no doubt harder to find the mix experienced developers in a big IT shop, but in the situation described in the article, they can worth more than their weight in gold.
For SaaS especially, I’ve started to seriously consider investing time in a POC. It allows me to test their support teams and also see if the software does some of my most common use cases as easily as claimed.
The other one that should be on the checklist now is security. Since most software is now cloudy, there is an entirely different security risk than in your own data center. Does the software support SSO or multifactor authentication? Where is the data kept? Does the vendor have a SOC 2? Etc.
Don’t “ospreyize” the project just to get funding. The Osprey jet/plane/helicopter “all things to all pilots” was funded but doesn’t actually meet any specific need. The reason the U.S. still uses B-52 bombers is because the requirements for the new ones were so ludicrous they couldn’t actually work.
Speaking of bombers, are your requirements even realistic or useful or profitable?
A stealth bomber that sneaks up is kind of cool but once they drop the bomb, everybody knows you’re there.
1. You can do the same thing with a cruise missile
2. Let’s talk about the effect of a fleet of 20 big-ass noisy B-52s coming in low and dropping bombs. If you want to subdue a civilian population, seems like you’d want attention instead of secret sniping. Shoot, you could probably fly over the city without dropping any bombs and change a lot of people’s minds real quick.
Of course, looking well at your over-all requirements is the same thing as realizing you’re talking about how the business actually operates.
To be fair to stealth bombers, I think the point is that they’re less likely to be shot down before delivering their payload.
Extreme license/cost modifications on renewal (cheap drugs til you’re hooked…)
The VENDOR has the right skills and experience on prior projects – but are they putting experienced PM and programmers on your possibly-not-their-flagship-project contract?
Are they staffed deeply enough to swap out incompentants on request?
How many other commitments has the vendor made – will they use delaying tactics due to lack of bandwidth? Is there any disincentive for delays beyond delay of payment – which doesn’t bother most large vendors one bit.
Are you getting a project manager that will be able to listen to people they may consider their inferiors (race, sex, age, whatever) or will you have to provide a “translator”?
For custom work, is the vendor’s Vision so strong that they will be unable to provide what our company needs rather than what they think we should need?
SaaS – are they going to upgrade your software routinely, or only when you renew your license/contract?
If doing an agile development under a contract, no matter how good the contract is, if the vendor PM doesn’t have experience/understand Agile, it’s going to be rough.
Get web intrusion test and VPAT results on the application during evaluation…if they coded it poorly, remediation may be near impossible. It’s way too easy for a bidder to check a box and commit to remediation without considering the potential delay and cost.
Are we willing to adapt our business processes to use the product the way it is intended to be used? Or will we complain because the product does not work the way we do business? Sometimes the software is just fine and we need to adjust, or even worse, change.
Unwillingness to adapt. I like that. Some folks view adaptation as failure and some think if they are contracting, then things ought to work the way they want them to regardless.
But one of the memes I see in many successful startups is a willingness to adapt and to discover their own business. A brewer who eventually ends up welding kegs or running a beer distribution company; game developers that can’t sell their game but can sell their development platform or their artwork; the cupcake baker who starts selling the management software she wrote and hasn’t baked in 4 years.
Unwillingness to adapt or inability to adapt? If you’ve been in business for any length of time, you almost certainly have other business systems you depend upon. Now you decide to add another (or replace one you have). The issue that arises is that no system lives in isolation. You’ve undoubtedly got data being fed to some aggregation system (for dashboards and reports) and you’re likely to have myriad other data feeds, some involving on-the-fly conversion. Adding or replacing a system is dependent upon being able to continue to support those data flows. “Adaptation” may not be practical. It’s really entertaining to watch what happens when a company decides to replace its ERP system! There is not enough time or expertise to understand the interdependencies before some naive executive makes the decision. (Worse, they are likely to have made the decision without a clue that there are existing interdependencies.) The entertainment value escalates rapidly if the selected solution is “in the cloud” (which likely means its SaaS). You are then faced with limited ability to customize processes and data elements, restricted ability to import/export data, and almost no ability to interactively exchange data with other business systems, all while learning a new operating environment and having to understand the limitations of someone else’s security architecture. But the vendor has promised to help… In the worst case, the vendor has a wildly successful offering and their implementation support personnel are undergoing “on the job training.”