To get my PDA working properly I had to change my e-mail address. It’s an irritating story.

I’ve been using CompuServe 2000, an IMAP e-mail platform, with Outlook as my e-mail client. Everything worked fine … most of the time, anyway … until my recent change in employment, which moved my calendar from my office copy of Outlook, which was attached to Microsoft Exchange. My PDA (a Palm device) synced just fine to that, but not to my home copy of Outlook. The problem: I’d installed Outlook using the “Internet only” installation option, but synchronization requires “Corporate/Workgroup” installation instead.

So I switched over to Corporate/Workgroup and voilà! … CompuServe 2000 stopped working. Some genius at Microsoft decided IMAP e-mail should only work in Internet only mode. Which means CompuServe 2000 (along with all other IMAP e-mail services) and PDA synchronization are incompatible.

Please note my new e-mail address.

While annoying, the experience was also timely, because I was in the throes of helping a client select an e-commerce vendor. As with most vendor evaluations, we asked questions, made note of the answer, and compared the vendors on the basis of their responses.

Were I to evaluate PDAs and e-mail systems this way, I’d have easily reached the conclusion that my PDA, e-mail provider and e-mail client would work together in perfect harmony. The incompatibilities were, after all, buried pretty deep.

Which leads to a suggestion: Follow all vendor selections with a pilot implementation. The ideal pilot is simultaneously technically challenging, small in scale, and politically simple. You’re looking for a relatively small group that’s enthusiastic about the new technology — you’ll have plenty to worry about without dealing with resistance to change –that will exercise it hard to test as many complexities as possible.

This should be obvious, but for one reason or another … go-for-broke schedules are a common culprit … companies often skip the pilot. They invariably lose more time than they ever hoped to gain, because problems are easier to fix under controlled circumstances.

That’s one tip. Want a few more?

Tip #2: Be balanced. I divide evaluation criteria into four categories: Technology, Services, Business/Marketplace, and Pricing. With technology, don’t try to design the vendor’s product. You’re watching out for bad architecture, reliance on immature or fringe platforms, and obsolescence, and checking whether the vendor will commit to service levels. To define the others: Services covers features and functionality; Business/Marketplace helps you predict whether your vendor will in business next year; and Pricing covers not just the price itself but the pricing structure.

Tip #3: If you can make your decision from the numbers, somebody rigged the evaluation. Use a three-point weighting factor for the evaluation criteria, where 1=Useful, 2=Important, and 3=Crucial. Score responses on a five point scale (with no fractions!). When you tote everything up, scores within 5 percent are a tie. If one vendor is the clear winner, you either compared the preferred vendor to only its weakest competitors, the preferred vendor provided the evaluation criteria, or both. The numbers screen out clearly inferior choices. Use the details of a vendor’s responses, and the tone of each conversation (I prefer interviews to written responses) to differentiate among the remainder.

Tip #4: Watch out for second-stage selling. Every losing vendor will ask you where they came up short. If you answer the question you’re in for an argument. Your correct response is something like this: “You didn’t lose on any single question. We were confident your solution would have worked, but in balance it wasn’t as good a fit for our needs as our first choice. To give you any more detail I’d have to violate the confidentiality of the other vendors we talked to, so I’m afraid that’s a much information as I can provide.”

Tip #5: Don’t worry. Be happy. You’ll never know if you picked the best solution, even after you’re operational. No matter how good an evaluation you performed, you’ll run into gotchas during the pilot, and more during the full roll-out. If they aren’t insurmountable you made a good choice, so don’t fall into the grass-is-greener mentality.

Remember token ring vs Ethernet? Both led to headaches during implementation, but in the end, either net worked. That may be a wisecrack, but I doubt it.