Often, when something new comes along, the skills you have to jettison outweigh the new ones you have to acquire.

I am, of course, writing about artificial intelligence and what IT has to do to cope with it. Are there any other topics for a Recognized Industry Pundit (RIP) to write about right now?

Sure there are, but not this week. This week’s topic is AI, and specifically the AI-driven need to rewrite the rules of IT quality assurance.

As an IT professional you’re familiar with software quality assurance (SQA) and its role in making sure the organization’s applications do what they’re supposed to do.

You’re also familiar with DQA – data quality assurance, while you might not use the acronym in your everyday conversations. You should, because what seems to be missing in IT AI methodology-land is the complete re-write we need of the DQA handbook.

Do some googling (or co-piloting, or whatever) and you’ll find quite a few suggestions for using AI to improve your DQA practices. But these get things backward.

In pre-AI IT, quality (to oversimplify) comes from SQA, a search for situations in which a program doesn’t turn its inputs into the right outputs.

Bring generative AI into the conversation and the day-to-day need for SQA goes away. Generative AI’s neural-network-based application logic is fixed – neural network nodes are, to oversimplify some more, multivariate correlation engines.

With generative AI it’s the data, not application logic, that drives output quality.

Trying to override this dynamic can be a cure that’s worse than the disease, as Google recently discovered to its corporate embarrassment.

When old-school DQA was in charge, biased data meant the company’s data repositories didn’t accurately reflect the underlying statistical universe.

What ran Google’s Gemini off the road was its attempt to inject bias into its outputs.

The problem Gemini ran afoul of was that The World isn’t what we want it to be. With Gemini, Google tried to fix what’s wrong with The World by superimposing its preferences on the Gemini’s outputs.

As explained by Prabhakar Raghavan, Google’s executive in charge:

Three weeks ago, we launched a new image generation feature for the Gemini conversational app (formerly known as Bard), which included the ability to create images of people.

It’s clear that this feature missed the mark. Some of the images generated are inaccurate or even offensive. We’re grateful for users’ feedback and are sorry the feature didn’t work well.

I’m pretty sure the situation is much, much worse than Raghavan’s apology suggests, because we can expect future image, video, audio, and text generation products to be just as problematic as Gemini is.

Fixing Gemini and its generative AI brethren amounts to trying to fix The World.

Imagine you asked Gemini to, as did The Verge, “… generate an image of a 1943 German Soldier. It should be an illustration.” Programmed to avoid generating offensively biased images, Gemini produced a picture showing a demographically diverse WWII-era German military workforce (click here).

Raghavan was right about it being an offensive output (or, more accurately, an output that would offend some viewers). But it wasn’t Gemini that was offensive. It’s how Google tried to teach Gemini how to respond when The World is offensive that ended up being offensive.

It could have worked, if it weren’t, that is, for two thorny questions: (1) who gets to define “ought to be?” and (2) if we’re going to tell AI what the right answer is, what’s the point?

We already have AI systems where humans tell the AI the right answer. They’re called “expert systems,” and they’ve been around since the 1970s.

One way of looking at generative AI is that (oversimplifying yet again) it’s just like expert systems except we’re trying to make machines the experts. In traditional analytics, data quality is something you take care of so you can draw reliable conclusions when you analyze the data with programs you’ve subjected to software quality assurance.

Data quality isn’t what it once was. Now, it’s what you need so that the data whose quality you’re assuring properly trains your generative AI.

In generative AI, that is, the data aren’t something you process with programmed logic. In a very real sense, the data are the program logic.

Bob’s last word: One more thing. The Gemini team produced its problematic results despite having Google’s resources to draw on. But AI vendors are starting to peddle the benefits of connecting your company’s internal data to the same AI technologies. It’s tempting, but if Google, with far deeper pockets than its customers have, couldn’t figure out the DQA practices it needed to stay out of trouble, how are its customers supposed to do so?

And while we’re on the subject, this week CIO.com’s CIO Survival Guide is: “A CIO primer on addressing perceived AI risks.” It’s about real and perceived AI risks you probably haven’t read about anyplace else.