All Posts by Bob Lewis

|In: App Dev Methodologies|Last Updated:

Faced with a discipline that looks too much like hard work, I generally compromise by memorizing a handful of magic buzzwords and their definitions. That lets me acknowledge the discipline’s importance without having to actually learn a trade that looks like it would give me a migraine were I to pursue it.

Which gets us to testing … software quality assurance (SQA) … which I know consists of unit testing, integration testing, regression testing, user acceptance testing, and stress testing.

Although from the developer’s perspective, user acceptance testing and stress testing are one and the same thing – developers tend to find watching end-users try to use their software deeply stressful.

More to the point, I also “know” test automation is a key factor in successful SQA, even though I have no hands-on experience with it at all.

Speaking of no hands-on experience with testing stuff, the headline read, “Bombshell Stanford study finds ChatGPT and Google’s Bard answer medical questions with racist, debunked theories that harm Black patients.” (Garance Burke, Matt O’Brien and the Associated Press, October 20, 2023).

Which gets us to this week’s subject, AI testing. Short version: It’s essential. Longer version: For most IT organizations it’s a new competency, one that’s quite different from what we’re accustomed to. Especially, unlike app dev, where SQA is all about making sure the code does what it’s supposed to do, for the current crop of AI technologies SQA isn’t really SQA at all. It’s “DQA” (Data Quality Assurance) because, as the above-mentioned Stanford study documents, when AI reaches the wrong conclusion it isn’t because of bad code. It’s because the AI is being fed bad data.

In this, AI resembles human intelligence.

If you’re looking for a good place to start putting together an AI testing regime, Wipro has a nice introduction to the subject: “Testing of AI/ML-based systems,” (Sanjay Nambiar and Prashanth Davey, 2023). And no, I’m not affiliated or on commission.

Rather than continuing down the path of AI nuts and bolts, some observations:

Many industry commentators are fond of pointing out that “artificial intelligence” doesn’t really deal with intelligence, because what machines do doesn’t resemble human thinking.

Just my opinion: This is both bad logic and an incorrect statement.

The bad logic part is the contention that what AI does doesn’t resemble human thinking. The fact of the matter is that we don’t have a good enough grasp of how humans think to be so certain it isn’t what machines are doing when it looks like they’re thinking.

It’s an incorrect statement because decades ago, computers were able to do what we humans do when we think we’re thinking.

Revisit Thinking, Fast and Slow, (Daniel Kahneman, 2011). Kahneman identifies two modes of cognition, which he monosyllabically labels “fast” and “slow.”

The fast mode is the one you use when you recognize a friend’s face. You don’t expend much time and effort to think fast, which is why it’s fast. But you can’t rely on its results, something you’d find out if you tried to get your friend into a highly secure facility on the strength of you having recognized their face.

In security circles, identification and authentication are difficult to do reliably, specifically because doing them the fast way isn’t a reliable way to determine what access rights should be granted to the person trying to prove who they are.

Fast thinking, also known as “trusting your gut,” is quick but unreliable, unlike slow thinking, which is what you do when you apply evidence and logic to try to reach a correct conclusion.

One of life’s little ironies is that just about every bit of AI research and development is invested in achieving fast thinking – the kind of thinking whose results we can’t actually trust.

AI researchers aren’t focused on slow thinking – what we do when we say, “I’ve researched and thought about this a lot. Here’s what I concluded and why I reached that conclusion.” They aren’t because we already won that war. Slow thinking is the kind of artificial intelligence we achieved with expert systems in the late 1980s with their rule-based processing architectures.

Bob’s last word: For some reason, we shallow human beings want fast thinking to win out over slow thinking. Whether it’s advising someone faced with a tough decision to “trust your gut,” Obi Wan Kenobi telling Luke to shut off his targeting computer, or some beer-sodden opinionator at your local watering hole sharing what they incorrectly term their “thinking” on a subject. When we aren’t careful we end up promulgating the wit and wisdom of Spiro Agnew. “Ah,” he once rhetorically asked, “What do the experts know?”

Bob’s bragging rights: I just learned that TABPI – the Trade Association Business Publications International – has recognized Jason Snyder, my long-suffering editor at CIO.com and me a Silver Tabbie Award for our monthly feature, the CIO Survival Guide. Regarding the award, they say, “This blog scores highly for the consistent addressing of the readers’ challenges, backed by insightful examples and application to current events.“

Gratifying.

Speaking of which, On CIO.com’s CIO Survival Guide: “The CIO’s fatal flaw: Too much leadership, not enough management.” Its point: Compared to management, leadership is what has the mystique. But mystique isn’t what gets work out the door.

Comments

|In: Industry Commentary|Last Updated:

What follows is dedicated to all of you who asked me to replace KJR with a political blog. As always, you should be careful what you ask for. – Bob

When I was growing up, antisemitism was a joke, and the experience most of us in my community had with actual antisemites was on a par with our real-world experiences with Big Foot and the Yeti. I did once hear someone use “Jew” as a verb – a stereotype that from time to time I wished was more accurate – and in my college years once overhead an inebriated patron in a local bar complaining about f***ing k*** lawyers.

But everyone in earshot was content to ignore him, and he eventually went away.

I was in high school when, in 1967, in response to Egypt closing the Straits of Tiran to Israeli shipping, Israel launched the so-called “6-Day War.” Most in my high-school community thought of Mideast policy in much the same way that we thought of the Chicago Cubs – a team to root for, even though we couldn’t articulate why. I had no idea what even a single Strait of Tiran was, but it didn’t matter. Israel was my team and I rooted for it.

My views on the subject have, I hope, become a bit more nuanced than that, and, on the grounds of my being Jewish, I’ve been asked about them. So here goes:

Where it started: WWII and the concentration camps, in which something like 12 million people were slaughtered, half of them Jews. One reason I stopped thinking in terms of MOTs (Members of Tribe) was how many of my fellow Jewish MOTs ignored or trivialized the 6 million or so non-Jews also murdered by the Nazis.

Regardless, public awareness of the camps led to a widespread perception that fair-is-fair: Jews deserved a homeland in which they could feel safe.

Which is why, in 1948, the Jewish residents in Palestine declared the founding of Israel as a modern sovereign entity, at which time, with no noticeable delay, the nations surrounding it launched an invasion with a goal of destroying it.

And it’s then that the historical record and assessments of cause and effect become confused. Some historians claim Israel expelled the Palestinians. Others assert that the Palestinians fled because they were urged to do so by the Arab leaders of the time.

What has been lost in the dueling narratives is that no matter the reason Palestinians left Israel, the nations they fled to – especially Lebanon, Syria, Jordan, and Egypt – settled them into refugee camps and radicalized them rather than welcoming them and providing assistance.

Nor did Israel do anything to encourage the refugees to return.

Which is why facile good-guys/bad-guys storytelling is of no value in thinking through what should happen next. Nothing can excuse Hamas’s recent invasion. Read about Hamas and it’s clear it doesn’t represent the Palestinian community. It has more in common with an organized crime syndicate than a political entity.

Read about Israel’s response to the invasion and a cynic might think it’s Netanyahu’s way of maintaining his leadership position, not of creating a just peace.

Read about the war and its contribution to resurgent antisemitism. It has underscored, in no uncertain terms, that just as is true of all other forms of bigotry, all antisemites needed to crawl out of the woodwork was an excuse.

Far from being the jokes I thought antisemites were when I was a youth, they were just as much MOTs as I was, just members of a different tribe.

And most of them understood that, back then, belonging to that tribe was socially unacceptable.

Do I have a solution? Not hardly. I do, however, have a notion, for all the good having a notion ever has. It’s for the entertainment industry to take The Blues Brothers as an exemplar: Create entertaining fare that ridicules bigots of all tribes and stripes.

Not the earnest, preachy fare that’s usually paraded in front of us to “raise our awareness.” Entertainment.

Because raising our consciousness asks us to acknowledge that our consciousness needs raising, and to be willing to expend cognitive effort – work – to raise it.

Entertainment, in contrast, is, by definition, fun.

Maybe fun enough to embarrass the MOTs who are, for some reason, proud of their idiocracy.

Bob’s sales pitch: In a more traditional KJR vein, I’m keynoting OSICON 2035 this coming Wednesday. It’s free. If you happen to be in Toledo you can catch it in person. Otherwise, it, along with the rest of the program, will be streamed.

Check it out!

Comments