Industry Commentary – Page 23 – IS Survivor Publishing

Modeling is always in fashion

Rank the most-reported aspects of COVID-19, in descending order of worst-explained-ness. Modeling is, if not at the top, close to it.

Which is a shame, because beyond improving our public policy discussions, better coverage would also help all of us who think in terms of business strategy and tactics think more deeply and, perhaps, more usefully about the role modeling might play in business planning.

For those interested in the public-health dimension, “COVID-19 Models: Can They Tell Us What We Want to Know?” Josh Michaud, Jennifer Kates, and Larry Levitt, KFF (Kaiser Family Foundation), Apr 16, 2020 provides a useful summary. It discusses three types of model that, translated to business planning terms, we might call actuarial, simulation, and multivariate-statistical.

Actuarial models divide a population into groups (cohorts) and move numbers of members of each cohort to other cohorts based on a defined set of rules. If you run an insurance company that needs to price risk (there’s no other kind), actuarial models are a useful alternative to throwing darts.

Imagine that instead you’re responsible for managing a business process of some kind. A common mistake process designers make is describing processes as collections of interconnected boxes.

It’s a mistake because most business processes consist of queues, not boxes. Take a six-step process, where each step takes an hour to execute. Add the steps and the cycle time should be six hours.

Measure cycle time and it’s more likely to be six days. That’s because each item tossed into a queue has to wait its turn before anyone starts tow work on it.

Think of these queues as actuarial cohorts and you stand a much better chance of accurately forecasting process cycle time and throughput — an outcome process managers presumably might find useful.

Truth in advertising: I don’t know if anyone has ever tried applying actuarial techniques to process analysis. But queue-to-queue vs box-to-box process analysis? It’s one of Lean’s most important contributions.

Simulation models are as the name implies. They define a collection of “agents” that behave like entities in the situation being simulated. The more accurately they describe agent behaviors, estimate the numbers of each type of agent, the probability distributions of different behaviors for each type, and the outcomes of these behaviors … including the outcomes of encounters among agents … the more accurate the model’s predictions.

For years, business strategists have talked about a company’s “business model.” These have mostly been narratives rather than true models. That is, they’ve been qualitative accounts of the buttons and levers business managers can push and pull to get the outcomes they want.

There’s no reason to think sophisticated modelers couldn’t develop equivalent simulation models to forecast the impact of different business strategies and tactics on, say, customer retention, mindshare, and walletshare.

If one of your modeling goals is understanding how something works, simulation is just the ticket.

The third type of model, multivariate-statistical, applies such techniques as multiple regression analysis, analysis of variance, and multidimensional scaling to large datasets to determine how strongly different hypothesized input factors correlate with the outputs that matter. For COVID-19, input factors are such well-known variables as adherence to social distancing, use of masks and gloves, and not pressuring a cohabiter to join you in your kale and beet salad diet. Outputs are correlations to rates of infection and strangulation.

In business, multivariate-statistical modeling is how most analytics gets done. It’s also more or less how neural-network-based machine learning works. It works better for interpolation than extrapolation, and depends on figuring out which way the arrow of causality points when an analysis discovers a correlation.

As with all programming, model value depends on testing, although model testing is more about consistency and calibration than defect detection. And COVID-19 models have brought the impact of data limitations on model outputs into sharp focus.

For clarity’s sake: Models are consistent when output metrics improve and get worse in step with reality. They’re calibrated when the output metrics match real-world measurements.

With COVID-19 testers have to balance clinical and statistical needs. Clinically, testing is how physicians determine which disease they’re treating, leading to the exact opposite of random sampling. With non-random samples, testing for consistency is possible, but calibration testing is, at best, contorted.

Lacking enough testing capacity to satisfy clinical demands, which for most of us must come first as an ethical necessity. Modelers are left to de-bias their non-random datasets — an inexact practice at best that limits their ability to calibrate models. That they yield different forecasts is unsurprising.

And guess what: Your own data scientists face a similar challenge: Their datasets are piles of business transactions that are, by their very nature, far from random.

Exercise suitable caution.

The only Subject

As a writer, right now I have, only one Subject. Which doesn’t mean I’m the only writer worth reading about the Subject (yes, I do have some remaining tattered shreds of humility).

Nor, it occurred to me, is the Subject the only subject that can provide insights into the Subject.

To that end, as I imagine you’re finding yourself with more reading time on your hands than usual, allow me to recommend Lynne Olson’s marvelous Citizens of London.

Even without its relevance, this book would belong on your reading shortlist. It’s history few of us know anything about — how three Americans — Edward R. Murrow, Averell Harriman, and John Gilbert Winant helped save England before the United States joined the war against the Axis powers.

As we chafe at the self-imposed restrictions on our now-on-hold daily routines, it should be required reading: Citizens of London, as it relates the previously untold joint biography of Murrow, Harriman, and Winant, also tells the story of the Blitz in stark, unblinking terms.

Understanding the day-to-day stoicism, courage, and resilience with which the British dealt with two solid months of daily bombings should quell any sense we have of daily aggravation.

We find ourselves shopping on-line from the comfort of our homes. Our inconvenience is shortages brought about mostly because of shoppers who expect shortages. The British crowded into air raid shelters, wondering if this would be the time a German bomb turned their shelter into a mass tomb. And, it found them dealing with very real, painful, severe and constant shortages of food and basic commodities.

If, though, fiction is more to your liking than biography, and you’d like to read about the Subject without reading about it as daily … or hourly, or up-to-the-minute-breaking … news, get hold of the incomparable Connie Willis’s World War II duology, Blackout and All Clear. While not historically accurate in detail, Willis paints a vivid picture of Great Britain in World War II, and especially the Blitz, through the lens of time-traveling historians who find themselves caught up in it all.

If you don’t regularly read science fiction reader don’t be put off. The science-fictional elements don’t dominate the story. The everyday heroism of so-called “ordinary” people does.

Speaking only for myself, reading and thinking about the Blitz gives me something to aspire to.

Parallels between the Blitz and our current government-encouraged, self-imposed quarantines and isolation are, of course, inexact. Especially, the Blitz caused the British to crowd in together. They might have wished for a protective solution that let them isolate themselves, just as we wish for one that would let us congregate.

I will, by the way, allow you some escapism as you wait things out (no, don’t thank me!). While I’m touting Connie Willis, that means you should save The Doomsday Book — a grim and unrelenting telling of the bubonic plague — for more cheerful times.

Instead, seek out her utterly delightful To Say Nothing of the Dog. It’s about a time traveling historian’s (of course!) search for Jerome K Jerome, author of Three Men in a Boat (To Say Nothing of the Dog),

Which is, if anything, even more delightful than Willis’s book about it.

Add it to your list.

# # #

Those of us who write have, I think, a responsibility to make sure we rely on trustworthy sources of information; also to monitor ourselves to make sure the trustworthy sources we rely on aren’t limited to trustworthy sources who reliably tell us what we want to read (or hear).

Last week, while using the Subject as a springboard for writing about risk management, I highlighted a source of reliably untrustworthy information. Several subscribers objected to my bringing politics into KJR.

Interestingly, these same subscribers did not complain when I used the Democratic primaries as a springboard for writing about sexism in the workplace, even though I was less than kind in my comments about some of the candidates.

In any event, it’s fair to say that identifying trustworthy sources of information about the Subject is probably more useful than trying to list the near-endless sources of misinformation. Hint: if a source endorses bleach cocktails or homemade hydroxychloroquine, it’s untrustworthy.

Better to rely on the World Health Organization (WHO) and the Centers for Disease Control and Prevention (CDC). Yes, the CDC’s early attempts at establishing a testing program were pretty ragged. No, that doesn’t disqualify it as a source of reliable information.

One more: while I’m sometimes less than complimentary regarding McKinsey & Company, its COVID-19 Executive Briefing for business management is quite good.

And, of course, there’s Snopes.