What should we do when the experts change their minds?
Last week, KJR talked about NIST changing (or is it “updating”?) its recommendation regarding its longstanding advice to change passwords frequently.
The question of the hour is, does NIST changing its recommendation make it a more trustworthy source of expertise, or less?
The two obvious and most popular answers boil down to:
More worthwhile: I’d rather take advice from someone who’s constantly learning more about their field, than from someone who learned something once and decided that’s all they need to know.
Less worthwhile: Why should I rely on advice that’s constantly changing? I’d rather rely on positions that don’t change with the time of day, phase of the moon, and the sun’s position in the zodiac.
Before continuing down this path on the information security front, let’s explore a better-known subject of ongoing controversy — the role of dietary fat in personal health.
There’s been a lot written on all sides of this question, so much so that it’s easy to figure that with no medical consensus, what the hell, I’m in the mood for a cheeseburger!
Me, I take a different position: I’m in the mood for a cheeseburger! Isn’t that what pills are for?
No, say the skeptics. There’s published research showing that statins don’t provide much medical benefit and, for that matter, that saturated fats aren’t at all toxic.
As my pre-statin LDLs were way out of whack, I have a personal stake in this, and so here are my personal guidelines for making sense of personal health, information security, or pretty much any other highly technical subject:
Ignore the divisive. Divisive language is easy to spot. Phrases like “The x crowd,” with x = a position you disagree with (“The first amendment crowd,” or, adding 1, “The second amendment crowd” are easy examples.
This sort of ridicule might be fun (strike that — it is fun) but it isn’t illuminating. Quite the opposite, it’s one of the many ways of dividing the world into us and them, and defining the “right answer” as the one “we” endorse.
Fools vs the informed vs experts. Fools believe what’s convenient. The informed read widely. Experts read original sources.
Fools … perhaps a better designation would be “the easily fooled” … have made confirmation bias a lifestyle choice. Faced with two opposing points of view they’ll accept without question the one they find agreeable while nitpicking the opposing perspective to death.
Those of us who try to remain informed read widely. We choose sources without obvious and extreme biases; that go beyond quoting experts to explaining the evidence and logic they cite; and that provide links or citations to the original sources they drew on.
Especially, we deliberately counter our own confirmation biases by looking skeptically at any material that tells us what we want to believe.
Experts? They don’t form opinions from secondary sources. They read and evaluate the original works to understand their quality and reliability in detail.
There’s always an expert. Want to believe the earth is flat? There’s an “expert” out there with impressive credentials who will attest to it. Likewise the proposition that cigarettes are good for you, and, for that matter, that Wisconsin has jurisdiction over the moon on the grounds that the moon is made of cheese.
Just because someone is able to cite a lone expert is no reason to accept nonsense … see “confirmation bias,” above.
Preliminary studies are interesting, not definitive. For research purposes, statistical significance at the .05 level is sufficient for publication. But statistically, one in every 20 results significant at that level is due to random chance.
Desire to learn vs fondness for squirrels. Ignoring new ideas and information is a sign of ossification, not expertise. But being distracted by every squirrel — changing metaphors, jumping on every new bandwagon because it’s new and exciting — isn’t all that smart either. Automatic rejection and bandwagoning have a lot in common, especially when the rejection or bandwagon appeals to your … yes, you know what’s coming … confirmation bias.
Ignoring changing conditions. No matter what opinion you hold and what policies you advocate, they’re contextual. Situations change. When they do they make the answers we worked so hard to master wrong.
The world has no shortage of people who refuse to acknowledge change because of this. But relying on answers designed for the world as it used to be leads to the well-known military mistake known as “fighting the last war.”
Except that nobody ever fights the last war. They prepare to fight the last war. That’s why they lose the next war.
These are my guidelines. Use them as you like, but please remember:
I’m no expert.
Until you experience a heart attack. Then the expert is the cardiologist at the heart failure clinic.
It’s possible to prepare for the last war and win the next war – it requires one of 3 things – time to course correct, such overwhelming odds that it doesn’t really matter what you do, or a willingness to use tactics or means that your opponent won’t or can’t.
Staying away from fats isn’t a bad idea for 2 reasons. 1. The “experts” that recommend that might be right. 2. Fats are high density calories. Even if reason 1 turns out to be false, reason 2 is sufficient reason for most people in the U.S. to not eat a lot of fats – they need to reduce caloric intake.
A cardiologist in Cleveland Ohio has found that vessel health can be improved regardless of your family history – with a diet that has no meat or dairy, and is devoid of most oils. Blockages of 60% go to 1% or 2% with no surgery. He admits that people that are most likely to stick to that diet are those that have been told that further surgery is no longer an option. Esselstein is the name on a lot of peer-reviewed articles.
Great column and guidelines.
I think the difference between taking your doctor’s advice and taking NIST’s advice is that you and your doctor have a great deal more knowledge about your physiology and behaviors both as an individual, and as a member of several demographic groups, than NIST does about state sponsored hackers.
Usually, listen to the experts. But, when the experts are missing large amounts of new or unpredictable data, a person’s own generalist experience and perception should probably carry more weight, unless they are completely ignorant in the topic. Then, listening to the experts is probably best, even though they don’t yet really know the answer.
Wow Bob, after a great piece of advice about avoiding divisiveness you further the fundamental us vs. them problem we have today by writing:
“Fools vs the informed vs experts. Fools believe what’s convenient. The informed read widely. Experts read original sources.”
How about digging into WHY some have made confirmation bias a way of life?
This is well-explored territory without my weighing in. Short version: Confirmation bias is the default. It takes effort and self-awareness to overcome it. Which is why, in the next paragraph, I amended “fools” to “the easily fooled.”
Which, I think, is accurate without being divisive.
Great advice Bob! You sound like a graduate student, research scientist, or philosopher. So many people fall into tbhe traps you’ve identified. It’s too bad that more people don’t read your column.
Advice that _never_ changes is only useful if
1) conditions never change and
2) the advice was correct to begin with.
The only saying that comes to mind is Ben Franklin’s “moderation in all things” which to me means taking everything with a grain of salt and being immoderate sometimes (since you also need to moderate your moderation).
I generally ignore advice from doctors and health studies, assuming they’ve probably misinterpreted the statistics. For example, my doctor dropped my after my refusal to have a colonoscopy just because that’s standard. I told him I’d do a swab stool test like I had been doing but that colonoscopies were far too invasive and in my opinion, probably causing as many problems as they were preventing (but no one wants to study that because there’s no money in it).
He said I really needed to do it because “colon cancer is bad and my wife had a colonoscopy and–they missed it–and she died.” They people performing the colonoscopy missed the cancer, yet that’s the reason my doc wants me to have one.
Right. I know some management positions his ‘logic’ qualifies for.
Another example. Most folks say the reason Americans are so fat nowadays is because of laziness, television/internet and poor food choices. Nothing in there about antibiotics even tho the reason farmers feed them to livestock is because antibiotics make mammals put on weight.
Personal to Bob.
Don’t worry about your ldl levels.
Don’t even measure them.
And if the doc complains, ask him which is worse: a high ldl level or stress from worrying pointlessly about stuff you can’t change and that may or may not affect your health.
There was some management consultant I read who said you need to measure the right stuff or else you’ll worry about the wrong stuff.