Al Gore was right.
Oh, don’t be like that. When a man is right, he’s right whether you like him, hate him, or feel intense apathy about him. A couple of years ago Gore published The Assault on Reason, and every day brings another example of People We’re Supposed to Take Seriously swinging another baseball bat at Reason’s head.
Take, for example, Jonah Lehrer’s “Trials and Errors: Why Science Is Failing Us,” in the January 2012 edition of Wired. Lehrer strings together some high profile examples of scientific theories turning out to be wrong, adds erudite-sounding invocations of philosophers like David Hume, and concludes that all scientific accounts of causation are just empty story-telling.
As an antidote, read the Wikipedia entry on Karl Popper. He’s the father of modern scientific epistemology, but somehow didn’t rate even a mention in Lehrer’s article. What you’ll learn is that in our search for truth, the best we’re able to manage is a collection of ideas nobody has managed to prove false.
That’s what science is — causal “stories” (they’re called theories). Some are new and have been subjected to just a few challenges. Others have been battle-tested over a span of decades or even centuries by large numbers of researchers, some brighter than anyone reading these words (not to mention the person writing them); a few brighter than everyone reading these words put together.
Fail to prove an idea wrong enough different times in enough different ways (fail to falsify it in Popper’s terminology) and scientists start to feel confident they’re on the right track.
Not certain, but confident, because it isn’t a certain process. Even when scientists get something right, “right” can turn out to be a special case of a theory that covers more ground. That’s how it turned out for Newton’s theory of gravity. It’s useful enough when you need to build, say, a building that doesn’t fall down, but not sufficient for something more complicated, like, say, a global positioning system.
Citing science’s limitations is easy, which has led the easily fooled to the foolish conclusion that we should ignore what the scientific method tells us — foolish because no one has offered an alternative that works anywhere near as well, let alone better.
It’s New Year’s Resolutions time and I have one for you. (Yes, I have some for myself too, but suggesting ways for other people to improve themselves is so much more fun …) It’s to foster a culture of honest inquiry … in the business culture you influence (KJR’s proper scope) and also in your social circles and family, if that isn’t too much to ask.
It’s harder than you might think, for two interconnected reasons: (1) All culture change starts with changes to your own behavior; and (2) we’re all wired to reach the wrong conclusion under a scarily wide variety of circumstances. For example:
Did you know that in the United States, the highest incidences of kidney cancer are found in small towns?
It’s true. What conclusion do you draw from this? Probably not that it has to be this way as a matter of pure, random chance, but that’s the actual explanation. Don’t believe me? Here’s another, equally true statement: The lowest incidences of kidney cancer are found in small towns.
The way randomness works is that small samples exhibit more variation than large samples. So large metropolitan areas … big samples of the U.S. population … will all have incidences of kidney cancer very close to the overall national mean. Small towns, each a small sample, will vary more widely, so some will have an incidence much lower than the national mean while others will have an incidence much higher.
Even professional statisticians get this sort of thing wrong if they aren’t on their guard for it, as is documented, along with about a zillion other places our ability to draw the correct conclusion falls by the wayside, by your must-read book for 2012 — Daniel Kahneman’s Thinking, Fast and Slow.
The title comes from the author’s well-tested and thus-far not falsified theory that humans rely on two separate decision-making systems. One is quick, heuristic, error-prone, and so effortless we’re often unaware it’s even operating. The other is slow, requires significant effort and concentration, and takes proper stock of evidence and logic.
Do you want to encourage a culture of honest inquiry? It means engaging the slow, effortful system as much as you can. That requires an environment that provides enough time, and sufficiently few distractions, to allow it.
It will be an uphill battle, but one eminently worth fighting.
Why did you need Al Gore’s name for this column? Because he wrote a book with a title related to what your were going to talk about?
In all fairness, I went to Amazon and read some of the excerpts of Gore’s book see if he was really on to something. Alas, the excerpts were complete garbage! In part of one he talks about people being swayed by fear, and how dangerous that is. Of course its true, but this is coming from the man who’s poster about his climate change documentary had a category 5 hurricane on it! He even says its bad to succumb to baseless fears when other big real fears exist. Well, that’s all fine and good, now who gets to say which fears are baseless and which are real? (not trying to get into a global warming debate, I’m just saying that determining correct fears to push people with is IMHO a path to the dark side).
Gore also goes into great detail about the use of fear by the Republican party to gather power over all branches of government. Real honest inquiry there. Every president in the last 40 years, regardless of party, has strived to increase presidential power! Both political parties have always wanted a dominant position! That’s the honest view, not one where your party (doing exactly the same thing!) is the only hope of the people.
Gore’s book may have a good title, but in my opinion he’s too partisan to tell his story straight. And I think those making a big deal over how badly we are being governed telling us that “their” party will fix it are just as disengenuous as the writer of the misguided Wired article about causation.
To answer your question, there’s a basic principle of good manners I do my best to follow: Give credit where it’s due. Gore’s account of the subject might not have been “fair and balanced” but it was quite well documented. And while my first mention of intellectual relativism preceded publication of “Assault on Reason” (2004 vs 2008), Gore’s book was both better-researched and more broadly visible than anything I was able to manage in Keep the Joint Running.
Was it partisan? Looking at the policy positions of both political parties, it’s hard to escape the conclusion that the Republican party has become consistently anti-science while the Democratic party has not — the positions of party leaders on global warming and evolution by natural selection are the two most obvious examples, but far from the only ones.
Your point about both parties invoking fear is on target. Fear based on pure fantasy, though, does seem more prevalent among Republican leaders — Michele Bachmann’s concerns about the HPV vaccine causing mental retardation is a prominent example but far from the only one, and I’m unable to discover anything remotely parallel among Democratic fear-mongers.
The failings of the Democratic party are certainly as egregious. An obvious example: finding an actual consistent principle that animates the party is quite difficult. That its failings are as large doesn’t make its failings the same.
And while Gore’s partisanship makes his portrayal of the Republican party as “the bad guys” suspect, it doesn’t invalidate his larger point, which is on target — from Star Wars’ “trust in the force, Luke” to “Trust Your Gut: How the Power of Intuition Can Grow Your Business,” the number of loud voices advising us to ignore evidence and logic is large and growing.
I read the Wired article before reading this one and found it quite reasonable. What I got from it was to be sceptical of what is hyped as causation when it is really correlation, especially by those who stand to profit from more studies (e.g., grant seekers) or those selling drugs or supplements or medical testing procedures. I am old enough to remember the rise and fall of tonsellectomies and appendectomies at the least indication (with the latter sometimes occuring “while we are in here for some other reason”. The most recent examples are the PSA test for prostate cancer and mammograms for breast cancer in younger women. Statins for cholesterol control to reduce the risk of heart disease are also overdone. It doesn’t mean that one should stop looking but it does suggest that one should be careful jumping into what may very well be a “fad” with poorly understood long range effects but which generate profits now. I am not opposed to profits and have no problem with a free people being able to choose to use or abstain from taking various drugs or supplements or procedures. I just think they should be aware that when these “new” things come out that there is a non-trivial probability that they will later be found to be of little value and some risk.
Oh dear. “… especially by those who stand to profit from more studies (e.g., grant seekers).”
Without more studies, scientific knowledge is static. Without grant money, there’s no more research.
Here’s the challenge: Determining when there’s been enough research to be sufficiently confident that we can base decisions on it.
Example: The initial findings of correlation between high levels of serum cholesterol and heart disease were enough to warrant investing in more research to understand the subject better. The more detailed knowledge demonstrated a causal connection between high LDL levels and arterial plaque formation in convincing ways. That led to the conclusion that taking drugs to reduce LDL levels made medical sense. There was nothing nefarious in any of this.
There is now preliminary evidence suggesting that statin-induced reduction in LDL levels does not result in decreased risk of heart disease. Should this evidence be borne out by further research it will tell us that something else interesting is going on, which presumably will lead to more effective prevention of heart disease in the future.
It won’t tell us to ignore the grant-seeking scientists, and from now on to have a juicy cheeseburger and lots of french fries every day, because it won’t affect our health one bit.
Human physiology is quite complicated. This doesn’t mean we’ve learned nothing since the days when physicians bled patients to heal them. It means we aren’t done yet.
On the profit-seeking dimension: It’s one reason the current crop of complaints about the FDA approval process concerns me. We certainly do need an independent review of the research, especially when large profits are involved, for all the obvious reasons.
Bob, I think grant seeking is an honorable profession practiced by honorable people. However, big money will always be able to manipulate reported results. Example: Big tobacco paid for a multitude of studies that concluded that, short of nicotine poisoning, smoking was harmless. I share your concern about a lack of independent review. It is our only hope when the financial stakes are high.
With respect to grant seekers, I am not against grant seekers at all. Anyone should have the right to propose whatever they want. My concern is the grant awarder, especially when it is with tax money and more especially when much of it is borrowed. The results of grants may indeed add to the net knowledge base. But in an era of enormous debt that threaten our economy, it would seem that we would be very judicious about what federal gramts were awarded. The notion of diminishing returns for tax-payer funded grants should be a key consideration. In my opinion, it is not the purpose of grant programs to simply fund all the post-docs in the country. Private donors have lots of money out there (e.g., Gates Foundation, Ford Foundation, Howard Huges Medical Institute). Remember that Craig Ventner pushed the human genome project much faster than the NIH would have produced a result; maybe it was just luck, maybe not.
In my opinion, the statin drugs were lept on as a panacea by those people who wanted instant gratification without effort and there were industries around to provide them. Coronary heart disease is not a problem for those who watch their diets and get proper exercise. And those practices cost next to nothing relative to medical care expenditures (the cost of statin drugs like Lipitor are built into the medical care insurance premiums as well as out-of pocket costs). The observational evidence has been clear for years with the Japanese before and after they started adopting the Western (mostly US) lifestyle – See http://eurheartjsupp.oxfordjournals.org/content/6/suppl_A/A8.full Furthermore, statins and many other such drugs have significant side effects (read the labels on those drugs). Good diet and exercise have no such side effects.
In any case, my initial point was that there are dozens if not hundreds of correlation studies that come out yearly (Google will send you daily messages summarizing studies on such topics as Prostate Cancer, Diabetes and any number of other diseases) and most should be viewed with scepticism when they suggest magic bullets. Of course they have learned not to use such words; rather they say things like “promising” and “encouraging”; and almost all end with “needs more study”. As we have seen, pretty much anything can be characterized that way – the trick for the potential study approver providing the funding is to make a reasonable assessment of the potential reutrn on that investment vs. something else.
Personal experience: My serum cholesterol topped 290. Watched my diet very carefully for a year. Very carefully. Exercised more, too. Got it down to 235 – as low as it would go. Statins got it to where it needed to be. Diet is a factor. It isn’t the only factor.
Bob:
Thanks for an insightful column and a great start to 2012.
Hey Bob,
Thanks for the book recommendation.
I loved Daniel Ariely’s Predictably Irrational.
Other books, like “Sway: The Irresistible Pull of Irrational Behavior”, “Asking the Right Questions: A Guide to Critical Thinking”, and “The Drunkard’s Walk: How Randomness Rules Our Lives” have caused me to pause and really THINK about my assumptions and conclusions.
Perhaps what we (as a society) should require in High School is 2 years of critical thinking. Of course, it would help if those who taught it were capable of critical thinking themselves…
Happy New Year Bob.
High school would be nice. I’ve sometimes thought we should make this the basis of a four-year college degree. Call it an ABS (for “Anti-B.S.”) degree.
Bob,
I would like to second your recommendation of Kahneman’s “Thinking, Fast and Slow”. It’s an excellent book that definitely belongs on everyone’s must read list for 2012.
Tom
Bob,
By bringing up the crucially-important but oft overlooked subject of sample size effects, you help us understand the much maligned US public educational system. Many nationally prominent writers bemoan the US math and science test score results which appear to rank the US somewhere near the middle of “countries” giving out average math test scores. But this ranking system is totally bogus, because it ignores the sample size and socioeconomic characteristics of the tested samples of students taking the tests. Countries like Singapore and China which show high-ranking score averages in math and science have a test-taking student populations which are confined to small urban elites of high socioeconomic status.
A couple of careful analyses of student standard math test scores which stratify the results by economic status of the student population consistently show the US results for well-off students ranking right at the top of list of all test takers. Lumping all US test takers together, however, creates a super large sample, which predictably approaches the overall mean results (i.e. ranks the US closer to the middle of all countries giving results). Size of test-taking sample fully explains this difference, rather than any failure of the public high school education system in the US. High socioeconomic students in the US test out at the very top of the range of results, above their peers in many countries.
Bob:
I do NOT read Kahneman to say that “slow thinking” is always better, and to use it as much as we can. I read him to say each has different strengths and weaknesses, and we should try to use each appropriately. To me, that’s a big important difference from your last line.
Rollie Cole PhD, JD
Founder, Fertile Ground for Startups and Small Firms
Helping Build Environments Where Multiple Startups and Small Firms Can Thrive