Give me evidence or give me death! (GUEST BLOG)
Hey fellow platypophiliacs (platypus lovers in sloppy latin).
“Is that a potato in your purse?”
This is your last platypus offering written by an interloper, composed while Amy is taking a sabbatical to work on her novel. I am Rob, Amy’s father, and have long been blessed with shareable opinions. For background, I was a full service small town General Practitioner for a few decades, before I crossed to the dark side to work with big industry. I left my practice with a wealth of wonderful relationships, having shared so many important events and decisions in peoples lives. A key part of my role was to offer people the best information I could to help inform their decisions, while keeping an appropriate distance to allow them the autonomy to make their own choice. However I remain with a concern about how we make decisions, and in particular how we evaluate evidence.
So what is with the ‘potato in your purse’? That comes from an actual office visit with a patient, Louise, an RN, who explained that her neighbour told her to carry a potato in her purse to help control her arthritic pains. She chose to act on this advice and, sure enough, there was a potato in her purse. It is easy to dismiss this as innocent, harmless, and her choice.
Or another true story about Joyce, age 62, came in for a checkup and we found a nasty breast cancer. I took a fair bit of time with Joyce to inform her about some current research done by leaders in the field at some of the best universities (who had no vested interest apart from academic stature, and no product to sell). The study involved nearly 300,000 women and was considered the state of the knowledge at that time. After hearing that information, Joyce then countered with contradictory and alarming advice that she had heard from her sister’s esthetician. She seemed to be treating these two pieces of information as equally valid. A wrong decision could increase her mortal risk.
So the question becomes, how do any of us know what we know or believe, and of equal importance, how do we know what we do not believe. What kinds of evidence persuade you, because clearly not all evidence is equal. The standards for evidence – and thus for your opinions and beliefs are ultimately personal and individual. Bad information elevates the risk of poor decisions. The crux of this little essay is to reflect on our personal criteria for evaluating evidence.
From my education, I was thoroughly imbued with the perspective that the only real route to truth was via the scientific method. “Evidence based practice” should be governed by highest quality scientific evidence. There is enormous variability among individuals and professionals about what they will consider as valid information. An absolutist would assert that there is only one single truth if only we could know it, but that is not a very useful observation. What criteria do you personally use in deciding that this piece of advice is valid and instructive, but other advice is junk, dangerous, and that should be tossed on the garbage heap.
A brief digression is to contrast the process of science versus the process of politics (or many beliefs systems). Science begins with a question and then seeks objective information to try and inform the question. Conversely, politics (and many belief systems) start with a conclusion/ answer/ or point of dogma, and then searches for information which will affirm it. The need for information is much less when you start with an answer rather than a question. As a simple illustration, think back to the cancellation of the long form census when the politicians began with an opinion/ conclusion so thus felt no need to collect data to inform the question – they already knew their preferred policy answer. “I know what I believe so don’t confuse me with the facts.”
In science there is a hierarchy of evidence pyramid which ranks types of evidence from higher to lower in quality, and higher to lower risk in of bias. My thesis is that we all need to develop a personal hierarchy for rating evidence to help us process the flood of opinions, advice, and ‘facts’ that we encounter. The strongest evidence comes from systematic reviews and aggregated meta analysis of randomized control trials. Instructively, the lowest quality of scientific evidence with highest risk of bias comes from expert opinions, to say nothing of non-expert gratuitous opinion. Anecdotes and testimonials are very popular but are at extraordinary risk of bias and should be freely challenged.
There is a heresy arising in medicine. The rigors of scientific inquiry are being increasingly challenged as being too narrowminded, and thereby missing out on other useful sources of useful information. John Raulston Saul’s excellent book Voltaire’s Bastards is a very vigorous attack on the tyranny and closemindedness of the scientific method. An important read for anal scientists. From inside medicine there is a growing movement to be open to qualitative evidence rather than just quantitative evidence. What this means is being open to ‘softer’ information, concepts, metaphors, observations, untestable hypotheses. Qualitative research advocates being open to more than just numerical data, but qualitative evidence struggles to figure out what is false.
The risk of bias is pervasive. Do you think the used car salesman is totally objective about whether or not you need to buy a new car. Do you think the hawker of a wonderful new potion is a trustworthy source of evidence about its effectiveness and safety. Risk of bias is so high that information contaminated by self interest should be low on our personal hierarchy of believable evidence, and should be treated with caution and skepticism. We are biased by hope and what we wish the facts to be - but remember the American military adage “Hope is not a strategy.”
The slope, it gets so slippery! A fundamental tenet is that not everything is true! And your personal task is to decide what you don’t believe, what is snake oil, versus what is golden. Once you open the floodgates to non-scientific evidence, it becomes very tough to figure out what you do not believe in the wild west of non-discriminatory evidence. Our information plate is full with fake news and junk science and anti-intellectual bias. And lots of strident opinions by ill informed people. In your personal hierarchy of evidence, where do you place intuition, hunch, common sense, gut feelings, the opinion of your neighbour, barber, National Enquirer etc. So often we are encouraged to be much more ecumenical and open minded about sources of evidence. Malcom Gladwell’s book wants us to “Blink” rather than think. Alternate health practitioners like Andrew Weil want us to embrace placebos because they are ‘harmless’ – (maybe they do damage integrity and clarity).
Many bright people happily and knowingly expose themselves to sloppy thinking. Some people seem indifferent to the risk of being duped. We are getting ever close to magical thinking. Many people seem to have no interest in separating the wheat from the chaff – all ideas are invited as equals to the intellectual dance. Some feel that any attempt to figure out what is incorrect is tantamount to intolerance and lack of openness. “It is their choice” becomes the mantra, damn the risk.
And we become suckers for a good narrative. A wonderful example is the story of shark cartilage as a supplement. A couple of impoverished capitalists wanted to develop a product that would make them rich. So they came up with a narrative about sharks as a strong primal force, disease resistant, evolutionary survivors, (and lots of other evocative positive descriptors). They developed a shark cartilage extract and marketed it very successfully – with a great narrative but not a shred of evidence. Serious researchers (with nothing to sell) evaluated the product and concluded it was useless. The only thing the shark cartilage cured was the developer’s poverty. The seductive power of a narrative is huge.
I could go on, but expect that I am exceeding Amy’s word limit. Thank you for coming for a ride with me on my hobby horse.
As a final observation, the Earnest Platypus always closes with random thought or idea – a ‘literary palate cleanser’ to quote Amy. Ken Burns makes excellent documentaries (Civil War, Jazz, baseball etc) that offer insights into the American culture. He recently produced a 16 hour series on the history of country music which did a reasonable job of explaining, defending, and gentrifying that genre. As a counterpoint, I offer you this link as evidence that not all country music has been sanitized.
Closing thought from Tao Te Ching:
If I were possessed of the slightest knowledge, my only fear would be to go astray.
From one scientifically-inclined person to another, I am glad that we can learn together. I hope you enjoyed this dispatch, and have an evidence-filled week!
Until next Sunday,
By Guest Author Rob, on behalf of the Earnest Platypus