Wednesday, December 8, 2010

Twisting facts to suit "theory". How smart people can fool themselves.

              “It is a capital mistake to theorise before one has data. Insensibly one begins to twist facts to suit theories instead of theories to suit facts.”
Sir Arthur Conan Doyle

   It often seems strange how frequently smart, well educated people seem to fall for pseudoscience. Doctors and Veterinarians are certainly no exception, and it is not unusual to see veterinarians recommending poorly evidenced treatments or even treatments with evidence against their effectiveness. People often ask how a trained medical professional can fall for such things and then promote them. In addition to regression to the mean, selection bias and other cognitive errors, we are all prone to the illusion of control. A recent paper in the British Journal of Psychology (discussion by Ben Goldacre here as the entire paper is behind a paywall) helps to demonstrate how this illusion can develop in medical professionals and suggests some ways to avoid it. Unfortunately, many veterinary school curricula and continuing education rely heavily on a fairly authoritarian presentation of information and rote memorization that do not help students to learn to evaluate claims critically and understand the evidence base behind various treatments.

  In recent comments on The SkeptVet Blog, several different people have suggested that skeptics should not criticize alternative treatments until they have tried them.This advice is given without an awareness of how easily we all make cognitive errors and how those errors can give us an illusory view of the effectiveness of our treatments. To be fair, we are all just as susceptible to this type of cognitive error no matter what type of treatment we are using, and we should always be wary of misdiagnosis or inappropriate treatment selection, situations in which these types of cognitive errors are common in both evidence based and alternative medical treatments. It underlines the importance of blinded, controlled trials, and the evaluation of the strength of the evidence supporting any treatment in  as objective a way as possible. "Trying it for yourself" is just a good way to fool yourself, and the more confident you are in your experience and ability, the more likely you are to believe that an ineffective treatment works. This new psychological paper just demonstrates the idea again in a relevant way.


  1. Of course, skeptics also shouldn't comment about murder, suicide, hard drugs, incest, extraterrestrial life etc etc until they've tried those as well.