Anecdotal Evidence

Anecdotal evidence is evidence that we draw from our own lives and direct personal experience. People typically utilise anecdotal evidence when making decisions about what products to buy, which people or organisations to trust, what medical treatments to get, what is the best way of doing something, etc. Anecdotal evidence is typically discussed using language such as ‘in my experience’, ‘I have found that this works best’, ‘I have seen this happen’, ‘I know from personal experience’, and similar such phrases. Anecdotal evidence can be very valuable and is essential in making many ordinary decisions of daily life. However, when applied in inappropriate circumstances, anecdotal evidence can be very unreliable and lead to many false inferences and poor decisions.

There are a number of important weaknesses and limitations of anecdotal evidence:

  • Anecdotes cannot control for possible confounding variables. For example, if I were to eat a new type of fruit and later that day become sick, I might conclude that the fruit made me sick. However, there are many other possibilities as to what could have caused me to become sick, including anything else I may have eaten, something I touched, or some place that I visited recently. Since my single anecdotal experience cannot rule out these other possibilities, we say that it cannot control for confounding factors.
  • Anecdotal evidence is subject to confirmation bias. Confirmation bias is a cognitive bias which describes the human tendency to remember information or events which confirm one’s preexisting beliefs, and forget facts or events which challenge those beliefs. Thus, if we believe that a particular food makes us ill, we are liable to remember those times when we felt ill after having eaten that food, but forget the times when we do not feel ill after having eaten it. This tendency has been demonstrated in numerous psychological studies in a variety of settings.
  • Anecdotal evidence is often not representative of broader trends. We can only directly experience or observe events which come to our attention as part of our lives. By the nature of human existence, however, we only ever experience a very specific, narrow portion of reality, and often this will not be representative of the broader whole. So, for example, I might believe that because neither I nor my friends have lost their jobs, that therefore the economy isn’t really in a recession. This would be an invalid inference because I and my friends may work in a particular industry that is doing well, while other people in very different occupations might be having a much harder time finding work.

Whenever one or more of these difficulties are likely to be relevant we should not rely solely upon anecdotal evidence for decision making. A very common example of reliance on anecdotal evidence leading to poor decisions occurs when people justify using demonstrably ineffective alternative medical treatments such as homeopathy, chiropractic, and various herbal medicines on the basis of their own positive experiences in using such treatments. People can come to believe that such treatments have helped them because of the impact of confounding variables (other treatments they may have also been taking, the natural history of the disease, etc), confirmation bias (they remember times the treatment seemed to help but forget the times when it did not), and non-representative reports (only people who experience some improvement after taking the medication come forward to report a success, while those who do not experience any improvement just try some other treatment and don’t speak out).

It is precisely in order to overcome the defects of anecdotal evidence that scientific research has adopted methodologies which carefully control for extraneous variables, rigorously record all relevant observations so that all evidence is included in anlayses, and test outcomes or observations in a variety of settings to ensure that results are representative and not isolated to one unique context. Taking proper care to rigorously eliminate as many of the failings of anecdotal evidence as possible is one of the hallmarks of scientific investigation, and is one of the primary reasons why science is able to produce more reliable knowledge than is possible through mere unstructured observation and personal experience.

Further Reading

The role of anecdotes in evidence based medicine: An excellent introduction to anecdotes and the many problems with relying on them as evidence, with an application to medicine

Why anecdotal evidence is unreliable: An explanation of the problems with relying on anecdotes, and how many of these can be overcome by using scientific controls

How anecdotal evidence can undermine scientific results: Scientific American article discussing the pitfalls of relying on anecdotal evidence

Confirmation bias: A brief introductory article discussing the concept and giving examples