You would think by now that we could say unequivocally what causes what. But the question of cause, which has haunted science and philosophy from their earliest days, still dogs our heels for numerous reasons.
Humans are evolutionarily predisposed to see patterns, and psychologically inclined to gather information that supports preexisting views, a trait known as confirmation bias. We confuse coincidence with correlation, and correlation with causality.
For A to cause B, we tend to say that, at a minimum, A must precede B, the two must covary (vary together), and no competing explanation can better explain the covariance of A and B. Taken alone, however, these three requirements cannot prove cause; they are, as philosophers say, necessary but not sufficient. In any case, not everyone agrees with them.
Speaking of philosophers, David Hume argued that causation doesn’t exist in any provable sense [source: Cook]. Karl Popper and the Falsificationists maintained that we cannot prove a relationship, only disprove it, which explains why statistical analyses do not try to prove a correlation; instead, they pull a double negative and disprove that the data are uncorrelated, a process known as rejecting the null hypothesis [source: McLeod].
With such considerations in mind, scientists must carefully design and control their experiments to weed out bias, circular reasoning, self-fulfilling prophecies and hidden variables. They must respect the requirements and limitations of the methods used, draw from representative samples where possible, and not overstate their results.
Ready to read about 10 instances where that wasn’t so easy?
10: The Trouble With Henry (and Hawthorne)
People are a pain to research. They react not only to the stimulus being studied, but also to the experiment itself. Researchers today try to design experiments to control for such factors, but that wasn’t always the case.
Take the Hawthorne Works in Cicero, Illinois. In a series of experiments from 1924-1932, researchers studied worker productivity effects associated with altering the Illinois factory’s environment, including changing light levels, tidying up the place and moving workstations around. Just when they thought they were on to something, they noticed a problem: The observed increases in productivity dropped almost as soon as the researchers left the works, indicating that the workers’ knowledge of the experiment — not the researchers’ changes — had fueled the boost. Researchers still call this phenomenon the Hawthorne Effect [source: Obrenović].
A related concept, the John Henry effect, occurs when members of a control group try to beat the experimental group by kicking their efforts into overdrive. They need not know about the experiment; they need only see one group receive new tools or additional instruction. Like the steel-driving man of legend, they want to prove their capabilities and earn respect [sources: Saretsky; Vogt].
9: Always Bet on Black?
The titular characters of Tom Stoppard’s film “Rosencrantz and Guildenstern Are Dead” begin the film baffled, confused and finally frightened as each of 157 consecutive flips of a coin comes up heads. Guildenstern’s explanations of this phenomenon range from time loops to “a spectacular vindication of the principle that each individual coin, spun individually, is as likely to come down heads as tails … “
Evolution wired humans to see patterns, and our ability to properly process that urge seems to short-circuit the longer we spend gambling. We can rationally accept that independent events like coin flips keep the same odds no matter how many times you perform them. But we also view those events, less rationally, as streaks, making false mental correlations between randomized events. Viewing the past as prelude, we keep thinking the next flip ought to be tails.
Statisticians call this the gambler’s fallacy, aka the Monte Carlo fallacy, after a particularly illustrative example that occurred in that famed Monaco resort town. During the summer of 1913, bettors watched in increasing amazement as a casino’s roulette wheel landed on black 26 times in a row. Inflamed by the certainty that red was “due,” the punters kept plunking down their chips. The casino made a mint [sources: Lehrer; Oppenheimer and Monin; Vogt].
8: The Hot Hand and the Monkey’s Paw
No discussion of streaks, magical thinking or false causation would be complete without a flip through the sports pages. Stellar sports seasons arise from such a mysterious interplay of factors — natural ability, training, confidence, the occasional X factor — that we imagine patterns in performance, even though studies repeatedly reject streak shooting and “successful” superstitions as anything more than imaginary.
The belief in streaks or slumps implies that success “causes” success and failure “causes” failure or, perhaps more reasonably, that variation in some common factor, such as confidence, causes both. But study after study fails to bear this out [source: Gilovich, et al]. The same holds true for superstitions, although that never stopped retired NBA player and Dallas Mavericks guard Jason Terry from sleeping in the opposing teams’ game shorts before each game, or NHL center and retired Ottawa Senators player Bruce Gardiner from dunking his hockey stick in the toilet to break the occasional slump [source: Exact Sports].
The sophomore slump, too, typically arises from a too-good first year. Performance swings tend to even out in the long run, a phenomenon statisticians call regression toward the mean [source: Barnett, et al]. In sports, this averaging out is aided by the opposition, which adjusts to counter the new player’s successful skill set.
7: Hormonal Imbalance
Randomized controlled trials are the gold standard in statistics, but sometimes — in epidemiology, for example — ethical and practical considerations force researchers to analyze available cases. Unfortunately, such observational studies risk bias, hidden variables and, worst of all, study groups that might not accurately reflect the population. Studying a representative sample is vital; it allows researchers to apply results to people outside of the study, like the rest of us.
A case in point: hormone replacement therapy (HRT) for women. Beyond treating symptoms associated with menopause, it was once hailed for potentially reducing coronary heart disease (CHD) risk, thanks to a much-ballyhooed 1991 observational study [source: Stampfer and Colditz]. But later randomized controlled studies, including the large-scale Women’s Health Initiative, revealed either a negative relationship, or a statistically insignificant one, between HRT and CHD [source: Lawlor, et al.].
Why the difference? For one thing, women who use HRT tend to come from higher socioeconomic strata and receive better quality of diet and exercise — a hidden explanatory relationship for which the observational study failed to fully account [source: Lawlor, et al].
6: Super Bowl Stock Market Shuffle
In 1978, sports reporter and columnist Leonard Koppett mocked the causation-correlation confusion by wryly suggesting that Super Bowl outcomes could predict the stock market. It backfired: Not only did people believe him, but it worked — with frightful frequency.
The proposal, now commonly known as the Super Bowl Indicator, went as follows: If one of the 16 original National Football League teams — those in existence before the NFL’s 1966 merger with the American Football League — won the Super Bowl, the stock market would rise throughout the rest of the year. If a former AFL team won, it would go down [source: Bonsal].
From 1967 to 1978, Koppett’s system went 12 for 12; up through 1997, it boasted a 95 percent success rate. It stumbled during the dot-com era (1998 -2001) and notably in 2008, when the Great Recession hit, despite a win by the New York Giants (NFC). Still, as of February 2021, the indicator had a 74 percent success rate [source: Chen].
Some have argued that the pattern exists, driven by belief; it works, they say, because investors believe it does, or because they believe that other investors believe it. This notion, though clever in a regressive sort of way, hardly explains the 12 years of successful correlations predating Koppett’s article. Others argue that a more relevant pattern lies in the stock market’s large-scale upward trend, barring some short-term major and minor fluctuations [source: Johnson].
5: Big Data, Little Clarity
Big data — the process of looking for patterns in data sets so large they resist traditional methods of analysis — rates big buzz in the boardroom these days [source: Arthur]. But is bigger always better?
It’s a rule that’s drummed into most researchers in their first stats class: When encountering a sea of data, resist the urge to go on a fishing expedition. Given enough data, patience and methodological leeway, correlations are almost inevitable, if unethical and largely useless.
After all, the mere correlation between two variables does not imply causation; nor does it, in many cases, point to much of a relationship. For one thing, researchers cannot use statistical measures of correlation willy-nilly; each contains certain assumptions and limitations that fishing expeditions too often ignore, to say nothing of the hidden variables, sampling problems and flaws in interpretation that can gum up a poorly designed study.
But big data is increasingly being used and hailed for its invaluable contributions to areas such as education (e.g., creating customized learning programs); health care (e.g., wearable devices that provide real-time feed to your electronic health records); and entertainment (e.g., on-demand music services giving you targeted recommendations) [source: IntelliPaat]. Just don’t expect too much out of big data in the causality department.
4: Minimum Wage Equals Maximum Unemployment
Any issue dealing with money is bound to be deeply divisive and highly politicized, and minimum wage increases are no exception. The arguments are varied and complex, but essentially one side contends that a higher minimum wage hurts businesses, which drives down job availability, which hurts the poor. The other side responds that there’s little evidence for this claim, and that the 76 million Americans working at or below minimum wage, which some argue is not a living wage, would benefit from such an increase. They argue that the federal minimum wage for covered, nonexempt employees ($7.25 per hour in April 2022) has lowered Americans’ purchasing power by more than 20 percent [sources: U.S. Department of Labor; Cooper, et al].
As literary critic George Shaw reportedly quipped, “If all the economists were laid end to end, they’d never reach a conclusion,” and the minimum-wage debate seems to bear that out [source: Quote Investigator]. For every analyst who says minimum wage increases drive jobs away, there is another who argues against such a correlation.
In the end, both sides share a fundamental problem: namely, the abundance of anecdotal evidence many of their talking heads rely on for support. Secondhand stories and cherry-picked data make for weak tea in any party, even when presented in pretty bar charts.
3: Breakfast Beats Obesity, Dinner Denies Drugs
Between fitness apps, drugs and surgeries, weight loss in the United States is a $78 billion-per-year industry, with millions of Americans bellying up to the weight-loss bar annually [source: Research and Markets]. Not surprisingly, weight loss studies — good, bad or ugly — get a lot of press in the U.S.
Take the popular idea that eating breakfast beats obesity, a sugar-frosted nugget derived from two main studies: One, a 1992 Vanderbilt University randomized controlled study, showed that reversing normal breakfast habits, whether by eating or not eating, correlated with weight loss; the other, a 2002 observational study by the National Weight Control Registry, correlated breakfast-eating with successful weight-losers — which is not the same as correlating it with weight loss [sources: Brown, et al.; Schlundt, et al.; Wyatt, et al.].
Unfortunately, the NWCR study failed to control for other factors — or, indeed, establish any causal connection from its correlation. For example, a person who wants to lose weight might work out more, eat breakfast or go whole-hog protein, but without an experimental design capable of dialing in causal links, such behaviors amount to nothing more than commonly co-occurring characteristics [source: Brown, et al].
A similar problem plagues the numerous studies linking family dinners with a decreased risk of drug addiction for teens. Although attractive for their simple, appealing strategy, these studies frequently fail to control for related factors, such as strong family connections or deep parental involvement in a child’s life [source: Miller, et al].
2: The Suicidal Sex
We often hear that men, especially young men, are more likely to commit suicide than are women. In truth, such statements partake of empirical generalization — the act of making a broad statement about a common pattern without attempting to explain it — and mask several known and potential confounding factors.
Take, for example, a Youth Risk Behaviors Survey from 2019 (the most recent year available) found that girls in grades 9-12 attempted suicide almost twice as often as male students (11 percent vs. 6.6 percent) [source: American Foundation for Suicide Prevention].
How, then, can a higher correlation exist between the opposite sex and suicide? The answer lies in suicide attempts by methodology: While the most common method of suicide for both sexes in 2019 was by firearm (55.6 percent for men and 31.4 percent for women), women were almost equally likely to die by poisoning or suffocation [source: National Institute of Mental Health].
Even if we could dispose of such confounding factors, the fact would remain that maleness, per se, is not a cause. To explain the trend, we need to instead identify factors common to men, or at least suicidal ones. The same point applies to the comparatively high rates of suicide reported among divorced men. Divorce doesn’t cause men to commit suicide; if anything, the causal variable hides among related factors such as male role inflexibility, their social networks, the increasing importance of child care and men’s desire for control in relationships [source: Scourfield and Evans].
1: Vaccination Vexation
No correlation/causation list would be complete without discussing parental concerns over vaccination safety. Before the COVID-19 pandemic hit the world in 2020, the main issue was a fear among some parents that the measles, mumps and rubella vaccination was causally linked to autism spectrum disorders. This notion was popularized by celebrities like Jenny McCarthy.
Despite the medical community debunking the 1998 Andrew Wakefield paper that inspired the falsehood, and despite subsequent studies showing no causal link, some parents remain fearful of an autism connection or other vaccine-related dangers [sources: Park; Sifferlin; Szabo].
Then COVID-19 arrived, and to date has killed millions around the globe. Scientists raced to create an effective vaccine and they succeeded; the first U.S. COVID-19 vaccine was available in December 2020 under the FDA’s emergency use authorization [source: FDA]. But it also quickly became intertwined with the extreme polarization of U.S. politics and misinformation.
Many parents, especially Republicans, feared the vaccines were unsafe because they were developed so quickly, and because there might be as-yet-unknown long-term side effects. There were also incorrect fears about the vaccine affecting future fertility. Those have now been proven false [source: Kelen and Maragakis].
These are no harmless misunderstandings. Despite debunking a link between autism and childhood vaccines, many parents remain leery of the shots. In 2019, there were 1,282 cases of measles in 31 states, the highest number in the U.S. since 1992. The majority of these cases were among the unvaccinated [source: CDC]. Whether that correspondence is coincidental, correlative or causal is well worth considering. And the effects of the current COVID-19 vaccination hesitation remain to be seen.
Originally Published: Dec 23, 2013
Lots More Information
- American Foundation for Suicide Prevention. “Suicide statistics.” (Feb. 21, 2022) https://afsp.org/suicide-statistics/
- Arthur, Lisa. “What is Big Data?” Forbes. Aug. 15, 2013. (Dec. 18, 2013) http://www.forbes.com/sites/lisaarthur/2013/08/15/what-is-big-data/
- Bansal, Ayush. “Everything you want to know about The Super Bowl Indicator.” TechStory. Feb. 14, 2022. (Feb. 16, 2022)
- Barnett, Adrian, et al. “Regression to the mean: what it is and how to deal with it.” International Journal of Epidemiology. Aug. 27, 2004. (Feb. 16, 2022)
- Brown, Andrew W., Michelle M, Bohan Brown and David B. Allison. “Belief Beyond the Evidence: Using the Proposed Effect of Breakfast on Obesity to Show 2 Practices that Distort Scientific Evidence.” American Journal of Clinical Nutrition. Vol. 98, no. 5. Page 1298. (Dec. 18, 2013) http://ajcn.nutrition.org/content/early/2013/09/04/ajcn.113.064410
- Centers for Disease Control and Prevention. “Measles Cases and Outbreaks.” Nov. 19, 2021. (Feb. 21, 2022)
- Centers for Disease Control and Prevention. “Recommended Immunization Schedule for Persons Age 0 through 18 Years: United States, 2013.” (Dec. 18, 2013) http://www.cdc.gov/vaccines/schedules/hcp/imz/child-adolescent.html
- Chen, James. “Super Bowl Indicator.” Investopedia. Feb. 10, 2022. (Feb. 16, 2022) https://www.investopedia.com/terms/s/superbowlindicator.asp
- Cooper, David, et al. “Twenty-one states raised their minimum wages on New Year’s Day.” Economic Policy Institute. Jan. 6, 2022. (Feb. 21, 2022) https://www.epi.org/blog/states-minimum-wage-increases-jan-2022/
- ESPN. “Michael Jordan Stats.” (Dec. 18, 2013) http://espn.go.com/nba/player/stats/_/id/1035/michael-jordan
- Exact Sports. “Pre-game rituals and superstitions.” July 19, 2010. (Feb. 16, 2022) https://exactsports.com/blog/pregame-superstitions/2010/07/19/
- FDA. “FDA Approves First COVID-19 Vaccine.” Aug. 23, 2021. (Feb. 21, 2022) https://www.fda.gov/news-events/press-announcements/fda-approves-first-covid-19-vaccine
- Gilovich, Thomas, Robert Vallone and Amos Tversky. “The hot hand in basketball: On the misperception of random sequences.” Cognitive Psychology. Vol. 17. Page 295. (Feb. 16, 2022) https://www.sciencedirect.com/science/article/abs/pii/0010028585900106
- Hamel, Liz, et al. “KFF COVID-19 Vaccine Monitor: October 2021.” KFF. Oct. 28, 2021. (Feb. 21, 2022) https://www.kff.org/coronavirus-covid-19/poll-finding/kff-covid-19-vaccine-monitor-october-2021/
- Hill, Kashmir. “How Target Figured Out a Teen Girl was Pregnant Before Her Father Did.” Forbes. Feb. 16, 2012. (Dec. 18, 2013) http://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/
- HoopData. “League Average Stats.” (Dec. 18, 2013) http://hoopdata.com/regstats.aspx
- IntelliPaat. “10 Big Data Applications in Real Life.” (Feb. 21, 2022) https://intellipaat.com/blog/10-big-data-examples-application-of-big-data-in-real-life/
- Johnson, Bob. “Would you base your market bets on the Super Bowl?” CNBC. Jan. 27, 2015. (Feb. 16, 2022) https://www.cnbc.com/2015/01/26/would-you-base-your-market-bets-on-the-super-bowl.html
- Johnson, Eric O. and Lonni Schultz. “Forward Telescoping Bias in Reported Age of Onset: An Example from Cigarette Smoking.” International Journal of Methods in Psychiatric Research. Vol. 14, no.3. Page 119. (Dec. 18, 2013) http://www.ncbi.nlm.nih.gov/pubmed/16389888
- Kates, Jennifer, et al. “Update on COVID-19 Vaccination of 5-22 Year Olds in the U.S.” KFF. Jan. 20, 2022. (Feb. 21, 2022) https://www.kff.org/coronavirus-covid-19/issue-brief/update-on-covid-19-vaccination-of-5-11-year-olds-in-the-u-s/
- Lawlor, Debbie A., George Davey Smith and Shah Ebrahim. “Commentary: The Hormone Replacement-Coronary Heart Disease Conundrum: Is this the Death of Observational Epidemiology?” International Journal of Epidemiology. Vol. 33, no. 3. Page 464. (Dec. 18, 2013) http://ije.oxfordjournals.org/content/33/3/464.full#ref-4
- Lehrer, Jonah. “How We Decide.” Houghton Mifflin Harcourt. 2009.
- Lorkowski, C.M. “David Hume: Causation.” Internet Encyclopedia of Philosophy. (Feb. 16, 2022) https://www.simplypsychology.org/Karl-Popper.html
- Mandell, David S., Maytali M. Novak and Cynthia D. Zubritsky. “Factors Associated With Age of Diagnosis Among Children With Autism Spectrum Disorders.” Pediatrics. Vol. 116, no. 6. Page 1480. (Dec. 18, 2013) http://pediatrics.aappublications.org/content/116/6/1480.abstract
- Manson, JoAnn, et al. “Menopausal Hormone Therapy and Health Outcomes During the Intervention and Extended Poststopping Phases of the Women’s Health Initiative Randomized Trials.” Journal of the American Medical Association. Vol. 310, no. 13. Page 1353. (Dec. 18, 2013) http://jama.jamanetwork.com/article.aspx?articleid=1745676
- McLeod, Dr. Saul. “Karl Popper — Theory of Falsification.” SimplyPsychology. 2020. (Feb. 16, 2022) https://www.simplypsychology.org/Karl-Popper.html
- Miller, Daniel, et al. “Family Meals and Child Academic and Behavioral Outcomes.” NCBI. Aug. 7, 2012. (Feb. 21, 2022) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3498594/
- National Institute of Mental Health. “Suicide.” (Feb. 21, 2022) https://www.nimh.nih.gov/health/statistics/suicide
- Oppenheimer, Daniel and Benoît Monin. “The retrospective gambler’s fallacy: Unlikely events, constructing the past, and multiple universes.” Judgment and Decision Making. Vol. 4, no. 5. Page 326. (Feb. 16, 2022) https://psycnet.apa.org/record/2009-12960-001
- Ozonoff, Sally, Kelly Heung, Robert Byrd, Robin Hansen and Irva Hertz-Picciotto. The Onset of Autism: Patterns of Symptom Emergence in the First Years of Life. Autism Research. Vol. 1, no. 6. Page 320. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2857525/
- Quote Investigator. “If All the Economists Were Laid End to End, They Would Not Reach a Conclusion.” Sept. 13, 2016. (Feb. 21, 2022) https://quoteinvestigator.com/2016/09/13/economists/
- Research and Markets. “The U.S. Weight Loss & Diet Control Market.” March 2021. (Feb. 21, 2022) https://www.researchandmarkets.com/reports/5313560/the-u-s-weight-loss-and-diet-control-market
- Saretsky, Gary. “The OEO P.C. Experiment and the John Henry Effect.” The Phi Delta Kappan. Vol. 53, no. 9. Page 579. (Dec. 18, 2013) http://www.jstor.org/stable/20373317
- Schlundt, David G., James O. Hill, Tracy Sbrocco, Jamie Pope-Cordle and Teresa Sharp. “The Role of Breakfast in the Treatment of Obesity: A Randomized Clinical Trial.” American Journal of Clinical Nutrition. Vol. 55, no. 3. Page 645. (Dec. 18, 2013) http://ajcn.nutrition.org/content/55/3/645.abstract?sid=2ef145a4-5324-40d0-bead-e669e51756d1
- Scourfield, Jonathan and Rhiannon Evans. “Why Might Men Be More at Risk of Suicide After a Relationship Breakdown? Sociological Insights.” American Journal of Men’s Health. Aug. 26, 2014. (Feb. 21, 2022) https://journals.sagepub.com/doi/full/10.1177/1557988314546395
- Shute, Nancy. “The Last Word on Hormone Therapy from the Women’s Health Initiative.” NPR. Oct. 4, 2013. (Dec. 18, 2013) http://www.npr.org/blogs/health/2013/10/04/229171477/the-last-word-on-hormone-therapy-from-the-womens-health-initiative
- Sifferlin, Alexandra. “Multiple Vaccinations on Same Day Does Not Raise Autism Risk.” Time magazine. March 29, 2013. (Dec. 18, 2013) http://healthland.time.com/2013/03/29/multiple-vaccinations-on-same-day-does-not-raise-autism-risk/
- Sports Reference. “NBA & ABA Year-by-Year Leaders and Records for Points per Game.” (Dec. 18, 2013) http://www.basketball-reference.com/leaders/pts_per_g_yearly.html
- Stampfer Meir J. and Graham A. Colditz. “Estrogen Replacement Therapy and Coronary Heart Disease: A Quantitative Assessment of the Epidemiologic Evidence.” Preventative Medicine. Vol. 20, no. 1. Page 47. (Dec. 18, 2013) http://www.sciencedirect.com/science/article/pii/009174359190006P
- Szabo, Liz. “Full Vaccine Schedule Safe for Kids, No Link to Autism.” USA Today. March 29, 2013. (Dec. 18, 2013) http://www.usatoday.com/story/news/nation/2013/03/29/vaccine-schedule-autism/2026617/
- Toporek, Bryan. “Kobe Bryant vs. Michael Jordan: Comparing the G.O.A.T. to the Birthday Boy.” Bleacher Report. Aug. 23, 2013. (Dec. 18, 2013) http://bleacherreport.com/articles/1732165-kobe-bryant-vs-michael-jordan-comparing-the-goat-to-the-birthday-boy
- U.S. Bureau of Labor Statistics. “Labor Force Statistics from the Current Population Survey.” Jan. 20, 2022. (Feb. 21, 2022https://www.bls.gov/cps/cpsaat44.htm
- U.S. Department of Labor. “Minimum Wage.” (Feb. 21, 2022) https://www.dol.gov/general/topic/wages/minimumwage
- Vogt, W. Paul. “Dictionary of Statistics and Methodology.” Sage. 1993.
- Wyatt, Holly R., Gary K. Grunwald, Cecilia L. Mosca, Mary L. Klem, Rena R. Wing and James O. Hill. “Long-Term Weight Loss and Breakfast in Subjects in the National Weight Control Registry.” Obesity Research Vol. 10, no. 2. Page 78. (Dec. 18, 2013) http://onlinelibrary.wiley.com/doi/10.1038/oby.2002.13/pdf
Please copy/paste the following text to properly cite this HowStuffWorks.com article: