Alternately, this post could be titled:
There is Good Science, Bad Science, and Plain Ole JUNK Science
As a follow-up to my recent study on the incredible safety of fluoride in water and dental products, this is a perfect opportunity to talk about why science journalism, and even some research, can do far more harm than good by inaccurate reporting, or by reporting single studies without putting them in context of the total body of research on a subject, etc. The article which I am rebutting here was published in Newsweek Magazine on March 10th, 2015 and titled: Water Fluoridation Linked to Higher ADHD Rates.
As someone who was diagnosed with ADD (without hyperactivity) in 2005 ago at age 33, as well as being a dentist, this naturally caught my attention. However, as I read through it, so many glaring errors jumped out at me, it became clear that the study is essentially meaningless, and the article DRASTICALLY overstated the conclusion. There is one statement mid-way through the article that basically tells you how meaningless this study is on its own:
But scientists were quick to point out that this is just one study, and doesn’t prove that there is necessarily a causal link between fluoridation and ADHD. They also noted a number of important limitations: Individual fluoride exposures weren’t measured, ADHD diagnoses weren’t independently verified and there may be other unknown confounding factors that explain the link.
Summary of the Major Flaws in the Study and the Newsweek Report
Let’s break that down to be really clear:
- Individual fluoride exposures weren’t measured (there is no way to tell if the people diagnosed with ADHD had higher or lower levels of fluoride)
- ADHD diagnoses weren’t independently verified (meaning that we don’t actually know if the people reported as ADHD have it or not)
- There may be other unknown confounding factors that explain the link. (they only included 1 possible confounding factor)
Based on some additional research into this topic, I will point out a number of other glaring weaknesses:
- The study is based solely on surveys! There is no control group, no placebo comparison, no double-blind controls to ensure the investigators aren’t minimizing their own biases.
- Neither of the study’s authors has any expertise or credibility in this field.
- Dr. Christine Tillman is not an epidemiologist, and her primary area of research is “…focused on understanding the relation between brain-related changes and cognition in patients with pediatric-onset multiple sclerosis.” 1 Not only is this not her field, this is her only paper on the subject.
- Ashley Malin is still a Ph.D candidate in Psychology, has only published 2 other papers, and this is her only publication on the subject.2 She also is not an epidemiologist.
- The ONLY possible confounding factor that the authors controlled for was Socioeconomic Status (SES). Really? This is a VERY poor study design.
- Click here to learn more about confounding factors and why they’re so important
- Even the Newsweek article clearly states early on: “It’s the first time that scientists have systematically studied the relationship between the behavioral disorder and fluoridation.”
- IOW – if this is the very first study on the subject, and there are a lot of weaknesses, then Newsweek is being highly irresponsible with their headline, which implies a much stronger connection than is justified.
- The journal in which the paper was published, Environmental Health, has a paltry Impact Factor of 2.71. Basically, that means that this journal is very rarely cited in other journals or papers, and is a low-quality journal. As a comparison, the Impact Factor for the highly respected journal Nature is 31.4.
- It should be noted that the Impact Factor is only 1 rating system, and not everyone agrees on its importance. At the bottom of the page I linked just above on Wikipedia, there is a good discussion of the strengths and weaknesses of it.
- Environmental Health is not even included in the top 100 Psychology journals.
- Throughout the rest of the article, you will see a lot of links to studies and people that support this claim, but except for a few quotes (which I’m including here for emphasis), there is very little included in the way of rebuttal. IOW, the author seems to have a bias towards one side of the argument (based on the number of references) that is unjustified given the weakness of the initial study. As the British would say, “Poor form, old chap. Poor form.”
Reading just a little bit further into the article, one comes to this quotation that sums up the study well:
Dr. Benedetto Vitiello, a researcher at the National Institutes of Health, says that the link between the two may not be a causal one and could be explained by regional or cultural factors. Charles Poole, an epidemiologist at the University of North Carolina, says that this research suggests fluoride should be more carefully studied, but doesn’t show much of anything by itself. “I think the authors were quite cautious in their interpretation… and [fusion_builder_container hundred_percent=”yes” overflow=”visible”][fusion_builder_row][fusion_builder_column type=”1_1″ background_position=”left top” background_color=”” border_size=”” border_color=”” border_style=”solid” spacing=”yes” background_image=”” background_repeat=”no-repeat” padding=”” margin_top=”0px” margin_bottom=”0px” class=”” id=”” animation_type=”” animation_speed=”0.3″ animation_direction=”left” hide_on_mobile=”no” center_content=”no” min_height=”none”][accurate] in their statement of the study’s limitations,” he says. “So it would be ludicrous to draw a strong conclusion based on this study alone.” [Emphasis in the last quote is mine.]
And as an example of just how weak the results in this study are, given that the only possible confounding factor for which the authors controlled was socioeconomic status, I found these rebuttals quite apt: ADHD Linked to Elevation Not Fluoridation and More poor-quality research promoted by anti-fluoride activists. The author of that blog highlights Points 1 and 2 in my first list with this quote about the flawed methods in the study reported in Newsweek:
Note, they did not use data for individual children exhibiting symptoms of ADHD determined by a health professional. The data was from random surveys “in which parents were contacted via telephone and asked about the emotional and physical well-being of a randomly selected child from their household.” Similarly they did not use data for dietary intake of fluoride by individual children but used “the percentage of the U.S. population on public water systems that receives optimally fluoridated drinking water.”
In simplest terms, this means that they really have no proof of anything at all! They don’t know if the people actually had ADHD; they didn’t know if those people actually had higher levels of fluoride intake, there were no controls, no placebo, no double-blinds. This study stinks!
One of the authors of Making Sense of Fluoride did a little further “research” similar to the study authors but including several more confounding factors, and here are several other correlated factors. Was it fluoridated water, or was it maybe the elevation? Or poverty? Or age? See how weak a study is when it doesn’t control for as many factors as possible?
You can also read this well-written article by pediatrician Dr. Chad Hayes, which complements my article and adds more thoughtful analysis. Highly recommended! Drink the Water, Not the Kool-Aid
Correlation Does Not Equal Causation
One of the must frustrating issues that scientists, physicians, and dentists deal with in public outreach and communication about science is a misunderstanding about the difference between correlation and causation. They can be connected and relevant, but often they are not, so let’s spell them out:
- Correlation: a mutual relationship or connection between two or more things which may be coincidental, somewhat related, or causal.
- Causation: the relationship between cause and effect, when it is determined that the first event was directly responsible for the second.
A favorite FB meme of skeptics and scientists is this one illustrating the “clear” cause of the increase in autism: organic foods! Now, before anyone takes me to task – NO, I am NOT claiming that the increase in sales of organic food is what caused the increase in autism; it should be noted that the Correlation is factual, but the Causality is not.
Check out Spurious Correlations to see some really bizarre things that correlate, but which are obviously not causal. Hopefully, they will encourage you to be highly skeptical of correlations and demand more evidence. But some of them are pretty funny!
How To Figure Out If the Research is Good or Bad
For the public, it can be really difficult to know when a news article is based on good science or bad, or even if the reporting is good or not. Here are some guidelines and resources to help you navigate that difficult path. For starters, try using this infographic to count up how many problems you can find in the Newsweek article and the study:
- How to read and understand a scientific paper: a guide for non-scientists
- Carl Sagan’s Baloney Detection Guide
- What are Logical Fallacies and How to Avoid Them in Scientific Research and Discussion
WOW – if you’ve made it all the way to the end, congratulations! I’d love to hear what you think in the comments, as long as they’re respectful and supported by evidence. Conspiracy theories don’t count BTW. 🙂
In summary? The study is basically worthless, and there is currently no link between fluoride and ADHD. Period.[/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]
