In November of 2012, the journal “Food and Chemical Toxicology” published a paper by French scientist Gilles-Eric Séralini and his colleagues called “Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize.” The study was developed because Séralini and his team were interested in indications of toxicity found in the raw data of an earlier 90-day study done by Monsanto. The results of Séralini’s two year study showed as much as 5 times more liver and kidney disease, and 2 to 3 times more tumors in rats that had eaten Genetically Modified (GM) corn and that had drunk water containing Roundup.
In August of 2013, the journal retracted the article after reviewing the study design and the data it was based upon. Among the flaws they noted was that there were too few animals in the study, and the type of rat used, the Sprague Dawley, is prone to cancers and tumors. Folks on both sides of the issue have cried foul, some for the publication of the article in the first place, and others for its retraction. Both sides continue to present data and arguments to support what they believe to be the truth. And that’s where this gets interesting, pointing out the importance of understanding the scientific process and how difficult it can be to get everything just right. It also brings up the specter of profit and politics, two issues that are hard to remove from the daily lives of humans, leaving us with the question: Who should we believe?
We’ll try to help you with that question by using this controversy as an example of how to dig through the data and the claims to get a little closer to that elusive creature: THE ABSOLUTE TRUTH.
How To Read A Scientific Paper
Scientific papers are not meant to be light, entertaining reading. They follow a formula that makes sure you know precisely what steps were taken as the study was carried out, how the data was gathered, and what kind of statistical analysis was done on the data to determine how powerful the results are. It’s the scientist equivalent of Hansel and Gretal leaving breadcrumbs: If everything was done correctly, we should be able to do the exact same steps again and get pretty much the same results. By understanding exactly what was done, it also makes it possible for researchers to build on the results of a study by doing something slightly different, or for a longer timeframe, or with more animals. Well-done research will also be completely transparent about the funding of the research and the affiliations of the authors so that we can view the work while keeping in mind any biases that the authors may have had. It’s a system of checks and balances.
Most studies answer one small question, but raise many more questions. Thus most research papers end by saying something like “More research is needed.” This isn’t simply a ploy to get more money for more research. It’s actually a sign of humility and an acceptance that one paper is not going to give us the answer to life the universe and everything.
Here are the parts of a good scientific paper:
- Authors and their affiliations: This is generally everyone who participated in making the study possible and writing up results. Because we’re known by the company we keep, this gives you a first look at any kind of bias that might be included in the paper.
- Abstract: This is basically a summary describing what the study was and its findings.
- Introduction: Here’s the background leading up to the study. It often cites previous studies so you know what the authors were basing their work on and what they considered as they developed their experimental design. By showing us this, they’re allowing us once again to screen for bias.
- Materials and Methods: The devil is in the details and this is where the devil resides in a research paper. If you want to do the test again, just follow the instructions you find here. Though it can be dull reading, it’s incredibly important to determining the validity of the data.
- Results: This is the data and the statistical analysis of it.
- Discussion: Here’s where the authors tell you what they derived from their study.
- References: This is like the genealogy of the paper. It tells you what other papers the authors used to inform their process, study design and discussion. They are noted in parentheses throughout the paper, and then the full citation is shown so that readers can check out the papers on their own.
All in all, the scientific process and the papers that come out of the studies are like building a step-by-step map that shows what a group of researchers did and what they learned, and it allows anyone to look at all the information to see if their results and conclusions were valid. Papers are reviewed by scientific peers and if they think that everything was done properly, the paper is published in a journal. Publication is like a certification that says, “Yes, you can use this information to inform other projects or changes in management.”
It is not unusual for papers to be controversial even after publication. Discussions, like the one that is ongoing about Séralini’s paper, are good because they ensure that no one is making things up and that we continue to work through problems until we have something truly substantial on which to base our decisions. These discussions can unveil additional information and biases. That’s why THE ABSOLUTE TRUTH is so elusive. Everyone has a bias, and there’s always more to know.
Let’s see how this all works in the Seralini paper controversy.
What About That Rat?
Let’s start with the type of rat used for this study. There are about 200 strains of rats created through breeding so that they have traits in common. This reduces variability among individuals in a study design and allows researchers to be more precise. For example, the Zucker rat is a strain developed to study obesity and high blood pressure. Then there’s the Lewis rat strain, commonly used for transplantation and arthritis/inflammation research. For research on macular degeneration, researchers prefer the RCS rat which is the first known animal with inherited retinal degeneration.
The Sprague Dawley rat, which was used for this study, is an albino rat. They live about 2 to 3 years, with females typically surviving longer than males. Research done on this rat strain indicates that at about 1.5 to 2 years of age 87% of them will have developed some sort of tumor. These tumors are found in pituitary and adrenal glands and in the case of female rats, in the mammary glands.
The Sprague Dawley is the most widely used rat in biomedical research, particularly for toxicology research when we’re trying to understand the effects of chemicals on living creatures. The National Toxicity Program in the U.S. uses the same rat, from the same source as the Séralini study, for long-term toxicity and carcinogenic studies. The journal that retracted Séralini’s article had previously published a 90-day study by Monsanto using the same rat and showing no negative effects of eating GM corn. Monsanto’s study was also used to inform the European Union’s decision to authorize the use of glyphosate (Roundup) in member countries.
So, if everybody uses the Sprague Dawley rat for this kind of research, how do we know that changes in the number of tumors reported by Séralini are due to the chemicals they’re eating, or simply a result of their tendency to get tumors as they get older?
You probably remember this from your science classes in school. If you want to know if something you’re doing is making a difference, you need to have a “control” to which nothing is done. Then you can compare the results of a change to doing nothing. As a good scientist should do, Séralini included a control group in his study. When the study was finished, he could compare the number of tumors between the female rats in the “treatment” groups and the female rats in the control group. His study showed that by the beginning of the 24th month, and depending on which level of chemicals the animals had been fed, 50 to 80% of the rats had tumors. Meanwhile, only 30% of the control group had tumors. His conclusion was that a treated rat was 2 times more likely to have a mammary tumor.
Ah! But any time you come to a conclusion, it’s based on statistics, or as many people like to say: “Lies, Damn Lies, and Statistics.” If you’re going to do a study, you have to be sure that the size of your treatment and control will give you “statistically valid” results. Critics of Séralini’s study say that he used too few rats and thus his results are useless. Séralini pushes back by saying “I used the same number of rats as Monsanto used in their study,” adding that in fact Monsanto did have 20 rats in each group, as compared to the 10 he had in each group, but that Monsanto only reported on the results of 10 members in each group. No one knows how Monsanto chose which 10 rats to use and which to throw out in each group, leaving us with interesting questions about that study’s validity.
Hmmmm……it seems like there’s more to look into here, and that’s exactly what we’ll do next week. It might seem that statistics is a boring topic, but in this case the stakes are high. How well scientists are doing their jobs will tell us if we’re safe eating all the foods that come with GM ingredients. People’s lives and livelihoods are on the line, and that’s why it’s so important for us to be able to drill down into the data, and the politics. We may not come up with THE ABSOLUTE TRUTH, but by understanding the process, and the ongoing discussion, we can be part of the community of science referees that keeps everyone honest.
Stay tuned for our next, exciting installment: Lies, Damn Lies, and Statistics: GM Corn, Roundup, and Tumors