Here’s a quick recap of Part I of this series: In November of 2012, the journal “Food and Chemical Toxicology” published a paper by French scientist Gilles-Eric Séralini and his team called “Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize.” Séralini and his team undertook the study because they were interested in what they saw as indications of toxicity found in the raw data of an earlier 90-day study done by Monsanto. The results of Séralini’s two year study showed as much as 5 times more liver and kidney disease, and 2 to 3 times more tumors in rats that had eaten Genetically Modified (GM) corn and that had drunk water containing Roundup. The journal retracted the article in August of 2013, in spite of the objections of its authors, citing problems with the study caused by using too few animals, and because the type of rat used is prone to tumors. The Editor-in-chief said, “Ultimately the results presented (while not incorrect) are inconclusive, and therefore do not reach the threshold of publication for ‘Food and Chemical Toxicology.'”
Though the Sprague Dawley rat is commonly used for this type of study, and was used for the original Monsanto study, this fact may not overcome concerns about the statistical validity of Séralini’s results. So let’s take a look at statistics and the suggested guidelines for the number of animals to be used in studies like this.
What Is Statistics?
Statistics is way of testing the likelihood of something happening. Your insurance company uses statistics to decide how much to charge you for car insurance using information about drivers similar to you and how often they are in accidents. Weather forecasters use statistical models that compare prior weather conditions with current weather conditions to predict future weather. If you’re taking any kind of medication, it has been evaluated based on the likelihood that it will help someone like you and the likelihood that you might suffer some kind of side effect.
What all of these uses have in common is that insurance companies don’t have information on all the drivers on the planet, forecasters don’t look at the entire history of weather, and medical researchers don’t study every person taking a drug. Instead, they use a sample population. So if you’re running a study, like the one that Séralini’s team did, you need to find similar individuals to represent a larger population and then, you need to figure out how many individuals make up a good sample. Scientists have solved the first issue to some degree by using rats with very similar genetic make ups, like the Sprague Dawley rat. As far as the number of individual rats used in an experiment, it depends on what you’re trying to find out.
Statistics is not an easy discipline. So, just like every other science, statistics builds on the lessons of researchers who went before. That means there are protocols that researchers must follow to ensure that they are getting good results. After all, no one wants to do a 2-year study only to find that they’ve wasted their time. This also helps those of us non-statisticians to evaluate a study without having to go back to school and get another degree.
Séralini’s Statistics
The Séralini team was primarily interested in potential liver and kidney damage, so they followed the guidance for studies on toxicity presented by the Organization for Economic Cooperation and Development’s (OECD) protocol 453. For this protocol, the researchers are to use 10 rats per group, at three dose levels for 12 months. Had they been doing a study on cancer they would have needed 50 rats per group for two years. The reason for including more animals in the cancer studies is that it gives us a larger sample so that we can tell the difference between animals who were going to get cancer anyway, and animals who were affected by the test substance. What this means to us is that we can look at the effects on livers and kidneys, but we really can’t say much about whether or not what the rats were eating and drinking caused their tumors.
But the use of statistics goes beyond simply making sure you have enough animals in your sample size. You have to properly analyze the data that you collect using accepted mathematical models. The Séralini team used a new type of statistical analysis that left many researchers and professional statisticians scratching their heads. Since Séralini will not release his raw data, statisticians have taken what they can from the published paper to run their own, more standard analyses and have found a very low likelihood that the rats got tumors as a result of what they were eating.
Many of the critiques I’ve read about the Séralini study over the last month or so point out that this is the only study among many that has ever shown negative results for animals fed Genetically Modified grains. On the other hand, I’ve found articles that describe problems caused by Roundup itself. Farm families suffered from more miscarriages and premature birth when males were exposed to Roundup. Subsequent research found that it kills human placental cells at concentrations far below what is used in fields, and that Roundup was at least twice as toxic as glyphosate alone. Brazilian researchers concluded that Roundup is toxic to rat mothers and causes “developmental retardation of the fetal skeleton” and liver damage to pregnant rats and their fetuses. But while it may cause those kinds of problems, a study of 57,311 licensed pesticide applicators in Iowa and North Carolina found that glyphosate exposure was not associated with cancer incidence overall.
What I’m most disturbed by is the emotional, and often insulting nature of the comments about and even some of the critiques of the Séralini team’s article. From my perspective, an opinion about a scientific study has no value if it’s based simply on what the opiner wanted the results to be. Yes, politics does play a role in science, and yes, it’s possible that the science on both sides of this argument has been tainted from time to time by politics. That’s part of what it means to be human. But we have ideals, and we can attempt to live up to them. We can all play a part in this by paying attention to our own biases, and understanding what science should look like when it has been done well.
As for this study, we really can’t tell what the answer is. More research seems to be in order.