Summary answer:
According to standard carcinogenicity protocols, Séralini did not have enough animals to do a statistical analysis on the increased rates of tumours and mortality. He did have enough for a statistical analysis on the biochemical measurements, which confirmed that disturbances in hormonal function in treatment groups resulted from ingestion of GM maize NK603 and Roundup, with a 95% confidence level (the likelihood that a repeat of the experiment with a different group of animals would provide the same result). These hormonal disturbances are known to cause mammary tumours.
Detailed answer:
Critics of Séralini’s study have focused heavily on finding fault with his statistical approach. But none of the critics have explained why his approach was scientifically unacceptable or why the methods they prefer are superior. None have published their criticisms in a peer-reviewed journal. And none of these critics are as eminent in the field as Paul Deheuvels, the statistician of the French Academy of Sciences, who has strongly defended the statistical strength of Séralini’s study. Deheuvels wrote an article for the French press whose title translates as “Séralini study on GMOs: Why the methodology is statistically sound”.1
Statistical approach chosen
Part of the criticism may be due to a lack of familiarity with the statistical method Séralini used, OPLS-DA. This method is newer and more sophisticated than the methods often used, for example, in industry tests on GMOs.2 Seralini’s team chose OPLS-DA because it is suited to analyzing complex data sets on many different effects.3
Peter Saunders, emeritus professor of mathematics at King’s College London and an expert in mathematical biology, commented on the critics’ arguments, “There is often more than one way of analyzing data and they can give different results. In order to decide which is better, you need a good understanding of the assumptions that underlie each of them, and even then there can be room for argument.
“What you often find, however, and I think that’s what happened here, is that people will insist that if the simple method they were taught as undergraduates doesn’t give a result and a more sophisticated one does, then they won’t accept the latter.”
How does Seralini’s statistical approach stand up in comparison to that used in industry tests on GMOs? In response to Séralini’s critics, Jack Heinemann, professor of genetics and molecular biology in the School of Biological Sciences at the University of Canterbury, Christchurch, New Zealand, performed a quick review of the statistical approaches used in industry studies on GMOs published in the same journal that published Séralini’s paper. His verdict was:
“The statistics used in these other [industry] studies passed anonymous peer-review. Aside from that, there is no other peer-reviewed evidence that these statistical approaches are either uniquely appropriate or validated for their use in this kind of study. On those fronts, I find Séralini et al.’s statistical analysis equally valid. I would encourage both the scientific community and the regulatory community to engage in an exercise of validation of statistical analyses if this remains an issue of contention.”4
The critics’ arguments
The details of the critics’ arguments can be summarized as follows: that Séralini’s study has too little statistical power to draw any conclusions; that there are no Kaplan Meier’s curves (a measure of survival); that the variations seen could be due to chance; that only raw data are presented in figures 1, 2, 3 and table 2, with no statistical analysis; and that there are no means and standard deviations (now much variation or dispersal exists from the average) in Table 3.
Séralini’s response to critics
Séralini’s team has addressed the critics’ points in a response3 published in the same journal that published his original research.5 The authors state, “Statistics do not tell the truth, but may help the understanding.” The important factor is the biological interpretation of the many different effects and how they developed over time.
With regard to mortality and tumour rates, due to the relatively small numbers of animals, there was not enough statistical power for a Kaplan Meier’s analysis to conclude on whether there was a treatment-related effect. The authors explain, “Taking into account these limits, we decided to be factual and thus to describe the chronology and incidence of tumours and deaths.”3 So only the raw data are presented.
There was enough statistical power for the analytical method used for the biochemical measurements, OPLS-DA. Using this method, the authors analyzed around 50 parameters measured 11 times for 200 rats.3 Some of this data is presented in Figure 5 of the original study for the female treatment group fed 33% NK603 GM maize (without Roundup application) compared to controls.5 More data will be published in future papers.3
The statistical analysis confirmed that disturbances in hormonal function in treatment groups resulted from ingestion of GM maize NK603 and Roundup, with a 95% confidence level (the likelihood that a repeat of the experiment with a different group of animals would provide the same result). These hormonal disturbances are known to cause mammary tumours of the type that Séralini found at increased rates in treatment groups.
The authors summarized their statistical approach to the biochemical measurements and how they interpreted them to reach their conclusions:
“These discriminant biochemical markers were for instance sexual hormones (at 95% for females at month 15) when the differences in hormone-dependent tumour incidences with the control group began. The disabled pituitaries were the second most reached [affected] organ as certified independently by the pathologists in a blinded manner in treated female groups. This is known to elicit mammary tumours in rats and pituitary is a target of endocrine disrupting chemicals… The pathologists explained most of the mortality in females by the tumours, which leaded to euthanasia independently of the grade of cancer. This is why we did not detail the grades of tumours in our research, and the cancerous nature of major ones are described in our study… These observations together with microscopy reinforced our conclusions.”3
In reply to the criticism that there are no means and standard deviations in Table 3 (showing percentage variation of parameters indicating kidney failures of female animals and sex hormone levels), Séralini’s team explained that OPLS-DA is not a method of comparing mean differences.3
Low statistical power of industry tests
Critics who argue that Séralini’s experiment lacks the statistical power to prove that GM maize NK603 and Roundup caused the toxic effects seen must apply the same standards to the industry tests that underpin regulatory authorizations. If they are correct that Séralini’s study lacks the statistical power to show that these substances caused toxic effects, then the industry tests certainly lack the statistical power to show that they are safe.
These industry tests analyzed the blood and urine chemistry of the same number of animals that Séralini used. But the tests were much shorter – 90 days2 6 7 – meaning that they had no power to detect long-term effects. Séralini’s team commented that the statistical power of the Monsanto test on NK603 maize2 was “extremely low” to conclude safety.”3
Those who dismiss Séralini’s experiment on grounds of statistical power must apply the same standards to the industry studies concluding safety for GMOs and show that these conclusions are reliable.
What conclusions can be drawn from Séralini’s statistical analysis?
The statistical analysis confirmed that disturbances in hormonal function in treatment groups resulted from ingestion of GM maize NK603 and Roundup, with a 95% confidence level. Critics appear to demand that Séralini must prove at the 5% level, i.e. beyond reasonable doubt, that GM maize and/or Roundup are hazardous. Even if he has not done that, he has clearly done enough to raise doubts in the mind of any reasonable person.
The EU claims to accept the precautionary principle in cases “where preliminary objective scientific evidence indicates that there are reasonable grounds for concern”.8
In other words, if there is preliminary evidence, you cannot simply ignore it. At this stage, it is not for Séralini to prove beyond any doubt that there is a danger. In a reasonable world, another independent group would be funded to replicate and extend the scope of the experiments, following an accepted protocol for carcinogenesis and taking on board any of the critics’ points that have scientific validity. Some do, but there is a difference between seeing how an experiment could have been done better and dismissing it out of hand.
References:
1. Deheuvels P. Étude de Séralini sur les OGM: Pourquoi sa méthodologie est statistiquement bonne [Seralini study on GMOs: Why the methodology is statistically sound]. Le Nouvel Observateur. 9 October 2012. http://bit.ly/RtPivG
2. Hammond B, Dudek R, Lemen J, Nemeth M. Results of a 13 week safety assurance study with rats fed grain from glyphosate tolerant corn. Food Chem Toxicol. Jun 2004; 42(6): 1003-1014.
3. Séralini GE, Mesnage R, Defarge N, et al. Answers to critics: Why there is a long term toxicity due to NK603 Roundup-tolerant genetically modified maize and to a Roundup herbicide. Food and Chemical Toxicology. 9 November 2012.
4. Heinemann J. Letter to the editor. Food and Chemical Toxicology. 6 November 2012; 50(11): 4221-4231.
5. Séralini GE, Clair E, Mesnage R, et al. Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize. Food and Chemical Toxicology. November 2012; 50(11): 4221-4231.
6. Hammond B, Lemen J, Dudek R, et al. Results of a 90-day safety assurance study with rats fed grain from corn rootworm-protected corn. Food Chem Toxicol. Feb 2006; 44(2): 147-160.
7. Hammond BG, Dudek R, Lemen JK, Nemeth MA. Results of a 90-day safety assurance study with rats fed grain from corn borer-protected corn. Food Chem Toxicol. Jul 2006; 44(7): 1092-1099.
8. European Commission (DG SANCO). Commission adopts Communication on precautionary principle. 2 February 2000. http://ec.europa.eu/dgs/health_consumer/library/press/press38_en.print.html
Source of criticism:
European Food Safety Authority
http://www.efsa.europa.eu/en/efsajournal/pub/2910.htm
http://www.efsa.europa.eu/en/efsajournal/pub/2986.htm
BfR (German federal institute for risk assessment)
Monsanto
http://www.monsanto.com/products/Documents/ProductSafety/seralini-sept-2012-monsanto-comments.pdf
