Summary answer:
Many toxins, especially those that affect the hormonal system (these include Roundup), have nonlinear dose-response patterns. Scientists have published papers about nonlinear dose-response patterns since the 1990s, but industry and some risk assessment bodies cling to the outdated toxicological model of linear dose-response.
Detailed answer:
Risk assessment of chemicals and toxins is based on an old, outdated assumption of toxicology: that increasing the dose always increases toxic effects in a regular and predictable manner. This is known as linear dose-response.
Since the 1990s, scientists have published numerous peer-reviewed papers showing that linear dose-response does not hold true with all toxins. In particular, substances that affect the hormonal system (called endocrine disruptors) often have effects that follow a nonlinear pattern. For example, a low dose can have a toxic effect, a higher dose can have no effect, and then an even higher dose can again produce a toxic effect (U-shape response).1
With regard to Séralini’s study, a nonlinear dose-response to Roundup was to be expected, as previous experiments have shown that glyphosate herbicides are endocrine disruptors in rats2 3 and that Roundup is an endocrine disruptor in human cells.4
Séralini’s team observed a particular type of nonlinear dose-response known as a threshold effect, where a low dose appeared to saturate the system and a higher dose had no additional effect. A similar threshold effect was found with NK603 maize. Séralini did not identify a dose so low that it had no effect; further work would be needed to establish such a “no-effect” dose.
References:
1. Vandenberg LN, Colborn T, Hayes TB, et al. Hormones and endocrine-disrupting chemicals: Low-dose effects and nonmonotonic dose responses. Endocrine reviews. 14 March 2012; 33(3): 378-455.
2. Dallegrave E, Mantese FD, Oliveira RT, Andrade AJ, Dalsenter PR, Langeloh A. Pre- and postnatal toxicity of the commercial glyphosate formulation in Wistar rats. Arch Toxicol. Sep 2007; 81: 665–673.
3. Romano RM, Romano MA, Bernardi MM, Furtado PV, Oliveira CA. Prepubertal exposure to commercial formulation of the herbicide Glyphosate alters testosterone levels and testicular morphology. Archives of Toxicology. 2010; 84(4): 309-317.
4. Richard S, Moslemi S, Sipahutar H, Benachour N, Seralini GE. Differential effects of glyphosate and roundup on human placental cells and aromatase. Environ Health Perspect. Jun 2005; 113(6): 716-720.
Sources of criticism:
Monsanto
http://www.monsanto.com/products/Documents/ProductSafety/seralini-sept-2012-monsanto-comments.pdf
Emily Willingham
http://www.emilywillinghamphd.com/2012/09/was-it-gmos-or-bpa-that-did-in-those.html
GMO Compass
http://www.gmo-compass.org/eng/news/555.docu.html