Reproducibility of preclinical data: one man's poison is another man's meat

Anton Bespalov, Christoph H Emmerich, Björn Gerlach, Martin C Michel


Limited reproducibility of preclinical data is increasingly discussed in the literature. Failure of drug devel-opment programs due to lack of clinical efficacy is also of growing concern. The two phenomena may share an important root cause — a lack of robustness in preclinical research. Such a lack of robustness can be a relevant cause of fail-ure in translating preclinical findings into clinical efficacy and hence attrition, and exaggerated cost in drug develop-ment. Apart from the study design and data analysis factors (e.g., insufficient sample sizes, failure to implement blind-ing, and randomization), heterogeneity among experimental models (e.g., animal strains) and the conditions of the study used between different laboratories is a major contributor to the lacking of robustness of research findings. The flipside of this coin is that the understanding of the causes of heterogeneity across experimental models may lead to the identification of relevant factors for defining the responder populations. Thus, this heterogeneity within preclinical find-ings could be an asset, rather than an obstacle, for precision medicine. To enable this paradigm shift, several steps need to be taken to identify conditions under which drugs do not work. An improved granularity in the reporting of preclini-cal studies is central among them (i.e., details about the study design, experimental conditions, quality of tools and rea-gents, validation of assay conditions, etc.). These actions need to be discussed jointly by the research communities in-terested in preclinical data robustness and precision medicine. Thus, we propose that a lack of robustness due to the heterogeneity across models and conditions of the study is not necessarily a liability for biomedical research but can be transformed into an asset of precision medicine.


animal models; data reproducibility; heterogeneity; precision medicine; translational research

Full Text:



Scannell J W, Blanckley A, Boldon H, et al., 2012, Diagnosing the decline in pharmaceutical R&D efficiency. Nature Reviews Drug Discovery, vol.11: 191–200.

Herper M, 2012, The truly staggering cost of inventing new drugs, viewed April 16, 2016,

Grainger D, 2015, Why too many clinical trials fail -- And a simple solution that could increase returns on pharma R&D, viewed April 16, 2016,

Kola I and Landis J, 2004, Can the pharmaceutical industry reduce attrition rates? Nature Reviews Drug Discovery, vol.3: 711–716.

Garner J P, 2014, The significance of meaning: why do over 90% of behavioral neuroscience results fail to trans-late to humans, and what can we do to fix it? ILAR Jour-nal, vol.55(3): 438–456.

Millan M J, Goodwin G M, Meyer-Lindenberg A, et al., 2015, Learning from the past and looking to the future: Emerging perspectives for improving the treatment of psychiatric disorders. European Neuropsychopharmacology, vol.25(5): 599–656.

Köster U, Nolte I and Michel M C, 2016, Preclinical re-search strategies for newly approved drugs as reflected in early publication patterns. Naunyn-Schmiedeberg's Arc-hives of Pharmacology, vol.389(2): 187–199.

Insel T R, Voon V, Nye J S, et al., 2013, Innovative solutions to novel drug development in mental health. Neuroscience and Biobehavioral Reviews, vol.37(10 Part 1): 2438–2444.

Tymianski M, 2015, Neuroprotective therapies: preclinical reproducibility is only part of the problem. Science Translational Medicine, vol.7(299): 299fs32.

Williams M, 2011, Productivity shortfalls in drug discovery: contributions from the preclinical sciences? The Journal of Pharmacology and Experimental Therapeutics, vol.336(1): 3–8.

Prinz F, Schlange T and Asadullah K, 2011, Believe it or not: how much can we rely on published data on potential drug targets? Nature Reviews Drug Discovery, vol.10: 712–713.

Begley C G and Ellis L M, 2012, Drug development: raise standards for preclinical cancer research. Nature, vol.483: 533–533.

Collins F S and Tabak L A, 2014, Policy: NIH plans to enhance reproducibility. Nature, vol.505(7485): 612–613.

Freedman L P, Cockburn I M and Simcoe T S, 2015, The economics of reproducibility in preclinical research. PLoS Biology, vol.13(6): e1002165.

Colquhoun D, 2014, An investigation of the false discovery rate and the misinterpretation of p-values. Royal Society Open Science, vol.1: 140216.

Motulsky H J, 2014, Common misconceptions about data analysis and statistics. Naunyn-Schmiedeberg's Archives of Pharmacology, vol.387(11): 1017–1023.

McGrath J C, Drummond G B, McLachlan E M, et al., 2010, Guidelines for reporting experiments involving animals: the ARRIVE guidelines. British Journal of Pharmacology, vol.160(7): 1573–1576.

Curtis M J, Bond R A, Spina D, et al., 2015, Experimental design and analysis and their reporting: new guidance for publication in BJP. British Journal of Pharmacology, vol.172(14): 3461–3471.

Jarvis M F and Williams M, 2016, Irreproducibility in preclinical biomedical research: perceptions, uncertain-ties, and knowledge gaps. Trends in Pharmacological Sciences, vol.37(4): 290–302.

Alberts B, Cicerone R J, Fienberg S E, et al., 2015, Self-correction in science at work. Science, vol.348(6242): 1420–1422.

Clemens M A, 2015, The meaning of failed replications: a review and proposal. Journal of Economic Surveys, 1–17.

Kenett R S and Shmueli G, 2015, Clarifying the terminology that describes scientific reproducibility. Nature Methods, vol.12: 699.

Llovera G, Hofmann K, Roth S, et al., 2015, Results of a preclinical randomized controlled multicenter trial (pRCT): anti-CD49d treatment for acute brain ischemia. Science Translational Medicine, vol.7(299): 299ra121.

Dale P R, Cernecka H, Schmidt M, et al., 2014, The pharmacological rationale for combining muscarinic receptor antagonists and β-adrenoceptor agonists in the treatment of airway and bladder disease. Current Opinion in Pharmacology, vol.16: 31–42.

Maman K, Aballea S, Nazir J, et al., 2014, Comparative efficacy and safety of medical treatments for the management of overactive bladder: a systematic literature review and mixed treatment comparison. European Urology, vol.65(4): 755–765.

Michel M C, Brunner H R, Foster C, et al., 2016, Angiotensin II type 1 receptor antagonists in animal models of vascular, cardiac, metabolic and renal disease. Pharmacology and Therapeutics (in press).

Lindpaintner K, Kreutz R and Ganten D, 1992, Genetic variation in hypertensive and ‘control’ strains: what are we controlling for anyway? Hypertension, vol.19: 428–430.

Kumar G, Talpos J and Steckler T, 2015, Strain-dependent effects on acquisition and reversal of visual and spatial

tasks in a rat touchscreen battery of cognition. Physiology and Behavior, vol.144: 26–36.

Bottger A, den Bieman M, Lankhorst Æ, et al., 1996, Strain-specific response to hypercholesterolaemic diets in the rat. Laboratory Animals, vol.30(2): 149–157.

Bouxsein M L, Myers K S, Shultz K L, et al., 2005, Ovariectomy-induced bone loss varies among inbred strains of mice. Journal of Bone and Mineral Research, vol.20(7): 1085–1092.

Iwaniec U T, Yuan D, Power R A, et al., 2006, Strain- dependent variations in the response of cancellous bone to ovariectomy in mice. Journal of Bone and Mineral Research, vol.21(7): 1068–1074.

Cui Q, Hodgetts S I, Hu Y, et al., 2007, Strain-specific differences in the effects of cyclosporin A and FK506 on the survival and regeneration of axotomized retinal ganglion cells in adult rats. Neuroscience, vol.146(3): 986– 999.

Coruzzi G, Pozzoli C, Adami M, et al., 2012, Strain-dependent effects of the histamine H₄ receptor antagonist JNJ7777120 in a murine model of acute skin inflammation. Experimental Dermatology, vol.21(1): 32–37.

Maronpota R R, Thoolen R J M M and Hansen B, 2015, Two-year carcinogenicity study of acrylamide in Wistar Han rats with in utero exposure. Experimental and Toxicologic Pathology, vol.67(2): 189–195.

Kawedia J D, Janke L, Funk A J, et al., 2012, Sub-strain-specific differences in survival and osteonecrosis incidence in a mouse model. Comparative Medicine, vol.62(6): 466–471.

Palm S, Roman E and Nylander I, 2012, Differences in basal and ethanol-induced levels of opioid peptides in Wistar rats from five different suppliers. Peptides, vol.36(1): 1–8.

International Committee on Standardized Genetic Nomenclature for Mice and Rat Genome and Nomenclature Committee, Guidelines for nomenclature of mouse and rat strains, revised January 2016, viewed April 16, 2016,

Lindemann L L, Gatti S, Ballard T, et al., 2006, Neuroscience meeting planner, October 14–18, 2006: Pharmacological and biochemical characterization of a population of Wistar rats with reduced expression of metabotropic glutamate receptor 2 protein (mGluR2). Society for Neuroscience, Atlanta, GA.

Ceolin L, Kantamneni S, Barker G R I, et al., 2011, Study of novel selective mGlu2 agonist in the temporoammonic input to CA1 neurons reveals reduced mGlu2 receptor expression in a Wistar substrain with an anxiety-like phenotype. The Journal of Neuroscience, vol.31(18): 6721–6731.

Wood G K, Marcotte E R, Quirion R, et al., 2001, Strain differences in the behavioural outcome of neonatal ventral hippocampal lesions are determined by the postnatal environment and not genetic factors. European Journal of Neuroscience, vol.14(6): 1030–1034.

Klengel T and Binder E B, 2013, Gene × environment interactions in the prediction of response to antidepressant treatment. International Journal of Neuropsycho-pharmacology, vol.16(3): 701–711.

Alkermes announces topline results of FORWARD-3 and FORWARD-4, two phase 3 studies of ALKS 5461 in major depressive disorder, n.d., viewed April 16, 2016,

Carr G V, Bangasser D A, Bethea T, et al., 2010, Antidepressant-like effects of κ-opioid receptor antagonists in Wistar Kyoto rats. Neuropsychopharmacology, vol.35: 752–763.

Kinon B J, Millen B A, Zhang L, et al., 2015, Exploratory analysis for a targeted patient population responsive to the metabotropic glutamate 2/3 receptor agonist pomaglumetad methionil in schizophrenia. Biological Psychiatry, vol.78(11): 754–762.

Zhou Z, Karlsson C, Liang T, et al., 2013, Loss of metabotropic glutamate receptor 2 escalates alcohol consumption. Proceedings of the National Academy of Sciences of the United States of America, vol.110(42): 16963–16968.

Meinhardt M W, Hansson A C, Perreau-Lenz S, et al., 2013, Rescue of infralimbic mGluR2 deficit restores control over drug-seeking behavior in alcohol dependence. The Journal of Neuroscience, vol.33(7): 2794–2806.

Michel M C and Korstanje C, 2016, β3-Adrenoceptor agonists for overactive bladder syndrome: role of translational pharmacology in a repositioning drug development project. Pharmacology and Therapeutics, vol.159: 66–82.

Larsen T M, Toubro S, van Baak M A, et al., 2002, Effect of a 28-d treatment with L-796568, a novel β3-adrenergic receptor agonist, on energy expenditure and body composition in obese men. The American Journal of Clinical Nutrition, vol.76(4): 780–788.

Preclinical Reproducibility and Robustness, n.d., F1000 Research, viewed June 14, 2016,

bioRxiv, n.d., Cold Spring Harbor Laboratory, viewed June 16, 2016,

PubMed Commons, n.d., viewed June 17, 2016,

Preclinical Data Forum Network, n.d., ECNP, viewed June 17, 2016,

Multi-PART (Multicentre Preclinical Animal Research Team), n.d., viewed June 8, 2016,

Interventions Testing Program (ITP), n.d., National Institute on Aging, viewed June 11, 2016,

Richter S H, Garner J P, Auer C, et al., 2010, Systematic variation improves reproducibility of animal experiments. Nature Methods, vol.7: 167–168.

Lauschke V M and Ingelman-Sundberg M, 2016, Precision medicine and rare genetic variants. Trends in Pharmacological Science, vol.37(2): 85–86.

Berthonneche C, Peter B, Schüpfer F, et al., 2009, Cardiovascular response to beta-adrenergic blockade or activation in 23 inbred mouse strains. PLoS One, vol.4(8): e6610.

Haibe-Kains B, El-Hachem N, Birkbak N J, et al., 2013, Inconsistency in large pharmacogenomic studies. Nature, vol.504: 389–393.

Landeck L, Lessl M, Reischl J, et al., 2016, Collaboration for success: the value of strategic collaborations for precision medicine and biomarker discovery. Advances in Precision Medicine, vol.1(1): 25–33.

Kannt A and Wieland T, 2016, Managing risks in drug discovery: reproducibility of published findings. Naunyn-Schmiedeberg's Archives of Pharmacology, vol.389(4): 353–360.

Gilman A G, 1995, G proteins and regulation of adenylyl cyclase. Bioscience Reports, vol.15(2): 65–97.

(324 Abstract Views, 162 PDF Downloads)


  • There are currently no refbacks.

Copyright (c) 2016 Anton Bespalov, Christoph H Emmerich, Björn Gerlach, Martin C Michel

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.




Advances in Precision Medicine is a peer-reviewed, open-access journal. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.