IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/0030189.html
   My bibliography  Save this article

Universally Sloppy Parameter Sensitivities in Systems Biology Models

Author

Listed:
  • Ryan N Gutenkunst
  • Joshua J Waterfall
  • Fergal P Casey
  • Kevin S Brown
  • Christopher R Myers
  • James P Sethna

Abstract

Quantitative computational models play an increasingly important role in modern biology. Such models typically involve many free parameters, and assigning their values is often a substantial obstacle to model development. Directly measuring in vivo biochemical parameters is difficult, and collectively fitting them to other experimental data often yields large parameter uncertainties. Nevertheless, in earlier work we showed in a growth-factor-signaling model that collective fitting could yield well-constrained predictions, even when it left individual parameters very poorly constrained. We also showed that the model had a “sloppy” spectrum of parameter sensitivities, with eigenvalues roughly evenly distributed over many decades. Here we use a collection of models from the literature to test whether such sloppy spectra are common in systems biology. Strikingly, we find that every model we examine has a sloppy spectrum of sensitivities. We also test several consequences of this sloppiness for building predictive models. In particular, sloppiness suggests that collective fits to even large amounts of ideal time-series data will often leave many parameters poorly constrained. Tests over our model collection are consistent with this suggestion. This difficulty with collective fits may seem to argue for direct parameter measurements, but sloppiness also implies that such measurements must be formidably precise and complete to usefully constrain many model predictions. We confirm this implication in our growth-factor-signaling model. Our results suggest that sloppy sensitivity spectra are universal in systems biology models. The prevalence of sloppiness highlights the power of collective fits and suggests that modelers should focus on predictions rather than on parameters.: Dynamic systems biology models typically involve many kinetic parameters, the quantitative determination of which has been a serious obstacle to using these models. Previous work showed for a particular model that useful predictions could be extracted from a fit long before the experimental data constrained the parameters, even to within orders of magnitude. This was attributed to a “sloppy” pattern in the model's parameter sensitivities; the sensitivity eigenvalues were roughly evenly spaced over many decades. Consequently, the model behavior depended effectively on only a few “stiff” parameter combinations. Here we study the converse problem, showing that direct parameter measurements are very inefficient at constraining the model's behavior. To yield effective predictions, such measurements must be very precise and complete; even a single imprecise parameter often destroys predictivity. We also show here that the characteristic sloppy eigenvalue pattern is reproduced in 16 other diverse models from the systems biology literature. The apparent universality of sloppiness suggests that predictions from most models will be very fragile to single uncertain parameters and that collective parameters fits can often yield tight predictions with loose parameters. Together these results argue that focusing on parameter values may be a very inefficient route to useful models.

Suggested Citation

  • Ryan N Gutenkunst & Joshua J Waterfall & Fergal P Casey & Kevin S Brown & Christopher R Myers & James P Sethna, 2007. "Universally Sloppy Parameter Sensitivities in Systems Biology Models," PLOS Computational Biology, Public Library of Science, vol. 3(10), pages 1-8, October.
  • Handle: RePEc:plo:pcbi00:0030189
    DOI: 10.1371/journal.pcbi.0030189
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.0030189
    Download Restriction: no

    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.0030189&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pcbi.0030189?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Hiroaki Kitano, 2002. "Computational systems biology," Nature, Nature, vol. 420(6912), pages 206-210, November.
    2. George von Dassow & Eli Meir & Edwin M. Munro & Garrett M. Odell, 2000. "The segment polarity network is a robust developmental module," Nature, Nature, vol. 406(6792), pages 188-192, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Samuel Bandara & Johannes P Schlöder & Roland Eils & Hans Georg Bock & Tobias Meyer, 2009. "Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model," PLOS Computational Biology, Public Library of Science, vol. 5(11), pages 1-12, November.
    2. Adel Dayarian & Madalena Chaves & Eduardo D Sontag & Anirvan M Sengupta, 2009. "Shape, Size, and Robustness: Feasible Regions in the Parameter Space of Biochemical Networks," PLOS Computational Biology, Public Library of Science, vol. 5(1), pages 1-12, January.
    3. Mark Read & Paul S. Andrews & Jon Timmis & Vipin Kumar, 2011. "Techniques for grounding agent-based simulations in the real domain: a case study in experimental autoimmune encephalomyelitis," Mathematical and Computer Modelling of Dynamical Systems, Taylor & Francis Journals, vol. 18(1), pages 67-86, May.
    4. Chandra, Yanto & Wilkinson, Ian F., 2017. "Firm internationalization from a network-centric complex-systems perspective," Journal of World Business, Elsevier, vol. 52(5), pages 691-701.
    5. Jacobo Ayensa-Jiménez & Marina Pérez-Aliacar & Teodora Randelovic & José Antonio Sanz-Herrera & Mohamed H. Doweidar & Manuel Doblaré, 2020. "Analysis of the Parametric Correlation in Mathematical Modeling of In Vitro Glioblastoma Evolution Using Copulas," Mathematics, MDPI, vol. 9(1), pages 1-22, December.
    6. Qing-Ju Jiao & Yan-Kai Zhang & Lu-Ning Li & Hong-Bin Shen, 2011. "BinTree Seeking: A Novel Approach to Mine Both Bi-Sparse and Cohesive Modules in Protein Interaction Networks," PLOS ONE, Public Library of Science, vol. 6(11), pages 1-12, November.
    7. Tom C Freeman & Leon Goldovsky & Markus Brosch & Stijn van Dongen & Pierre Mazière & Russell J Grocock & Shiri Freilich & Janet Thornton & Anton J Enright, 2007. "Construction, Visualisation, and Clustering of Transcription Networks from Microarray Expression Data," PLOS Computational Biology, Public Library of Science, vol. 3(10), pages 1-11, October.
    8. Yuksel Bayraktar & Esme Isik & Ibrahim Isik & Ayfer Ozyilmaz & Metin Toprak & Fatma Kahraman Guloglu & Serdar Aydin, 2022. "Analyzing of Alzheimer’s Disease Based on Biomedical and Socio-Economic Approach Using Molecular Communication, Artificial Neural Network, and Random Forest Models," Sustainability, MDPI, vol. 14(13), pages 1-15, June.
    9. Mika Gustafsson & Michael Hörnquist, 2010. "Gene Expression Prediction by Soft Integration and the Elastic Net—Best Performance of the DREAM3 Gene Expression Challenge," PLOS ONE, Public Library of Science, vol. 5(2), pages 1-8, February.
    10. Diego Fernández Slezak & Cecilia Suárez & Guillermo A Cecchi & Guillermo Marshall & Gustavo Stolovitzky, 2010. "When the Optimal Is Not the Best: Parameter Estimation in Complex Biological Models," PLOS ONE, Public Library of Science, vol. 5(10), pages 1-10, October.
    11. Matthew A Hibbs & Chad L Myers & Curtis Huttenhower & David C Hess & Kai Li & Amy A Caudy & Olga G Troyanskaya, 2009. "Directing Experimental Biology: A Case Study in Mitochondrial Biogenesis," PLOS Computational Biology, Public Library of Science, vol. 5(3), pages 1-12, March.
    12. Alan A Cohen & Emmanuel Milot & Qing Li & Patrick Bergeron & Roxane Poirier & Francis Dusseault-Bélanger & Tamàs Fülöp & Maxime Leroux & Véronique Legault & E Jeffrey Metter & Linda P Fried & Luigi Fe, 2015. "Detection of a Novel, Integrative Aging Process Suggests Complex Physiological Integration," PLOS ONE, Public Library of Science, vol. 10(3), pages 1-26, March.
    13. Cummings, F.W, 2004. "A model of morphogenesis," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 339(3), pages 531-547.
    14. Niki Vermeulen, 2018. "The choreography of a new research field: Aggregation, circulation and oscillation," Environment and Planning A, , vol. 50(8), pages 1764-1784, November.
    15. Armaghan W Naik & Joshua D Kangas & Christopher J Langmead & Robert F Murphy, 2013. "Efficient Modeling and Active Learning Discovery of Biological Responses," PLOS ONE, Public Library of Science, vol. 8(12), pages 1-1, December.
    16. Joep P J Schmitz & Jeroen A L Jeneson & Joep W M van Oorschot & Jeanine J Prompers & Klaas Nicolay & Peter A J Hilbers & Natal A W van Riel, 2012. "Prediction of Muscle Energy States at Low Metabolic Rates Requires Feedback Control of Mitochondrial Respiratory Chain Activity by Inorganic Phosphate," PLOS ONE, Public Library of Science, vol. 7(3), pages 1-14, March.
    17. Marco S Nobile & Paolo Cazzaniga & Daniela Besozzi & Dario Pescini & Giancarlo Mauri, 2014. "cuTauLeaping: A GPU-Powered Tau-Leaping Stochastic Simulator for Massive Parallel Analyses of Biological Systems," PLOS ONE, Public Library of Science, vol. 9(3), pages 1-20, March.
    18. Fabian Fröhlich & Barbara Kaltenbacher & Fabian J Theis & Jan Hasenauer, 2017. "Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks," PLOS Computational Biology, Public Library of Science, vol. 13(1), pages 1-18, January.
    19. Markus J. Buehler & Theodor Ackbarow, 2008. "Nanomechanical strength mechanisms of hierarchical biological materials and tissues," Computer Methods in Biomechanics and Biomedical Engineering, Taylor & Francis Journals, vol. 11(6), pages 595-607.
    20. Luca Marchetti & Rosario Lombardo & Corrado Priami, 2017. "HSimulator: Hybrid Stochastic/Deterministic Simulation of Biochemical Reaction Networks," Complexity, Hindawi, vol. 2017, pages 1-12, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:0030189. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.