IDEAS home Printed from https://ideas.repec.org/a/wly/riskan/v40y2020i2p352-369.html
   My bibliography  Save this article

Recognizing Structural Nonidentifiability: When Experiments Do Not Provide Information About Important Parameters and Misleading Models Can Still Have Great Fit

Author

Listed:
  • Philip J. Schmidt
  • Monica B. Emelko
  • Mary E. Thompson

Abstract

In the quest to model various phenomena, the foundational importance of parameter identifiability to sound statistical modeling may be less well appreciated than goodness of fit. Identifiability concerns the quality of objective information in data to facilitate estimation of a parameter, while nonidentifiability means there are parameters in a model about which the data provide little or no information. In purely empirical models where parsimonious good fit is the chief concern, nonidentifiability (or parameter redundancy) implies overparameterization of the model. In contrast, nonidentifiability implies underinformativeness of available data in mechanistically derived models where parameters are interpreted as having strong practical meaning. This study explores illustrative examples of structural nonidentifiability and its implications using mechanistically derived models (for repeated presence/absence analyses and dose–response of Escherichia coli O157:H7 and norovirus) drawn from quantitative microbial risk assessment. Following algebraic proof of nonidentifiability in these examples, profile likelihood analysis and Bayesian Markov Chain Monte Carlo with uniform priors are illustrated as tools to help detect model parameters that are not strongly identifiable. It is shown that identifiability should be considered during experimental design and ethics approval to ensure generated data can yield strong objective information about all mechanistic parameters of interest. When Bayesian methods are applied to a nonidentifiable model, the subjective prior effectively fabricates information about any parameters about which the data carry no objective information. Finally, structural nonidentifiability can lead to spurious models that fit data well but can yield severely flawed inferences and predictions when they are interpreted or used inappropriately.

Suggested Citation

  • Philip J. Schmidt & Monica B. Emelko & Mary E. Thompson, 2020. "Recognizing Structural Nonidentifiability: When Experiments Do Not Provide Information About Important Parameters and Misleading Models Can Still Have Great Fit," Risk Analysis, John Wiley & Sons, vol. 40(2), pages 352-369, February.
  • Handle: RePEc:wly:riskan:v:40:y:2020:i:2:p:352-369
    DOI: 10.1111/risa.13386
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/risa.13386
    Download Restriction: no

    File URL: https://libkey.io/10.1111/risa.13386?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Philip J. Schmidt, 2015. "Norovirus Dose–Response: Are Currently Available Data Informative Enough to Determine How Susceptible Humans Are to Infection from a Single Virus?," Risk Analysis, John Wiley & Sons, vol. 35(7), pages 1364-1383, July.
    2. Tim Maiwald & Helge Hass & Bernhard Steiert & Joep Vanlier & Raphael Engesser & Andreas Raue & Friederike Kipkeew & Hans H Bock & Daniel Kaschek & Clemens Kreutz & Jens Timmer, 2016. "Driving the Model to Its Limit: Profile Likelihood Based Model Reduction," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-18, September.
    3. Philip J. Schmidt & Katarina D. M. Pintar & Aamir M. Fazil & Edward Topp, 2013. "Harnessing the Theoretical Foundations of the Exponential and Beta‐Poisson Dose‐Response Models to Quantify Parameter Uncertainty Using Markov Chain Monte Carlo," Risk Analysis, John Wiley & Sons, vol. 33(9), pages 1677-1693, September.
    4. Mark P Little & Wolfgang F Heidenreich & Guangquan Li, 2010. "Parameter Identifiability and Redundancy: Theoretical Considerations," PLOS ONE, Public Library of Science, vol. 5(1), pages 1-6, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael J. Messner & Philip Berger, 2016. "Cryptosporidium Infection Risk: Results of New Dose‐Response Modeling," Risk Analysis, John Wiley & Sons, vol. 36(10), pages 1969-1982, October.
    2. Steven Duret & Régis Pouillot & Wendy Fanaselle & Efstathia Papafragkou & Girvin Liggans & Laurie Williams & Jane M. Van Doren, 2017. "Quantitative Risk Assessment of Norovirus Transmission in Food Establishments: Evaluating the Impact of Intervention Strategies and Food Employee Behavior on the Risk Associated with Norovirus in Food," Risk Analysis, John Wiley & Sons, vol. 37(11), pages 2080-2106, November.
    3. Shoya Iwanami & Kosaku Kitagawa & Hirofumi Ohashi & Yusuke Asai & Kaho Shionoya & Wakana Saso & Kazane Nishioka & Hisashi Inaba & Shinji Nakaoka & Takaji Wakita & Odo Diekmann & Shingo Iwami & Koichi , 2020. "Should a viral genome stay in the host cell or leave? A quantitative dynamics study of how hepatitis C virus deals with this dilemma," PLOS Biology, Public Library of Science, vol. 18(7), pages 1-17, July.
    4. Barraquand, Frédéric & Gimenez, Olivier, 2021. "Fitting stochastic predator–prey models using both population density and kill rate data," Theoretical Population Biology, Elsevier, vol. 138(C), pages 1-27.
    5. Michael J. Messner & Philip Berger & Sharon P. Nappier, 2014. "Fractional Poisson—A Simple Dose‐Response Model for Human Norovirus," Risk Analysis, John Wiley & Sons, vol. 34(10), pages 1820-1829, October.
    6. Tucker Burch, 2019. "Validation of Quantitative Microbial Risk Assessment Using Epidemiological Data from Outbreaks of Waterborne Gastrointestinal Disease," Risk Analysis, John Wiley & Sons, vol. 39(3), pages 599-615, March.
    7. Mario Castro & Rob J de Boer, 2020. "Testing structural identifiability by a simple scaling method," PLOS Computational Biology, Public Library of Science, vol. 16(11), pages 1-15, November.
    8. Nicole Van Abel & Mary E. Schoen & John C. Kissel & J. Scott Meschke, 2017. "Comparison of Risk Predicted by Multiple Norovirus Dose–Response Models and Implications for Quantitative Microbial Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 37(2), pages 245-264, February.
    9. Gang Xie, 2017. "Authors’ Response to Comment on “Guidelines for Use of the Approximate Beta‐Poisson Dose–Response Model” – Previously Published Guidelines Continue to Be Ignored," Risk Analysis, John Wiley & Sons, vol. 37(2), pages 201-203, February.
    10. Sanjana Gupta & Robin E C Lee & James R Faeder, 2020. "Parallel Tempering with Lasso for model reduction in systems biology," PLOS Computational Biology, Public Library of Science, vol. 16(3), pages 1-22, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:riskan:v:40:y:2020:i:2:p:352-369. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1539-6924 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.