IDEAS home Printed from https://ideas.repec.org/a/wly/riskan/v14y1994i4p441-447.html
   My bibliography  Save this article

An Improved Framework for Uncertainty Analysis: Accounting for Unsuspected Errors

Author

Listed:
  • Alexander I. Shlyakhter

Abstract

I use an analogy with the history of physical measurements, population and energy projections, and analyze the trends in several data sets to quantify the overconfidence of the experts in the reliability of their uncertainty estimates. Data sets include (i) time trends in the sequential measurements of the same physical quantity; (ii) national population projections; and (iii) projections for the U.S., energy sector. Probabilities of large deviations for the true values are parametrized by an exponential distribution with the slope determined by the data. Statistics of past errors can be used in probabilistic risk assessment to hedge against unsuspected uncertainties and to include the possibility of human error into the framework of uncertainty analysis. By means of a sample Monte Carlo simulation of cancer risk caused by ingestion of benzene in soil, I demonstrate how the upper 95th percentiles of risk are changed when unsuspected uncertainties are included. I recommend to inflate the estimated uncertainties by default safety factors determined from the relevant historical data sets.

Suggested Citation

  • Alexander I. Shlyakhter, 1994. "An Improved Framework for Uncertainty Analysis: Accounting for Unsuspected Errors," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 441-447, August.
    • Handle: RePEc:wly:riskan:v:14:y:1994:i:4:p:441-447
    DOI: 10.1111/j.1539-6924.1994.tb00262.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1539-6924.1994.tb00262.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1539-6924.1994.tb00262.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Shlyakhter, Alexander I. & Kammen, Daniel M. & Broido, Claire L. & Wilson, Richard, 1994. "Quantifying the credibility of energy projections from trends in past data : The US energy sector," Energy Policy, Elsevier, vol. 22(2), pages 119-130, February.
    2. Kimberly M. Thompson & David E. Burmaster & Edmund A.C. Crouch3, 1992. "Monte Carlo Techniques for Quantitative Uncertainty Analysis in Public Health Risk Assessments," Risk Analysis, John Wiley & Sons, vol. 12(1), pages 53-63, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kash Barker & Kaycee J. Wilson, 2012. "Decision Trees with Single and Multiple Interval-Valued Objectives," Decision Analysis, INFORMS, vol. 9(4), pages 348-358, December.
    2. Per Sander & Bo Bergbäck & Tomas Öberg, 2006. "Uncertain Numbers and Uncertainty in the Selection of Input Distributions—Consequences for a Probabilistic Risk Assessment of Contaminated Land," Risk Analysis, John Wiley & Sons, vol. 26(5), pages 1363-1375, October.
    3. Barker, Kash & Haimes, Yacov Y., 2009. "Assessing uncertainty in extreme events: Applications to risk-based decision making in interdependent infrastructure sectors," Reliability Engineering and System Safety, Elsevier, vol. 94(4), pages 819-829.
    4. Lin, Shi-Woei & Bier, Vicki M., 2008. "A study of expert overconfidence," Reliability Engineering and System Safety, Elsevier, vol. 93(5), pages 711-721.
    5. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    6. S. N. Rai & D. Krewski, 1998. "Uncertainty and Variability Analysis in Multiplicative Risk Models," Risk Analysis, John Wiley & Sons, vol. 18(1), pages 37-45, February.
    7. Dale Hattis & David E. Burmaster, 1994. "Assessment of Variability and Uncertainty Distributions for Practical Risk Analyses," Risk Analysis, John Wiley & Sons, vol. 14(5), pages 713-730, October.
    8. Victor R. Vasquez & Wallace B. Whiting, 2005. "Accounting for Both Random Errors and Systematic Errors in Uncertainty Propagation Analysis of Computer Models Involving Experimental Measurements with Monte Carlo Methods," Risk Analysis, John Wiley & Sons, vol. 25(6), pages 1669-1681, December.
    9. Adam M. Finkel & George Gray, 2018. "Taking the reins: how regulatory decision-makers can stop being hijacked by uncertainty," Environment Systems and Decisions, Springer, vol. 38(2), pages 230-238, June.
    10. Ken Silver & Richard Clapp, 2006. "Environmental Surveillance at Los Alamos: An Independent Reassessment of Historical Data," Risk Analysis, John Wiley & Sons, vol. 26(4), pages 893-906, August.
    11. James K. Hammitt & Alexander I. Shlyakhter, 1999. "The Expected Value of Information and the Probability of Surprise," Risk Analysis, John Wiley & Sons, vol. 19(1), pages 135-152, February.
    12. Fritz A. Seiler & Joseph L. Alvarez, 1996. "On the Selection of Distributions for Stochastic Variables," Risk Analysis, John Wiley & Sons, vol. 16(1), pages 5-18, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. A. E. Ades & Karl Claxton & Mark Sculpher, 2006. "Evidence synthesis, parameter correlation and probabilistic sensitivity analysis," Health Economics, John Wiley & Sons, Ltd., vol. 15(4), pages 373-381, April.
    2. Victor R. Fuchs & Alan B. Krueger & James M. Poterba, 1997. "Why do Economists Disagree About Policy?," NBER Working Papers 6151, National Bureau of Economic Research, Inc.
    3. Christopher Dickey & Regina M. Santella & Dale Hattis & Deliang Tang & Yanzhi Hsu & Tom Cooper & Tie‐Lan Young & Frederica P. Perera, 1997. "Variability in PAH‐DNA Adduct Measurements in Peripheral Mononuclear Cells: Implications for Quantitative Cancer Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 17(5), pages 649-656, October.
    4. Auffhammer, Maximilian, 2005. "The Rationality of EIA Forecasts under Symmetric and Asymmetric Loss," Department of Agricultural & Resource Economics, UC Berkeley, Working Paper Series qt2ts415ts, Department of Agricultural & Resource Economics, UC Berkeley.
    5. Brent Finley & Deborah Proctor & Paul Scott & Natalie Harrington & Dennis Paustenbach & Paul Price, 1994. "Recommended Distributions for Exposure Factors Frequently Used in Health Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 533-553, August.
    6. Nico Keilman & Dinh Quang Pham, 2004. "Empirical errors and predicted errors in fertility, mortality and migration forecasts in the European Economic Area," Discussion Papers 386, Statistics Norway, Research Department.
    7. Fischer, Carolyn & Herrnstadt, Evan & Morgenstern, Richard, 2009. "Understanding errors in EIA projections of energy demand," Resource and Energy Economics, Elsevier, vol. 31(3), pages 198-209, August.
    8. Laitner, J. A. & DeCanio, S. J. & Koomey, J. G. & Sanstad, A. H., 2003. "Room for improvement: increasing the value of energy modeling for policy analysis," Utilities Policy, Elsevier, vol. 11(2), pages 87-94, June.
    9. Maged M. Hamed & Philip B. Bedient, 1997. "On the Effect of Probability Distributions of Input Variables in Public Health Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 17(1), pages 97-105, February.
    10. Hou, Tianfeng & Nuyens, Dirk & Roels, Staf & Janssen, Hans, 2019. "Quasi-Monte Carlo based uncertainty analysis: Sampling efficiency and error estimation in engineering applications," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    11. Lin, Shi-Woei & Bier, Vicki M., 2008. "A study of expert overconfidence," Reliability Engineering and System Safety, Elsevier, vol. 93(5), pages 711-721.
    12. Frédéric Dor & Pascal Empereur‐Bissonnet & Denis Zmirou & Vincent Nedellec & Jean‐Marie Haguenoer & Frans Jongeneelen & Alain Person & William Dab & Colin Ferguson, 2003. "Validation of Multimedia Models Assessing Exposure to PAHs—The SOLEX Study," Risk Analysis, John Wiley & Sons, vol. 23(5), pages 1047-1057, October.
    13. K. Claxton & P. J. Neumannn & S. S. Araki & M. C. Weinstein, "undated". "Bayesian Value-of-Information Analysis: An Application to a Policy Model of Alzheimer's Disease," Discussion Papers 00/39, Department of Economics, University of York.
    14. James Hammitt, 2000. "Are The Costs of Proposed Environmental Regulations Overestimated? Evidence from the CFC Phaseout," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 16(3), pages 281-302, July.
    15. Alan H. Stern, 1993. "Re‐evaluation of the Reference Dose for Methylmercury and Assessment of Current Exposure Levels," Risk Analysis, John Wiley & Sons, vol. 13(3), pages 355-364, June.
    16. Julia J. Pet‐Armacost & Jose Sepulveda & Milton Sakude, 1999. "Monte Carlo Sensitivity Analysis of Unknown Parameters in Hazardous Materials Transportation Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 19(6), pages 1173-1184, December.
    17. Farmer, J. Doyne & Lafond, François, 2016. "How predictable is technological progress?," Research Policy, Elsevier, vol. 45(3), pages 647-665.
    18. Wout Slob, 1994. "Uncertainty Analysis in Multiplicative Models," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 571-576, August.
    19. Alison C. Cullen, 1994. "Measures of Compounding Conservatism in Probabilistic Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 389-393, August.
    20. Michael Greenberg & Charles Haas & Anthony Cox & Karen Lowrie & Katherine McComas & Warner North, 2012. "Ten Most Important Accomplishments in Risk Analysis, 1980–2010," Risk Analysis, John Wiley & Sons, vol. 32(5), pages 771-781, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:riskan:v:14:y:1994:i:4:p:441-447. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1539-6924 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.