IDEAS home Printed from https://ideas.repec.org/a/bpj/statpp/v4y2012i1p26n2.html
   My bibliography  Save this article

Synthetic Priors that Merge Opinion from Multiple Experts

Author

Listed:
  • Das Sourish
  • Yang Hongxia
  • Banks David

Abstract

Public policy relies strongly upon expert opinion, especially in risk assessment for rare events. But expert opinion is often inconsistent, both within and between experts. We therefore develop a statistical model for the elicited opinions, and use that to borrow strength across the responses through an exchangeable prior. Several versions of that prior are considered; the most advanced uses covariate information on the experts to characterize their areas of agreement and disagreement, which ultimately allows the estimation of the opinion of a synthetic expert whose covariates are selected by the analyst.This result depends upon a novel technique that incorporates the background information of the expert using hierarchical Dirichlet regression and a latent space model. As an illustration, in October 2008 we elicited opinions on the percentage of the popular vote that presidential candidate Barack Obama would win in North Carolina. Among the respondents, those who were conservative or who had lived in North Carolina for a longer period of time gave systematically lower percentages. Our method enables the analyst to infer the opinion of a respondent with a specified political inclination or number of years of residency.

Suggested Citation

  • Das Sourish & Yang Hongxia & Banks David, 2012. "Synthetic Priors that Merge Opinion from Multiple Experts," Statistics, Politics and Policy, De Gruyter, vol. 4(1), pages 1-26, December.
    • Handle: RePEc:bpj:statpp:v:4:y:2012:i:1:p:26:n:2
    DOI: 10.1515/2151-7509.1060
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/2151-7509.1060
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.1515/2151-7509.1060?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Wright, William F. & Anderson, Urton, 1989. "Effects of situation familiarity and financial incentives on use of the anchoring and adjustment heuristic for probability assessment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 44(1), pages 68-82, August.
    2. Dennis Lindley, 1983. "Reconciliation of Probability Distributions," Operations Research, INFORMS, vol. 31(5), pages 866-880, October.
    3. Lin, Shi-Woei & Bier, Vicki M., 2008. "A study of expert overconfidence," Reliability Engineering and System Safety, Elsevier, vol. 93(5), pages 711-721.
    4. Garthwaite, Paul H. & Kadane, Joseph B. & O'Hagan, Anthony, 2005. "Statistical Methods for Eliciting Probability Distributions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 680-701, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Erin Baker & Olaitan Olaleye, 2013. "Combining Experts: Decomposition and Aggregation Order," Risk Analysis, John Wiley & Sons, vol. 33(6), pages 1116-1127, June.
    2. Eggstaff, Justin W. & Mazzuchi, Thomas A. & Sarkani, Shahram, 2014. "The effect of the number of seed variables on the performance of Cooke′s classical model," Reliability Engineering and System Safety, Elsevier, vol. 121(C), pages 72-82.
    3. Lichtendahl Jr., Kenneth C., 2009. "Random quantiles of the Dirichlet process," Statistics & Probability Letters, Elsevier, vol. 79(4), pages 501-507, February.
    4. Hathout, Michel & Vuillet, Marc & Carvajal, Claudio & Peyras, Laurent & Diab, Youssef, 2019. "Expert judgments calibration and combination for assessment of river levee failure probability," Reliability Engineering and System Safety, Elsevier, vol. 188(C), pages 377-392.
    5. Barker, Kash & Haimes, Yacov Y., 2009. "Assessing uncertainty in extreme events: Applications to risk-based decision making in interdependent infrastructure sectors," Reliability Engineering and System Safety, Elsevier, vol. 94(4), pages 819-829.
    6. Eyting, Markus & Schmidt, Patrick, 2021. "Belief elicitation with multiple point predictions," European Economic Review, Elsevier, vol. 135(C).
    7. David R. Mandel & Daniel Irwin, 2021. "Tracking accuracy of strategic intelligence forecasts: Findings from a long‐term Canadian study," Futures & Foresight Science, John Wiley & Sons, vol. 3(3-4), September.
    8. Maarten Ijzerman & Lotte Steuten, 2011. "Early assessment of medical technologies to inform product development and market access," Applied Health Economics and Health Policy, Springer, vol. 9(5), pages 331-347, September.
    9. Claire Copeland & Britta Turner & Gareth Powells & Kevin Wilson, 2022. "In Search of Complementarity: Insights from an Exercise in Quantifying Qualitative Energy Futures," Energies, MDPI, vol. 15(15), pages 1-21, July.
    10. Robert Stewart & Marie Urban & Samantha Duchscherer & Jason Kaufman & April Morton & Gautam Thakur & Jesse Piburn & Jessica Moehl, 2016. "A Bayesian machine learning model for estimating building occupancy from open source data," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 81(3), pages 1929-1956, April.
    11. Morais, Caroline & Estrada-Lugo, Hector Diego & Tolo, Silvia & Jacques, Tiago & Moura, Raphael & Beer, Michael & Patelli, Edoardo, 2022. "Robust data-driven human reliability analysis using credal networks," Reliability Engineering and System Safety, Elsevier, vol. 218(PA).
    12. Arbel, Yuval & Ben-Shahar, Danny & Gabriel, Stuart, 2014. "Anchoring and housing choice: Results of a natural policy experiment," Regional Science and Urban Economics, Elsevier, vol. 49(C), pages 68-83.
    13. Anne Sofie Jore & James Mitchell & Shaun P. Vahey, 2010. "Combining forecast densities from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 621-634.
    14. Nicholas M. Kiefer, 2011. "Default estimation, correlated defaults, and expert information," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(2), pages 173-192, March.
    15. Tim R. Holcomb & R. Duane Ireland & R. Michael Holmes Jr. & Michael A. Hitt, 2009. "Architecture of Entrepreneurial Learning: Exploring the Link among Heuristics, Knowledge, and Action," Entrepreneurship Theory and Practice, , vol. 33(1), pages 167-192, January.
    16. Ross Gruetzemacher & Kang Bok Lee & David Paradice, 2024. "Calibration training for improving probabilistic judgments using an interactive app," Futures & Foresight Science, John Wiley & Sons, vol. 6(2), June.
    17. Miller, Joshua Benjamin & Sanjurjo, Adam, 2018. "How Experience Confirms the Gambler's Fallacy when Sample Size is Neglected," OSF Preprints m5xsk, Center for Open Science.
    18. Dai, Min & Jia, Yanwei & Kou, Steven, 2021. "The wisdom of the crowd and prediction markets," Journal of Econometrics, Elsevier, vol. 222(1), pages 561-578.
    19. Gove Allen & Jeffrey Parsons, 2010. "Is Query Reuse Potentially Harmful? Anchoring and Adjustment in Adapting Existing Database Queries," Information Systems Research, INFORMS, vol. 21(1), pages 56-77, March.
    20. A Zuashkiani & D Banjevic & A K S Jardine, 2009. "Estimating parameters of proportional hazards model based on expert knowledge and statistical data," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(12), pages 1621-1636, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:statpp:v:4:y:2012:i:1:p:26:n:2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.