IDEAS home Printed from https://ideas.repec.org/a/eee/tefoso/v105y2016icp63-76.html
   My bibliography  Save this article

Biases in future-oriented Delphi studies: A cognitive perspective

Author

Listed:
  • Winkler, Jens
  • Moser, Roger

Abstract

Delphi is an established information gathering and forecasting approach that has proven to deliver valuable results in a wide variety of specialist fields. Yet, Delphi studies have also continuously been subject to critique and doubt, particularly concerning its judgmental and forecasting accuracy. To a large part this can be attributed to the substantial discretion researchers have in their design and implementation. Awkwardly designed Delphi studies may lead to severe cognitive biases that adversely affect the research results. This paper takes a cognitive perspective by investigating how different cognitive biases take effect within future-oriented Delphi studies and how their unfavorable impacts can be mitigated by thoroughly adapting specific Delphi design features. The analysis addresses cognitive biases affecting panelists' initial estimates — namely framing and anchoring as well as the desirability bias — as well as such cognitive biases taking effect during feedback and revision loops — namely the bandwagon effect and belief perseverance.

Suggested Citation

  • Winkler, Jens & Moser, Roger, 2016. "Biases in future-oriented Delphi studies: A cognitive perspective," Technological Forecasting and Social Change, Elsevier, vol. 105(C), pages 63-76.
    • Handle: RePEc:eee:tefoso:v:105:y:2016:i:c:p:63-76
    DOI: 10.1016/j.techfore.2016.01.021
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0040162516000226
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.techfore.2016.01.021?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yaniv, Ilan, 2011. "Group diversity and decision quality: Amplification and attenuation of the framing effect," International Journal of Forecasting, Elsevier, vol. 27(1), pages 41-49, January.
    2. Eroglu, Cuneyt & Croxton, Keely L., 2010. "Biases in judgmental adjustments of statistical forecasts: The role of individual differences," International Journal of Forecasting, Elsevier, vol. 26(1), pages 116-133, January.
    3. Klayman, Joshua & Soll, Jack B. & Gonzalez-Vallejo, Claudia & Barlas, Sema, 1999. "Overconfidence: It Depends on How, What, and Whom You Ask, , , , , , , , ," Organizational Behavior and Human Decision Processes, Elsevier, vol. 79(3), pages 216-247, September.
    4. Yaniv, Ilan, 2011. "Group diversity and decision quality: Amplification and attenuation of the framing effect," International Journal of Forecasting, Elsevier, vol. 27(1), pages 41-49.
    5. Winkler, Jens & Kuklinski, Christian Paul Jian-Wei & Moser, Roger, 2015. "Decision making in emerging markets: The Delphi approach's contribution to coping with uncertainty and equivocality," Journal of Business Research, Elsevier, vol. 68(5), pages 1118-1126.
    6. Paese, Paul W. & Bieser, Mary & Tubbs, Mark E., 1993. "Framing Effects and Choice Shifts in Group Decision Making," Organizational Behavior and Human Decision Processes, Elsevier, vol. 56(1), pages 149-165, October.
    7. Graefe, Andreas & Armstrong, J. Scott, 2011. "Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task," International Journal of Forecasting, Elsevier, vol. 27(1), pages 183-195, January.
    8. Michail Tsikerdekis, 2013. "The effects of perceived anonymity and anonymity states on conformity and groupthink in online communities: A Wikipedia study," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(5), pages 1001-1015, May.
    9. Yaniv, Ilan & Kleinberger, Eli, 2000. "Advice Taking in Decision Making: Egocentric Discounting and Reputation Formation," Organizational Behavior and Human Decision Processes, Elsevier, vol. 83(2), pages 260-281, November.
    10. repec:eee:intfor:v:27:y:2011:i:1:p:1-13 is not listed on IDEAS
    11. van Exel, N.J.A. & Brouwer, W.B.F. & van den Berg, B. & Koopmanschap, M.A., 2006. "With a little help from an anchor: Discussion and evidence of anchoring effects in contingent valuation," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 35(5), pages 836-853, October.
    12. Michael R Czinkota & Ilkka A Ronkainen, 1997. "International Business and Trade in the Next Decade: Report from a Delphi Study," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 28(4), pages 827-844, December.
    13. Michael R Czinkota, 1986. "International Trade And Business In The Late 1980s: An Integrated U.S. Perspective," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 17(1), pages 127-134, March.
    14. Lin, Vera Shanshan & Goodwin, Paul & Song, Haiyan, 2014. "Accuracy and bias of experts’ adjusted forecasts," Annals of Tourism Research, Elsevier, vol. 48(C), pages 156-174.
    15. Michail Tsikerdekis, 2013. "The effects of perceived anonymity and anonymity states on conformity and groupthink in online communities: A Wikipedia study," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(5), pages 1001-1015, May.
    16. Richard L. Daft & Robert H. Lengel, 1986. "Organizational Information Requirements, Media Richness and Structural Design," Management Science, INFORMS, vol. 32(5), pages 554-571, May.
    17. Edward J. Russo & Kurt A. Carlson & Margaret G. Meloy & Kevyn Yong, 2008. "The goal of consistency as a cause of information distortion," Post-Print hal-00481326, HAL.
    18. Ayton, Peter & Ferrell, William R. & Stewart, Thomas R., 1999. "Commentaries on "The Delphi technique as a forecasting tool: issues and analysis" by Rowe and Wright," International Journal of Forecasting, Elsevier, vol. 15(4), pages 377-379, October.
    19. Oechssler, Jörg & Roider, Andreas & Schmitz, Patrick W., 2009. "Cognitive abilities and behavioral biases," Journal of Economic Behavior & Organization, Elsevier, vol. 72(1), pages 147-152, October.
    20. Tersine, Richard J. & Riggs, Walter E., 1976. "The delphi technique: A long-range planning tool," Business Horizons, Elsevier, vol. 19(2), pages 51-56, April.
    21. Kerr, Norbert L. & Tindale, R. Scott, 2011. "Group-based forecasting?: A social psychological analysis," International Journal of Forecasting, Elsevier, vol. 27(1), pages 14-40, January.
    22. Wright, George & Rowe, Gene, 2011. "Group-based judgmental forecasting: An integration of extant knowledge and the development of priorities for a new research agenda," International Journal of Forecasting, Elsevier, vol. 27(1), pages 1-13, January.
    23. Harvey, Nigel & Harries, Clare, 2004. "Effects of judges' forecasting on their later combination of forecasts for the same outcomes," International Journal of Forecasting, Elsevier, vol. 20(3), pages 391-409.
    24. Green, Kesten C. & Armstrong, J. Scott, 2011. "Role thinking: Standing in other people’s shoes to forecast decisions in conflicts," International Journal of Forecasting, Elsevier, vol. 27(1), pages 69-80.
    25. repec:cup:judgdm:v:4:y:2009:i:1:p:41-50 is not listed on IDEAS
    26. Mussweiler, Thomas & Strack, Fritz, 2001. "The Semantics of Anchoring," Organizational Behavior and Human Decision Processes, Elsevier, vol. 86(2), pages 234-255, November.
    27. Kauko, Karlo & Palmroos, Peter, 2014. "The Delphi method in forecasting financial markets— An experimental study," International Journal of Forecasting, Elsevier, vol. 30(2), pages 313-327.
    28. Chapman, Gretchen B. & Johnson, Eric J., 1999. "Anchoring, Activation, and the Construction of Values, , , , , ," Organizational Behavior and Human Decision Processes, Elsevier, vol. 79(2), pages 115-153, August.
    29. Windschitl, Paul D. & Scherer, Aaron M. & Smith, Andrew R. & Rose, Jason P., 2013. "Why so confident? The influence of outcome desirability on selective exposure and likelihood judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 120(1), pages 73-86.
    30. Thorsteinson, Todd J. & Breier, Jennifer & Atwell, Anna & Hamilton, Catherine & Privette, Monica, 2008. "Anchoring effects on performance judgments," Organizational Behavior and Human Decision Processes, Elsevier, vol. 107(1), pages 29-40, September.
    31. Rowe, Gene & Wright, George, 1999. "The Delphi technique as a forecasting tool: issues and analysis," International Journal of Forecasting, Elsevier, vol. 15(4), pages 353-375, October.
    32. Rowe, Gene & Wright, George, 1996. "The impact of task characteristics on the performance of structured group forecasting techniques," International Journal of Forecasting, Elsevier, vol. 12(1), pages 73-89, March.
    33. Bergman, Oscar & Ellingsen, Tore & Johannesson, Magnus & Svensson, Cicek, 2010. "Anchoring and cognitive ability," Economics Letters, Elsevier, vol. 107(1), pages 66-68, April.
    34. Kerr, Norbert L. & Tindale, R. Scott, 2011. "Group-based forecasting?: A social psychological analysis," International Journal of Forecasting, Elsevier, vol. 27(1), pages 14-40.
    35. Zaichkowsky, Judith Lynne, 1985. "Measuring the Involvement Construct," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 12(3), pages 341-352, December.
    36. Czinkota, Michael R. & Ronkainen, Ilkka A., 2005. "A forecast of globalization, international business and trade: report from a Delphi study," Journal of World Business, Elsevier, vol. 40(2), pages 111-123, May.
    37. repec:cup:judgdm:v:2:y:2007:i::p:48-53 is not listed on IDEAS
    38. Simon, Herbert A, 1979. "Rational Decision Making in Business Organizations," American Economic Review, American Economic Association, vol. 69(4), pages 493-513, September.
    39. Kray, Laura J. & Galinsky, Adam D., 2003. "The debiasing effect of counterfactual mind-sets: Increasing the search for disconfirmatory information in group decisions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 91(1), pages 69-81, May.
    40. Klenk, Nicole L. & Hickey, Gordon M., 2011. "A virtual and anonymous, deliberative and analytic participation process for planning and evaluation: The Concept Mapping Policy Delphi," International Journal of Forecasting, Elsevier, vol. 27(1), pages 152-165, January.
    41. DeKay, Michael L. & Patiño-Echeverri, Dalia & Fischbeck, Paul S., 2009. "Distortion of probability and outcome information in risky decisions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 109(1), pages 79-92, May.
    42. Windschitl, Paul D. & Smith, Andrew R. & Rose, Jason P. & Krizan, Zlatan, 2010. "The desirability bias in predictions: Going optimistic without leaving realism," Organizational Behavior and Human Decision Processes, Elsevier, vol. 111(1), pages 33-47, January.
    43. Zimmermann, Martin & Darkow, Inga-Lena & von der Gracht, Heiko A., 2012. "Integrating Delphi and participatory backcasting in pursuit of trustworthiness — The case of electric mobility in Germany," Technological Forecasting and Social Change, Elsevier, vol. 79(9), pages 1605-1621.
    44. Landeta, Jon & Barrutia, Jon, 2011. "People consultation to construct the future: A Delphi application," International Journal of Forecasting, Elsevier, vol. 27(1), pages 134-151, January.
    45. Sniezek, Janet A. & Van Swol, Lyn M., 2001. "Trust, Confidence, and Expertise in a Judge-Advisor System," Organizational Behavior and Human Decision Processes, Elsevier, vol. 84(2), pages 288-307, March.
    46. Dequech, David, 2006. "The new institutional economics and the theory of behaviour under uncertainty," Journal of Economic Behavior & Organization, Elsevier, vol. 59(1), pages 109-131, January.
    47. Campbell, Sean D. & Sharpe, Steven A., 2009. "Anchoring Bias in Consensus Forecasts and Its Effect on Market Prices," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 44(2), pages 369-390, April.
    48. Förster, Bernadette & von der Gracht, Heiko, 2014. "Assessing Delphi panel composition for strategic foresight — A comparison of panels based on company-internal and external participants," Technological Forecasting and Social Change, Elsevier, vol. 84(C), pages 215-229.
    49. Landeta, Jon & Barrutia, Jon, 2011. "People consultation to construct the future: A Delphi application," International Journal of Forecasting, Elsevier, vol. 27(1), pages 134-151.
    50. Fuller, Sally Riggs & Aldag, Ramon J., 1998. "Organizational Tonypandy: Lessons from a Quarter Century of the Groupthink Phenomenon," Organizational Behavior and Human Decision Processes, Elsevier, vol. 73(2-3), pages 163-184, February.
    51. Robert Fildes & Paul Goodwin, 2007. "Against Your Better Judgment? How Organizations Can Improve Their Use of Management Judgment in Forecasting," Interfaces, INFORMS, vol. 37(6), pages 570-576, December.
    52. Igor Álvarez & José Calvo & Araceli Mora, 2014. "Involving academics in the accounting standard setting process: an application of the Delphi methodology to the assessment of IASB proposals," Journal of Management & Governance, Springer;Accademia Italiana di Economia Aziendale (AIDEA), vol. 18(3), pages 765-791, August.
    53. Yaniv, Ilan, 2004. "Receiving other people's advice: Influence and benefit," Organizational Behavior and Human Decision Processes, Elsevier, vol. 93(1), pages 1-13, January.
    54. Russo, J. Edward & Medvec, Victoria Husted & Meloy, Margaret G., 1996. "The Distortion of Information during Decisions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 66(1), pages 102-110, April.
    55. Block, Richard A. & Harper, David R., 1991. "Overconfidence in estimation: Testing the anchoring-and-adjustment hypothesis," Organizational Behavior and Human Decision Processes, Elsevier, vol. 49(2), pages 188-207, August.
    56. Klenk, Nicole L. & Hickey, Gordon M., 2011. "A virtual and anonymous, deliberative and analytic participation process for planning and evaluation: The Concept Mapping Policy Delphi," International Journal of Forecasting, Elsevier, vol. 27(1), pages 152-165.
    57. Mochon, Daniel & Frederick, Shane, 2013. "Anchoring in sequential judgments," Organizational Behavior and Human Decision Processes, Elsevier, vol. 122(1), pages 69-79.
    58. Harvey, Nigel & Fischer, Ilan, 1997. "Taking Advice: Accepting Help, Improving Judgment, and Sharing Responsibility," Organizational Behavior and Human Decision Processes, Elsevier, vol. 70(2), pages 117-133, May.
    59. Milch, Kerry F. & Weber, Elke U. & Appelt, Kirstin C. & Handgraaf, Michel J.J. & Krantz, David H., 2009. "From individual preference construction to group decisions: Framing effects and group processes," Organizational Behavior and Human Decision Processes, Elsevier, vol. 108(2), pages 242-255, March.
    60. Sieck, Winston & Yates, J. Frank, 1997. "Exposition Effects on Decision Making: Choice and Confidence in Choice," Organizational Behavior and Human Decision Processes, Elsevier, vol. 70(3), pages 207-219, June.
    61. Yaniv, Ilan & Milyavsky, Maxim, 2007. "Using advice from multiple sources to revise and improve judgments," Organizational Behavior and Human Decision Processes, Elsevier, vol. 103(1), pages 104-120, May.
    62. Matthew Rabin & Joel L. Schrag, 1999. "First Impressions Matter: A Model of Confirmatory Bias," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(1), pages 37-82.
    63. Song, Haiyan & Gao, Bastian Z. & Lin, Vera S., 2013. "Combining statistical and judgmental forecasts via a web-based tourism demand forecasting system," International Journal of Forecasting, Elsevier, vol. 29(2), pages 295-310.
    64. Bonaccio, Silvia & Dalal, Reeshad S., 2006. "Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences," Organizational Behavior and Human Decision Processes, Elsevier, vol. 101(2), pages 127-151, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bonaccorsi, Andrea & Apreda, Riccardo & Fantoni, Gualtiero, 2020. "Expert biases in technology foresight. Why they are a problem and how to mitigate them," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    2. Förster, Bernadette & von der Gracht, Heiko, 2014. "Assessing Delphi panel composition for strategic foresight — A comparison of panels based on company-internal and external participants," Technological Forecasting and Social Change, Elsevier, vol. 84(C), pages 215-229.
    3. Wright, George & Rowe, Gene, 2011. "Group-based judgmental forecasting: An integration of extant knowledge and the development of priorities for a new research agenda," International Journal of Forecasting, Elsevier, vol. 27(1), pages 1-13, January.
    4. repec:eee:intfor:v:27:y:2011:i:1:p:1-13 is not listed on IDEAS
    5. Winkler, Jens & Kuklinski, Christian Paul Jian-Wei & Moser, Roger, 2015. "Decision making in emerging markets: The Delphi approach's contribution to coping with uncertainty and equivocality," Journal of Business Research, Elsevier, vol. 68(5), pages 1118-1126.
    6. Perera, H. Niles & Hurley, Jason & Fahimnia, Behnam & Reisi, Mohsen, 2019. "The human factor in supply chain forecasting: A systematic review," European Journal of Operational Research, Elsevier, vol. 274(2), pages 574-600.
    7. Barrios, Maite & Guilera, Georgina & Nuño, Laura & Gómez-Benito, Juana, 2021. "Consensus in the delphi method: What makes a decision change?," Technological Forecasting and Social Change, Elsevier, vol. 163(C).
    8. Philipp Ecken & Richard Pibernik, 2016. "Hit or Miss: What Leads Experts to Take Advice for Long-Term Judgments?," Management Science, INFORMS, vol. 62(7), pages 2002-2021, July.
    9. Bonaccio, Silvia & Dalal, Reeshad S., 2006. "Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences," Organizational Behavior and Human Decision Processes, Elsevier, vol. 101(2), pages 127-151, November.
    10. Albert E. Mannes, 2009. "Are We Wise About the Wisdom of Crowds? The Use of Group Judgments in Belief Revision," Management Science, INFORMS, vol. 55(8), pages 1267-1279, August.
    11. Spickermann, Alexander & Zimmermann, Martin & von der Gracht, Heiko A., 2014. "Surface- and deep-level diversity in panel selection — Exploring diversity effects on response behaviour in foresight," Technological Forecasting and Social Change, Elsevier, vol. 85(C), pages 105-120.
    12. Kauko, Karlo & Palmroos, Peter, 2014. "The Delphi method in forecasting financial markets— An experimental study," International Journal of Forecasting, Elsevier, vol. 30(2), pages 313-327.
    13. Soll, Jack B. & Mannes, Albert E., 2011. "Judgmental aggregation strategies depend on whether the self is involved," International Journal of Forecasting, Elsevier, vol. 27(1), pages 81-102.
    14. Soll, Jack B. & Mannes, Albert E., 2011. "Judgmental aggregation strategies depend on whether the self is involved," International Journal of Forecasting, Elsevier, vol. 27(1), pages 81-102, January.
    15. Pennings, Clint L.P. & van Dalen, Jan & Rook, Laurens, 2019. "Coordinating judgmental forecasting: Coping with intentional biases," Omega, Elsevier, vol. 87(C), pages 46-56.
    16. Palmeira, Mauricio, 2020. "Advice in the presence of external cues: The impact of conflicting judgments on perceptions of expertise," Organizational Behavior and Human Decision Processes, Elsevier, vol. 156(C), pages 82-96.
    17. Van Swol, Lyn M., 2011. "Forecasting another’s enjoyment versus giving the right answer: Trust, shared values, task effects, and confidence in improving the acceptance of advice," International Journal of Forecasting, Elsevier, vol. 27(1), pages 103-120.
    18. Kausel, Edgar E. & Culbertson, Satoris S. & Leiva, Pedro I. & Slaughter, Jerel E. & Jackson, Alexander T., 2015. "Too arrogant for their own good? Why and when narcissists dismiss advice," Organizational Behavior and Human Decision Processes, Elsevier, vol. 131(C), pages 33-50.
    19. Van Swol, Lyn M., 2011. "Forecasting another's enjoyment versus giving the right answer: Trust, shared values, task effects, and confidence in improving the acceptance of advice," International Journal of Forecasting, Elsevier, vol. 27(1), pages 103-120, January.
    20. Marcin Kozak & Olesia Iefremova, 2014. "Implementation Of The Delphi Technique In Finance," "e-Finanse", University of Information Technology and Management, Institute of Financial Research and Analysis, vol. 10(4), pages 36-45, May.
    21. Merfeld, Katrin & Wilhelms, Mark-Philipp & Henkel, Sven & Kreutzer, Karin, 2019. "Carsharing with shared autonomous vehicles: Uncovering drivers, barriers and future developments – A four-stage Delphi study," Technological Forecasting and Social Change, Elsevier, vol. 144(C), pages 66-81.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:tefoso:v:105:y:2016:i:c:p:63-76. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.