IDEAS home Printed from https://ideas.repec.org/a/eee/anture/v48y2014icp156-174.html
   My bibliography  Save this article

Accuracy and bias of experts’ adjusted forecasts

Author

Listed:
  • Lin, Vera Shanshan
  • Goodwin, Paul
  • Song, Haiyan

Abstract

This study investigates whether experts’ group-based judgmental adjustments to econometric forecasts of tourism demand improve the accuracy of the forecasts and whether the adjusted forecasts are unbiased. The Delphi method was used to aggregate experts’ judgmental adjustments and a range of error measures and statistical tests were employed to evaluate forecast accuracy. Regression analysis was used to investigate whether the statistical and judgmentally-adjusted forecasts were unbiased. The hypothesis tests suggested that, on average, the adjustments of the Delphi panel improved forecast accuracy though the group-adjusted forecasts were found to be biased for some of the individual markets. In-depth interviews with the Delphi panellists provided further insights into the biases that were associated with the Delphi surveys.

Suggested Citation

  • Lin, Vera Shanshan & Goodwin, Paul & Song, Haiyan, 2014. "Accuracy and bias of experts’ adjusted forecasts," Annals of Tourism Research, Elsevier, vol. 48(C), pages 156-174.
    • Handle: RePEc:eee:anture:v:48:y:2014:i:c:p:156-174
    DOI: 10.1016/j.annals.2014.06.005
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0160738314000814
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.annals.2014.06.005?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Willemain, Thomas R., 1989. "Graphical adjustment of statistical forecasts," International Journal of Forecasting, Elsevier, vol. 5(2), pages 179-185.
    2. Nada R. Sanders & Karl B. Manrodt, 1994. "Forecasting Practices in US Corporations: Survey Results," Interfaces, INFORMS, vol. 24(2), pages 92-100, April.
    3. Klassen, Robert D. & Flores, Benito E., 2001. "Forecasting practices of Canadian firms: Survey results and comparisons," International Journal of Production Economics, Elsevier, vol. 70(2), pages 163-174, March.
    4. Goodwin, Paul & Wright, George, 1993. "Improving judgmental time series forecasting: A review of the guidance provided by research," International Journal of Forecasting, Elsevier, vol. 9(2), pages 147-161, August.
    5. Eggleton, Irc, 1982. "Intuitive Time-Series Extrapolation," Journal of Accounting Research, Wiley Blackwell, vol. 20(1), pages 68-102.
    6. Lawrence, Michael & Makridakis, Spyros, 1989. "Factors affecting judgmental forecasts and confidence intervals," Organizational Behavior and Human Decision Processes, Elsevier, vol. 43(2), pages 172-187, April.
    7. Lawrence, Michael & Goodwin, Paul & O'Connor, Marcus & Onkal, Dilek, 2006. "Judgmental forecasting: A review of progress over the last 25 years," International Journal of Forecasting, Elsevier, vol. 22(3), pages 493-518.
    8. Witt, Stephen F. & Witt, Christine A., 1995. "Forecasting tourism demand: A review of empirical research," International Journal of Forecasting, Elsevier, vol. 11(3), pages 447-475, September.
    9. Richard D. F. Harris, 1999. "The Accuracy, Bias and Efficiency of Analysts’ Long Run Earnings Growth Forecasts," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 26(5‐6), pages 725-755, June.
    10. Makridakis, Spyros & Chatfield, Chris & Hibon, Michele & Lawrence, Michael & Mills, Terence & Ord, Keith & Simmons, LeRoy F., 1993. "The M2-competition: A real-time judgmentally based forecasting study," International Journal of Forecasting, Elsevier, vol. 9(1), pages 5-22, April.
    11. Song, Haiyan & Lin, Shanshan & Witt, Stephen F. & Zhang, Xinyan, 2011. "Impact of financial/economic crisis on demand for hotel rooms in Hong Kong," Tourism Management, Elsevier, vol. 32(1), pages 172-186.
    12. Lawrence, Michael & O'Connor, Marcus & Edmundson, Bob, 2000. "A field study of sales forecasting accuracy and processes," European Journal of Operational Research, Elsevier, vol. 122(1), pages 151-160, April.
    13. Fildes, Robert & Goodwin, Paul & Lawrence, Michael & Nikolopoulos, Konstantinos, 2009. "Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning," International Journal of Forecasting, Elsevier, vol. 25(1), pages 3-23.
    14. Collopy, Fred & Armstrong, J. Scott, 1992. "Expert opinions about extrapolation and the mystery of the overlooked discontinuities," International Journal of Forecasting, Elsevier, vol. 8(4), pages 575-582, December.
    15. Robert Fildes & Paul Goodwin, 2007. "Against Your Better Judgment? How Organizations Can Improve Their Use of Management Judgment in Forecasting," Interfaces, INFORMS, vol. 37(6), pages 570-576, December.
    16. Willemain, Thomas R., 1991. "The effect of graphical adjustment on forecast accuracy," International Journal of Forecasting, Elsevier, vol. 7(2), pages 151-154, August.
    17. Dilek Onkal & M. Sinan Gonul, 2005. "Judgmental Adjustment: A Challenge for Providers and Users of Forecasts," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 1, pages 13-17, June.
    18. Robert Carbone & Allan Andersen & Yvan Corriveau & Paul Piat Corson, 1983. "Comparing for Different Time Series Methods the Value of Technical Expertise Individualized Analysis, and Judgmental Adjustment," Management Science, INFORMS, vol. 29(5), pages 559-566, May.
    19. Goodwin, Paul, 2000. "Correct or combine? Mechanically integrating judgmental forecasts with statistical methods," International Journal of Forecasting, Elsevier, vol. 16(2), pages 261-275.
    20. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    21. Roy Batchelor, 2001. "How useful are the forecasts of intergovernmental agencies? The IMF and OECD versus the consensus," Applied Economics, Taylor & Francis Journals, vol. 33(2), pages 225-235.
    22. Flores, Benito E. & Olson, David L. & Wolfe, Christopher, 1992. "Judgmental adjustment of forecasts: A comparison of methods," International Journal of Forecasting, Elsevier, vol. 7(4), pages 421-433, March.
    23. O'Connor, Marcus & Remus, William & Griggs, Ken, 1993. "Judgemental forecasting in times of change," International Journal of Forecasting, Elsevier, vol. 9(2), pages 163-172, August.
    24. Sanders, NR, 1992. "Accuracy of judgmental forecasts: A comparison," Omega, Elsevier, vol. 20(3), pages 353-364, May.
    25. Alberto Musso & Steven Phillips, 2002. "Comparing Projections and Outcomes of IMF-Supported Programs," IMF Staff Papers, Palgrave Macmillan, vol. 49(1), pages 1-3.
    26. Rowe, Gene & Wright, George, 1999. "The Delphi technique as a forecasting tool: issues and analysis," International Journal of Forecasting, Elsevier, vol. 15(4), pages 353-375, October.
    27. Song, Haiyan & Gao, Bastian Z. & Lin, Vera S., 2013. "Combining statistical and judgmental forecasts via a web-based tourism demand forecasting system," International Journal of Forecasting, Elsevier, vol. 29(2), pages 295-310.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vera Shanshan Lin, 2019. "Judgmental adjustments in tourism forecasting practice: How good are they?," Tourism Economics, , vol. 25(3), pages 402-424, May.
    2. Liu, Anyu & Vici, Laura & Ramos, Vicente & Giannoni, Sauveur & Blake, Adam, 2021. "Visitor arrivals forecasts amid COVID-19: A perspective from the Europe team," Annals of Tourism Research, Elsevier, vol. 88(C).
    3. Arvan, Meysam & Fahimnia, Behnam & Reisi, Mohsen & Siemsen, Enno, 2019. "Integrating human judgement into quantitative forecasting methods: A review," Omega, Elsevier, vol. 86(C), pages 237-252.
    4. Woodside, Arch G., 2020. "Interventions as experiments: Connecting the dots in forecasting and overcoming pandemics, global warming, corruption, civil rights violations, misogyny, income inequality, and guns," Journal of Business Research, Elsevier, vol. 117(C), pages 212-218.
    5. Zhang, Hanyuan & Song, Haiyan & Wen, Long & Liu, Chang, 2021. "Forecasting tourism recovery amid COVID-19," Annals of Tourism Research, Elsevier, vol. 87(C).
    6. Anyu Liu & Laura Vici & Vicente Ramos & Sauveur Giannoni & Adam Blake, 2021. "Visitor arrivals forecasts amid COVID-19: A perspective from the Europe team," Post-Print hal-04653783, HAL.
    7. Toppinen, Anne & Röhr, Axel & Pätäri, Satu & Lähtinen, Katja & Toivonen, Ritva, 2018. "The future of wooden multistory construction in the forest bioeconomy – A Delphi study from Finland and Sweden," Journal of Forest Economics, Elsevier, vol. 31(C), pages 3-10.
    8. Winkler, Jens & Moser, Roger, 2016. "Biases in future-oriented Delphi studies: A cognitive perspective," Technological Forecasting and Social Change, Elsevier, vol. 105(C), pages 63-76.
    9. George Athanasopoulos & Rob J Hyndman & Mitchell O'Hara-Wild, 2021. "The Road to Recovery from COVID-19 for Australian Tourism," Monash Econometrics and Business Statistics Working Papers 1/21, Monash University, Department of Econometrics and Business Statistics.
    10. Larissa Koupriouchina & Jean-Pierre van der Rest & Zvi Schwartz, 2023. "Judgmental Adjustments of Algorithmic Hotel Occupancy Forecasts: Does User Override Frequency Impact Accuracy at Different Time Horizons?," Tourism Economics, , vol. 29(8), pages 2143-2164, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Leitner, Johannes & Leopold-Wildburger, Ulrike, 2011. "Experiments on forecasting behavior with several sources of information - A review of the literature," European Journal of Operational Research, Elsevier, vol. 213(3), pages 459-469, September.
    2. Perera, H. Niles & Hurley, Jason & Fahimnia, Behnam & Reisi, Mohsen, 2019. "The human factor in supply chain forecasting: A systematic review," European Journal of Operational Research, Elsevier, vol. 274(2), pages 574-600.
    3. Lawrence, Michael & Goodwin, Paul & O'Connor, Marcus & Onkal, Dilek, 2006. "Judgmental forecasting: A review of progress over the last 25 years," International Journal of Forecasting, Elsevier, vol. 22(3), pages 493-518.
    4. Arvan, Meysam & Fahimnia, Behnam & Reisi, Mohsen & Siemsen, Enno, 2019. "Integrating human judgement into quantitative forecasting methods: A review," Omega, Elsevier, vol. 86(C), pages 237-252.
    5. Webby, Richard & O'Connor, Marcus, 1996. "Judgemental and statistical time series forecasting: a review of the literature," International Journal of Forecasting, Elsevier, vol. 12(1), pages 91-118, March.
    6. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    7. Petropoulos, Fotios & Goodwin, Paul & Fildes, Robert, 2017. "Using a rolling training approach to improve judgmental extrapolations elicited from forecasters with technical knowledge," International Journal of Forecasting, Elsevier, vol. 33(1), pages 314-324.
    8. Vera Shanshan Lin, 2019. "Judgmental adjustments in tourism forecasting practice: How good are they?," Tourism Economics, , vol. 25(3), pages 402-424, May.
    9. JS Armstrong & Fred Collopy, 2004. "Integration of Statistical Methods and Judgment for Time Series," General Economics and Teaching 0412024, University Library of Munich, Germany.
    10. Goodwin, Paul, 2002. "Integrating management judgment and statistical methods to improve short-term forecasts," Omega, Elsevier, vol. 30(2), pages 127-135, April.
    11. Eksoz, Can & Mansouri, S. Afshin & Bourlakis, Michael, 2014. "Collaborative forecasting in the food supply chain: A conceptual framework," International Journal of Production Economics, Elsevier, vol. 158(C), pages 120-135.
    12. F Caniato & M Kalchschmidt & S Ronchi, 2011. "Integrating quantitative and qualitative forecasting approaches: organizational learning in an action research case," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(3), pages 413-424, March.
    13. De Baets, Shari & Harvey, Nigel, 2020. "Using judgment to select and adjust forecasts from statistical models," European Journal of Operational Research, Elsevier, vol. 284(3), pages 882-895.
    14. Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
    15. Welch, Eric & Bretschneider, Stuart & Rohrbaugh, John, 1998. "Accuracy of judgmental extrapolation of time series data: Characteristics, causes, and remediation strategies for forecasting," International Journal of Forecasting, Elsevier, vol. 14(1), pages 95-110, March.
    16. Song, Haiyan & Gao, Bastian Z. & Lin, Vera S., 2013. "Combining statistical and judgmental forecasts via a web-based tourism demand forecasting system," International Journal of Forecasting, Elsevier, vol. 29(2), pages 295-310.
    17. Han, Weiwei & Wang, Xun & Petropoulos, Fotios & Wang, Jing, 2019. "Brain imaging and forecasting: Insights from judgmental model selection," Omega, Elsevier, vol. 87(C), pages 1-9.
    18. Petropoulos, Fotios & Fildes, Robert & Goodwin, Paul, 2016. "Do ‘big losses’ in judgmental adjustments to statistical forecasts affect experts’ behaviour?," European Journal of Operational Research, Elsevier, vol. 249(3), pages 842-852.
    19. Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2015. "Golden rule of forecasting: Be conservative," Journal of Business Research, Elsevier, vol. 68(8), pages 1717-1731.
    20. Syntetos, Aris A. & Kholidasari, Inna & Naim, Mohamed M., 2016. "The effects of integrating management judgement into OUT levels: In or out of context?," European Journal of Operational Research, Elsevier, vol. 249(3), pages 853-863.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:anture:v:48:y:2014:i:c:p:156-174. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/annals-of-tourism-research/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.