IDEAS home Printed from https://ideas.repec.org/a/inm/orinte/v33y2003i6p91-111.html
   My bibliography  Save this article

The Ombudsman: Reaping Benefits from Management Research: Lessons from the Forecasting Principles Project

Author

Listed:
  • J. Scott Armstrong

    (The Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania 19104-6340)

  • Ruth Pagell

    (Goizueta Business Library, Emory University, Atlanta, Georgia 30322)

Abstract

It is often claimed that managers do not read serious research papers in journals. If true, this neglect would seem to pose a problem because journals are the dominant source of knowledge in management science. By examining results from the forecasting principles project, which was designed to summarize all useful knowledge in forecasting, we found that journals have provided 89 percent of the useful knowledge. However, journal papers relevant to practice are difficult to find because fewer than three percent of papers on forecasting contain useful findings. That turns out to be about one useful paper per month over the last half century. Once found, papers are difficult to interpret. Managers need low-cost, easily accessible sources that summarize advice (principles) from research; journals do not meet this need. To increase the rate of progress in developing and communicating principles, researchers, journal editors, textbook writers, software developers, Web-site designers, and practitioners should make some changes. We offer some examples: Researchers should directly study forecasting principles. Journal editors should actively solicit papers; invited submissions were about 20 times better than standard submissions at producing useful findings that were often cited, and they do so at lower cost. Textbook writers should focus on principles so that readers can apply knowledge. Web-site and software developers should provide practitioners with low-cost ways to use principles. Practitioners should apply the principles that are currently available.

Suggested Citation

  • J. Scott Armstrong & Ruth Pagell, 2003. "The Ombudsman: Reaping Benefits from Management Research: Lessons from the Forecasting Principles Project," Interfaces, INFORMS, vol. 33(6), pages 91-111, December.
  • Handle: RePEc:inm:orinte:v:33:y:2003:i:6:p:91-111
    DOI: 10.1287/inte.33.6.91.25180
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/inte.33.6.91.25180
    Download Restriction: no

    File URL: https://libkey.io/10.1287/inte.33.6.91.25180?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Green, Kesten C., 2002. "Forecasting decisions in conflict situations: a comparison of game theory, role-playing, and unaided judgement," International Journal of Forecasting, Elsevier, vol. 18(3), pages 321-344.
    2. Lomborg,Bjørn, 2001. "The Skeptical Environmentalist," Cambridge Books, Cambridge University Press, number 9780521010689, October.
    3. J. Scott Armstrong, 1996. "The Ombudsman: Management Folklore and Management Science—On Portfolio Planning, Escalation Bias, and Such," Interfaces, INFORMS, vol. 26(4), pages 25-55, August.
    4. Chatfield, Chris, 1993. "Calculating Interval Forecasts: Reply," Journal of Business & Economic Statistics, American Statistical Association, vol. 11(2), pages 143-144, April.
    5. Chatfield, Chris, 1993. "Calculating Interval Forecasts," Journal of Business & Economic Statistics, American Statistical Association, vol. 11(2), pages 121-135, April.
    6. Juan Miguel Campanario, 1996. "The competition for journal space among referees, editors, and other authors and its influence on journals' impact factors," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 47(3), pages 184-192, March.
    7. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
    8. Hubbard, Raymond & Vetter, Daniel E., 1996. "An empirical comparison of published replication research in accounting, economics, finance, management, and marketing," Journal of Business Research, Elsevier, vol. 35(2), pages 153-164, February.
    9. Armstrong, J. Scott, 2003. "Discovery and communication of important marketing findings: Evidence and proposals," Journal of Business Research, Elsevier, vol. 56(1), pages 69-84, January.
    10. Laband, David N & Piette, Michael J, 1994. "Favoritism versus Search for Good Papers: Empirical Evidence Regarding the Behavior of Journal Editors," Journal of Political Economy, University of Chicago Press, vol. 102(1), pages 194-203, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. JS Armstrong, 2005. "Incentives for Developing and Communicating Principles: A Reply," General Economics and Teaching 0502049, University Library of Munich, Germany.
    2. Armstrong, J. Scott, 2006. "Findings from evidence-based forecasting: Methods for reducing forecast error," International Journal of Forecasting, Elsevier, vol. 22(3), pages 583-598.
    3. Robert Fildes & Paul Goodwin, 2007. "Against Your Better Judgment? How Organizations Can Improve Their Use of Management Judgment in Forecasting," Interfaces, INFORMS, vol. 37(6), pages 570-576, December.
    4. Rianne Legerstee & Philip Hans Franses, 2014. "Do Experts’ SKU Forecasts Improve after Feedback?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 69-79, January.
    5. Benda, Wim G.G. & Engels, Tim C.E., 2011. "The predictive validity of peer review: A selective review of the judgmental forecasting qualities of peers, and implications for innovation in science," International Journal of Forecasting, Elsevier, vol. 27(1), pages 166-182.
    6. Armstrong, J. Scott & Fildes, Robert, 2006. "Making progress in forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 433-441.
    7. JS Armstrong & Ruth Pagell, 2005. "Reaping Benefits from Management Research: Lessons from the Forecasting Principles Project, with Reply to Commentators," General Economics and Teaching 0502048, University Library of Munich, Germany.
    8. Ortinau, David J., 2011. "Writing and publishing important scientific articles: A reviewer's perspective," Journal of Business Research, Elsevier, vol. 64(2), pages 150-156, February.
    9. Benda, Wim G.G. & Engels, Tim C.E., 2011. "The predictive validity of peer review: A selective review of the judgmental forecasting qualities of peers, and implications for innovation in science," International Journal of Forecasting, Elsevier, vol. 27(1), pages 166-182, January.
    10. Geuens, Maggie, 2011. "Where does business research go from here? Food-for-thought on academic papers in business research," Journal of Business Research, Elsevier, vol. 64(10), pages 1104-1107, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mirakyan, Atom & Meyer-Renschhausen, Martin & Koch, Andreas, 2017. "Composite forecasting approach, application for next-day electricity price forecasting," Energy Economics, Elsevier, vol. 66(C), pages 228-237.
    2. Meira, Erick & Cyrino Oliveira, Fernando Luiz & de Menezes, Lilian M., 2021. "Point and interval forecasting of electricity supply via pruned ensembles," Energy, Elsevier, vol. 232(C).
    3. De Gooijer, Jan G. & Hyndman, Rob J., 2006. "25 years of time series forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 443-473.
    4. Taylor, James W., 2007. "Forecasting daily supermarket sales using exponentially weighted quantile regression," European Journal of Operational Research, Elsevier, vol. 178(1), pages 154-167, April.
    5. Armstrong, J. Scott, 2003. "Discovery and communication of important marketing findings: Evidence and proposals," Journal of Business Research, Elsevier, vol. 56(1), pages 69-84, January.
    6. Gardner, Everette Jr., 2006. "Exponential smoothing: The state of the art--Part II," International Journal of Forecasting, Elsevier, vol. 22(4), pages 637-666.
    7. JS Armstrong & Ruth Pagell, 2005. "Reaping Benefits from Management Research: Lessons from the Forecasting Principles Project, with Reply to Commentators," General Economics and Teaching 0502048, University Library of Munich, Germany.
    8. Clements Michael P. & Hendry David F., 2008. "Economic Forecasting in a Changing World," Capitalism and Society, De Gruyter, vol. 3(2), pages 1-20, October.
    9. Tashman, Leonard J., 2000. "Out-of-sample tests of forecasting accuracy: an analysis and review," International Journal of Forecasting, Elsevier, vol. 16(4), pages 437-450.
    10. Berrin Aytac & S. Wu, 2013. "Characterization of demand for short life-cycle technology products," Annals of Operations Research, Springer, vol. 203(1), pages 255-277, March.
    11. JS Armstrong, 2005. "Incentives for Developing and Communicating Principles: A Reply," General Economics and Teaching 0502049, University Library of Munich, Germany.
    12. Litsiou, Konstantia & Polychronakis, Yiannis & Karami, Azhdar & Nikolopoulos, Konstantinos, 2022. "Relative performance of judgmental methods for forecasting the success of megaprojects," International Journal of Forecasting, Elsevier, vol. 38(3), pages 1185-1196.
    13. Lee, Yun Shin & Scholtes, Stefan, 2014. "Empirical prediction intervals revisited," International Journal of Forecasting, Elsevier, vol. 30(2), pages 217-234.
    14. repec:lan:wpaper:470 is not listed on IDEAS
    15. Charles, Amelie & Darne, Olivier & Kim, Jae, 2016. "Stock Return Predictability: Evaluation based on Prediction Intervals," MPRA Paper 70143, University Library of Munich, Germany.
    16. Dhaoui, Iyad, 2015. "Climat des Affaires et Compétitivité de l’Entreprise Tunisienne Après la Révolution : Analyses et Perspectives [Business Climate and Competitiveness of the Tunisian Enterprise After the Revolution:," MPRA Paper 87331, University Library of Munich, Germany.
    17. Pascual, Lorenzo & Romo, Juan & Ruiz, Esther, 2005. "Bootstrap prediction intervals for power-transformed time series," International Journal of Forecasting, Elsevier, vol. 21(2), pages 219-235.
    18. Li, Yushu & Andersson, Jonas, 2014. "A Likelihood Ratio and Markov Chain Based Method to Evaluate Density Forecasting," Discussion Papers 2014/12, Norwegian School of Economics, Department of Business and Management Science.
    19. Klaus Abberger, 2006. "Kernel smoothed prediction intervals for ARMA models," Statistical Papers, Springer, vol. 47(1), pages 1-15, January.
    20. Roberto Buizza & James W. Taylor, 2004. "A comparison of temperature density forecasts from GARCH and atmospheric models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(5), pages 337-355.
    21. Yushu Li & Jonas Andersson, 2020. "A likelihood ratio and Markov chain‐based method to evaluate density forecasting," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(1), pages 47-55, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orinte:v:33:y:2003:i:6:p:91-111. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.