IDEAS home Printed from https://ideas.repec.org/p/bep/unimip/unimi-1109.html
   My bibliography  Save this paper

Forecast Evaluation in Call Centers: Combined Forecasts, Flexible Loss Functions and Economic Criteria

Author

Listed:
  • Andrea Bastianin

    (University of Milan-Bicocca)

  • Marzio Galeotti

    (University of Milan)

  • Matteo Manera

    (University of Milan-Bicocca)

Abstract

Accurate forecasts of incoming calls are crucial to optimal staffing decisions in call centers. This paper evaluates a wide range of models and forecast combination techniques by means of statistical and economic criteria. Relative to the previous literature, this paper is novel in several respects. In particular, the statistical evaluation of competing models is carried out by using a flexible loss function as input to pairwise and joint forecast diagnostic checks. Informative rankings across alternative single models and different groups of models are obtained. Moreover, models are evaluated from the perspective of a manager, who needs reliable forecasts to dimension the call center. Money metrics of forecasting performance are computed, which are based on the economic value of information and the certainty equivalent.

Suggested Citation

  • Andrea Bastianin & Marzio Galeotti & Matteo Manera, 2011. "Forecast Evaluation in Call Centers: Combined Forecasts, Flexible Loss Functions and Economic Criteria," UNIMI - Research Papers in Economics, Business, and Statistics unimi-1109, Universitá degli Studi di Milano.
    • Handle: RePEc:bep:unimip:unimi-1109
    Note: oai:cdlib1:unimi-1109
    as

    Download full text from publisher

    File URL: http://services.bepress.com/unimi/economics/art41
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Tych, Wlodek & Pedregal, Diego J. & Young, Peter C. & Davies, John, 2002. "An unobserved component model for multi-rate forecasting of telephone call demand: the design of a forecasting support system," International Journal of Forecasting, Elsevier, vol. 18(4), pages 673-695.
    2. Edward E. Gbur & Robert A. Collins, 1989. "A Small-Sample Comparison of Estimators in the EU-MGF Approach to Decision Making," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 71(1), pages 202-210.
    3. James W. Taylor, 2012. "Density Forecasting of Intraday Call Center Arrivals Using Models Based on Exponential Smoothing," Management Science, INFORMS, vol. 58(3), pages 534-549, March.
    4. Ivana Komunjer & Michael T. Owyang, 2012. "Multivariate Forecast Evaluation and Rationality Testing," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1066-1080, November.
    5. Andrea BASTIANIN & Marzio GALEOTTI & Matteo MANERA, 2011. "Forecast evaluation in call centers: combined forecasts, flexible loss functions and economic criteria," Departmental Working Papers 2011-08, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    6. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    7. Robert Jung & A. Tremayne, 2011. "Useful models for time series of counts or simply wrong ones?," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 95(1), pages 59-91, March.
    8. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
    9. Taylor, James W., 2010. "Reply to the discussion of: Exponentially weighted methods for forecasting intraday time series with multiple seasonal cycles," International Journal of Forecasting, Elsevier, vol. 26(4), pages 658-660, October.
    10. Robert Engle, 2002. "New frontiers for arch models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 17(5), pages 425-446.
    11. Bruce H. Andrews & Shawn M. Cunningham, 1995. "L. L. Bean Improves Call-Center Forecasting," Interfaces, INFORMS, vol. 25(6), pages 1-13, December.
    12. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    13. Graham Elliott & Allan Timmermann & Ivana Komunjer, 2005. "Estimation and Testing of Forecast Rationality under Flexible Loss," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(4), pages 1107-1125.
    14. Howard E. Thompson & George C. Tiao, 1971. "Analysis of Telephone Data: A Case Study of Forecasting Seasonal Time Series," Bell Journal of Economics, The RAND Corporation, vol. 2(2), pages 515-541, Autumn.
    15. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    16. Noah Gans & Ger Koole & Avishai Mandelbaum, 2003. "Telephone Call Centers: Tutorial, Review, and Research Prospects," Manufacturing & Service Operations Management, INFORMS, vol. 5(2), pages 79-141, September.
    17. Jeff Fleming & Chris Kirby & Barbara Ostdiek, 2001. "The Economic Value of Volatility Timing," Journal of Finance, American Finance Association, vol. 56(1), pages 329-352, February.
    18. Elamin H. Elbasha, 2005. "Risk aversion and uncertainty in cost‐effectiveness analysis: the expected‐utility, moment‐generating function approach," Health Economics, John Wiley & Sons, Ltd., vol. 14(5), pages 457-470, May.
    19. Taylor, James W., 2010. "Exponentially weighted methods for forecasting intraday time series with multiple seasonal cycles," International Journal of Forecasting, Elsevier, vol. 26(4), pages 627-646, October.
    20. Skouras, Spyros, 2007. "Decisionmetrics: A decision-based approach to econometric modelling," Journal of Econometrics, Elsevier, vol. 137(2), pages 414-440, April.
    21. Philip Hans Franses, 2008. "Forecasting Seasonal Time Series," World Scientific Book Chapters, in: Roberto S Mariano & Yiu-Kuen Tse (ed.), Econometric Forecasting And High-Frequency Data Analysis, chapter 3, pages 93-130, World Scientific Publishing Co. Pte. Ltd..
    22. Norman R. Swanson & Halbert White, 1997. "A Model Selection Approach To Real-Time Macroeconomic Forecasting Using Linear Models And Artificial Neural Networks," The Review of Economics and Statistics, MIT Press, vol. 79(4), pages 540-550, November.
    23. Dorfman, Jeffrey H. & Mcintosh, Christopher S., 1997. "Economic Criteria for Evaluating Commodity Price Forecasts," Journal of Agricultural and Applied Economics, Cambridge University Press, vol. 29(2), pages 337-345, December.
    24. Andrea BASTIANIN & Marzio GALEOTTI & Matteo MANERA, 2011. "Forecast evaluation in call centers: combined forecasts, flexible loss functions and economic criteria," Departmental Working Papers 2011-08, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    25. Graham Elliott & Ivana Komunjer & Allan Timmermann, 2008. "Biases in Macroeconomic Forecasts: Irrationality or Asymmetric Loss?," Journal of the European Economic Association, MIT Press, vol. 6(1), pages 122-157, March.
    26. Granger, Clive W.J. & Machina, Mark J., 2006. "Forecasting and Decision Theory," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 2, pages 81-98, Elsevier.
    27. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
    28. Robert F. Engle & Jeffrey R. Russell, 1998. "Autoregressive Conditional Duration: A New Model for Irregularly Spaced Transaction Data," Econometrica, Econometric Society, vol. 66(5), pages 1127-1162, September.
    29. Zeng Tian & Swanson Norman R., 1998. "Predictive Evaluation of Econometric Forecasting Models in Commodity Futures Markets," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 2(4), pages 1-21, January.
    30. Robert Neil Collender & James A. Chalfant, 1986. "An Alternative Approach to Decisions under Uncertainty Using the Empirical Moment-Generating Function," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 68(3), pages 727-731.
    31. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    32. James W. Taylor, 2008. "A Comparison of Univariate Time Series Methods for Forecasting Intraday Arrivals at a Call Center," Management Science, INFORMS, vol. 54(2), pages 253-265, February.
    33. Franses, Philip Hans & van Dijk, Dick, 2005. "The forecasting performance of various models for seasonality and nonlinearity for quarterly industrial production," International Journal of Forecasting, Elsevier, vol. 21(1), pages 87-102.
    34. Bianchi, Lisa & Jarrett, Jeffrey & Choudary Hanumara, R., 1998. "Improving forecasting for telemarketing centers by ARIMA modeling with intervention," International Journal of Forecasting, Elsevier, vol. 14(4), pages 497-504, December.
    35. Ibrahim, Rouba & Ye, Han & L’Ecuyer, Pierre & Shen, Haipeng, 2016. "Modeling and forecasting call center arrivals: A literature survey and a case study," International Journal of Forecasting, Elsevier, vol. 32(3), pages 865-874.
    36. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    37. Garratt A. & Lee K. & Pesaran M.H. & Shin Y., 2003. "Forecast Uncertainties in Macroeconomic Modeling: An Application to the U.K. Economy," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 829-838, January.
    38. Taylor, James W., 2008. "Exponentially weighted information criteria for selecting among forecasting models," International Journal of Forecasting, Elsevier, vol. 24(3), pages 513-524.
    39. Fildes, Robert & Kumar, V., 2002. "Telecommunications demand forecasting--a review," International Journal of Forecasting, Elsevier, vol. 18(4), pages 489-522.
    40. Lawrence Brown & Noah Gans & Avishai Mandelbaum & Anat Sakov & Haipeng Shen & Sergey Zeltyn & Linda Zhao, 2005. "Statistical Analysis of a Telephone Call Center: A Queueing-Science Perspective," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 36-50, March.
    41. G. Elliott & C. Granger & A. Timmermann (ed.), 2006. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 1, number 1.
    42. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
    43. Mark W. Watson & James H. Stock, 2004. "Combination forecasts of output growth in a seven-country data set," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(6), pages 405-430.
    44. A Antipov & N Meade, 2002. "Forecasting call frequency at a financial services call centre," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 53(9), pages 953-960, September.
    45. Leitch, Gordon & Tanner, J Ernest, 1991. "Economic Forecast Evaluation: Profits versus the Conventional Error Measures," American Economic Review, American Economic Association, vol. 81(3), pages 580-590, June.
    46. Haipeng Shen & Jianhua Z. Huang, 2008. "Interday Forecasting and Intraday Updating of Call Center Arrivals," Manufacturing & Service Operations Management, INFORMS, vol. 10(3), pages 391-410, July.
    47. David E. Rapach & Jack K. Strauss, 2008. "Forecasting US employment growth using forecast combining methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 27(1), pages 75-93.
    48. Swanson, Norman R & White, Halbert, 1995. "A Model-Selection Approach to Assessing the Information in the Term Structure Using Linear Models and Artificial Neural Networks," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 265-275, July.
    49. Nabil Channouf & Pierre L’Ecuyer & Armann Ingolfsson & Athanassios Avramidis, 2007. "The application of forecasting techniques to modeling emergency medical system calls in Calgary, Alberta," Health Care Management Science, Springer, vol. 10(1), pages 25-45, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andrea Bastianin & Marzio Galeotti & Matteo Manera, 2019. "Statistical and economic evaluation of time series models for forecasting arrivals at call centers," Empirical Economics, Springer, vol. 57(3), pages 923-955, September.
    2. Andrea BASTIANIN & Marzio GALEOTTI & Matteo MANERA, 2011. "Forecast evaluation in call centers: combined forecasts, flexible loss functions and economic criteria," Departmental Working Papers 2011-08, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    3. Andrea BASTIANIN & Marzio GALEOTTI & Matteo MANERA, 2011. "Forecast evaluation in call centers: combined forecasts, flexible loss functions and economic criteria," Departmental Working Papers 2011-08, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    4. Andrea Bastianin & Marzio Galeotti & Matteo Manera, 2011. "Forecast Evaluation in Call Centers: Combined Forecasts, Flexible Loss Functions and Economic Criteria," Working Papers 20110301, Università degli Studi di Milano-Bicocca, Dipartimento di Statistica.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andrea Bastianin & Marzio Galeotti & Matteo Manera, 2019. "Statistical and economic evaluation of time series models for forecasting arrivals at call centers," Empirical Economics, Springer, vol. 57(3), pages 923-955, September.
    2. Andrea Bastianin & Marzio Galeotti & Matteo Manera, 2011. "Forecast Evaluation in Call Centers: Combined Forecasts, Flexible Loss Functions and Economic Criteria," Working Papers 20110301, Università degli Studi di Milano-Bicocca, Dipartimento di Statistica.
    3. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    4. Barrow, Devon K., 2016. "Forecasting intraday call arrivals using the seasonal moving average method," Journal of Business Research, Elsevier, vol. 69(12), pages 6088-6096.
    5. Meade, Nigel & Islam, Towhidul, 2015. "Forecasting in telecommunications and ICT—A review," International Journal of Forecasting, Elsevier, vol. 31(4), pages 1105-1126.
    6. Barrow, Devon & Kourentzes, Nikolaos, 2018. "The impact of special days in call arrivals forecasting: A neural network approach to modelling special days," European Journal of Operational Research, Elsevier, vol. 264(3), pages 967-977.
    7. Theresa Maria Rausch & Tobias Albrecht & Daniel Baier, 2022. "Beyond the beaten paths of forecasting call center arrivals: on the use of dynamic harmonic regression with predictor variables," Journal of Business Economics, Springer, vol. 92(4), pages 675-706, May.
    8. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    9. Ibrahim, Rouba & Ye, Han & L’Ecuyer, Pierre & Shen, Haipeng, 2016. "Modeling and forecasting call center arrivals: A literature survey and a case study," International Journal of Forecasting, Elsevier, vol. 32(3), pages 865-874.
    10. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    11. Graham Elliott & Allan Timmermann, 2016. "Forecasting in Economics and Finance," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 81-110, October.
    12. Taylor, James W. & Snyder, Ralph D., 2012. "Forecasting intraday time series with multiple seasonal cycles using parsimonious seasonal exponential smoothing," Omega, Elsevier, vol. 40(6), pages 748-757.
    13. Lahiri, Kajal & Yang, Liu, 2013. "Forecasting Binary Outcomes," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1025-1106, Elsevier.
    14. Albrecht, Tobias & Rausch, Theresa Maria & Derra, Nicholas Daniel, 2021. "Call me maybe: Methods and practical implementation of artificial intelligence in call center arrivals’ forecasting," Journal of Business Research, Elsevier, vol. 123(C), pages 267-278.
    15. Ricardo P. Masini & Marcelo C. Medeiros & Eduardo F. Mendes, 2023. "Machine learning advances for time series forecasting," Journal of Economic Surveys, Wiley Blackwell, vol. 37(1), pages 76-111, February.
    16. Daniel Borup & Jonas N. Eriksen & Mads M. Kjær & Martin Thyrsgaard, 2024. "Predicting Bond Return Predictability," Management Science, INFORMS, vol. 70(2), pages 931-951, February.
    17. James W. Taylor, 2008. "A Comparison of Univariate Time Series Methods for Forecasting Intraday Arrivals at a Call Center," Management Science, INFORMS, vol. 54(2), pages 253-265, February.
    18. Capistrán, Carlos & Timmermann, Allan, 2009. "Forecast Combination With Entry and Exit of Experts," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 428-440.
    19. Thomadakis, Apostolos, 2016. "Do Combination Forecasts Outperform the Historical Average? Economic and Statistical Evidence," MPRA Paper 71589, University Library of Munich, Germany.
    20. De Gooijer, Jan G. & Hyndman, Rob J., 2006. "25 years of time series forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 443-473.

    More about this item

    Keywords

    Combining forecasts; Decision making; Loss function; Seasonality; Telecommunications forecasting;
    All these keywords.

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C25 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions; Probabilities
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty
    • M15 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - IT Management

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bep:unimip:unimi-1109. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum (email available below). General contact details of provider: https://edirc.repec.org/data/damilit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.