IDEAS home Printed from https://ideas.repec.org/a/taf/japsta/v28y2001i6p759-773.html
   My bibliography  Save this article

Hits-and-misses for the evaluation and combination of forecasts

Author

Listed:
  • Thomas Wenzel

Abstract

Error measures for the evaluation of forecasts are usually based on the size of the forecast errors. Common measures are, e.g. the mean squared error (MSE), the mean absolute deviation (MAD) or the mean absolute percentage error (MAPE). Alternative measures for the comparison of forecasts are turning points or hits-and-misses, where an indicator loss function is used to decide if a forecast is of high quality or not. Here, we discuss the latter to obtain reliable combined forecasts. We apply several combination techniques to a set of German macroeconomic data. Furthermore, we perform a small simulation study for the combination of two biased forecasts.

Suggested Citation

  • Thomas Wenzel, 2001. "Hits-and-misses for the evaluation and combination of forecasts," Journal of Applied Statistics, Taylor & Francis Journals, vol. 28(6), pages 759-773.
    • Handle: RePEc:taf:japsta:v:28:y:2001:i:6:p:759-773
    DOI: 10.1080/02664760120059282
    as

    Download full text from publisher

    File URL: http://www.tandfonline.com/doi/abs/10.1080/02664760120059282
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/02664760120059282?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ed Dammers, 1993. "Measurement in the ex post evaluation of forecasts," Quality & Quantity: International Journal of Methodology, Springer, vol. 27(1), pages 31-45, February.
    2. Henriksson, Roy D & Merton, Robert C, 1981. "On Market Timing and Investment Performance. II. Statistical Procedures for Evaluating Forecasting Skills," The Journal of Business, University of Chicago Press, vol. 54(4), pages 513-533, October.
    3. Flores, Benito E., 1989. "The utilization of the Wilcoxon test to compare forecasting methods: A note," International Journal of Forecasting, Elsevier, vol. 5(4), pages 529-535.
    4. Flores, Benito E., 1986. "Use of the sign test to supplement the percentage better statistic," International Journal of Forecasting, Elsevier, vol. 2(4), pages 477-489.
    5. Wenzel, Thomas, 1998. "Pitman-closeness and the linear combination of multivariate forecasts," Technical Reports 1998,34, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    6. Wenzel, Thomas, 1999. "Combination of biased forecasts: Bias correction or bias based weights?," Technical Reports 1999,50, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    7. Cumby, Robert E. & Modest, David M., 1987. "Testing for market timing ability : A framework for forecast evaluation," Journal of Financial Economics, Elsevier, vol. 19(1), pages 169-189, September.
    8. Schneider, Carsten & Klapper, Matthias & Wenzel, Thomas, 1999. "An evaluation of forecasting methods and forecast combination methods in goods management systems," Technical Reports 1999,31, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    9. Klapper, Matthias, 1998. "Combining German macro economic forecasts using rank-based techniques," Technical Reports 1998,19, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    10. Cicarelli, James, 1982. "A new method of evaluating the accuracy of economic forecasts," Journal of Macroeconomics, Elsevier, vol. 4(4), pages 469-475.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. López Menéndez, Ana Jesús & Pérez Suárez, Rigoberto, 2017. "Forecasting Performance and Information Measures. Revisiting the M-Competition /Evaluación de Predicciones y Medidas de Información. Reexamen de la M-Competición," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 35, pages 299-314, Mayo.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wenzel, Thomas, 2000. "Hits-and-misses for the evaluation and combination of forecasts," Technical Reports 2000,26, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    2. Blaskowitz, Oliver & Herwartz, Helmut, 2014. "Testing the value of directional forecasts in the presence of serial correlation," International Journal of Forecasting, Elsevier, vol. 30(1), pages 30-42.
    3. Garcia, Philip & Irwin, Scott H. & Leuthold, Raymond M. & Yang, Li, 1997. "The value of public information in commodity futures markets," Journal of Economic Behavior & Organization, Elsevier, vol. 32(4), pages 559-570, April.
    4. Mark T. Leung & An-Sing Chen, 2005. "Performance evaluation of neural network architectures: the case of predicting foreign exchange correlations," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 24(6), pages 403-420.
    5. Lim, Terence & Lo, Andrew W. & Merton, Robert C. & Scholes, Myron S., 2006. "The Derivatives Sourcebook," Foundations and Trends(R) in Finance, now publishers, vol. 1(5–6), pages 365-572, April.
    6. Stanislav Anatolyev, 2006. "Testing for predictability (in Russian)," Quantile, Quantile, issue 1, pages 39-42, September.
    7. Barnhart, Scott W. & Giannetti, Antoine, 2009. "Negative earnings, positive earnings and stock return predictability: An empirical examination of market timing," Journal of Empirical Finance, Elsevier, vol. 16(1), pages 70-86, January.
    8. Peter Christoffersen & Francis X. Diebold, 2002. "Financial Asset Returns, Market Timing, and Volatility Dynamics," CIRANO Working Papers 2002s-02, CIRANO.
    9. repec:hum:wpaper:sfb649dp2008-073 is not listed on IDEAS
    10. Peter F. Christoffersen & Francis X. Diebold, 2006. "Financial Asset Returns, Direction-of-Change Forecasting, and Volatility Dynamics," Management Science, INFORMS, vol. 52(8), pages 1273-1287, August.
    11. Hsin-Min Lu & Chia-Shang J. Chu, 2006. "Random walk hypothesis in exchange rate reconsidered," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(4), pages 275-290.
    12. Rapach, David E. & Wohar, Mark E., 2006. "The out-of-sample forecasting performance of nonlinear models of real exchange rate behavior," International Journal of Forecasting, Elsevier, vol. 22(2), pages 341-361.
    13. Jaehun Chung & Yongmiao Hong, 2007. "Model-free evaluation of directional predictability in foreign exchange markets," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(5), pages 855-889.
    14. Helmut Herwartz & Leonardo Morales-Arias, 2009. "In-sample and out-of-sample properties of international stock return dynamics conditional on equilibrium pricing factors," The European Journal of Finance, Taylor & Francis Journals, vol. 15(1), pages 1-28.
    15. Pesaran, M. Hashem & Timmermann, Allan, 2009. "Testing Dependence Among Serially Correlated Multicategory Variables," Journal of the American Statistical Association, American Statistical Association, vol. 104(485), pages 325-337.
    16. Luigi Buzzacchi & Luca Ghezzi, 2021. "The Odds of Profitable Market Timing," JRFM, MDPI, vol. 14(6), pages 1-14, June.
    17. Aubry, Mathilde & Renou-Maissant, Patricia, 2014. "Semiconductor industry cycles: Explanatory factors and forecasting," Economic Modelling, Elsevier, vol. 39(C), pages 221-231.
    18. Stekler, H. O. & Petrei, G., 2003. "Diagnostics for evaluating the value and rationality of economic forecasts," International Journal of Forecasting, Elsevier, vol. 19(4), pages 735-742.
    19. Wenzel, Thomas, 1999. "Using different Pitman-closeness techniques for the linear combination of multivariate forecasts," Technical Reports 1999,18, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    20. Bruno Solnik, 1991. "Finance Theory and Investment Management," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 127(III), pages 303-324, September.
    21. Mark T. Leung & An‐Sing Chen & Ruben Mancha, 2009. "Making trading decisions for financial‐engineered derivatives: a novel ensemble of neural networks using information content," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 16(4), pages 257-277, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:japsta:v:28:y:2001:i:6:p:759-773. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/CJAS20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.