IDEAS home Printed from https://ideas.repec.org/a/jof/jforec/v24y2005i6p403-420.html
   My bibliography  Save this article

Performance evaluation of neural network architectures: the case of predicting foreign exchange correlations

Author

Listed:
  • Mark T. Leung

    (University of Texas at San Antonio, USA)

  • An-Sing Chen

    (National Chung Cheng University, Taiwan)

Abstract

In the last decade, neural networks have emerged from an esoteric instrument in academic research to a rather common tool assisting auditors, investors, portfolio managers and investment advisors in making critical financial decisions. It is apparent that a better understanding of the network's performance and limitations would help both researchers and practitioners in analysing real-world problems. Unlike many existing studies which focus on a single type of network architecture, this study evaluates and compares the performance of models based on two competing neural network architectures, the multi-layered feedforward neural network (MLFN) and general regression neural network (GRNN). Our empirical evaluation measures the network models' strength on the prediction of currency exchange correlation with respect to a variety of statistical tests including RMSE, MAE, U statistic, Theil's decomposition test, Henriksson-Merton market timing test and Fair-Shiller informational content test. Results of experiments suggest that the selection of proper architectural design may contribute directly to the success in neural network forecasting. In addition, market timing tests indicate that both MLFN and GRNN models have economically significant values in predicting the exchange rate correlation. On the other hand, informational content tests discover that the neural network models based on different architectures capture useful information not found in each other and the information sets captured by the two network designs are independent of one another. An auxiliary experiment is developed and confirms the possible synergetic effect from combining forecasts made by the two different network architectures and from incorporating information from an implied correlation model into the neural network forecasts. Implied correlation and random walk models are also included in our empirical experiment for benchmark comparison. Copyright © 2005 John Wiley & Sons, Ltd.

Suggested Citation

  • Mark T. Leung & An-Sing Chen, 2005. "Performance evaluation of neural network architectures: the case of predicting foreign exchange correlations," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 24(6), pages 403-420.
    • Handle: RePEc:jof:jforec:v:24:y:2005:i:6:p:403-420
    DOI: 10.1002/for.967
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1002/for.967
    File Function: Link to full text; subscription required
    Download Restriction: no
    File URL: https://libkey.io/10.1002/for.967?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jose A. Lopez & Christian Walter, 1997. "Is implied correlation worth calculating? Evidence from foreign exchange options and historical data," Research Paper 9730, Federal Reserve Bank of New York.
    2. Henriksson, Roy D & Merton, Robert C, 1981. "On Market Timing and Investment Performance. II. Statistical Procedures for Evaluating Forecasting Skills," The Journal of Business, University of Chicago Press, vol. 54(4), pages 513-533, October.
    3. Hian Koh & Sen Tan, 1999. "A neural network approach to the prediction of going concern status," Accounting and Business Research, Taylor & Francis Journals, vol. 29(3), pages 211-216.
    4. Cumby, Robert E. & Modest, David M., 1987. "Testing for market timing ability : A framework for forecast evaluation," Journal of Financial Economics, Elsevier, vol. 19(1), pages 169-189, September.
    5. Kar Yan Tam & Melody Y. Kiang, 1992. "Managerial Applications of Neural Networks: The Case of Bank Failure Predictions," Management Science, INFORMS, vol. 38(7), pages 926-947, July.
    6. Campa, Jose Manuel & Chang, P. H. Kevin, 1998. "The forecasting ability of correlations implied in foreign exchange options," Journal of International Money and Finance, Elsevier, vol. 17(6), pages 855-880, December.
    7. Altman, Edward I. & Marco, Giancarlo & Varetto, Franco, 1994. "Corporate distress diagnosis: Comparisons using linear discriminant analysis and neural networks (the Italian experience)," Journal of Banking & Finance, Elsevier, vol. 18(3), pages 505-529, May.
    8. Merton, Robert C, 1981. "On Market Timing and Investment Performance. I. An Equilibrium Theory of Value for Market Forecasts," The Journal of Business, University of Chicago Press, vol. 54(3), pages 363-406, July.
    9. Alpaydin, Ethem & Altinel, I. Kuban & Aras, Necati, 1996. "Parametric distance functions vs. nonparametric neural networks for estimating road travel distances," European Journal of Operational Research, Elsevier, vol. 93(2), pages 230-243, September.
    10. Wittkemper, Hans-Georg & Steiner, Manfred, 1996. "Using neural networks to forecast the systematic risk of stocks," European Journal of Operational Research, Elsevier, vol. 90(3), pages 577-588, May.
    11. Fair, Ray C & Shiller, Robert J, 1990. "Comparing Information in Forecasts from Econometric Models," American Economic Review, American Economic Association, vol. 80(3), pages 375-389, June.
    12. Udo Broll & Kit Pong Wong, 1999. "Hedging with mismatched currencies," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 19(8), pages 859-875, December.
    13. Longin, Francois & Solnik, Bruno, 1995. "Is the correlation in international equity returns constant: 1960-1990?," Journal of International Money and Finance, Elsevier, vol. 14(1), pages 3-26, February.
    14. Iebeling Kaastra & Milton S. Boyd, 1995. "Forecasting futures trading volume using neural networks," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 15(8), pages 953-970, December.
    15. De Jong, Frank & Mahieu, Ronald & Schotman, Peter, 1998. "Price discovery in the foreign exchange market: an empirical analysis of the yen/dmark rate1, 2," Journal of International Money and Finance, Elsevier, vol. 17(1), pages 5-27, February.
    16. Leitch, Gordon & Tanner, J Ernest, 1991. "Economic Forecast Evaluation: Profits versus the Conventional Error Measures," American Economic Review, American Economic Association, vol. 81(3), pages 580-590, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Oscar Claveria & Enric Monte & Salvador Torra, 2017. "“Regional tourism demand forecasting with machine learning models: Gaussian process regression vs. neural network models in a multiple-input multiple-output setting"," IREA Working Papers 201701, University of Barcelona, Research Institute of Applied Economics, revised Jan 2017.
    2. Marek Vochozka & Jakub Horák & Petr Šuleř, 2019. "Equalizing Seasonal Time Series Using Artificial Neural Networks in Predicting the Euro–Yuan Exchange Rate," JRFM, MDPI, vol. 12(2), pages 1-17, April.
    3. Parisi, Antonino & Parisi, Franco & Díaz, David, 2008. "Forecasting gold price changes: Rolling and recursive neural network models," Journal of Multinational Financial Management, Elsevier, vol. 18(5), pages 477-487, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peter Christoffersen & Francis X. Diebold, 2002. "Financial Asset Returns, Market Timing, and Volatility Dynamics," CIRANO Working Papers 2002s-02, CIRANO.
    2. Francis X. Diebold & Jose A. Lopez, 1995. "Forecast evaluation and combination," Research Paper 9525, Federal Reserve Bank of New York.
    3. Peter F. Christoffersen & Francis X. Diebold, 2006. "Financial Asset Returns, Direction-of-Change Forecasting, and Volatility Dynamics," Management Science, INFORMS, vol. 52(8), pages 1273-1287, August.
    4. Mark T. Leung & An‐Sing Chen & Ruben Mancha, 2009. "Making trading decisions for financial‐engineered derivatives: a novel ensemble of neural networks using information content," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 16(4), pages 257-277, October.
    5. Sandrine Lardic & Auguste Mpacko Priso, 1999. "Une comparaison des prévisions des experts à celles issues des modèles B VAR," Économie et Prévision, Programme National Persée, vol. 140(4), pages 161-180.
    6. Jaehun Chung & Yongmiao Hong, 2013. "Model-Free Evaluation of Directional Predictability in Foreign Exchange," Working Papers 2013-10-14, Wang Yanan Institute for Studies in Economics (WISE), Xiamen University.
    7. McCracken,M.W. & West,K.D., 2001. "Inference about predictive ability," Working papers 14, Wisconsin Madison - Social Systems.
    8. Greer, Mark, 2003. "Directional accuracy tests of long-term interest rate forecasts," International Journal of Forecasting, Elsevier, vol. 19(2), pages 291-298.
    9. repec:wyi:journl:002068 is not listed on IDEAS
    10. Bespalova, Olga, 2018. "Forecast Evaluation in Macroeconomics and International Finance. Ph.D. thesis, George Washington University, Washington, DC, USA," MPRA Paper 117706, University Library of Munich, Germany.
    11. Garcia, Philip & Irwin, Scott H. & Leuthold, Raymond M. & Yang, Li, 1997. "The value of public information in commodity futures markets," Journal of Economic Behavior & Organization, Elsevier, vol. 32(4), pages 559-570, April.
    12. Lim, Terence & Lo, Andrew W. & Merton, Robert C. & Scholes, Myron S., 2006. "The Derivatives Sourcebook," Foundations and Trends(R) in Finance, now publishers, vol. 1(5–6), pages 365-572, April.
    13. Baghestani, Hamid & Toledo, Hugo, 2017. "Do analysts' forecasts of term spread differential help predict directional change in exchange rates?," International Review of Economics & Finance, Elsevier, vol. 47(C), pages 62-69.
    14. repec:hum:wpaper:sfb649dp2008-073 is not listed on IDEAS
    15. Christodoulakis, George A., 2007. "Common volatility and correlation clustering in asset returns," European Journal of Operational Research, Elsevier, vol. 182(3), pages 1263-1284, November.
    16. T. Hendricks & B. Kempa & C. Pierdzioch, 2010. "Do local analysts have an informational advantage in forecasting stock returns? Evidence from the German DAX30," Financial Markets and Portfolio Management, Springer;Swiss Society for Financial Market Research, vol. 24(2), pages 137-158, June.
    17. Jaehun Chung & Yongmiao Hong, 2007. "Model-free evaluation of directional predictability in foreign exchange markets," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(5), pages 855-889.
    18. Norman R. Swanson & Halbert White, 1997. "A Model Selection Approach To Real-Time Macroeconomic Forecasting Using Linear Models And Artificial Neural Networks," The Review of Economics and Statistics, MIT Press, vol. 79(4), pages 540-550, November.
    19. Misik, Sándor, 2023. "Korrelációbecslés a forintpiacon [Correlation forecasting on the Hungarian forint market]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(7), pages 772-794.
    20. Stekler, H. O. & Petrei, G., 2003. "Diagnostics for evaluating the value and rationality of economic forecasts," International Journal of Forecasting, Elsevier, vol. 19(4), pages 735-742.
    21. Hamid Baghestani, 2017. "Do US consumer survey data help beat the random walk in forecasting mortgage rates?," Cogent Economics & Finance, Taylor & Francis Journals, vol. 5(1), pages 1343017-134, January.
    22. Mr. Aasim M. Husain & Chakriya Bowman, 2004. "Forecasting Commodity Prices: Futures Versus Judgment," IMF Working Papers 2004/041, International Monetary Fund.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jof:jforec:v:24:y:2005:i:6:p:403-420. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley-Blackwell Digital Licensing or Christopher F. Baum (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/2966 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.