IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v63y2022i1d10.1007_s00362-021-01233-4.html
   My bibliography  Save this article

Quantifying the data-dredging bias in structural break tests

Author

Listed:
  • Yannick Hoga

    (University of Duisburg-Essen)

Abstract

Structural break tests are often applied as a pre-step to ensure the validity of subsequent statistical analyses. Without any a priori knowledge of the type of breaks to expect, eye-balling the data can indicate changes in some parameter, e.g., the mean. This, however, can distort the result of a structural break test for that parameter, because the data themselves suggested the hypothesis. In this paper, we formalize the eye-balling procedure and theoretically derive the implied size distortion of the structural break test. We also show that eye-balling a stretch of historical data for possible changes in a parameter does not invalidate the subsequent procedure that monitors for structural change in new incoming observations. An empirical application to Bitcoin returns shows that taking into account the data-dredging bias, which is incurred by looking at the data, can lead to different test decisions.

Suggested Citation

  • Yannick Hoga, 2022. "Quantifying the data-dredging bias in structural break tests," Statistical Papers, Springer, vol. 63(1), pages 143-155, February.
    • Handle: RePEc:spr:stpapr:v:63:y:2022:i:1:d:10.1007_s00362-021-01233-4
    DOI: 10.1007/s00362-021-01233-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-021-01233-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00362-021-01233-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Davidson, James, 1994. "Stochastic Limit Theory: An Introduction for Econometricians," OUP Catalogue, Oxford University Press, number 9780198774037.
    2. Daniel J. Benjamin & James O. Berger & Magnus Johannesson & Brian A. Nosek & E.-J. Wagenmakers & Richard Berk & Kenneth A. Bollen & Björn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Chr, 2018. "Redefine statistical significance," Nature Human Behaviour, Nature, vol. 2(1), pages 6-10, January.
      • Daniel Benjamin & James Berger & Magnus Johannesson & Brian Nosek & E. Wagenmakers & Richard Berk & Kenneth Bollen & Bjorn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Christopher Chambe, 2017. "Redefine Statistical Significance," Artefactual Field Experiments 00612, The Field Experiments Website.
    3. Badi Baltagi & Chihwa Kao & Sanggon Na, 2013. "Testing for cross-sectional dependence in a panel factor model using the wild bootstrap $$F$$ test," Statistical Papers, Springer, vol. 54(4), pages 1067-1094, November.
    4. Robert M. De Jong & James Davidson, 2000. "Consistency of Kernel Estimators of Heteroscedastic and Autocorrelated Covariance Matrices," Econometrica, Econometric Society, vol. 68(2), pages 407-424, March.
    5. Carmela Quintos & Zhenhong Fan & Peter C. B. Phillips, 2001. "Structural Change Tests in Tail Behaviour and the Asian Crisis," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 68(3), pages 633-663.
    6. Wied, Dominik & Krämer, Walter & Dehling, Herold, 2012. "Testing For A Change In Correlation At An Unknown Point In Time Using An Extended Functional Delta Method," Econometric Theory, Cambridge University Press, vol. 28(3), pages 570-589, June.
    7. Y Hoga, 2018. "A structural break test for extremal dependence in β-mixing random vectors," Biometrika, Biometrika Trust, vol. 105(3), pages 627-643.
    8. Cheah, Eng-Tuck & Fry, John, 2015. "Speculative bubbles in Bitcoin markets? An empirical investigation into the fundamental value of Bitcoin," Economics Letters, Elsevier, vol. 130(C), pages 32-36.
    9. Matei Demetrescu & Christoph Hanck, 2013. "Nonlinear IV panel unit root testing under structural breaks in the error variance," Statistical Papers, Springer, vol. 54(4), pages 1043-1066, November.
    10. Vogelsang, Timothy J, 1998. "Testing for a Shift in Mean without Having to Estimate Serial-Correlation Parameters," Journal of Business & Economic Statistics, American Statistical Association, vol. 16(1), pages 73-80, January.
    11. Urquhart, Andrew, 2016. "The inefficiency of Bitcoin," Economics Letters, Elsevier, vol. 148(C), pages 80-82.
    12. Ke‐Li Xu, 2015. "Testing for structural change under non‐stationary variances," Econometrics Journal, Royal Economic Society, vol. 18(2), pages 274-305, June.
    13. Rainer Böhme & Nicolas Christin & Benjamin Edelman & Tyler Moore, 2015. "Bitcoin: Economics, Technology, and Governance," Journal of Economic Perspectives, American Economic Association, vol. 29(2), pages 213-238, Spring.
    14. Giles, Judith A & Giles, David E A, 1993. "Pre-test Estimation and Testing in Econometrics: Recent Developments," Journal of Economic Surveys, Wiley Blackwell, vol. 7(2), pages 145-197, June.
    15. Dominik Wied & Matthias Arnold & Nicolai Bissantz & Daniel Ziggel, 2012. "A new fluctuation test for constant variances with applications to finance," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 75(8), pages 1111-1127, November.
    16. Harvey, David I. & Leybourne, Stephen J. & Taylor, A.M. Robert, 2013. "Testing for unit roots in the possible presence of multiple trend breaks using minimum Dickey–Fuller statistics," Journal of Econometrics, Elsevier, vol. 177(2), pages 265-284.
    17. Carrasco, Marine & Chen, Xiaohong, 2002. "Mixing And Moment Properties Of Various Garch And Stochastic Volatility Models," Econometric Theory, Cambridge University Press, vol. 18(1), pages 17-39, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christie Smith & Aaron Kumar, 2018. "Crypto‐Currencies – An Introduction To Not‐So‐Funny Moneys," Journal of Economic Surveys, Wiley Blackwell, vol. 32(5), pages 1531-1559, December.
    2. Parthajit Kayal & Purnima Rohilla, 2021. "Bitcoin in the economics and finance literature: a survey," SN Business & Economics, Springer, vol. 1(7), pages 1-21, July.
    3. Greg W. Hunter & Craig Kerr, 2019. "Virtual Money Illusion and the Fundamental Value of Non-Fiat Anonymous Digital Payment Methods," International Advances in Economic Research, Springer;International Atlantic Economic Society, vol. 25(2), pages 151-164, May.
    4. Aurelio F. Bariviera & Ignasi Merediz‐Solà, 2021. "Where Do We Stand In Cryptocurrencies Economic Research? A Survey Based On Hybrid Analysis," Journal of Economic Surveys, Wiley Blackwell, vol. 35(2), pages 377-407, April.
    5. Dunbar, Kwamie & Owusu-Amoako, Johnson, 2022. "Cryptocurrency returns under empirical asset pricing," International Review of Financial Analysis, Elsevier, vol. 82(C).
    6. Kajtazi, Anton & Moro, Andrea, 2019. "The role of bitcoin in well diversified portfolios: A comparative global study," International Review of Financial Analysis, Elsevier, vol. 61(C), pages 143-157.
    7. Khanh Hoang & Cuong C. Nguyen & Kongchheng Poch & Thang X. Nguyen, 2020. "Does Bitcoin Hedge Commodity Uncertainty?," JRFM, MDPI, vol. 13(6), pages 1-14, June.
    8. Ishtiaq Ahmad Bajwa & Shafiq Ur Rehman & Abid Iqbal & Zaheer Anwer & Murtaza Ashiq & Muhammad Ajmal Khan, 2022. "Past, Present and Future of FinTech Research: A Bibliometric Analysis," SAGE Open, , vol. 12(4), pages 21582440221, October.
    9. Dyhrberg, Anne H. & Foley, Sean & Svec, Jiri, 2018. "How investible is Bitcoin? Analyzing the liquidity and transaction costs of Bitcoin markets," Economics Letters, Elsevier, vol. 171(C), pages 140-143.
    10. Pedro Bação & António Portugal Duarte & Helder Sebastião & Srdjan Redzepagic, 2018. "Information Transmission Between Cryptocurrencies: Does Bitcoin Rule the Cryptocurrency World?," Scientific Annals of Economics and Business (continues Analele Stiintifice), Alexandru Ioan Cuza University, Faculty of Economics and Business Administration, vol. 65(2), pages 97-117, June.
    11. Flori, Andrea, 2019. "News and subjective beliefs: A Bayesian approach to Bitcoin investments," Research in International Business and Finance, Elsevier, vol. 50(C), pages 336-356.
    12. ORĂȘTEAN Ramona & MĂRGINEAN Silvia Cristina & SAVA Raluca, 2019. "Bitcoin In The Scientific Literature – A Bibliometric Study," Studies in Business and Economics, Lucian Blaga University of Sibiu, Faculty of Economic Sciences, vol. 14(3), pages 160-174, December.
    13. Zhou, Siwen, 2018. "Exploring the Driving Forces of the Bitcoin Exchange Rate Dynamics: An EGARCH Approach," MPRA Paper 89445, University Library of Munich, Germany.
    14. Fang, Wen & Tian, Shaolin & Wang, Jun, 2018. "Multiscale fluctuations and complexity synchronization of Bitcoin in China and US markets," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 512(C), pages 109-120.
    15. Lennart Ante, 2020. "A place next to Satoshi: foundations of blockchain and cryptocurrency research in business and economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1305-1333, August.
    16. Wied, Dominik & Dehling, Herold & van Kampen, Maarten & Vogel, Daniel, 2014. "A fluctuation test for constant Spearman’s rho with nuisance-free limit distribution," Computational Statistics & Data Analysis, Elsevier, vol. 76(C), pages 723-736.
    17. Dominik Wied & Matthias Arnold & Nicolai Bissantz & Daniel Ziggel, 2012. "A new fluctuation test for constant variances with applications to finance," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 75(8), pages 1111-1127, November.
    18. Theodore Pelagidis & Eleftheria Kostika, 2022. "Investigating the role of central banks in the interconnection between financial markets and cryptoassets," Economia e Politica Industriale: Journal of Industrial and Business Economics, Springer;Associazione Amici di Economia e Politica Industriale, vol. 49(3), pages 481-507, September.
    19. Symitsi, Efthymia & Chalvatzis, Konstantinos J., 2019. "The economic value of Bitcoin: A portfolio analysis of currencies, gold, oil and stocks," Research in International Business and Finance, Elsevier, vol. 48(C), pages 97-110.
    20. Lambrecht, Maxime & Larue, Louis, 2018. "After the (virtual) gold rush: is Bitcoin more than a speculative bubble?," Internet Policy Review: Journal on Internet Regulation, Alexander von Humboldt Institute for Internet and Society (HIIG), Berlin, vol. 7(4), pages 1-22.

    More about this item

    Keywords

    Data-dredging bias; Hypothesis test; Monitoring; Structural breaks;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:63:y:2022:i:1:d:10.1007_s00362-021-01233-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.