IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v63y2022i1d10.1007_s00362-021-01233-4.html
   My bibliography  Save this article

Quantifying the data-dredging bias in structural break tests

Author

Listed:
  • Yannick Hoga

    (University of Duisburg-Essen)

Abstract

Structural break tests are often applied as a pre-step to ensure the validity of subsequent statistical analyses. Without any a priori knowledge of the type of breaks to expect, eye-balling the data can indicate changes in some parameter, e.g., the mean. This, however, can distort the result of a structural break test for that parameter, because the data themselves suggested the hypothesis. In this paper, we formalize the eye-balling procedure and theoretically derive the implied size distortion of the structural break test. We also show that eye-balling a stretch of historical data for possible changes in a parameter does not invalidate the subsequent procedure that monitors for structural change in new incoming observations. An empirical application to Bitcoin returns shows that taking into account the data-dredging bias, which is incurred by looking at the data, can lead to different test decisions.

Suggested Citation

  • Yannick Hoga, 2022. "Quantifying the data-dredging bias in structural break tests," Statistical Papers, Springer, vol. 63(1), pages 143-155, February.
    • Handle: RePEc:spr:stpapr:v:63:y:2022:i:1:d:10.1007_s00362-021-01233-4
    DOI: 10.1007/s00362-021-01233-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-021-01233-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00362-021-01233-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Davidson, James, 1994. "Stochastic Limit Theory: An Introduction for Econometricians," OUP Catalogue, Oxford University Press, number 9780198774037.
    2. Daniel J. Benjamin & James O. Berger & Magnus Johannesson & Brian A. Nosek & E.-J. Wagenmakers & Richard Berk & Kenneth A. Bollen & Björn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Chr, 2018. "Redefine statistical significance," Nature Human Behaviour, Nature, vol. 2(1), pages 6-10, January.
      • Daniel Benjamin & James Berger & Magnus Johannesson & Brian Nosek & E. Wagenmakers & Richard Berk & Kenneth Bollen & Bjorn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Christopher Chambe, 2017. "Redefine Statistical Significance," Artefactual Field Experiments 00612, The Field Experiments Website.
    3. Carmela Quintos & Zhenhong Fan & Peter C. B. Phillips, 2001. "Structural Change Tests in Tail Behaviour and the Asian Crisis," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 68(3), pages 633-663.
    4. Y Hoga, 2018. "A structural break test for extremal dependence in β-mixing random vectors," Biometrika, Biometrika Trust, vol. 105(3), pages 627-643.
    5. Matei Demetrescu & Christoph Hanck, 2013. "Nonlinear IV panel unit root testing under structural breaks in the error variance," Statistical Papers, Springer, vol. 54(4), pages 1043-1066, November.
    6. Vogelsang, Timothy J, 1998. "Testing for a Shift in Mean without Having to Estimate Serial-Correlation Parameters," Journal of Business & Economic Statistics, American Statistical Association, vol. 16(1), pages 73-80, January.
    7. Urquhart, Andrew, 2016. "The inefficiency of Bitcoin," Economics Letters, Elsevier, vol. 148(C), pages 80-82.
    8. Dominik Wied & Matthias Arnold & Nicolai Bissantz & Daniel Ziggel, 2012. "A new fluctuation test for constant variances with applications to finance," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 75(8), pages 1111-1127, November.
    9. Robert M. De Jong & James Davidson, 2000. "Consistency of Kernel Estimators of Heteroscedastic and Autocorrelated Covariance Matrices," Econometrica, Econometric Society, vol. 68(2), pages 407-424, March.
    10. Harvey, David I. & Leybourne, Stephen J. & Taylor, A.M. Robert, 2013. "Testing for unit roots in the possible presence of multiple trend breaks using minimum Dickey–Fuller statistics," Journal of Econometrics, Elsevier, vol. 177(2), pages 265-284.
    11. Carrasco, Marine & Chen, Xiaohong, 2002. "Mixing And Moment Properties Of Various Garch And Stochastic Volatility Models," Econometric Theory, Cambridge University Press, vol. 18(1), pages 17-39, February.
    12. Badi Baltagi & Chihwa Kao & Sanggon Na, 2013. "Testing for cross-sectional dependence in a panel factor model using the wild bootstrap $$F$$ test," Statistical Papers, Springer, vol. 54(4), pages 1067-1094, November.
    13. Wied, Dominik & Krämer, Walter & Dehling, Herold, 2012. "Testing For A Change In Correlation At An Unknown Point In Time Using An Extended Functional Delta Method," Econometric Theory, Cambridge University Press, vol. 28(3), pages 570-589, June.
    14. Cheah, Eng-Tuck & Fry, John, 2015. "Speculative bubbles in Bitcoin markets? An empirical investigation into the fundamental value of Bitcoin," Economics Letters, Elsevier, vol. 130(C), pages 32-36.
    15. Ke‐Li Xu, 2015. "Testing for structural change under non‐stationary variances," Econometrics Journal, Royal Economic Society, vol. 18(2), pages 274-305, June.
    16. Rainer Böhme & Nicolas Christin & Benjamin Edelman & Tyler Moore, 2015. "Bitcoin: Economics, Technology, and Governance," Journal of Economic Perspectives, American Economic Association, vol. 29(2), pages 213-238, Spring.
    17. Giles, Judith A & Giles, David E A, 1993. "Pre-test Estimation and Testing in Econometrics: Recent Developments," Journal of Economic Surveys, Wiley Blackwell, vol. 7(2), pages 145-197, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christie Smith & Aaron Kumar, 2018. "Crypto‐Currencies – An Introduction To Not‐So‐Funny Moneys," Journal of Economic Surveys, Wiley Blackwell, vol. 32(5), pages 1531-1559, December.
    2. Parthajit Kayal & Purnima Rohilla, 2021. "Bitcoin in the economics and finance literature: a survey," SN Business & Economics, Springer, vol. 1(7), pages 1-21, July.
    3. Greg W. Hunter & Craig Kerr, 2019. "Virtual Money Illusion and the Fundamental Value of Non-Fiat Anonymous Digital Payment Methods," International Advances in Economic Research, Springer;International Atlantic Economic Society, vol. 25(2), pages 151-164, May.
    4. Dunbar, Kwamie & Owusu-Amoako, Johnson, 2022. "Cryptocurrency returns under empirical asset pricing," International Review of Financial Analysis, Elsevier, vol. 82(C).
    5. Kajtazi, Anton & Moro, Andrea, 2019. "The role of bitcoin in well diversified portfolios: A comparative global study," International Review of Financial Analysis, Elsevier, vol. 61(C), pages 143-157.
    6. Dyhrberg, Anne H. & Foley, Sean & Svec, Jiri, 2018. "How investible is Bitcoin? Analyzing the liquidity and transaction costs of Bitcoin markets," Economics Letters, Elsevier, vol. 171(C), pages 140-143.
    7. Pedro Bação & António Portugal Duarte & Helder Sebastião & Srdjan Redzepagic, 2018. "Information Transmission Between Cryptocurrencies: Does Bitcoin Rule the Cryptocurrency World?," Scientific Annals of Economics and Business (continues Analele Stiintifice), Alexandru Ioan Cuza University, Faculty of Economics and Business Administration, vol. 65(2), pages 97-117, June.
    8. Flori, Andrea, 2019. "News and subjective beliefs: A Bayesian approach to Bitcoin investments," Research in International Business and Finance, Elsevier, vol. 50(C), pages 336-356.
    9. Zhou, Siwen, 2018. "Exploring the Driving Forces of the Bitcoin Exchange Rate Dynamics: An EGARCH Approach," MPRA Paper 89445, University Library of Munich, Germany.
    10. Dominik Wied & Matthias Arnold & Nicolai Bissantz & Daniel Ziggel, 2012. "A new fluctuation test for constant variances with applications to finance," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 75(8), pages 1111-1127, November.
    11. Theodore Pelagidis & Eleftheria Kostika, 2022. "Investigating the role of central banks in the interconnection between financial markets and cryptoassets," Economia e Politica Industriale: Journal of Industrial and Business Economics, Springer;Associazione Amici di Economia e Politica Industriale, vol. 49(3), pages 481-507, September.
    12. Saji Thazhungal Govindan Nair, 2021. "On extreme value theory in the presence of technical trend: pre and post Covid-19 analysis of cryptocurrency markets," Journal of Financial Economic Policy, Emerald Group Publishing Limited, vol. 14(4), pages 533-561, December.
    13. Pape, Katharina & Wied, Dominik & Galeano, Pedro, 2016. "Monitoring multivariate variance changes," Journal of Empirical Finance, Elsevier, vol. 39(PA), pages 54-68.
    14. Merediz-Solà, Ignasi & Bariviera, Aurelio F., 2019. "A bibliometric analysis of bitcoin scientific production," Research in International Business and Finance, Elsevier, vol. 50(C), pages 294-305.
    15. Corbet, Shaen & Lucey, Brian & Urquhart, Andrew & Yarovaya, Larisa, 2019. "Cryptocurrencies as a financial asset: A systematic analysis," International Review of Financial Analysis, Elsevier, vol. 62(C), pages 182-199.
    16. Zhengyang Bao & Andreas Leibbrandt & ple391, 2019. "Thar she resurges: The case of assets that lack positive fundamental value," Monash Economics Working Papers 12-19, Monash University, Department of Economics.
    17. Yue, Yao & Li, Xuerong & Zhang, Dingxuan & Wang, Shouyang, 2021. "How cryptocurrency affects economy? A network analysis using bibliometric methods," International Review of Financial Analysis, Elsevier, vol. 77(C).
    18. Górecki, Tomasz & Horváth, Lajos & Kokoszka, Piotr, 2018. "Change point detection in heteroscedastic time series," Econometrics and Statistics, Elsevier, vol. 7(C), pages 63-88.
    19. Vidal-Tomás, David, 2022. "Which cryptocurrency data sources should scholars use?," International Review of Financial Analysis, Elsevier, vol. 81(C).
    20. Vidal-Tomás, David, 2021. "The entry and exit dynamics of the cryptocurrency market," Research in International Business and Finance, Elsevier, vol. 58(C).

    More about this item

    Keywords

    Data-dredging bias; Hypothesis test; Monitoring; Structural breaks;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:63:y:2022:i:1:d:10.1007_s00362-021-01233-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.