IDEAS home Printed from https://ideas.repec.org/a/eee/stapro/v165y2020ics0167715220301474.html
   My bibliography  Save this article

On the expected runtime of multiple testing algorithms with bounded error

Author

Listed:
  • Hahn, Georg

Abstract

Consider testing multiple hypotheses in the setting where the p-values of all hypotheses are unknown and thus have to be approximated using Monte Carlo simulations. One class of algorithms published in the literature for this scenario provides guarantees on the correctness of their testing result through the computation of confidence statements on all approximated p-values. This article focuses on the expected runtime of such algorithms and derives a variety of finite and infinite expected runtime results.

Suggested Citation

  • Hahn, Georg, 2020. "On the expected runtime of multiple testing algorithms with bounded error," Statistics & Probability Letters, Elsevier, vol. 165(C).
    • Handle: RePEc:eee:stapro:v:165:y:2020:i:c:s0167715220301474
    DOI: 10.1016/j.spl.2020.108844
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167715220301474
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.spl.2020.108844?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Axel Gandy & Georg Hahn, 2014. "MMCTest—A Safe Algorithm for Implementing Multiple Monte Carlo Tests," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(4), pages 1083-1101, December.
    2. Russell Davidson & James MacKinnon, 2000. "Bootstrap tests: how many bootstraps?," Econometric Reviews, Taylor & Francis Journals, vol. 19(1), pages 55-68.
    3. Guo Wenge & Peddada Shyamal, 2008. "Adaptive Choice of the Number of Bootstrap Samples in Large Scale Multiple Testing," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 7(1), pages 1-21, March.
    4. Silva, Ivair R. & Assunção, Renato M., 2013. "Optimal generalized truncated sequential Monte Carlo test," Journal of Multivariate Analysis, Elsevier, vol. 121(C), pages 33-49.
    5. Axel Gandy & Georg Hahn, 2016. "A Framework for Monte Carlo based Multiple Testing," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(4), pages 1046-1063, December.
    6. Donald W. K. Andrews & Moshe Buchinsky, 2000. "A Three-Step Method for Choosing the Number of Bootstrap Repetitions," Econometrica, Econometric Society, vol. 68(1), pages 23-52, January.
    7. Gandy, Axel, 2009. "Sequential Implementation of Monte Carlo Tests With Uniformly Bounded Resampling Risk," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1504-1511.
    8. van Wieringen, Wessel N & van de Wiel, Mark A & van der Vaart, Aad W, 2008. "A Test for Partial Differential Expression," Journal of the American Statistical Association, American Statistical Association, vol. 103(483), pages 1039-1049.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dong Ding & Axel Gandy & Georg Hahn, 2020. "A simple method for implementing Monte Carlo tests," Computational Statistics, Springer, vol. 35(3), pages 1373-1392, September.
    2. Georg Hahn, 2018. "Closure properties of classes of multiple testing procedures," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 102(2), pages 167-178, April.
    3. Axel Gandy & Georg Hahn & Dong Ding, 2020. "Implementing Monte Carlo tests with p‐value buckets," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 47(3), pages 950-967, September.
    4. Giuseppe Cavaliere & Heino Bohn Nielsen & Anders Rahbek, 2017. "On the Consistency of Bootstrap Testing for a Parameter on the Boundary of the Parameter Space," Journal of Time Series Analysis, Wiley Blackwell, vol. 38(4), pages 513-534, July.
    5. Guo Wenge & Peddada Shyamal, 2008. "Adaptive Choice of the Number of Bootstrap Samples in Large Scale Multiple Testing," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 7(1), pages 1-21, March.
    6. Giuseppe Cavaliere & Anders Rahbek, 2019. "A Primer On Bootstrap Testing Of Hypotheses In Time Series Models: With An Application To Double Autoregressive Models," Discussion Papers 19-03, University of Copenhagen. Department of Economics.
    7. Boswijk, H. Peter & Cavaliere, Giuseppe & Rahbek, Anders & Taylor, A.M. Robert, 2016. "Inference on co-integration parameters in heteroskedastic vector autoregressions," Journal of Econometrics, Elsevier, vol. 192(1), pages 64-85.
    8. Gospodinov, Nikolay, 2002. "Median unbiased forecasts for highly persistent autoregressive processes," Journal of Econometrics, Elsevier, vol. 111(1), pages 85-101, November.
    9. Bruggeman, Annick & Donati, Paola & Warne, Anders, 2003. "Is the demand for euro area M3 stable?," Working Paper Series 255, European Central Bank.
    10. Brian P. Poi, 2004. "From the help desk: Some bootstrapping techniques," Stata Journal, StataCorp LP, vol. 4(3), pages 312-328, September.
    11. Awartani, Basel M.A. & Corradi, Valentina, 2005. "Predicting the volatility of the S&P-500 stock index via GARCH models: the role of asymmetries," International Journal of Forecasting, Elsevier, vol. 21(1), pages 167-183.
    12. Daiki Maki & Yasushi Ota, 2021. "Testing for Time-Varying Properties Under Misspecified Conditional Mean and Variance," Computational Economics, Springer;Society for Computational Economics, vol. 57(4), pages 1167-1182, April.
    13. Matos, José M.A. & Ramos, Sandra & Costa, Vítor, 2019. "Stochastic simulated rents in Portuguese public-private partnerships," Transportation Research Part A: Policy and Practice, Elsevier, vol. 130(C), pages 107-117.
    14. Axel Gandy & Georg Hahn, 2016. "A Framework for Monte Carlo based Multiple Testing," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(4), pages 1046-1063, December.
    15. Cavaliere, Giuseppe & Nielsen, Morten Ørregaard & Taylor, A.M. Robert, 2015. "Bootstrap score tests for fractional integration in heteroskedastic ARFIMA models, with an application to price dynamics in commodity spot and futures markets," Journal of Econometrics, Elsevier, vol. 187(2), pages 557-579.
    16. Burridge, Peter & Robert Taylor, A. M., 2004. "Bootstrapping the HEGY seasonal unit root tests," Journal of Econometrics, Elsevier, vol. 123(1), pages 67-87, November.
    17. Axel Gandy & Georg Hahn, 2014. "MMCTest—A Safe Algorithm for Implementing Multiple Monte Carlo Tests," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(4), pages 1083-1101, December.
    18. Corradi, Valentina & Swanson, Norman R., 2006. "Bootstrap conditional distribution tests in the presence of dynamic misspecification," Journal of Econometrics, Elsevier, vol. 133(2), pages 779-806, August.
    19. Andrews, Donald W. K. & Buchinsky, Moshe, 2001. "Evaluation of a three-step method for choosing the number of bootstrap repetitions," Journal of Econometrics, Elsevier, vol. 103(1-2), pages 345-386, July.
    20. Daiki Maki & Yasushi Ota, 2019. "Testing for time-varying properties under misspecified conditional mean and variance," Papers 1907.12107, arXiv.org, revised Aug 2019.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:stapro:v:165:y:2020:i:c:s0167715220301474. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.