IDEAS home Printed from https://ideas.repec.org/a/tsj/stataj/v10y2010i4p568-584.html
   My bibliography  Save this article

Frequentist q-values for multiple-test procedures

Author

Listed:
  • Roger B. Newson

    (National Heart and Lung Institute, Imperial College London)

Abstract

Multiple-test procedures are increasingly important as technology in- creases scientists’ ability to make large numbers of multiple measurements, as they do in genome scans. Multiple-test procedures were originally defined to input a vector of input p-values and an uncorrected critical p-value, interpreted as a fami- lywise error rate or a false discovery rate, and to output a corrected critical p-value and a discovery set, defined as the subset of input p-values that are at or below the corrected critical p-value. A range of multiple-test procedures is implemented us- ing the smileplot package in Stata (Newson and the ALSPAC Study Team 2003, Stata Journal 3: 109–132; 2010, Stata Journal 10: 691–692). The qqvalue com- mand uses an alternative formulation of multiple-test procedures, which is also used by the R function p.adjust. qqvalue inputs a variable of p-values and out- puts a variable of q-values that are equal in each observation to the minimum familywise error rate or false discovery rate that would result in the inclusion of the corresponding p-value in the discovery set if the specified multiple-test pro- cedure was applied to the full set of input p-values. Formulas and examples are presented. Copyright 2010 by StataCorp LP.

Suggested Citation

  • Roger B. Newson, 2010. "Frequentist q-values for multiple-test procedures," Stata Journal, StataCorp LP, vol. 10(4), pages 568-584, December.
  • Handle: RePEc:tsj:stataj:v:10:y:2010:i:4:p:568-584
    Note: to access software from within Stata, net describe http://www.stata-journal.com/software/sj10-4/st0209/
    as

    Download full text from publisher

    File URL: http://www.stata-journal.com/article.html?article=st0209
    File Function: link to article purchase
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Roger Newson & The ALSPAC Study Team, 2003. "Multiple-test procedures and smile plots," Stata Journal, StataCorp LP, vol. 3(2), pages 109-132, June.
    2. Roger Newson, 2003. "Confidence intervals and p-values for delivery to the end user," Stata Journal, StataCorp LP, vol. 3(3), pages 245-269, September.
    3. Christopher Genovese & Larry Wasserman, 2002. "Operating characteristics and extensions of the false discovery rate procedure," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(3), pages 499-517, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Naci H. Mocan & Samantha Bielen & Wim Marneffe, 2018. "Quality of Judicial Institutions, Crimes, Misdemeanors, and Dishonesty," NBER Working Papers 24396, National Bureau of Economic Research, Inc.
    2. Han Yu & Naci Mocan, 2018. "The Impact of High School Curriculum on Confidence, Academic Success, and Mental and Physical Well-Being of University Students," NBER Working Papers 24573, National Bureau of Economic Research, Inc.
    3. Mocan, Naci & Bielen, Samantha & Marneffe, Wim, 2020. "Quality of judicial institutions, crimes, misdemeanors, and dishonesty," European Journal of Political Economy, Elsevier, vol. 61(C).
    4. Huong Thu Le & Ha Trong Nguyen, 2017. "Parental health and children's cognitive and noncognitive development: New evidence from the longitudinal survey of Australian children," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1767-1788, December.
    5. Dang, Hai-Anh H & Malesky, Edmund & Nguyen, Cuong Viet, 2020. "Inequality and Support for Government Responses to COVID-19," IZA Discussion Papers 13816, Institute of Labor Economics (IZA).
    6. Dannenberg, Astrid & Weingärtner, Eva, 2023. "The effects of observability and an information nudge on food choice," Journal of Environmental Economics and Management, Elsevier, vol. 120(C).
    7. Damian Clarke & Joseph P. Romano & Michael Wolf, 2020. "The Romano–Wolf multiple-hypothesis correction in Stata," Stata Journal, StataCorp LP, vol. 20(4), pages 812-843, December.
    8. Pelin Akyol & Naci Mocan, 2023. "Education and Consanguineous Marriage," Journal of Human Capital, University of Chicago Press, vol. 17(1), pages 114-171.
    9. Mira Bierbaum & Eleonora E M Nillesen, 2021. "Sustaining the integrity of the threatened self: A cluster-randomised trial among social assistance applicants in the Netherlands," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-21, June.
    10. Leight, Jessica & Awonon, Josué & Pedehombga, Abdoulaye & Ganaba, Rasmané & Gelli, Aulo, 2022. "How light is too light touch: The effect of a short training-based intervention on household poultry production in Burkina Faso," Journal of Development Economics, Elsevier, vol. 155(C).
    11. Dannenberg, Astrid & Klatt, Charlotte & Weingärtner, Eva, 2024. "The effects of social norms and observability on food choice," Food Policy, Elsevier, vol. 125(C).
    12. Joseph Flavian Gomes, 2020. "The health costs of ethnic distance: evidence from sub-Saharan Africa," Journal of Economic Growth, Springer, vol. 25(2), pages 195-226, June.
    13. Han Yu & Naci Mocan, 2019. "The Impact of High School Curriculum on Confidence, Academic Success, and Mental and Physical Well-Being of University Students," Journal of Labor Research, Springer, vol. 40(4), pages 428-462, December.
    14. Maite D. Laméris & Pierre-Guillaume Méon & Anne-Marie Prooijen, 2023. "What have we done?! The impact of economics on the beliefs and values of business students," Journal of Business Economics, Springer, vol. 93(3), pages 433-483, April.
    15. Cotet-Grecu, Anca M., 2016. "On the estimation of the effect of stressors on birth outcomes," Economics Letters, Elsevier, vol. 147(C), pages 148-152.
    16. Martine D. Hansen & Therese H. Nøst & Eldbjørg S. Heimstad & Anita Evenset & Alexey A. Dudarev & Arja Rautio & Päivi Myllynen & Eugenia V. Dushkina & Marta Jagodic & Guttorm N. Christensen & Erik E. A, 2017. "The Impact of a Nickel-Copper Smelter on Concentrations of Toxic Elements in Local Wild Food from the Norwegian, Finnish, and Russian Border Regions," IJERPH, MDPI, vol. 14(7), pages 1-16, June.
    17. Dursun, Bahadır & Cesur, Resul & Mocan, Naci, 2018. "The Impact of Education on Health Outcomes and Behaviors in a Middle-Income, Low-Education Country," Economics & Human Biology, Elsevier, vol. 31(C), pages 94-114.
    18. Roger B. Newson, 2013. "Bonferroni and Holm approximations for Sidak and Holland–Copenhaver q-values," Stata Journal, StataCorp LP, vol. 13(2), pages 379-381, June.
    19. Hanson, Andrew & Hawley, Zackary & Martin, Hal & Liu, Bo, 2016. "Discrimination in mortgage lending: Evidence from a correspondence experiment," Journal of Urban Economics, Elsevier, vol. 92(C), pages 48-65.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roger Newson, 2008. "parmest and extensions," United Kingdom Stata Users' Group Meetings 2008 07, Stata Users Group.
    2. Roger B. Newson, 2013. "Bonferroni and Holm approximations for Sidak and Holland–Copenhaver q-values," Stata Journal, StataCorp LP, vol. 13(2), pages 379-381, June.
    3. B. Moerkerke & E. Goetghebeur & J. De Riek & I. Roldán‐Ruiz, 2006. "Significance and impotence: towards a balanced view of the null and the alternative hypotheses in marker selection for plant breeding," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(1), pages 61-79, January.
    4. Gómez-Villegas Miguel A. & Salazar Isabel & Sanz Luis, 2014. "A Bayesian decision procedure for testing multiple hypotheses in DNA microarray experiments," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 13(1), pages 49-65, February.
    5. Ben Jann, 2013. "Plotting regression coefficients and other estimates in Stata," University of Bern Social Sciences Working Papers 1, University of Bern, Department of Social Sciences, revised 18 Sep 2017.
    6. Arianna Dondi & Jacopo Lenzi & Egidio Candela & Sugitha Sureshkumar & Francesca Morigi & Carlotta Biagi & Marcello Lanari, 2022. "Social Determinants of Health and Distance Learning in Italy in the Era of the SARS-CoV-2 Pandemic," IJERPH, MDPI, vol. 19(9), pages 1-12, May.
    7. G�nther Fink & Margaret McConnell & Sebastian Vollmer, 2014. "Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction procedures," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(1), pages 44-57, January.
    8. Long Qu & Dan Nettleton & Jack C. M. Dekkers, 2012. "Improved Estimation of the Noncentrality Parameter Distribution from a Large Number of t-Statistics, with Applications to False Discovery Rate Estimation in Microarray Data Analysis," Biometrics, The International Biometric Society, vol. 68(4), pages 1178-1187, December.
    9. Gisselquist, Rachel M. & Leiderer, Stefan & Niño-Zarazúa, Miguel, 2016. "Ethnic Heterogeneity and Public Goods Provision in Zambia: Evidence of a Subnational “Diversity Dividend”," World Development, Elsevier, vol. 78(C), pages 308-323.
    10. Anti, Sebastian, 2021. "Land grabs and labor in Cambodia," Journal of Development Economics, Elsevier, vol. 149(C).
    11. Cipolli III, William & Hanson, Timothy & McLain, Alexander C., 2016. "Bayesian nonparametric multiple testing," Computational Statistics & Data Analysis, Elsevier, vol. 101(C), pages 64-79.
    12. Hofmann, Sarah & Mühlenweg, Andrea, 2018. "Learning intensity effects in students’ mental and physical health – Evidence from a large scale natural experiment in Germany," Economics of Education Review, Elsevier, vol. 67(C), pages 216-234.
    13. Dazard, Jean-Eudes & Sunil Rao, J., 2012. "Joint adaptive mean–variance regularization and variance stabilization of high dimensional data," Computational Statistics & Data Analysis, Elsevier, vol. 56(7), pages 2317-2333.
    14. Zehetmayer Sonja & Graf Alexandra C. & Posch Martin, 2015. "Sample size reassessment for a two-stage design controlling the false discovery rate," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 14(5), pages 429-442, November.
    15. Dunsch, Felipe A. & Evans, David K. & Eze-Ajoku, Ezinne & Macis, Mario, 2017. "Management, Supervision, and Health Care: A Field Experiment," IZA Discussion Papers 10967, Institute of Labor Economics (IZA).
    16. John Deke, "undated". "Causal Validity Considerations for Including High Quality Non-Experimental Evidence in Systematic Reviews," Mathematica Policy Research Reports 676a04feb19e4904a052ba2e7, Mathematica Policy Research.
    17. Michael A. Langston & Robert S. Levine & Barbara J. Kilbourne & Gary L. Rogers & Anne D. Kershenbaum & Suzanne H. Baktash & Steven S. Coughlin & Arnold M. Saxton & Vincent K. Agboto & Darryl B. Hood &, 2014. "Scalable Combinatorial Tools for Health Disparities Research," IJERPH, MDPI, vol. 11(10), pages 1-25, October.
    18. Yongqiang Tang & Subhashis Ghosal & Anindya Roy, 2007. "Nonparametric Bayesian Estimation of Positive False Discovery Rates," Biometrics, The International Biometric Society, vol. 63(4), pages 1126-1134, December.
    19. Habiger, Joshua D. & Adekpedjou, Akim, 2014. "Optimal rejection curves for exact false discovery rate control," Statistics & Probability Letters, Elsevier, vol. 94(C), pages 21-28.
    20. David R. Bickel, 2013. "Minimax-Optimal Strength of Statistical Evidence for a Composite Alternative Hypothesis," International Statistical Review, International Statistical Institute, vol. 81(2), pages 188-206, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tsj:stataj:v:10:y:2010:i:4:p:568-584. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum or Lisa Gilmore (email available below). General contact details of provider: http://www.stata-journal.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.