IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v184y2021i3p791-811.html
   My bibliography  Save this article

Removing the influence of group variables in high‐dimensional predictive modelling

Author

Listed:
  • Emanuele Aliverti
  • Kristian Lum
  • James E. Johndrow
  • David B. Dunson

Abstract

In many application areas, predictive models are used to support or make important decisions. There is increasing awareness that these models may contain spurious or otherwise undesirable correlations. Such correlations may arise from a variety of sources, including batch effects, systematic measurement errors or sampling bias. Without explicit adjustment, machine learning algorithms trained using these data can produce out‐of‐sample predictions which propagate these undesirable correlations. We propose a method to pre‐process the training data, producing an adjusted dataset that is statistically independent of the nuisance variables with minimum information loss. We develop a conceptually simple approach for creating an adjusted dataset in high‐dimensional settings based on a constrained form of matrix decomposition. The resulting dataset can then be used in any predictive algorithm with the guarantee that predictions will be statistically independent of the nuisance variables. We develop a scalable algorithm for implementing the method, along with theory support in the form of independence guarantees and optimality. The method is illustrated on some simulation examples and applied to two case studies: removing machine‐specific correlations from brain scan data, and removing ethnicity information from a dataset used to predict recidivism. That the motivation for removing undesirable correlations is quite different in the two applications illustrates the broad applicability of our approach.

Suggested Citation

  • Emanuele Aliverti & Kristian Lum & James E. Johndrow & David B. Dunson, 2021. "Removing the influence of group variables in high‐dimensional predictive modelling," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 791-811, July.
  • Handle: RePEc:bla:jorssa:v:184:y:2021:i:3:p:791-811
    DOI: 10.1111/rssa.12613
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssa.12613
    Download Restriction: no

    File URL: https://libkey.io/10.1111/rssa.12613?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Richard Berk & Hoda Heidari & Shahin Jabbari & Michael Kearns & Aaron Roth, 2021. "Fairness in Criminal Justice Risk Assessments: The State of the Art," Sociological Methods & Research, , vol. 50(1), pages 3-44, February.
    2. Jeffrey T Leek & John D Storey, 2007. "Capturing Heterogeneity in Gene Expression Studies by Surrogate Variable Analysis," PLOS Genetics, Public Library of Science, vol. 3(9), pages 1-12, September.
    3. Dunson, David B., 2018. "Statistics in the big data era: Failures of the machine," Statistics & Probability Letters, Elsevier, vol. 136(C), pages 4-9.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chakraborty, Tanujit & Chakraborty, Ashis Kumar & Murthy, C.A., 2019. "A nonparametric ensemble binary classifier and its statistical properties," Statistics & Probability Letters, Elsevier, vol. 149(C), pages 16-23.
    2. Lu, Xuefei & Borgonovo, Emanuele, 2023. "Global sensitivity analysis in epidemiological modeling," European Journal of Operational Research, Elsevier, vol. 304(1), pages 9-24.
    3. Arjun Bhattacharya & Anastasia N. Freedman & Vennela Avula & Rebeca Harris & Weifang Liu & Calvin Pan & Aldons J. Lusis & Robert M. Joseph & Lisa Smeester & Hadley J. Hartwell & Karl C. K. Kuban & Car, 2022. "Placental genomics mediates genetic associations with complex health traits and disease," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    4. Anna Langenberg & Shih-Chi Ma & Tatiana Ermakova & Benjamin Fabian, 2023. "Formal Group Fairness and Accuracy in Automated Decision Making," Mathematics, MDPI, vol. 11(8), pages 1-25, April.
    5. repec:jss:jstsof:40:i14 is not listed on IDEAS
    6. Won Jun Lee & Sang Cheol Kim & Jung-Ho Yoon & Sang Jun Yoon & Johan Lim & You-Sun Kim & Sung Won Kwon & Jeong Hill Park, 2016. "Meta-Analysis of Tumor Stem-Like Breast Cancer Cells Using Gene Set and Network Analysis," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-20, February.
    7. Youmi Suk & Kyung T. Han, 2024. "A Psychometric Framework for Evaluating Fairness in Algorithmic Decision Making: Differential Algorithmic Functioning," Journal of Educational and Behavioral Statistics, , vol. 49(2), pages 151-172, April.
    8. Marron, J.S., 2017. "Big Data in context and robustness against heterogeneity," Econometrics and Statistics, Elsevier, vol. 2(C), pages 73-80.
    9. Seungchul Baek & Yen‐Yi Ho & Yanyuan Ma, 2020. "Using sufficient direction factor model to analyze latent activities associated with breast cancer survival," Biometrics, The International Biometric Society, vol. 76(4), pages 1340-1350, December.
    10. Olhede, Sofia C. & Wolfe, Patrick J., 2018. "The future of statistics and data science," Statistics & Probability Letters, Elsevier, vol. 136(C), pages 46-50.
    11. Griffin, Maryclare & Hoff, Peter D., 2019. "Lasso ANOVA decompositions for matrix and tensor data," Computational Statistics & Data Analysis, Elsevier, vol. 137(C), pages 181-194.
    12. Yunfeng Li & Jarrett Morrow & Benjamin Raby & Kelan Tantisira & Scott T Weiss & Wei Huang & Weiliang Qiu, 2017. "Detecting disease-associated genomic outcomes using constrained mixture of Bayesian hierarchical models for paired data," PLOS ONE, Public Library of Science, vol. 12(3), pages 1-16, March.
    13. Zhaohui Qin & Ben Li & Karen N. Conneely & Hao Wu & Ming Hu & Deepak Ayyala & Yongseok Park & Victor X. Jin & Fangyuan Zhang & Han Zhang & Li Li & Shili Lin, 2016. "Statistical Challenges in Analyzing Methylation and Long-Range Chromosomal Interaction Data," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 8(2), pages 284-309, October.
    14. Zemin Zheng & Jinchi Lv & Wei Lin, 2021. "Nonsparse Learning with Latent Variables," Operations Research, INFORMS, vol. 69(1), pages 346-359, January.
    15. Chee Ho H’ng & Shanika L. Amarasinghe & Boya Zhang & Hojin Chang & Xinli Qu & David R. Powell & Alberto Rosello-Diez, 2024. "Compensatory growth and recovery of cartilage cytoarchitecture after transient cell death in fetal mouse limbs," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    16. Mark Reimers, 2010. "Making Informed Choices about Microarray Data Analysis," PLOS Computational Biology, Public Library of Science, vol. 6(5), pages 1-7, May.
    17. Leek Jeffrey T & Storey John D., 2011. "The Joint Null Criterion for Multiple Hypothesis Tests," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 10(1), pages 1-22, June.
    18. Federico Fioravanti & Iyad Rahwan & Fernando Abel Tohm'e, 2022. "Classes of Aggregation Rules for Ethical Decision Making in Automated Systems," Papers 2206.05160, arXiv.org, revised Jun 2023.
    19. Christos Miliotis & Yuling Ma & Xanthi-Lida Katopodi & Dimitra Karagkouni & Eleni Kanata & Kaia Mattioli & Nikolas Kalavros & Yered H. Pita-Juárez & Felipe Batalini & Varune R. Ramnarine & Shivani Nan, 2024. "Determinants of gastric cancer immune escape identified from non-coding immune-landscape quantitative trait loci," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    20. Nicoló Fusi & Oliver Stegle & Neil D Lawrence, 2012. "Joint Modelling of Confounding Factors and Prominent Genetic Regulators Provides Increased Accuracy in Genetical Genomics Studies," PLOS Computational Biology, Public Library of Science, vol. 8(1), pages 1-9, January.
    21. Sarah Friedrich & Gerd Antes & Sigrid Behr & Harald Binder & Werner Brannath & Florian Dumpert & Katja Ickstadt & Hans A. Kestler & Johannes Lederer & Heinz Leitgöb & Markus Pauly & Ansgar Steland & A, 2022. "Is there a role for statistics in artificial intelligence?," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 16(4), pages 823-846, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:184:y:2021:i:3:p:791-811. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.