IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v13y2022i1d10.1038_s41467-022-31347-8.html
   My bibliography  Save this article

A guided multiverse study of neuroimaging analyses

Author

Listed:
  • Jessica Dafflon

    (King’s College London)

  • Pedro F. Da Costa

    (King’s College London
    Birkbeck College)

  • František Váša

    (King’s College London)

  • Ricardo Pio Monti

    (University College London)

  • Danilo Bzdok

    (McGill University
    Mila - Quebec Artificial Intelligence Institute)

  • Peter J. Hellyer

    (King’s College London)

  • Federico Turkheimer

    (King’s College London)

  • Jonathan Smallwood

    (Queen’s University)

  • Emily Jones

    (Birkbeck College)

  • Robert Leech

    (King’s College London)

Abstract

For most neuroimaging questions the range of possible analytic choices makes it unclear how to evaluate conclusions from any single analytic method. One possible way to address this issue is to evaluate all possible analyses using a multiverse approach, however, this can be computationally challenging and sequential analyses on the same data can compromise predictive power. Here, we establish how active learning on a low-dimensional space capturing the inter-relationships between pipelines can efficiently approximate the full spectrum of analyses. This approach balances the benefits of a multiverse analysis without incurring the cost on computational and predictive power. We illustrate this approach with two functional MRI datasets (predicting brain age and autism diagnosis) demonstrating how a multiverse of analyses can be efficiently navigated and mapped out using active learning. Furthermore, our presented approach not only identifies the subset of analysis techniques that are best able to predict age or classify individuals with autism spectrum disorder and healthy controls, but it also allows the relationships between analyses to be quantified.

Suggested Citation

  • Jessica Dafflon & Pedro F. Da Costa & František Váša & Ricardo Pio Monti & Danilo Bzdok & Peter J. Hellyer & Federico Turkheimer & Jonathan Smallwood & Emily Jones & Robert Leech, 2022. "A guided multiverse study of neuroimaging analyses," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
  • Handle: RePEc:nat:natcom:v:13:y:2022:i:1:d:10.1038_s41467-022-31347-8
    DOI: 10.1038/s41467-022-31347-8
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-022-31347-8
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-022-31347-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Mikail Rubinov, 2016. "Constraints and spandrels of interareal connectomes," Nature Communications, Nature, vol. 7(1), pages 1-11, December.
    2. J. Kruskal, 1964. "Nonmetric multidimensional scaling: A numerical method," Psychometrika, Springer;The Psychometric Society, vol. 29(2), pages 115-129, June.
    3. Marc-Andre Schulz & B. T. Thomas Yeo & Joshua T. Vogelstein & Janaina Mourao-Miranada & Jakob N. Kather & Konrad Kording & Blake Richards & Danilo Bzdok, 2020. "Different scaling of linear models and deep learning in UKBiobank brain images versus machine-learning datasets," Nature Communications, Nature, vol. 11(1), pages 1-15, December.
    4. Matthew F. Glasser & Timothy S. Coalson & Emma C. Robinson & Carl D. Hacker & John Harwell & Essa Yacoub & Kamil Ugurbil & Jesper Andersson & Christian F. Beckmann & Mark Jenkinson & Stephen M. Smith , 2016. "A multi-modal parcellation of human cerebral cortex," Nature, Nature, vol. 536(7615), pages 171-178, August.
    5. Ricardo Pio Monti & Alex Gibberd & Sandipan Roy & Matthew Nunes & Romy Lorenz & Robert Leech & Takeshi Ogawa & Motoaki Kawanabe & Aapo Hyvärinen, 2020. "Interpretable brain age prediction using linear latent variable models of functional connectivity," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-25, June.
    6. Romy Lorenz & Ines R. Violante & Ricardo Pio Monti & Giovanni Montana & Adam Hampshire & Robert Leech, 2018. "Dissociating frontoparietal brain networks with neuroadaptive Bayesian optimization," Nature Communications, Nature, vol. 9(1), pages 1-14, December.
    7. Rotem Botvinik-Nezer & Felix Holzmeister & Colin F. Camerer & Anna Dreber & Juergen Huber & Magnus Johannesson & Michael Kirchler & Roni Iwanir & Jeanette A. Mumford & R. Alison Adcock & Paolo Avesani, 2020. "Variability in the analysis of a single neuroimaging dataset by many teams," Nature, Nature, vol. 582(7810), pages 84-88, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andrea I. Luppi & Helena M. Gellersen & Zhen-Qi Liu & Alexander R. D. Peattie & Anne E. Manktelow & Ram Adapa & Adrian M. Owen & Lorina Naci & David K. Menon & Stavros I. Dimitriadis & Emmanuel A. Sta, 2024. "Systematic evaluation of fMRI data-processing pipelines for consistent functional connectomics," Nature Communications, Nature, vol. 15(1), pages 1-24, December.
    2. Audrey C. Luo & Valerie J. Sydnor & Adam Pines & Bart Larsen & Aaron F. Alexander-Bloch & Matthew Cieslak & Sydney Covitz & Andrew A. Chen & Nathalia Bianchini Esper & Eric Feczko & Alexandre R. Franc, 2024. "Functional connectivity development along the sensorimotor-association axis enhances the cortical hierarchy," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    3. Beniaich, Adnane & Guimarães, Danielle Vieira & Avanzi, Junior Cesar & Silva, Bruno Montoani & Acuña-Guzman, Salvador Francisco & dos Santos, Wharley Pereira & Silva, Marx Leandro Naves, 2023. "Spontaneous vegetation as an alternative to cover crops in olive orchards reduces water erosion and improves soil physical properties under tropical conditions," Agricultural Water Management, Elsevier, vol. 279(C).
    4. Giuseppe Arbia & Giovanni Lafratta, 2002. "Anisotropic spatial sampling designs for urban pollution," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 51(2), pages 223-234, May.
    5. Samuel Shye, 2010. "The Motivation to Volunteer: A Systemic Quality of Life Theory," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 98(2), pages 183-200, September.
    6. Muñoz-Mas, Rafael & Vezza, Paolo & Alcaraz-Hernández, Juan Diego & Martínez-Capel, Francisco, 2016. "Risk of invasion predicted with support vector machines: A case study on northern pike (Esox Lucius, L.) and bleak (Alburnus alburnus, L.)," Ecological Modelling, Elsevier, vol. 342(C), pages 123-134.
    7. Christoph Huber & Christian König-Kersting & Matteo M. Marini, 2022. "Experimenting with Financial Professionals," Working Papers 2022-07, Faculty of Economics and Statistics, Universität Innsbruck, revised Jun 2024.
    8. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    9. Dennis Bontempi & Leonard Nuernberg & Suraj Pai & Deepa Krishnaswamy & Vamsi Thiriveedhi & Ahmed Hosny & Raymond H. Mak & Keyvan Farahani & Ron Kikinis & Andrey Fedorov & Hugo J. W. L. Aerts, 2024. "End-to-end reproducible AI pipelines in radiology using the cloud," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    10. Simensen, Trond & Halvorsen, Rune & Erikstad, Lars, 2018. "Methods for landscape characterisation and mapping: A systematic review," Land Use Policy, Elsevier, vol. 75(C), pages 557-569.
    11. Silvia Vilčeková & Ilija Zoran Apostoloski & Ľudmila Mečiarová & Eva Krídlová Burdová & Jozef Kiseľák, 2017. "Investigation of Indoor Air Quality in Houses of Macedonia," IJERPH, MDPI, vol. 14(1), pages 1-12, January.
    12. Matthew Rosenblatt & Link Tejavibulya & Rongtao Jiang & Stephanie Noble & Dustin Scheinost, 2024. "Data leakage inflates prediction performance in connectome-based machine learning models," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    13. Funk, Patrick & Davis, Alex & Vaishnav, Parth & Dewitt, Barry & Fuchs, Erica, 2020. "Individual inconsistency and aggregate rationality: Overcoming inconsistencies in expert judgment at the technical frontier," Technological Forecasting and Social Change, Elsevier, vol. 155(C).
    14. Leon D. Lotter & Amin Saberi & Justine Y. Hansen & Bratislav Misic & Casey Paquola & Gareth J. Barker & Arun L. W. Bokde & Sylvane Desrivières & Herta Flor & Antoine Grigis & Hugh Garavan & Penny Gowl, 2024. "Regional patterns of human cortex development correlate with underlying neurobiology," Nature Communications, Nature, vol. 15(1), pages 1-21, December.
    15. Haewon Nam & Chongwon Pae & Jinseok Eo & Maeng-Keun Oh & Hae-Jeong Park, 2021. "Inter-species cortical registration between macaques and humans using a functional network property under a spherical demons framework," PLOS ONE, Public Library of Science, vol. 16(10), pages 1-22, October.
    16. Moris Triventi, 2014. "Higher education regimes: an empirical classification of higher education systems and its relationship with student accessibility," Quality & Quantity: International Journal of Methodology, Springer, vol. 48(3), pages 1685-1703, May.
    17. Karim Abou-Moustafa & Frank P. Ferrie, 2018. "Local generalized quadratic distance metrics: application to the k-nearest neighbors classifier," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 12(2), pages 341-363, June.
    18. Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," Ruhr Economic Papers 1055, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    19. Camacho, Maximo & Perez-Quiros, Gabriel & Saiz, Lorena, 2006. "Are European business cycles close enough to be just one?," Journal of Economic Dynamics and Control, Elsevier, vol. 30(9-10), pages 1687-1706.
    20. Daria E. A. Jensen & Klaus P. Ebmeier & Sana Suri & Matthew F. S. Rushworth & Miriam C. Klein-Flügge, 2024. "Nuclei-specific hypothalamus networks predict a dimensional marker of stress in humans," Nature Communications, Nature, vol. 15(1), pages 1-19, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:13:y:2022:i:1:d:10.1038_s41467-022-31347-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.