IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/1002280.html
   My bibliography  Save this article

Optimizing Experimental Design for Comparing Models of Brain Function

Author

Listed:
  • Jean Daunizeau
  • Kerstin Preuschoff
  • Karl Friston
  • Klaas Stephan

Abstract

This article presents the first attempt to formalize the optimization of experimental design with the aim of comparing models of brain function based on neuroimaging data. We demonstrate our approach in the context of Dynamic Causal Modelling (DCM), which relates experimental manipulations to observed network dynamics (via hidden neuronal states) and provides an inference framework for selecting among candidate models. Here, we show how to optimize the sensitivity of model selection by choosing among experimental designs according to their respective model selection accuracy. Using Bayesian decision theory, we (i) derive the Laplace-Chernoff risk for model selection, (ii) disclose its relationship with classical design optimality criteria and (iii) assess its sensitivity to basic modelling assumptions. We then evaluate the approach when identifying brain networks using DCM. Monte-Carlo simulations and empirical analyses of fMRI data from a simple bimanual motor task in humans serve to demonstrate the relationship between network identification and the optimal experimental design. For example, we show that deciding whether there is a feedback connection requires shorter epoch durations, relative to asking whether there is experimentally induced change in a connection that is known to be present. Finally, we discuss limitations and potential extensions of this work. Author Summary: During the past two decades, brain mapping research has undergone a paradigm switch. In addition to localizing brain regions that encode specific sensory, motor or cognitive processes, neuroimaging data is nowadays further exploited to ask questions about how information is transmitted through brain networks. The ambition here is to ask questions such as: “what is the nature of the information that region A passes on to region B”. This can be experimentally addressed by, e.g., showing that the influence that A exerts onto B depends upon specific sensory, motor or cognitive manipulations. This means one has to compare (in a statistical sense) candidate network models of the brain (with different modulations of effective connectivity, say), based on experimental data. The question we address here is how one should design the experiment in order to best discriminate such candidate models. We approach the problem from a statistical decision theoretical perspective, whereby the optimal design is the one that minimizes the model selection error rate. We demonstrate the approach using simulated and empirical data and show how it can be applied to any experimental question that can be framed as a model comparison problem.

Suggested Citation

  • Jean Daunizeau & Kerstin Preuschoff & Karl Friston & Klaas Stephan, 2011. "Optimizing Experimental Design for Comparing Models of Brain Function," PLOS Computational Biology, Public Library of Science, vol. 7(11), pages 1-18, November.
  • Handle: RePEc:plo:pcbi00:1002280
    DOI: 10.1371/journal.pcbi.1002280
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002280
    Download Restriction: no

    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1002280&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pcbi.1002280?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Will D Penny & Klaas E Stephan & Jean Daunizeau & Maria J Rosa & Karl J Friston & Thomas M Schofield & Alex P Leff, 2010. "Comparing Families of Dynamic Causal Models," PLOS Computational Biology, Public Library of Science, vol. 6(3), pages 1-14, March.
    2. Jean Daunizeau & Hanneke E M den Ouden & Matthias Pessiglione & Stefan J Kiebel & Klaas E Stephan & Karl J Friston, 2010. "Observing the Observer (I): Meta-Bayesian Models of Learning and Decision-Making," PLOS ONE, Public Library of Science, vol. 5(12), pages 1-10, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jean Daunizeau & Vincent Adam & Lionel Rigoux, 2014. "VBA: A Probabilistic Treatment of Nonlinear Models for Neurobiological and Behavioural Data," PLOS Computational Biology, Public Library of Science, vol. 10(1), pages 1-16, January.
    2. Marie Devaine & Jean Daunizeau, 2017. "Learning about and from others' prudence, impatience or laziness: The computational bases of attitude alignment," PLOS Computational Biology, Public Library of Science, vol. 13(3), pages 1-28, March.
    3. Alizée Lopez-Persem & Lionel Rigoux & Sacha Bourgeois-Gironde & Jean Daunizeau & Mathias Pessiglione, 2017. "Choose, rate or squeeze: Comparison of economic value functions elicited by different behavioral tasks," PLOS Computational Biology, Public Library of Science, vol. 13(11), pages 1-18, November.
    4. Filip Melinscak & Dominik R Bach, 2020. "Computational optimization of associative learning experiments," PLOS Computational Biology, Public Library of Science, vol. 16(1), pages 1-23, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dimitrije Marković & Jan Gläscher & Peter Bossaerts & John O’Doherty & Stefan J Kiebel, 2015. "Modeling the Evolution of Beliefs Using an Attentional Focus Mechanism," PLOS Computational Biology, Public Library of Science, vol. 11(10), pages 1-34, October.
    2. Andreea O Diaconescu & Christoph Mathys & Lilian A E Weber & Jean Daunizeau & Lars Kasper & Ekaterina I Lomakina & Ernst Fehr & Klaas E Stephan, 2014. "Inferring on the Intentions of Others by Hierarchical Bayesian Learning," PLOS Computational Biology, Public Library of Science, vol. 10(9), pages 1-19, September.
    3. Falk Lieder & Klaas E Stephan & Jean Daunizeau & Marta I Garrido & Karl J Friston, 2013. "A Neurocomputational Model of the Mismatch Negativity," PLOS Computational Biology, Public Library of Science, vol. 9(11), pages 1-14, November.
    4. Giovanni Leone & Charlotte Postel & Alison Mary & Florence Fraisse & Thomas Vallée & Fausto Viader & Vincent Sayette & Denis Peschanski & Jaques Dayan & Francis Eustache & Pierre Gagnepain, 2022. "Altered predictive control during memory suppression in PTSD," Nature Communications, Nature, vol. 13(1), pages 1-16, December.
    5. Fabien Vinckier & Lionel Rigoux & Irma T Kurniawan & Chen Hu & Sacha Bourgeois-Gironde & Jean Daunizeau & Mathias Pessiglione, 2019. "Sour grapes and sweet victories: How actions shape preferences," PLOS Computational Biology, Public Library of Science, vol. 15(1), pages 1-24, January.
    6. Jean Daunizeau & Vincent Adam & Lionel Rigoux, 2014. "VBA: A Probabilistic Treatment of Nonlinear Models for Neurobiological and Behavioural Data," PLOS Computational Biology, Public Library of Science, vol. 10(1), pages 1-16, January.
    7. L. Bonetti & G. Fernández-Rubio & F. Carlomagno & M. Dietz & D. Pantazis & P. Vuust & M. L. Kringelbach, 2024. "Spatiotemporal brain hierarchies of auditory memory recognition and predictive coding," Nature Communications, Nature, vol. 15(1), pages 1-23, December.
    8. Benjamin Patrick Evans & Mikhail Prokopenko, 2021. "A maximum entropy model of bounded rational decision-making with prior beliefs and market feedback," Papers 2102.09180, arXiv.org, revised May 2021.
    9. Moe Okayasu & Tensei Inukai & Daiki Tanaka & Kaho Tsumura & Reiko Shintaki & Masaki Takeda & Kiyoshi Nakahara & Koji Jimura, 2023. "The Stroop effect involves an excitatory–inhibitory fronto-cerebellar loop," Nature Communications, Nature, vol. 14(1), pages 1-18, December.
    10. Ahmed A Moustafa & Jony Sheynin & Catherine E Myers, 2015. "The Role of Informative and Ambiguous Feedback in Avoidance Behavior: Empirical and Computational Findings," PLOS ONE, Public Library of Science, vol. 10(12), pages 1-21, December.
    11. Sam Gijsen & Miro Grundei & Robert T Lange & Dirk Ostwald & Felix Blankenburg, 2021. "Neural surprise in somatosensory Bayesian learning," PLOS Computational Biology, Public Library of Science, vol. 17(2), pages 1-36, February.
    12. repec:dau:papers:123456789/9572 is not listed on IDEAS
    13. Benjamin Skerritt-Davis & Mounya Elhilali, 2018. "Detecting change in stochastic sound sequences," PLOS Computational Biology, Public Library of Science, vol. 14(5), pages 1-24, May.
    14. Amir Dezfouli & Bernard W Balleine, 2013. "Actions, Action Sequences and Habits: Evidence That Goal-Directed and Habitual Action Control Are Hierarchically Organized," PLOS Computational Biology, Public Library of Science, vol. 9(12), pages 1-14, December.
    15. Marie Devaine & Jean Daunizeau, 2017. "Learning about and from others' prudence, impatience or laziness: The computational bases of attitude alignment," PLOS Computational Biology, Public Library of Science, vol. 13(3), pages 1-28, March.
    16. Jean Daunizeau & Hanneke E M den Ouden & Matthias Pessiglione & Stefan J Kiebel & Karl J Friston & Klaas E Stephan, 2010. "Observing the Observer (II): Deciding When to Decide," PLOS ONE, Public Library of Science, vol. 5(12), pages 1-19, December.
    17. Marie Devaine & Guillaume Hollard & Jean Daunizeau, 2014. "The Social Bayesian Brain: Does Mentalizing Make a Difference When We Learn?," PLOS Computational Biology, Public Library of Science, vol. 10(12), pages 1-14, December.
    18. Eduardo A Aponte & Dario Schöbi & Klaas E Stephan & Jakob Heinzle, 2017. "The Stochastic Early Reaction, Inhibition, and late Action (SERIA) model for antisaccades," PLOS Computational Biology, Public Library of Science, vol. 13(8), pages 1-36, August.
    19. Payam Piray & Nathaniel D. Daw, 2021. "A model for learning based on the joint estimation of stochasticity and volatility," Nature Communications, Nature, vol. 12(1), pages 1-16, December.
    20. Melody K Morris & Julio Saez-Rodriguez & David C Clarke & Peter K Sorger & Douglas A Lauffenburger, 2011. "Training Signaling Pathway Maps to Biochemical Data with Constrained Fuzzy Logic: Quantitative Analysis of Liver Cell Responses to Inflammatory Stimuli," PLOS Computational Biology, Public Library of Science, vol. 7(3), pages 1-20, March.
    21. Alizée Lopez-Persem & Lionel Rigoux & Sacha Bourgeois-Gironde & Jean Daunizeau & Mathias Pessiglione, 2017. "Choose, rate or squeeze: Comparison of economic value functions elicited by different behavioral tasks," PLOS Computational Biology, Public Library of Science, vol. 13(11), pages 1-18, November.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1002280. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.