IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/1006157.html
   My bibliography  Save this article

Community-based benchmarking improves spike rate inference from two-photon calcium imaging data

Author

Listed:
  • Philipp Berens
  • Jeremy Freeman
  • Thomas Deneux
  • Nikolay Chenkov
  • Thomas McColgan
  • Artur Speiser
  • Jakob H Macke
  • Srinivas C Turaga
  • Patrick Mineault
  • Peter Rupprecht
  • Stephan Gerhard
  • Rainer W Friedrich
  • Johannes Friedrich
  • Liam Paninski
  • Marius Pachitariu
  • Kenneth D Harris
  • Ben Bolte
  • Timothy A Machado
  • Dario Ringach
  • Jasmine Stone
  • Luke E Rogerson
  • Nicolas J Sofroniew
  • Jacob Reimer
  • Emmanouil Froudarakis
  • Thomas Euler
  • Miroslav Román Rosón
  • Lucas Theis
  • Andreas S Tolias
  • Matthias Bethge

Abstract

In recent years, two-photon calcium imaging has become a standard tool to probe the function of neural circuits and to study computations in neuronal populations. However, the acquired signal is only an indirect measurement of neural activity due to the comparatively slow dynamics of fluorescent calcium indicators. Different algorithms for estimating spike rates from noisy calcium measurements have been proposed in the past, but it is an open question how far performance can be improved. Here, we report the results of the spikefinder challenge, launched to catalyze the development of new spike rate inference algorithms through crowd-sourcing. We present ten of the submitted algorithms which show improved performance compared to previously evaluated methods. Interestingly, the top-performing algorithms are based on a wide range of principles from deep neural networks to generative models, yet provide highly correlated estimates of the neural activity. The competition shows that benchmark challenges can drive algorithmic developments in neuroscience.Author summary: Two-photon calcium imaging is one of the major tools to study the activity of large populations of neurons in the brain. In this technique, a fluorescent calcium indicator changes its brightness when a neuron fires an action potential due to an associated increase in intracellular calcium. However, while a number of algorithms have been proposed for estimating spike rates from the measured signal, the problem is far from solved. We organized a public competition using a data set for which ground truth data was available. Participants were given a training set to develop new algorithms, and the performance of the algorithms was evaluated on a hidden test set. Here we report on the results of this competition and discuss the progress made towards better algorithms to infer spiking activity from imaging data.

Suggested Citation

  • Philipp Berens & Jeremy Freeman & Thomas Deneux & Nikolay Chenkov & Thomas McColgan & Artur Speiser & Jakob H Macke & Srinivas C Turaga & Patrick Mineault & Peter Rupprecht & Stephan Gerhard & Rainer , 2018. "Community-based benchmarking improves spike rate inference from two-photon calcium imaging data," PLOS Computational Biology, Public Library of Science, vol. 14(5), pages 1-13, May.
  • Handle: RePEc:plo:pcbi00:1006157
    DOI: 10.1371/journal.pcbi.1006157
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006157
    Download Restriction: no

    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1006157&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pcbi.1006157?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Thomas Deneux & Attila Kaszas & Gergely Szalay & Gergely Katona & Tamás Lakner & Amiram Grinvald & Balázs Rózsa & Ivo Vanzetta, 2016. "Accurate spike estimation from noisy calcium signals for ultrafast three-dimensional imaging of large neuronal populations in vivo," Nature Communications, Nature, vol. 7(1), pages 1-17, November.
    2. Johannes Friedrich & Pengcheng Zhou & Liam Paninski, 2017. "Fast online deconvolution of calcium imaging data," PLOS Computational Biology, Public Library of Science, vol. 13(3), pages 1-26, March.
    3. Tsai-Wen Chen & Trevor J. Wardill & Yi Sun & Stefan R. Pulver & Sabine L. Renninger & Amy Baohan & Eric R. Schreiter & Rex A. Kerr & Michael B. Orger & Vivek Jayaraman & Loren L. Looger & Karel Svobod, 2013. "Ultrasensitive fluorescent proteins for imaging neuronal activity," Nature, Nature, vol. 499(7458), pages 295-300, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Elaine Tring & Mario Dipoppa & Dario L. Ringach, 2023. "A power law describes the magnitude of adaptation in neural populations of primary visual cortex," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    2. Thanh-an Pham & Aleix Boquet-Pujadas & Sandip Mondal & Michael Unser & George Barbastathis, 2024. "Deep-prior ODEs augment fluorescence imaging with chemical sensors," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    3. Julien Corbo & O. Batuhan Erkat & John McClure & Hussein Khdour & Pierre-Olivier Polack, 2025. "Discretized representations in V1 predict suboptimal orientation discrimination," Nature Communications, Nature, vol. 16(1), pages 1-16, December.
    4. Yina Wei & Anirban Nandi & Xiaoxuan Jia & Joshua H. Siegle & Daniel Denman & Soo Yeun Lee & Anatoly Buchin & Werner Geit & Clayton P. Mosher & Shawn Olsen & Costas A. Anastassiou, 2023. "Associations between in vitro, in vivo and in silico cell classes in mouse primary visual cortex," Nature Communications, Nature, vol. 14(1), pages 1-20, December.
    5. Roberto Asín Achá & Dorit S. Hochbaum & Quico Spaen, 2020. "HNCcorr: combinatorial optimization for neuron identification," Annals of Operations Research, Springer, vol. 289(1), pages 5-32, June.
    6. Pierre-Marie Gardères & Sébastien Gal & Charly Rousseau & Alexandre Mamane & Dan Alin Ganea & Florent Haiss, 2024. "Coexistence of state, choice, and sensory integration coding in barrel cortex LII/III," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
    7. Tristan G. Heintz & Antonio J. Hinojosa & Sina E. Dominiak & Leon Lagnado, 2022. "Opposite forms of adaptation in mouse visual cortex are controlled by distinct inhibitory microcircuits," Nature Communications, Nature, vol. 13(1), pages 1-14, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Johannes Friedrich & Pengcheng Zhou & Liam Paninski, 2017. "Fast online deconvolution of calcium imaging data," PLOS Computational Biology, Public Library of Science, vol. 13(3), pages 1-26, March.
    2. Johannes Friedrich & Weijian Yang & Daniel Soudry & Yu Mu & Misha B Ahrens & Rafael Yuste & Darcy S Peterka & Liam Paninski, 2017. "Multi-scale approaches for high-speed imaging and analysis of large neural populations," PLOS Computational Biology, Public Library of Science, vol. 13(8), pages 1-24, August.
    3. Thanh-an Pham & Aleix Boquet-Pujadas & Sandip Mondal & Michael Unser & George Barbastathis, 2024. "Deep-prior ODEs augment fluorescence imaging with chemical sensors," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    4. Celia M. Gagliardi & Marc E. Normandin & Alexandra T. Keinath & Joshua B. Julian & Matthew R. Lopez & Manuel-Miguel Ramos-Alvarez & Russell A. Epstein & Isabel A. Muzzio, 2024. "Distinct neural mechanisms for heading retrieval and context recognition in the hippocampus during spatial reorientation," Nature Communications, Nature, vol. 15(1), pages 1-22, December.
    5. Daniel U Campos-Delgado & Omar Gutierrez-Navarro & Ricardo Salinas-Martinez & Elvis Duran & Aldo R Mejia-Rodriguez & Miguel J Velazquez-Duran & Javier A Jo, 2021. "Blind deconvolution estimation by multi-exponential models and alternated least squares approximations: Free-form and sparse approach," PLOS ONE, Public Library of Science, vol. 16(3), pages 1-29, March.
    6. Tristan G. Heintz & Antonio J. Hinojosa & Sina E. Dominiak & Leon Lagnado, 2022. "Opposite forms of adaptation in mouse visual cortex are controlled by distinct inhibitory microcircuits," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    7. Joseph D Taylor & Samuel Winnall & Alain Nogaret, 2020. "Estimation of neuron parameters from imperfect observations," PLOS Computational Biology, Public Library of Science, vol. 16(7), pages 1-22, July.
    8. Omer Mano & Damon A Clark, 2017. "Graphics Processing Unit-Accelerated Code for Computing Second-Order Wiener Kernels and Spike-Triggered Covariance," PLOS ONE, Public Library of Science, vol. 12(1), pages 1-11, January.
    9. Giovanni Diana & Thomas T J Sainsbury & Martin P Meyer, 2019. "Bayesian inference of neuronal assemblies," PLOS Computational Biology, Public Library of Science, vol. 15(10), pages 1-31, October.
    10. Caio Vaz Rimoli & Claudio Moretti & Fernando Soldevila & Enora Brémont & Cathie Ventalon & Sylvain Gigan, 2024. "Demixing fluorescence time traces transmitted by multimode fibers," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    11. Zengpeng Han & Nengsong Luo & Wenyu Ma & Xiaodong Liu & Yuxiang Cai & Jiaxin Kou & Jie Wang & Lei Li & Siqi Peng & Zihong Xu & Wen Zhang & Yuxiang Qiu & Yang Wu & Chaohui Ye & Kunzhang Lin & Fuqiang X, 2023. "AAV11 enables efficient retrograde targeting of projection neurons and enhances astrocyte-directed transduction," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    12. Daniel M. Virga & Stevie Hamilton & Bertha Osei & Abigail Morgan & Parker Kneis & Emiliano Zamponi & Natalie J. Park & Victoria L. Hewitt & David Zhang & Kevin C. Gonzalez & Fiona M. Russell & D. Grah, 2024. "Activity-dependent compartmentalization of dendritic mitochondria morphology through local regulation of fusion-fission balance in neurons in vivo," Nature Communications, Nature, vol. 15(1), pages 1-21, December.
    13. Jilt Sebastian & Mriganka Sur & Hema A Murthy & Mathew Magimai-Doss, 2021. "Signal-to-signal neural networks for improved spike estimation from calcium imaging data," PLOS Computational Biology, Public Library of Science, vol. 17(3), pages 1-19, March.
    14. Jina Yun & Simon Hansen & Otto Morris & David T. Madden & Clare Peters Libeu & Arjun J. Kumar & Cameron Wehrfritz & Aaron H. Nile & Yingnan Zhang & Lijuan Zhou & Yuxin Liang & Zora Modrusan & Michelle, 2023. "Senescent cells perturb intestinal stem cell differentiation through Ptk7 induced noncanonical Wnt and YAP signaling," Nature Communications, Nature, vol. 14(1), pages 1-19, December.
    15. Shinichiro Kira & Houman Safaai & Ari S. Morcos & Stefano Panzeri & Christopher D. Harvey, 2023. "A distributed and efficient population code of mixed selectivity neurons for flexible navigation decisions," Nature Communications, Nature, vol. 14(1), pages 1-28, December.
    16. Lloyd E. Russell & Mehmet Fişek & Zidan Yang & Lynn Pei Tan & Adam M. Packer & Henry W. P. Dalgleish & Selmaan N. Chettih & Christopher D. Harvey & Michael Häusser, 2024. "The influence of cortical activity on perception depends on behavioral state and sensory context," Nature Communications, Nature, vol. 15(1), pages 1-17, December.
    17. Teppei Matsui & Takayuki Hashimoto & Tomonari Murakami & Masato Uemura & Kohei Kikuta & Toshiki Kato & Kenichi Ohki, 2024. "Orthogonalization of spontaneous and stimulus-driven activity by hierarchical neocortical areal network in primates," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    18. Franziska Bierbuesse & Anaïs C. Bourges & Vincent Gielen & Viola Mönkemöller & Wim Vandenberg & Yi Shen & Johan Hofkens & Pieter Vanden Berghe & Robert E. Campbell & Benjamien Moeyaert & Peter Dedecke, 2022. "Absolute measurement of cellular activities using photochromic single-fluorophore biosensors and intermittent quantification," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    19. M. Angeles Rabadan & Estanislao Daniel De La Cruz & Sneha B. Rao & Yannan Chen & Cheng Gong & Gregg Crabtree & Bin Xu & Sander Markx & Joseph A. Gogos & Rafael Yuste & Raju Tomer, 2022. "An in vitro model of neuronal ensembles," Nature Communications, Nature, vol. 13(1), pages 1-17, December.
    20. Jimin Wu & Yuzhi Chen & Ashok Veeraraghavan & Eyal Seidemann & Jacob T. Robinson, 2024. "Mesoscopic calcium imaging in a head-unrestrained male non-human primate using a lensless microscope," Nature Communications, Nature, vol. 15(1), pages 1-14, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1006157. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.