IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/1006135.html
   My bibliography  Save this article

beachmat: A Bioconductor C++ API for accessing high-throughput biological data from a variety of R matrix types

Author

Listed:
  • Aaron T L Lun
  • Hervé Pagès
  • Mike L Smith

Abstract

Biological experiments involving genomics or other high-throughput assays typically yield a data matrix that can be explored and analyzed using the R programming language with packages from the Bioconductor project. Improvements in the throughput of these assays have resulted in an explosion of data even from routine experiments, which poses a challenge to the existing computational infrastructure for statistical data analysis. For example, single-cell RNA sequencing (scRNA-seq) experiments frequently generate large matrices containing expression values for each gene in each cell, requiring sparse or file-backed representations for memory-efficient manipulation in R. These alternative representations are not easily compatible with high-performance C++ code used for computationally intensive tasks in existing R/Bioconductor packages. Here, we describe a C++ interface named beachmat, which enables agnostic data access from various matrix representations. This allows package developers to write efficient C++ code that is interoperable with dense, sparse and file-backed matrices, amongst others. We evaluated the performance of beachmat for accessing data from each matrix representation using both simulated and real scRNA-seq data, and defined a clear memory/speed trade-off to motivate the choice of an appropriate representation. We also demonstrate how beachmat can be incorporated into the code of other packages to drive analyses of a very large scRNA-seq data set.

Suggested Citation

  • Aaron T L Lun & Hervé Pagès & Mike L Smith, 2018. "beachmat: A Bioconductor C++ API for accessing high-throughput biological data from a variety of R matrix types," PLOS Computational Biology, Public Library of Science, vol. 14(5), pages 1-15, May.
  • Handle: RePEc:plo:pcbi00:1006135
    DOI: 10.1371/journal.pcbi.1006135
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006135
    Download Restriction: no

    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1006135&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pcbi.1006135?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Eddelbuettel, Dirk & Sanderson, Conrad, 2014. "RcppArmadillo: Accelerating R with high-performance C++ linear algebra," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 1054-1063.
    2. Jason D. Buenrostro & Beijing Wu & Ulrike M. Litzenburger & Dave Ruff & Michael L. Gonzales & Michael P. Snyder & Howard Y. Chang & William J. Greenleaf, 2015. "Single-cell chromatin accessibility reveals principles of regulatory variation," Nature, Nature, vol. 523(7561), pages 486-490, July.
    3. Bates, Douglas & Eddelbuettel, Dirk, 2013. "Fast and Elegant Numerical Linear Algebra Using the RcppEigen Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 52(i05).
    4. Kane, Michael & Emerson, John W. & Weston, Stephen, 2013. "Scalable Strategies for Computing with Massive Data," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 55(i14).
    5. Greg Finak & Jacob Frelinger & Wenxin Jiang & Evan W Newell & John Ramey & Mark M Davis & Spyros A Kalams & Stephen C De Rosa & Raphael Gottardo, 2014. "OpenCyto: An Open Source Infrastructure for Scalable, Robust, Reproducible, and Automated, End-to-End Flow Cytometry Data Analysis," PLOS Computational Biology, Public Library of Science, vol. 10(8), pages 1-12, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jos'e Vin'icius de Miranda Cardoso & Jiaxi Ying & Daniel Perez Palomar, 2020. "Algorithms for Learning Graphs in Financial Markets," Papers 2012.15410, arXiv.org.
    2. Bogdan Oancea & Tudorel Andrei & Raluca Mariana Dragoescu, 2015. "Accelerating R with high performance linear algebra libraries," Romanian Statistical Review, Romanian Statistical Review, vol. 63(3), pages 109-117, September.
    3. Wilson J. Wright & Peter N. Neitlich & Alyssa E. Shiel & Mevin B. Hooten, 2022. "Mechanistic spatial models for heavy metal pollution," Environmetrics, John Wiley & Sons, Ltd., vol. 33(8), December.
    4. François Bachoc & Marc G Genton & Klaus Nordhausen & Anne Ruiz-Gazen & Joni Virta, 2020. "Spatial blind source separation," Biometrika, Biometrika Trust, vol. 107(3), pages 627-646.
    5. Ferraro, Maria Brigida, 2024. "Fuzzy k-Means: history and applications," Econometrics and Statistics, Elsevier, vol. 30(C), pages 110-123.
    6. Napoleón Vargas Jurado & Kent M. Eskridge & Stephen D. Kachman & Ronald M. Lewis, 2018. "Using a Bayesian Hierarchical Linear Mixing Model to Estimate Botanical Mixtures," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 23(2), pages 190-207, June.
    7. James Joseph Balamuta & Steven Andrew Culpepper, 2022. "Exploratory Restricted Latent Class Models with Monotonicity Requirements under PÒLYA–GAMMA Data Augmentation," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 903-945, September.
    8. Athanasios C. Micheas & Jiaxun Chen, 2018. "sppmix: Poisson point process modeling using normal mixture models," Computational Statistics, Springer, vol. 33(4), pages 1767-1798, December.
    9. Pötscher, Benedikt M. & Preinerstorfer, David, 2023. "How Reliable Are Bootstrap-Based Heteroskedasticity Robust Tests?," Econometric Theory, Cambridge University Press, vol. 39(4), pages 789-847, August.
    10. Martinetti, Davide & Geniaux, Ghislain, 2017. "Approximate likelihood estimation of spatial probit models," Regional Science and Urban Economics, Elsevier, vol. 64(C), pages 30-45.
    11. Andrii ROSKLADKA & Roman BAIEV, 2021. "Digitalization of data analysis tools as the key for success in the online trading markets," Access Journal, Access Press Publishing House, vol. 2(3), pages 222-233, September.
    12. Etienne Côme & Nicolas Jouvin & Pierre Latouche & Charles Bouveyron, 2021. "Hierarchical clustering with discrete latent variable models and the integrated classification likelihood," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 15(4), pages 957-986, December.
    13. Mihai C. Giurcanu, 2017. "Oracle M-Estimation for Time Series Models," Journal of Time Series Analysis, Wiley Blackwell, vol. 38(3), pages 479-504, May.
    14. Songming Tang & Xuejian Cui & Rongxiang Wang & Sijie Li & Siyu Li & Xin Huang & Shengquan Chen, 2024. "scCASE: accurate and interpretable enhancement for single-cell chromatin accessibility sequencing data," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    15. Ross J Burton & Raya Ahmed & Simone M Cuff & Sarah Baker & Andreas Artemiou & Matthias Eberl, 2021. "CytoPy: An autonomous cytometry analysis framework," PLOS Computational Biology, Public Library of Science, vol. 17(6), pages 1-21, June.
    16. Tilman M. Davies & Sudipto Banerjee & Adam P. Martin & Rose E. Turnbull, 2022. "A nearest‐neighbour Gaussian process spatial factor model for censored, multi‐depth geochemical data," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(4), pages 1014-1043, August.
    17. Michael Braun & Paul Damien, 2016. "Scalable Rejection Sampling for Bayesian Hierarchical Models," Marketing Science, INFORMS, vol. 35(3), pages 427-444, May.
    18. Jean-Jacques Forneron, 2019. "A Sieve-SMM Estimator for Dynamic Models," Papers 1902.01456, arXiv.org, revised Jan 2023.
    19. Fulya Gokalp Yavuz & Barret Schloerke, 2020. "Parallel computing in linear mixed models," Computational Statistics, Springer, vol. 35(3), pages 1273-1289, September.
    20. Yu, Lining & Härdle, Wolfgang Karl & Borke, Lukas & Benschop, Thijs, 2017. "FRM: A financial risk meter based on penalizing tail events occurrence," SFB 649 Discussion Papers 2017-003, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1006135. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.