IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v35y2020i3d10.1007_s00180-019-00950-7.html
   My bibliography  Save this article

Parallel computing in linear mixed models

Author

Listed:
  • Fulya Gokalp Yavuz

    (Purdue University
    Middle East Technical University)

  • Barret Schloerke

    (Purdue University)

Abstract

In this study, we propose a parallel programming method for linear mixed models (LMM) generated from big data. A commonly used algorithm, expectation maximization (EM), is preferred for its use of maximum likelihood estimations, as the estimations are stable and simple. However, EM has a high computation cost. In our proposed method, we use a divide and recombine to split the data into smaller subsets, running the algorithm steps in parallel on multiple local cores and combining the results. The proposed method is used to fit LMM with dense and sparse parameters and for large number of observations. It is faster than the classical approach and generalizes for big data. Supplementary sources for the proposed method are available in the R package lmmpar.

Suggested Citation

  • Fulya Gokalp Yavuz & Barret Schloerke, 2020. "Parallel computing in linear mixed models," Computational Statistics, Springer, vol. 35(3), pages 1273-1289, September.
    • Handle: RePEc:spr:compst:v:35:y:2020:i:3:d:10.1007_s00180-019-00950-7
    DOI: 10.1007/s00180-019-00950-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00180-019-00950-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00180-019-00950-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Fulya Gokalp Yavuz & Olcay Arslan, 2018. "Linear mixed model with Laplace distribution (LLMM)," Statistical Papers, Springer, vol. 59(1), pages 271-289, March.
    2. Wickham, Hadley, 2011. "The Split-Apply-Combine Strategy for Data Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 40(i01).
    3. Kane, Michael & Emerson, John W. & Weston, Stephen, 2013. "Scalable Strategies for Computing with Massive Data," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 55(i14).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dolejš Martin & Forejt Michal, 2019. "Franziscean Cadastre in Landscape Structure Research: A Systematic Review," Quaestiones Geographicae, Sciendo, vol. 38(1), pages 131-144, March.
    2. Ferraro, Maria Brigida, 2024. "Fuzzy k-Means: history and applications," Econometrics and Statistics, Elsevier, vol. 30(C), pages 110-123.
    3. Ravic Nijbroek & Kristin Piikki & Mats Söderström & Bas Kempen & Katrine G. Turner & Simeon Hengari & John Mutua, 2018. "Soil Organic Carbon Baselines for Land Degradation Neutrality: Map Accuracy and Cost Tradeoffs with Respect to Complexity in Otjozondjupa, Namibia," Sustainability, MDPI, vol. 10(5), pages 1-20, May.
    4. repec:grz:wpsses:2021-03 is not listed on IDEAS
    5. Miller, Christine M.F. & Waterhouse, Hannah & Harter, Thomas & Fadel, James G. & Meyer, Deanne, 2020. "Quantifying the uncertainty in nitrogen application and groundwater nitrate leaching in manure based cropping systems," Agricultural Systems, Elsevier, vol. 184(C).
    6. Sarlas, Georgios & Páez, Antonio & Axhausen, Kay W., 2020. "Betweenness-accessibility: Estimating impacts of accessibility on networks," Journal of Transport Geography, Elsevier, vol. 84(C).
    7. Marin FOTACHE & Florin DUMITRU & Valerica GREAVU-SERBAN, 2015. "An Information Systems Master Programme in Romania. Some Commonalities and Specificities," Informatica Economica, Academy of Economic Studies - Bucharest, Romania, vol. 19(3), pages 5-18.
    8. Martijn Van Heel & Dinska Van Gucht & Koen Vanbrabant & Frank Baeyens, 2017. "The Importance of Conditioned Stimuli in Cigarette and E-Cigarette Craving Reduction by E-Cigarettes," IJERPH, MDPI, vol. 14(2), pages 1-18, February.
    9. Aaron T L Lun & Hervé Pagès & Mike L Smith, 2018. "beachmat: A Bioconductor C++ API for accessing high-throughput biological data from a variety of R matrix types," PLOS Computational Biology, Public Library of Science, vol. 14(5), pages 1-15, May.
    10. Sean McKenzie & Hilary Parkinson & Jane Mangold & Mary Burrows & Selena Ahmed & Fabian Menalled, 2018. "Perceptions, Experiences, and Priorities Supporting Agroecosystem Management Decisions Differ among Agricultural Producers, Consultants, and Researchers," Sustainability, MDPI, vol. 10(11), pages 1-19, November.
    11. Milad Abbasiharofteh & Tom Broekel, 2021. "Still in the shadow of the wall? The case of the Berlin biotechnology cluster," Environment and Planning A, , vol. 53(1), pages 73-94, February.
    12. Andee J. Kaplan & Eric R. Hare, 2019. "Putting down roots: a graphical exploration of community attachment," Computational Statistics, Springer, vol. 34(4), pages 1449-1464, December.
    13. Ahmad Alsaber & Jiazhu Pan & Adeeba Al-Herz & Dhary S. Alkandary & Adeeba Al-Hurban & Parul Setiya & on behalf of the KRRD Group, 2020. "Influence of Ambient Air Pollution on Rheumatoid Arthritis Disease Activity Score Index," IJERPH, MDPI, vol. 17(2), pages 1-17, January.
    14. Kane, Michael & Emerson, John W. & Weston, Stephen, 2013. "Scalable Strategies for Computing with Massive Data," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 55(i14).
    15. Haunschild, Robin & Bornmann, Lutz, 2023. "Which papers cited which tweets? An exploratory analysis based on Scopus data," Journal of Informetrics, Elsevier, vol. 17(2).
    16. Rebecca Hong & Monica Perkins & Belinda J. Gabbe & Lincoln M. Tracy, 2022. "Comparing Peak Burn Injury Times and Characteristics in Australia and New Zealand," IJERPH, MDPI, vol. 19(15), pages 1-9, August.
    17. Yu, Lining & Härdle, Wolfgang Karl & Borke, Lukas & Benschop, Thijs, 2017. "FRM: A financial risk meter based on penalizing tail events occurrence," SFB 649 Discussion Papers 2017-003, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    18. Merl, Robert & Stöckl, Thomas & Palan, Stefan, 2023. "Insider trading regulation and shorting constraints. Evaluating the joint effects of two market interventions," Journal of Banking & Finance, Elsevier, vol. 154(C).
    19. Ioannis Politis & Ioannis Fyrogenis & Efthymis Papadopoulos & Anastasia Nikolaidou & Eleni Verani, 2020. "Shifting to Shared Wheels: Factors Affecting Dockless Bike-Sharing Choice for Short and Long Trips," Sustainability, MDPI, vol. 12(19), pages 1-25, October.
    20. Paul J McMurdie & Susan Holmes, 2014. "Waste Not, Want Not: Why Rarefying Microbiome Data Is Inadmissible," PLOS Computational Biology, Public Library of Science, vol. 10(4), pages 1-12, April.
    21. Debanjan Mukherjee & Ângelo Ferreira Chora & Jean-Christophe Lone & Ricardo S. Ramiro & Birte Blankenhaus & Karine Serre & Mário Ramirez & Isabel Gordo & Marc Veldhoen & Patrick Varga-Weisz & Maria M., 2022. "Host lung microbiota promotes malaria-associated acute respiratory distress syndrome," Nature Communications, Nature, vol. 13(1), pages 1-14, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:35:y:2020:i:3:d:10.1007_s00180-019-00950-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.