IDEAS home Printed from https://ideas.repec.org/a/jss/jstsof/v077i13.html
   My bibliography  Save this article

Software for Distributed Computation on Medical Databases: A Demonstration Project

Author

Listed:
  • Narasimhan, Balasubramanian
  • Rubin, Daniel L.
  • Gross, Samuel M.
  • Bendersky, Marina
  • Lavori, Philip W.

Abstract

Bringing together the information latent in distributed medical databases promises to personalize medical care by enabling reliable, stable modeling of outcomes with rich feature sets (including patient characteristics and treatments received). However, there are barriers to aggregation of medical data, due to lack of standardization of ontologies, privacy concerns, proprietary attitudes toward data, and a reluctance to give up control over end use. Aggregation of data is not always necessary for model fitting. In models based on maximizing a likelihood, the computations can be distributed, with aggregation limited to the intermediate results of calculations on local data, rather than raw data. Distributed fitting is also possible for singular value decomposition. There has been work on the technical aspects of shared computation for particular applications, but little has been published on the software needed to support the "social networking" aspect of shared computing, to reduce the barriers to collaboration. We describe a set of software tools that allow the rapid assembly of a collaborative computational project, based on the flexible and extensible R statistical software and other open source packages, that can work across a heterogeneous collection of database environments, with full transparency to allow local officials concerned with privacy protections to validate the safety of the method. We describe the principles, architecture, and successful test results for the site-stratified Cox model and rank-k singular value decomposition.

Suggested Citation

  • Narasimhan, Balasubramanian & Rubin, Daniel L. & Gross, Samuel M. & Bendersky, Marina & Lavori, Philip W., 2017. "Software for Distributed Computation on Medical Databases: A Demonstration Project," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 77(i13).
    • Handle: RePEc:jss:jstsof:v:077:i13
    DOI: http://hdl.handle.net/10.18637/jss.v077.i13
    as

    Download full text from publisher

    File URL: https://www.jstatsoft.org/index.php/jss/article/view/v077i13/v77i13.pdf
    Download Restriction: no
    File URL: https://www.jstatsoft.org/index.php/jss/article/downloadSuppFile/v077i13/distcomp_1.0.tar.gz
    Download Restriction: no
    File URL: https://www.jstatsoft.org/index.php/jss/article/downloadSuppFile/v077i13/v77i13-replication.zip
    Download Restriction: no
    File URL: https://libkey.io/http://hdl.handle.net/10.18637/jss.v077.i13?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jeffrey T Leek & John D Storey, 2007. "Capturing Heterogeneity in Gene Expression Studies by Surrogate Variable Analysis," PLOS Genetics, Public Library of Science, vol. 3(9), pages 1-12, September.
    2. Eddelbuettel, Dirk & Stokely, Murray & Ooms, Jeroen, 2016. "RProtoBuf: Efficient Cross-Language Data Serialization in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 71(i02).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Arjun Bhattacharya & Anastasia N. Freedman & Vennela Avula & Rebeca Harris & Weifang Liu & Calvin Pan & Aldons J. Lusis & Robert M. Joseph & Lisa Smeester & Hadley J. Hartwell & Karl C. K. Kuban & Car, 2022. "Placental genomics mediates genetic associations with complex health traits and disease," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    2. repec:jss:jstsof:40:i14 is not listed on IDEAS
    3. Won Jun Lee & Sang Cheol Kim & Jung-Ho Yoon & Sang Jun Yoon & Johan Lim & You-Sun Kim & Sung Won Kwon & Jeong Hill Park, 2016. "Meta-Analysis of Tumor Stem-Like Breast Cancer Cells Using Gene Set and Network Analysis," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-20, February.
    4. Emanuele Aliverti & Kristian Lum & James E. Johndrow & David B. Dunson, 2021. "Removing the influence of group variables in high‐dimensional predictive modelling," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 791-811, July.
    5. Marron, J.S., 2017. "Big Data in context and robustness against heterogeneity," Econometrics and Statistics, Elsevier, vol. 2(C), pages 73-80.
    6. Seungchul Baek & Yen‐Yi Ho & Yanyuan Ma, 2020. "Using sufficient direction factor model to analyze latent activities associated with breast cancer survival," Biometrics, The International Biometric Society, vol. 76(4), pages 1340-1350, December.
    7. Griffin, Maryclare & Hoff, Peter D., 2019. "Lasso ANOVA decompositions for matrix and tensor data," Computational Statistics & Data Analysis, Elsevier, vol. 137(C), pages 181-194.
    8. Yunfeng Li & Jarrett Morrow & Benjamin Raby & Kelan Tantisira & Scott T Weiss & Wei Huang & Weiliang Qiu, 2017. "Detecting disease-associated genomic outcomes using constrained mixture of Bayesian hierarchical models for paired data," PLOS ONE, Public Library of Science, vol. 12(3), pages 1-16, March.
    9. Zhaohui Qin & Ben Li & Karen N. Conneely & Hao Wu & Ming Hu & Deepak Ayyala & Yongseok Park & Victor X. Jin & Fangyuan Zhang & Han Zhang & Li Li & Shili Lin, 2016. "Statistical Challenges in Analyzing Methylation and Long-Range Chromosomal Interaction Data," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 8(2), pages 284-309, October.
    10. Zemin Zheng & Jinchi Lv & Wei Lin, 2021. "Nonsparse Learning with Latent Variables," Operations Research, INFORMS, vol. 69(1), pages 346-359, January.
    11. Chee Ho H’ng & Shanika L. Amarasinghe & Boya Zhang & Hojin Chang & Xinli Qu & David R. Powell & Alberto Rosello-Diez, 2024. "Compensatory growth and recovery of cartilage cytoarchitecture after transient cell death in fetal mouse limbs," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    12. Mark Reimers, 2010. "Making Informed Choices about Microarray Data Analysis," PLOS Computational Biology, Public Library of Science, vol. 6(5), pages 1-7, May.
    13. Leek Jeffrey T & Storey John D., 2011. "The Joint Null Criterion for Multiple Hypothesis Tests," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 10(1), pages 1-22, June.
    14. Christos Miliotis & Yuling Ma & Xanthi-Lida Katopodi & Dimitra Karagkouni & Eleni Kanata & Kaia Mattioli & Nikolas Kalavros & Yered H. Pita-Juárez & Felipe Batalini & Varune R. Ramnarine & Shivani Nan, 2024. "Determinants of gastric cancer immune escape identified from non-coding immune-landscape quantitative trait loci," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    15. Nicoló Fusi & Oliver Stegle & Neil D Lawrence, 2012. "Joint Modelling of Confounding Factors and Prominent Genetic Regulators Provides Increased Accuracy in Genetical Genomics Studies," PLOS Computational Biology, Public Library of Science, vol. 8(1), pages 1-9, January.
    16. Jin Hyun Ju & Sushila A Shenoy & Ronald G Crystal & Jason G Mezey, 2017. "An independent component analysis confounding factor correction framework for identifying broad impact expression quantitative trait loci," PLOS Computational Biology, Public Library of Science, vol. 13(5), pages 1-26, May.
    17. Miecznikowski, Jeffrey C. & Gold, David & Shepherd, Lori & Liu, Song, 2011. "Deriving and comparing the distribution for the number of false positives in single step methods to control k-FWER," Statistics & Probability Letters, Elsevier, vol. 81(11), pages 1695-1705, November.
    18. Aline Talhouk & Stefan Kommoss & Robertson Mackenzie & Martin Cheung & Samuel Leung & Derek S Chiu & Steve E Kalloger & David G Huntsman & Stephanie Chen & Maria Intermaggio & Jacek Gronwald & Fong C , 2016. "Single-Patient Molecular Testing with NanoString nCounter Data Using a Reference-Based Strategy for Batch Effect Correction," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-18, April.
    19. Yuto Hasegawa & Juhyun Kim & Gianluca Ursini & Yan Jouroukhin & Xiaolei Zhu & Yu Miyahara & Feiyi Xiong & Samskruthi Madireddy & Mizuho Obayashi & Beat Lutz & Akira Sawa & Solange P. Brown & Mikhail V, 2023. "Microglial cannabinoid receptor type 1 mediates social memory deficits in mice produced by adolescent THC exposure and 16p11.2 duplication," Nature Communications, Nature, vol. 14(1), pages 1-19, December.
    20. Charlotte Soneson & Sarah Gerster & Mauro Delorenzi, 2014. "Batch Effect Confounding Leads to Strong Bias in Performance Estimates Obtained by Cross-Validation," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-13, June.
    21. Sudhir Varma, 2020. "Blind estimation and correction of microarray batch effect," PLOS ONE, Public Library of Science, vol. 15(4), pages 1-15, April.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jss:jstsof:v:077:i13. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum (email available below). General contact details of provider: http://www.jstatsoft.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.