IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v177y2023ics0167947322001670.html
   My bibliography  Save this article

Parallel-and-stream accelerator for computationally fast supervised learning

Author

Listed:
  • Hector, Emily C.
  • Luo, Lan
  • Song, Peter X.-K.

Abstract

Two dominant distributed computing strategies have emerged to overcome the computational bottleneck of supervised learning with big data: parallel data processing in the MapReduce paradigm and serial data processing in the online streaming paradigm. Despite the two strategies' common divide-and-combine approach, they differ in how they aggregate information, leading to different trade-offs between statistical and computational performance. The authors propose a new hybrid paradigm, termed a Parallel-and-Stream Accelerator (PASA), that uses the strengths of both strategies for computationally fast and statistically efficient supervised learning. PASA's architecture nests online streaming processing into each distributed and parallelized data process in a MapReduce framework. PASA leverages the advantages and mitigates the disadvantages of both the MapReduce and online streaming approaches to deliver a more flexible paradigm satisfying practical computing needs. The authors study the analytic properties and computational complexity of PASA, and detail its implementation for two key statistical learning tasks. PASA's performance is illustrated through simulations and a large-scale data example building a prediction model for online purchases from advertising data.

Suggested Citation

  • Hector, Emily C. & Luo, Lan & Song, Peter X.-K., 2023. "Parallel-and-stream accelerator for computationally fast supervised learning," Computational Statistics & Data Analysis, Elsevier, vol. 177(C).
  • Handle: RePEc:eee:csdana:v:177:y:2023:i:c:s0167947322001670
    DOI: 10.1016/j.csda.2022.107587
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947322001670
    Download Restriction: Full text for ScienceDirect subscribers only.

    File URL: https://libkey.io/10.1016/j.csda.2022.107587?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    2. Emily C. Hector & Peter X.-K. Song, 2021. "A Distributed and Integrated Method of Moments for High-Dimensional Correlated Data Analysis," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(534), pages 805-818, April.
    3. Lan Luo & Peter X.‐K. Song, 2020. "Renewable estimation and incremental inference in generalized linear models with streaming data sets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(1), pages 69-97, February.
    4. Fei Wang & Lu Wang & Peter X.-K. Song, 2012. "Quadratic inference function approach to merging longitudinal studies: validation and joint estimation," Biometrika, Biometrika Trust, vol. 99(3), pages 755-762.
    5. Xie, Minge & Singh, Kesar & Strawderman, William E., 2011. "Confidence Distributions and a Unifying Framework for Meta-Analysis," Journal of the American Statistical Association, American Statistical Association, vol. 106(493), pages 320-333.
    6. Min-ge Xie & Kesar Singh, 2013. "Confidence Distribution, the Frequentist Distribution Estimator of a Parameter: A Review," International Statistical Review, International Statistical Institute, vol. 81(1), pages 3-39, April.
    7. HaiYing Wang & Min Yang & John Stufken, 2019. "Information-Based Optimal Subdata Selection for Big Data Linear Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(525), pages 393-405, January.
    8. HaiYing Wang & Rong Zhu & Ping Ma, 2018. "Optimal Subsampling for Large Sample Logistic Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 829-844, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. La Vecchia, Davide & Moor, Alban & Scaillet, Olivier, 2023. "A higher-order correct fast moving-average bootstrap for dependent data," Journal of Econometrics, Elsevier, vol. 235(1), pages 65-81.
    2. Tianzhen Wang & Haixiang Zhang, 2022. "Optimal subsampling for multiplicative regression with massive data," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 76(4), pages 418-449, November.
    3. Guang Yang & Dungang Liu & Junyuan Wang & Min‐ge Xie, 2016. "Meta‐analysis framework for exact inferences with application to the analysis of rare events," Biometrics, The International Biometric Society, vol. 72(4), pages 1378-1386, December.
    4. Su, Miaomiao & Wang, Ruoyu & Wang, Qihua, 2022. "A two-stage optimal subsampling estimation for missing data problems with large-scale data," Computational Statistics & Data Analysis, Elsevier, vol. 173(C).
    5. Jun Yu & HaiYing Wang, 2022. "Subdata selection algorithm for linear model discrimination," Statistical Papers, Springer, vol. 63(6), pages 1883-1906, December.
    6. Duarte, Belmiro P.M. & Atkinson, Anthony C. & Oliveira, Nuno M.C., 2024. "Using hierarchical information-theoretic criteria to optimize subsampling of extensive datasets," LSE Research Online Documents on Economics 121641, London School of Economics and Political Science, LSE Library.
    7. J. Lars Kirkby & Dang H. Nguyen & Duy Nguyen & Nhu N. Nguyen, 2022. "Inversion-free subsampling Newton’s method for large sample logistic regression," Statistical Papers, Springer, vol. 63(3), pages 943-963, June.
    8. Ziyang Wang & HaiYing Wang & Nalini Ravishanker, 2023. "Subsampling in Longitudinal Models," Methodology and Computing in Applied Probability, Springer, vol. 25(1), pages 1-29, March.
    9. Xuhua Liu & Xingzhong Xu, 2016. "Confidence distribution inferences in one-way random effects model," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(1), pages 59-74, March.
    10. Amalan Mahendran & Helen Thompson & James M. McGree, 2023. "A model robust subsampling approach for Generalised Linear Models in big data settings," Statistical Papers, Springer, vol. 64(4), pages 1137-1157, August.
    11. Deng, Jiayi & Huang, Danyang & Ding, Yi & Zhu, Yingqiu & Jing, Bingyi & Zhang, Bo, 2024. "Subsampling spectral clustering for stochastic block models in large-scale networks," Computational Statistics & Data Analysis, Elsevier, vol. 189(C).
    12. Zhang, Haixiang & Wang, HaiYing, 2021. "Distributed subdata selection for big data via sampling-based approach," Computational Statistics & Data Analysis, Elsevier, vol. 153(C).
    13. Sokbae Lee & Serena Ng, 2020. "An Econometric Perspective on Algorithmic Subsampling," Annual Review of Economics, Annual Reviews, vol. 12(1), pages 45-80, August.
    14. Xiaohui Yuan & Yong Li & Xiaogang Dong & Tianqing Liu, 2022. "Optimal subsampling for composite quantile regression in big data," Statistical Papers, Springer, vol. 63(5), pages 1649-1676, October.
    15. Tang, Lu & Zhou, Ling & Song, Peter X.-K., 2020. "Distributed simultaneous inference in generalized linear models via confidence distribution," Journal of Multivariate Analysis, Elsevier, vol. 176(C).
    16. Wu, Suofei & Hannig, Jan & Lee, Thomas C.M., 2022. "Uncertainty quantification for honest regression trees," Computational Statistics & Data Analysis, Elsevier, vol. 167(C).
    17. Feifei Wang & Danyang Huang & Tianchen Gao & Shuyuan Wu & Hansheng Wang, 2022. "Sequential one‐step estimator by sub‐sampling for customer churn analysis with massive data sets," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(5), pages 1753-1786, November.
    18. Xiaokang Luo & Tirthankar Dasgupta & Minge Xie & Regina Y. Liu, 2021. "Leveraging the Fisher randomization test using confidence distributions: Inference, combination and fusion learning," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(4), pages 777-797, September.
    19. Jun Yu & Jiaqi Liu & HaiYing Wang, 2023. "Information-based optimal subdata selection for non-linear models," Statistical Papers, Springer, vol. 64(4), pages 1069-1093, August.
    20. Fei Wang & Lu Wang & Peter X.‐K. Song, 2016. "Fused lasso with the adaptation of parameter ordering in combining multiple studies with repeated measurements," Biometrics, The International Biometric Society, vol. 72(4), pages 1184-1193, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:177:y:2023:i:c:s0167947322001670. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.