IDEAS home Printed from https://ideas.repec.org/a/inm/orisre/v34y2023i4p1664-1685.html
   My bibliography  Save this article

Green Data Analytics of Supercomputing from Massive Sensor Networks: Does Workload Distribution Matter?

Author

Listed:
  • Zhiling Guo

    (School of Computing and Information Systems, Singapore Management University, Singapore 178902)

  • Jin Li

    (School of Management, Xi’an Jiaotong University, Xi’an 710049, China)

  • Ram Ramesh

    (Department of Management Science and Systems, State University of New York, Buffalo, New York 14260)

Abstract

Energy costs represent a significant share of the total cost of ownership in high-performance computing (HPC) systems. Using a unique data set collected by massive sensor networks in a petascale national supercomputing center, we first present an explanatory model to identify key factors that affect energy consumption in supercomputing. Our analytic results show that, not only does computing node utilization significantly affect energy consumption, workload distribution among the nodes also has significant effects and could effectively be leveraged to improve energy efficiency. Next, we establish the high model performance using in-sample and out-of-sample analyses. We then develop prescriptive models for energy-optimal runtime workload management and extend the models to consider energy consumption and job performance tradeoffs. Specifically, we present four dynamic resource management methodologies ( packing , load balancing , threshold-based switching, and energy optimization ), model their application at two levels (purely within-rack and jointly cross-rack resource allocation), and explore runtime resource redistribution policies for jobs under the emergent principle of computational steering and comparatively evaluate strategies that use computational steering with those that do not. Our experimental studies show that packing is preferred when the total workload of a rack is higher than a threshold and load balancing is preferred when it is lower. These results lead to a threshold strategy that yields near-optimal energy efficiency under all workload conditions. We further calibrate the energy-optimal resource allocations over the full range of workloads and present a bicriteria evaluation to consider energy consumption and job performance tradeoffs. We demonstrate significant energy savings of our proposed strategies under various workload conditions. We conclude with implementation guidelines and policy insights into energy-efficient computing resource management in large supercomputing data centers.

Suggested Citation

  • Zhiling Guo & Jin Li & Ram Ramesh, 2023. "Green Data Analytics of Supercomputing from Massive Sensor Networks: Does Workload Distribution Matter?," Information Systems Research, INFORMS, vol. 34(4), pages 1664-1685, December.
  • Handle: RePEc:inm:orisre:v:34:y:2023:i:4:p:1664-1685
    DOI: 10.1287/isre.2023.1208
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/isre.2023.1208
    Download Restriction: no

    File URL: https://libkey.io/10.1287/isre.2023.1208?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Xiao Fang & Olivia R. Liu Sheng & Paulo Goes, 2013. "When Is the Right Time to Refresh Knowledge Discovered from Data?," Operations Research, INFORMS, vol. 61(1), pages 32-44, February.
    2. Keely L. Croxton & Bernard Gendron & Thomas L. Magnanti, 2003. "A Comparison of Mixed-Integer Programming Models for Nonconvex Piecewise Linear Cost Minimization Problems," Management Science, INFORMS, vol. 49(9), pages 1268-1273, September.
    3. Maxime C. Cohen & Philipp W. Keller & Vahab Mirrokni & Morteza Zadimoghaddam, 2019. "Overcommitment in Cloud Services: Bin Packing with Chance Constraints," Management Science, INFORMS, vol. 65(7), pages 3255-3271, July.
    4. Giovanni S. F. Bruno, 2005. "Estimation and inference in dynamic unbalanced panel-data models with a small number of individuals," Stata Journal, StataCorp LP, vol. 5(4), pages 473-500, December.
    5. Gupta, Rohit & Asgari, Sahar & Moazamigoodarzi, Hosein & Down, Douglas G. & Puri, Ishwar K., 2021. "Energy, exergy and computing efficiency based data center workload and cooling management," Applied Energy, Elsevier, vol. 299(C).
    6. Yeliang Qiu & Congfeng Jiang & Yumei Wang & Dongyang Ou & Youhuizi Li & Jian Wan, 2019. "Energy Aware Virtual Machine Scheduling in Data Centers," Energies, MDPI, vol. 12(4), pages 1-21, February.
    7. Nickell, Stephen J, 1981. "Biases in Dynamic Models with Fixed Effects," Econometrica, Econometric Society, vol. 49(6), pages 1417-1426, November.
    8. Delorme, Maxence & Iori, Manuel & Martello, Silvano, 2016. "Bin packing and cutting stock problems: Mathematical models and exact algorithms," European Journal of Operational Research, Elsevier, vol. 255(1), pages 1-20.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eicher, Theo S. & Schreiber, Till, 2010. "Structural policies and growth: Time series evidence from a natural experiment," Journal of Development Economics, Elsevier, vol. 91(1), pages 169-179, January.
    2. Cornelius Hirsch & Harald Oberhofer, 2017. "Bilateral Trade Agreements and Trade Distortions in Agricultural Markets?," WIFO Working Papers 531, WIFO.
    3. Raf Van Gestel & Tobias Müller & Johan Bosmans, 2018. "Learning from failure in healthcare: Dynamic panel evidence of a physician shock effect," Health Economics, John Wiley & Sons, Ltd., vol. 27(9), pages 1340-1353, September.
    4. Michał Brzoza-Brzezina & Jacek Kotłowski & Kamil Wierus, 2015. "Can interest rate spreads stabilize the euro area?," Applied Economics, Taylor & Francis Journals, vol. 47(34-35), pages 3696-3709, July.
    5. Jean-Thomas Bernard & Ba Chu & Lynda Khalaf & Marcel Voia, 2019. "Non-Standard Confidence Sets for Ratios and Tipping Points with Applications to Dynamic Panel Data," Annals of Economics and Statistics, GENES, issue 134, pages 79-108.
    6. Mahir Binici & Yin-Wong Cheung & Kon S. Lai, 2011. "Trade Openness, Market Competition, and Inflation: Some Sectoral Evidence from OECD Countries," CESifo Working Paper Series 3690, CESifo.
    7. Federico M. Giesenow & Jakob de Haan, 2019. "The influence of government ideology on monetary policy: New cross‐country evidence based on dynamic heterogeneous panels," Economics and Politics, Wiley Blackwell, vol. 31(2), pages 216-239, July.
    8. Briglauer, Wolfgang & Cambini, Carlo & Melani, Sauro, 2016. "How to Fill the Digital Gap? The (Limited) Role of Regulation," VfS Annual Conference 2016 (Augsburg): Demographic Change 145480, Verein für Socialpolitik / German Economic Association.
    9. Aidt, Toke S. & Mooney, Graham, 2014. "Voting suffrage and the political budget cycle: Evidence from the London Metropolitan Boroughs 1902–1937," Journal of Public Economics, Elsevier, vol. 112(C), pages 53-71.
    10. Ali, Amin Masud & Savoia, Antonio, 2023. "Decentralisation or patronage: What determines government's allocation of development spending in a unitary country? Evidence from Bangladesh," European Journal of Political Economy, Elsevier, vol. 78(C).
    11. Beate Jochimsen & Robert Nuscheler, 2011. "The political economy of the German Lander deficits: weak governments meet strong finance ministers," Applied Economics, Taylor & Francis Journals, vol. 43(19), pages 2399-2415.
    12. Thanh Cong Nguyen, 2022. "The effects of financial crisis on income inequality," Development Policy Review, Overseas Development Institute, vol. 40(6), November.
    13. Briglauer, Wolfgang & Gugler, Klaus & Haxhimusa, Adhurim, 2016. "Facility- and service-based competition and investment in fixed broadband networks: Lessons from a decade of access regulations in the European Union member states," Telecommunications Policy, Elsevier, vol. 40(8), pages 729-742.
    14. Mar Delgado-Téllez & Esther Gordo & Iván Kataryniuk & Javier J. Pérez, 2022. "The decline in public investment: ``social dominance’’ or too-rigid fiscal rules?," Applied Economics, Taylor & Francis Journals, vol. 54(10), pages 1123-1136, February.
    15. John Martinovic & Markus Hähnel & Guntram Scheithauer & Waltenegus Dargie, 2022. "An introduction to stochastic bin packing-based server consolidation with conflicts," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(2), pages 296-331, July.
    16. Ansgar Belke & Ulrich Haskamp & Gunther Schnabl, 2018. "Beyond Balassa and Samuelson: real convergence, capital flows, and competitiveness in Greece," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 45(2), pages 409-424, May.
    17. Gehringer, Agnieszka & Prettner, Klaus, 2019. "Longevity And Technological Change," Macroeconomic Dynamics, Cambridge University Press, vol. 23(4), pages 1471-1503, June.
    18. Lechler, Marie, 2019. "Employment shocks and anti-EU sentiment," European Journal of Political Economy, Elsevier, vol. 59(C), pages 266-295.
    19. Ibrahim, Mansor H. & Salim, Kinan & Abojeib, Moutaz & Yeap, Lau Wee, 2019. "Structural changes, competition and bank stability in Malaysia’s dual banking system," Economic Systems, Elsevier, vol. 43(1), pages 111-129.
    20. Chris Belmert Milindi & Roula Inglesi-Lotz, 2023. "Impact of technological progress on carbon emissions in different country income groups," Energy & Environment, , vol. 34(5), pages 1348-1382, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orisre:v:34:y:2023:i:4:p:1664-1685. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.