IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2024i10p1601-d1398118.html
   My bibliography  Save this article

FedUB: Federated Learning Algorithm Based on Update Bias

Author

Listed:
  • Hesheng Zhang

    (School of Mathematics and Statistics, Henan University of Science and Technology, Luoyang 471023, China)

  • Ping Zhang

    (School of Mathematics and Statistics, Henan University of Science and Technology, Luoyang 471023, China
    Intelligent System Science and Technology Innovation Center, Longmen Laboratory, Luoyang 471023, China)

  • Mingkai Hu

    (School of Mathematics and Statistics, Henan University of Science and Technology, Luoyang 471023, China)

  • Muhua Liu

    (School of Mathematics and Statistics, Henan University of Science and Technology, Luoyang 471023, China)

  • Jiechang Wang

    (Sports Big Data Center, Department of Physical Education, Zhengzhou University, Zhengzhou 450001, China)

Abstract

Federated learning, as a distributed machine learning framework, aims to protect data privacy while addressing the issue of data silos by collaboratively training models across multiple clients. However, a significant challenge to federated learning arises from the non-independent and identically distributed (non-iid) nature of data across different clients. non-iid data can lead to inconsistencies between the minimal loss experienced by individual clients and the global loss observed after the central server aggregates the local models, affecting the model’s convergence speed and generalization capability. To address this challenge, we propose a novel federated learning algorithm based on update bias (FedUB). Unlike traditional federated learning approaches such as FedAvg and FedProx, which independently update model parameters on each client before direct aggregation to form a global model, the FedUB algorithm incorporates an update bias in the loss function of local models—specifically, the difference between each round’s local model updates and the global model updates. This design aims to reduce discrepancies between local and global updates, thus aligning the parameters of locally updated models more closely with those of the globally aggregated model, thereby mitigating the fundamental conflict between local and global optima. Additionally, during the aggregation phase at the server side, we introduce a metric called the bias metric, which assesses the similarity between each client’s local model and the global model. This metric adaptively sets the weight of each client during aggregation after each training round to achieve a better global model. Extensive experiments conducted on multiple datasets have confirmed the effectiveness of the FedUB algorithm. The results indicate that FedUB generally outperforms methods such as FedDC, FedDyn, and Scaffold, especially in scenarios involving partial client participation and non-iid data distributions. It demonstrates superior performance and faster convergence in tasks such as image classification.

Suggested Citation

  • Hesheng Zhang & Ping Zhang & Mingkai Hu & Muhua Liu & Jiechang Wang, 2024. "FedUB: Federated Learning Algorithm Based on Update Bias," Mathematics, MDPI, vol. 12(10), pages 1-26, May.
  • Handle: RePEc:gam:jmathe:v:12:y:2024:i:10:p:1601-:d:1398118
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/10/1601/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2227-7390/12/10/1601/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Martin Hilbert, 2016. "Big Data for Development: A Review of Promises and Challenges," Development Policy Review, Overseas Development Institute, vol. 34(1), pages 135-174, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luyu Liu & Harvey J Miller, 2021. "Measuring risk of missing transfers in public transit systems using high-resolution schedule and real-time bus location data," Urban Studies, Urban Studies Journal Limited, vol. 58(15), pages 3140-3156, November.
    2. Martin Hilbert, 2017. "Complementary Variety: When Can Cooperation in Uncertain Environments Outperform Competitive Selection?," Complexity, Hindawi, vol. 2017, pages 1-15, September.
    3. Anke Joubert & Matthias Murawski & Markus Bick, 2023. "Measuring the Big Data Readiness of Developing Countries – Index Development and its Application to Africa," Information Systems Frontiers, Springer, vol. 25(1), pages 327-350, February.
    4. Raymond Lang & Marguerite Schneider & Maria Kett & Ellie Cole & Nora Groce, 2019. "Policy development: An analysis of disability inclusion in a selection of African Union policies," Development Policy Review, Overseas Development Institute, vol. 37(2), pages 155-175, March.
    5. Makoza, Frank, 2023. "Analyzing policy change of Malawi ICT and Digitalization policy: Policy Assemblage Perspective," EconStor Preprints 273309, ZBW - Leibniz Information Centre for Economics.
    6. Chu Ping Lo, 2024. "Digitalization, AI Intensity, and International Trade," Annals of Economics and Finance, Society for AEF, vol. 25(1), pages 251-273, May.
    7. Yang, Jun & Yang, Dingjian & Cheng, Jixin, 2024. "The non-rivalry of data, directed technical change and the environment: A theoretical study incorporating data as a production factor," Economic Analysis and Policy, Elsevier, vol. 82(C), pages 417-448.
    8. Richard Heeks & Vanya Rakesh & Ritam Sengupta & Sumandro Chattapadhyay & Christopher Foster, 2021. "Datafication, value and power in developing countries: Big data in two Indian public service organizations," Development Policy Review, Overseas Development Institute, vol. 39(1), pages 82-102, January.
    9. Małgorzata Dobrowolska & Mariola Paruzel-Czachura & Marta Stasiła-Sieradzka, 2018. "Perception of Limitations by Individuals Threatened with Social Exclusion upon Entering Employment: Report on a Study of Individuals with Chronic Mental Illnesses," European Journal of Economics and Business Studies Articles, Revistia Research and Publishing, vol. 4, ejes_v4_i.
    10. Jingmei Gao & Zahid Sarwar, 2024. "How do firms create business value and dynamic capabilities by leveraging big data analytics management capability?," Information Technology and Management, Springer, vol. 25(3), pages 283-304, September.
    11. Jing Xu & Huijun Zhang, 2020. "Environmental Activism and Big Data: Building Green Social Capital in China," Sustainability, MDPI, vol. 12(8), pages 1-24, April.
    12. Secundo, Giustina & Riad Shams, S.M. & Nucci, Francesco, 2021. "Digital technologies and collective intelligence for healthcare ecosystem: Optimizing Internet of Things adoption for pandemic management," Journal of Business Research, Elsevier, vol. 131(C), pages 563-572.
    13. Rafael Prieto Curiel & Stefano Cresci & Cristina Ioana Muntean & Steven Richard Bishop, 2020. "Crime and its fear in social media," Palgrave Communications, Palgrave Macmillan, vol. 6(1), pages 1-12, December.
    14. Johannes Wachs & Mih'aly Fazekas & J'anos Kert'esz, 2019. "Corruption Risk in Contracting Markets: A Network Science Perspective," Papers 1909.08664, arXiv.org.
    15. Badr Bentalha, 2020. "Big-Data and Service Supply chain management: Challenges and opportunities [Big-Data et Service Supply chain management: Challenges et opportunités]," Post-Print hal-02680861, HAL.
    16. Muhammad Omar & Arif Mehmood & Gyu Sang Choi & Han Woo Park, 2017. "Global mapping of artificial intelligence in Google and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1269-1305, December.
    17. Bar-Gil, Oshri & Ron, Tom & Czerniak, Ofir, 2024. "AI for the people? Embedding AI ethics in HR and people analytics projects," Technology in Society, Elsevier, vol. 77(C).
    18. Hilbert, Martin, 2016. "The bad news is that the digital access divide is here to stay: Domestically installed bandwidths among 172 countries for 1986–2014," Telecommunications Policy, Elsevier, vol. 40(6), pages 567-581.
    19. Roland W. Scholz, 2016. "Sustainable Digital Environments: What Major Challenges Is Humankind Facing?," Sustainability, MDPI, vol. 8(8), pages 1-31, July.
    20. Haile Teklemariam, Mekuria & Kwon, Youngsun, 2018. "Reducing internet demand-side gap improves digital inclusion in low-income countries: - analysis that is more comprehensive," 22nd ITS Biennial Conference, Seoul 2018. Beyond the boundaries: Challenges for business, policy and society 190411, International Telecommunications Society (ITS).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2024:i:10:p:1601-:d:1398118. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.