Improving Training Performance in Federated Learning
Abstract
To utilize enormous data generated on numerous edge devices such as mobile phones to train a high-performance machine learning model while protecting users' data privacy, federated learning has been proposed and has become one of the most essential paradigms in distributed machine learning. Under the coordination of a central server, users collaboratively train a shared global model without sharing their data. They conduct local training with their data and only send their model updates to the central server for aggregating an improved global model. To improve training performance in federated learning, a myriad of new mechanisms have been proposed. However, our extensive evaluation indicated that most state-of-the-art mechanisms failed to perform as well as they claimed. We thus explored potential directions that can consistently decrease the elapsed training time for the global model to converge to a target accuracy, and found that changing the conventional synchronous aggregation paradigm, where the server does not conduct aggregation until it receives updates from all the selected users, to the asynchronous one, where the server aggregates without waiting for slow users, significantly improved the training performance. However, asynchronous federated learning has not been as widely studied as synchronous federated learning, and its existing mechanisms have not reached the best possible performance. We thus propose Blade, a new staleness-aware framework that seeks to…
Read the paper
Related studies
- Active Learning during Federated Learning for Object Detection
- Fairness-aware federated learning
- Decentralized Federated Learning
- Building a Personally Identifiable Information Recognizer in a Privacy Preserved Manner Using Automated Annotation and Federated Learning
- Preservation of Patient Level Privacy: Federated Classification and Calibration Models
- Efficient explainable federated learning
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.