Optimisation Algorithms for Federated Learning
Abstract
A more recent development within machine learning is a technique called federated learning. Instead of training a model on a central server, a model is instead collaboratively trained on several devices. The data on each device is not shared with the server but instead kept locally on the devices, which increases data privacy and security for the users. Federated learning was introduced with the optimisation algorithm Federated Averaging (FedAvg), that uses a local stochastic gradient descent before communicating the updated weights to the server. However, FedAvg struggles in setups where the data distribution is unbalanced, and therefore other optimisation algorithms have been developed to address these issues. In this project, we aim to test different optimisation algorithms and study their accuracy and loss on different datasets. Specifically, the focus has been on heterogeneous data and fairness of the model performance. The optimisers were analysed and compared on different levels of data heterogeneity. Our analysis identifies the optimisation algorithm best suited for certain setups and concludes the overall best performing algorithm to be FedAvg-M.
Read the paper
Related studies
- Active Learning during Federated Learning for Object Detection
- Fairness-aware federated learning
- Decentralized Federated Learning
- Building a Personally Identifiable Information Recognizer in a Privacy Preserved Manner Using Automated Annotation and Federated Learning
- Preservation of Patient Level Privacy: Federated Classification and Calibration Models
- Efficient explainable federated learning
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.