Federated learning model complexity vs robustness to non-IID data and selective federated learning
Abstract
Federated learning trains a global model using data distributed across local nodes, and differs from centralized machine learning by moving the computation to the data in order to address the challenges of data ownership, privacy, computational power, and data storage. Previous federated learning research has addressed the effect of non independent and identically distributed data on federated learning [6]. Meanwhile, local models may have better performance if the test set is also non-IID [7]. However, there may be insufficient data on a node to train a local model for every node; hence the purpose of federated learning. This research is the first, to our knowledge, to consider model performance on both a global test set and non-IID test set. Our experiments provide a original finding in that federated learning is only robust to non-IID data with constraints on the width and depth of a neural network. There is a tradeoff, however, between model complexity and feasibility of training the model on edge devices. Thus, we propose selective federated learning algorithm which greatly allows simpler models that fit on edge devices to be robust to highly non-IID data. For non-IID test sets, we prove that a converged federated model may converge to weights which do not provide the optimal local loss for an arbitrary chosen number of training samples on each node. Additionally, this thesis discusses the experiments that were conducted to examine the effects of model complexity,…
Read the paper
Related studies
- Active Learning during Federated Learning for Object Detection
- Fairness-aware federated learning
- Decentralized Federated Learning
- Building a Personally Identifiable Information Recognizer in a Privacy Preserved Manner Using Automated Annotation and Federated Learning
- Preservation of Patient Level Privacy: Federated Classification and Calibration Models
- Efficient explainable federated learning
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.