Efficient explainable federated learning
Abstract
The current AI revolution originated from the impressive deep learning methodology. Deep neural networks and in particular recent foundation models rely heavily on vast amounts of data and computing power, which poses the question of how to efficiently train large AI models from big data distributed geographically in multiple organisations. Because of privacy concerns and commercial interest of the data providers, as well as mandatory regulation rules such as General Data Protection Regulation (GDPR) of the European Union, sensitive local data must not be leaked to others. This restriction makes traditional distributed learning paradigms an unfeasible solution in which local information can freely flow among nodes. Federated learning is an emerging distributed learning paradigm that features privacy protection by utilising sophisticated algorithms such as differential privacy and aggregation schemes to generate global results. However, while federated learning has made great progress and been deployed in many industrial systems, there remain obstacles in the canonical paradigm that prevent its advance in the era of large models. In this thesis, two important difficulties are identified: low training efficiency and lack of explainability. Through experimental results from the literature and collected operating stat- istics of deployed federated systems, the main training bottleneck boils down to the transfer time of large messages, which could add up to more than one day in a…
Read the paper
Related studies
- Active Learning during Federated Learning for Object Detection
- Fairness-aware federated learning
- Decentralized Federated Learning
- Building a Personally Identifiable Information Recognizer in a Privacy Preserved Manner Using Automated Annotation and Federated Learning
- Preservation of Patient Level Privacy: Federated Classification and Calibration Models
- Machine learning for all : a methodology for choosing a federated learning approach
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.