Fed-XLM-R: a privacy-preserving federated framework with adapter-scoped differential privacy for mental health triage-level intent classification on resource-constrained edge devices
Abstract
Introduction: Mental health triage-level intent classification in linguistically diverse and resource-constrained clinical environments requires models that simultaneously meet strict patient data privacy regulations, cross-lingual generalization to low-resource languages, and real-time inference on edge hardware. No existing federated NLP system has jointly demonstrated this combination. This work proposes Fed-XLM-R, a unified federated learning framework designed to close that gap.
Methods: Fed-XLM-R integrates XLM-RoBERTa with lightweight bottleneck adapter layers, adapter-scoped differentially private stochastic gradient descent (DP-SGD), proximal regularization for straggler-robust aggregation (FedProx), and INT8 post-training quantization for edge deployment. Gradient perturbation is confined exclusively to the 0.87% trainable adapter subspace rather than the full 270-million-parameter model. The framework was evaluated in a simulated 10-client federation on a public benchmark, with intent classification trained and evaluated on English data. A cross-lingual diagnostic was additionally run on the XNLI benchmark across five languages (English, Spanish, Hindi, Swahili, Arabic) against an mBERT baseline, and canary phrase memorization analysis was conducted across all 50 federated rounds to assess data extraction risk.
Results: At the clinically meaningful privacy budget of ε = 1.0 (δ = 10-5, Fed-XLM-R achieves 96% accuracy retention, yielding an overall accuracy of…
Read the paper
Related studies
- Active Learning during Federated Learning for Object Detection
- Fairness-aware federated learning
- Decentralized Federated Learning
- Building a Personally Identifiable Information Recognizer in a Privacy Preserved Manner Using Automated Annotation and Federated Learning
- Preservation of Patient Level Privacy: Federated Classification and Calibration Models
- Efficient explainable federated learning
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.