Research on Federated Multimodal Human Morphology Recognition Integrating Convolutional and Graph Convolutional Networks
Abstract
Abstract To address the data privacy barriers and multimodal modeling bottlenecks faced by human morphology recognition in complex dynamic scenes, this paper proposes a Federated Hybrid Multimodal Recognition framework (FedHMR). The framework integrates the visual feature extraction capability of convolutional neural networks with the skeletal topology modeling advantage of graph convolutional networks, improving recognition accuracy for complex postures (e.g., scoliosis, gait abnormality) via cross-modal collaborative learning. Meanwhile, a blockchain-based federated learning architecture is designed to enable cross-institution collaborative training while preserving the privacy of raw RGB-D data. Experimental results show that FedHMR achieves 95.8\% accuracy and 95.7\% F1-score on a private dataset, outperforming the non-federated version by 0.6\%. On the public NTU RGB+D 120 dataset, the proposed method attains 94.5\% accuracy, surpassing the state-of-the-art method MMT (93.7\%). Moreover, multimodal fusion significantly outperforms single modalities, with the skeletal modality alone achieving at most 89.1\%. This study provides a scalable solution for collaborative optimization of medical AI under privacy constraints.
Read the paper
Related studies
- Active Learning during Federated Learning for Object Detection
- Fairness-aware federated learning
- Decentralized Federated Learning
- Building a Personally Identifiable Information Recognizer in a Privacy Preserved Manner Using Automated Annotation and Federated Learning
- Preservation of Patient Level Privacy: Federated Classification and Calibration Models
- Efficient explainable federated learning
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.