A Multimodal Machine Learning Approach to Diagnosis, Prognosis, and Treatment Prediction for Neurodegenerative Diseases and Cancer
Abstract
Despite early diagnosis and estimating the future course of neurodegenerative and cancerous diseases being integral for survival of patients, clinical and algorithmic methods fail to effectively utilize the multimodal data available, are time-inefficient, and expensive, making it difficult to access accurate screenings for these diseases. Therefore, a novel end-to-end quantum machine learning approach using multiple data modalities for the identification of diagnoses, prognoses, and effective treatments is proposed. In a procedural flow, data is sourced from one or more of the following: CT scan images, webcam, patient-physician audio, Whole Slide Images, and clinical data. For image data, a Convolutional Neural Network, is employed to detect high level features. With text-based clinical data (including audio-derived data), a Bidirectional Encoder Representation model is used to extract text embeddings. For video data, pupil progression and average fixation duration features are manually crafted. All feature vectors are concatenated, normalized, passed through a Deep Neural Network, and then mapped to one of 38 neurodegenerative and cancerous diseases. For prognosis, features are pooled, concatenated with the diagnosis feature vector, and passed through another neural network with an output of survival times. Treatment prediction involves an information-retrieval task matching feature vectors to treatments/drug lists from the FDA. The proposed approach was tested on 5,000…
Read the paper
Related studies
- In silico toxicology protocols
- QM7-X, a comprehensive dataset of quantum-mechanical properties spanning the chemical space of small organic molecules
- Quantum Informed Machine-Learning Potentials for Molecular Dynamics Simulations of CO2's Chemisorption and Diffusion in Mg-MOF-74
- In-silico strategies in nano-drug design: Bridging nanomaterials and pharmacological applications
- mRNA codon optimization with quantum computers
- The computational power of the human brain
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.