Differences in cohort study data affect external validation of artificial intelligence models for predictive diagnostics of dementia - lessons for translation into clinical practice
Abstract
Artificial intelligence (AI) approaches pose a great opportunity for individualized, pre-symptomatic disease diagnosis which plays a key role in the context of personalized, predictive, and finally preventive medicine (PPPM). However, to translate PPPM into clinical practice, it is of utmost importance that AI-based models are carefully validated. The validation process comprises several steps, one of which is testing the model on patient-level data from an independent clinical cohort study. However, recruitment criteria can bias statistical analysis of cohort study data and impede model application beyond the training data. To evaluate whether and how data from independent clinical cohort studies differ from each other, this study systematically compares the datasets collected from two major dementia cohorts, namely, the Alzheimer’s Disease Neuroimaging Initiative (ADNI) and AddNeuroMed. The presented comparison was conducted on individual feature level and revealed significant differences among both cohorts. Such systematic deviations can potentially hamper the generalizability of results which were based on a single cohort dataset. Despite identified differences, validation of a previously published, ADNI trained model for prediction of personalized dementia risk scores on 244 AddNeuroMed subjects was successful: External validation resulted in a high prediction performance of above 80% area under receiver operator characteristic curve up to 6 years before dementia…
How strong is the evidence?
Evidence Hub reads this as a cohort study, done in people. Its evidence level is 3 of 5 (Moderate · trial / cohort / secondary analysis). Trials without a large randomized design, cohorts and secondary analyses give moderate evidence. Reported size: 244 participants or records.
Read the paper
Related studies
- Improving sleep quality leads to better mental health: A meta-analysis of randomised controlled trials
- Tocilizumab in patients with severe COVID-19: a retrospective cohort study
- A new aging measure captures morbidity and mortality risk across diverse subpopulations from NHANES IV: A cohort study
- Fruit, vegetable, and legume intake, and cardiovascular disease and deaths in 18 countries (PURE): a prospective cohort study
- Are noise and air pollution related to the incidence of dementia? A cohort study in London, England
- Long COVID symptoms and duration in SARS-CoV-2 positive children — a nationwide cohort study
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.