Machine learning algorithms for systematic review: reducing workload in a preclinical review of animal studies and reducing human screening error
Abstract
BackgroundHere, we outline a method of applying existing machine learning (ML) approaches to aid citation screening in an on-going broad and shallow systematic review of preclinical animal studies. The aim is to achieve a high-performing algorithm comparable to human screening that can reduce human resources required for carrying out this step of a systematic review.MethodsWe applied ML approaches to a broad systematic review of animal models of depression at the citation screening stage. We tested two independently developed ML approaches which used different classification models and feature sets. We recorded the performance of the ML approaches on an unseen validation set of papers using sensitivity, specificity and accuracy. We aimed to achieve 95% sensitivity and to maximise specificity. The classification model providing the most accurate predictions was applied to the remaining unseen records in the dataset and will be used in the next stage of the preclinical biomedical sciences systematic review. We used a cross-validation technique to assign ML inclusion likelihood scores to the human screened records, to identify potential errors made during the human screening process (error analysis).ResultsML approaches reached 98.7% sensitivity based on learning from a training set of 5749 records, with an inclusion prevalence of 13.2%. The highest level of specificity reached was 86%. Performance was assessed on an independent validation dataset. Human errors in the training…
How strong is the evidence?
Evidence Hub reads this as a systematic review, done in computer models (in silico), not in people. Its evidence level is 5 of 5 (Very low · case / narrative / animal / in vitro / in silico). Case reports, narrative reviews, and animal, cell or computer-model studies give the weakest evidence for human health.
Read the paper
Related studies
- Challenges and opportunities with drug repurposing: finding strategies to find alternative uses of therapeutics
- Challenges in Chagas Disease Drug Development
- T Cell Specificity: A Great Challenge in Chagas Disease
- Benznidazole treatment leads to DNA damage in Trypanosoma cruzi and the persistence of rare widely dispersed non-replicative amastigotes in mice
- Rapid Molecular Detection of Terbinafine-resistant Dermatophytes
- Identifying novel drugs with new modes of action for neglected tropical fungal skin diseases (fungal skinNTDs) using an Open Source Drug discovery approach
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.