Does allowing access to electronic differential diagnosis support threaten the reliability of a licensing exam?
Abstract
INTRODUCTION: Newer electronic differential diagnosis supports (EDSs) are efficient and effective at improving diagnostic skill. Although these supports are encouraged in practice, they are prohibited in medical licensing examinations. The purpose of this study is to determine how using an EDS impacts examinees' results when answering clinical diagnosis questions.
METHOD: The authors recruited 100 medical students from McMaster University (Hamilton, Ontario) to answer 40 clinical diagnosis questions in a simulated examination in 2021. Of these, 50 were first-year students and 50 were final-year students. Participants from each year of study were randomised into one of two groups. During the survey, half of the students had access to Isabel (an EDS) and half did not. Differences were explored using analysis of variance (ANOVA), and reliability estimates were compared for each group.
RESULTS: Test scores were higher for final-year versus first-year students (53 ± 13% versus 29 ± 10, p < 0.001) and higher with the use of EDS (44 ± 28% versus 36 ± 26%, p < 0.001). Students using the EDS took longer to complete the test (p < 0.001). Internal consistency reliability (Cronbach's alpha) increased with EDS use among final-year students but was reduced among first-year students, although the effect was not significant. A similar pattern was noted in item discrimination, which was significant.
CONCLUSION: EDS use during diagnostic licensing style questions was associated with modest…
How strong is the evidence?
Evidence Hub reads this as a randomized trial, done in people. Its evidence level is 2 of 5 (High · randomized trial). Randomized trials of a reasonable size are the strongest single-study design.
Read the paper
Related studies
- The Matrisome: In Silico Definition and In Vivo Characterization by Proteomics of Normal and Tumor Extracellular Matrices
- The Escherichia coli MG1655 in silico metabolic genotype: Its definition, characteristics, and capabilities
- In silico pharmacology for drug discovery: methods for virtual ligand screening and profiling
- In silico prediction of splice-altering single nucleotide variants in the human genome
- In silico clinical trials: how computer simulation will transform the biomedical industry
- In silico optimization of a guava antimicrobial peptide enables combinatorial exploration for peptide design
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.