Preliminary Evaluation of a Stage-Structured Retrieval-Augmented Systematic Review Learning Agent for Undergraduate Nursing Students: A Proof-of-Concept Study
Abstract
BACKGROUND
Generative artificial intelligence (AI) may support systematic review learning, but general-purpose chatbots and workflow tools do not explicitly teach methodological reasoning.
PURPOSE
To develop a stage-structured retrieval-augmented generation learning agent and assess its perceived pedagogical fit and technical performance.
METHODS
The tool combined planning, query rewriting, iterative retrieval, and context-sufficiency checking across 6 stages: topic selection, review question framing, search strategy, screening and management, critical appraisal and data extraction, and synthesis and reporting. Six nurse educators and 4 undergraduate nursing students completed author-developed questionnaires assessing perceived pedagogical fit and AI performance.
RESULTS
The tool generated stage-aligned guidance across the review process. Participants reported moderate perceived pedagogical fit (M = 3.90, standard deviation = 0.45) and high perceived AI performance (M = 4.00, standard deviation = 0.25); personalization required improvement.
CONCLUSIONS
The tool was feasible as an instructional scaffold, but findings reflect perceptions from a small sample. Larger studies should assess objective learning outcomes, methodological accuracy, and responsible AI use.
How strong is the evidence?
Evidence Hub reads this as a systematic review, done in people. Its evidence level is 1 of 5 (Highest · pooled evidence). Pooled evidence from several studies sits at the top of the evidence pyramid.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.