Building CAR-E: A Novel Artificial Intelligence Agent for Coaching Conversations
Abstract
BACKGROUND AND NEED FOR INNOVATION: Coaching has many benefits in medical education, but large-scale implementation is constrained by faculty bandwidth, scheduling, cost, and power differentials. Artificial intelligence (AI), specifically large language models (LLMs), offers potential solutions to augment coaching, but limitations remain and refining performance is necessary to create valuable coaching conversations.
STEPS TAKEN FOR DEVELOPMENT AND IMPLEMENTATION OF INNOVATION: The authors created Coaching with AI-Reinforced Education (CAR-E), an AI coaching agent built using an LLM. Using layered architecture, CAR-E couples real-time speech input and output, retrieval-augmented generation (RAG) for evidence-based coaching, and dual memory system (short-term context and long-term history) to sustain longitudinal dialogue. An iterative design refined CAR-E with users who voluntarily engaged in coaching conversations and provided feedback. Transcripts and user feedback were analyzed to improve CAR-E.
EVALUATION OF INNOVATION: During the pilot, 37 medical trainees and faculty engaged in coaching conversations. Transcripts showed diverse topics discussed and many strengths and opportunities for improvement. Users reported that CAR-E facilitated self-reflection, clarified goals, and broke complex problems into actionable steps. They also noted formulaic questioning that felt repetitive, superficial attempts to display empathy and difficulty moving conversations forward. Many…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.