A Persona-based Automated Evaluation Framework for Intelligent AI Teaching Assistants
Abstract
To address global teacher shortages and the need for personalized learning, we develop an intelligent teaching assistant leveraging multimodal large language models, agentic retrieval-augmented generation, and function calling. Unlike standard models, our system provides verifiable pedagogical guidance by reasoning across heterogeneous resources, including lecture slides and instructional videos. To overcome the scarcity of real-world datasets and high human evaluation costs, we propose a scalable, human-annotation-free framework utilizing simulated learner agents grounded in the Big Five personality theory. This allows for systematic assessment across 18 distinct student personas. We introduce a novel process-level metric, the dialogue recovery rate, and a dynamic adaptive policy to mitigate conversational deadlocks. Experimental results across 1,800 simulated dialogues using the Qwen3 family (8B, 14B, 32B) and its Thai-variant (Typhoon 2.5) reveal that Qwen3-14B attains the highest robustness and aggregate tutor-performance score within the proposed framework (72.6%). Analysis demonstrates significant correlations between learner traits and performance: Conscientiousness positively correlates with success (r = 0.49), while Extraversion is negatively associated with structural adherence (r = −0.54). This work establishes a reproducible benchmarking protocol for persona-aware, adaptive intelligent tutoring systems.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.