Human–AI collaboration in deductive coding of classroom dialogue: prompt engineering and collaboration patterns
Abstract
Abstract Recent advances in large language models (LLMs) have created new opportunities for supporting qualitative analysis, yet little research has examined how such systems can be practically implemented for classroom dialogue analysis or how the human–artificial intelligence (AI) collaboration process unfolds in this context. This study investigates (1) how a lightweight configured conversational system can be developed to assist deductive dialogue coding and (2) what factors shape patterns of human–AI collaboration during its use. Using a design-based research (DBR) approach, the study iteratively developed and refined a Customised GPT Coding Assistant (CGCA) grounded in the Cambridge Dialogue Analysis Scheme (CDAS). The CGCA was tested and evaluated using standard classification metrics relative to human inter-coder agreement. To examine collaboration dynamics, a within-subject quasi-experimental design compared two workflow conditions (human-first vs. AI-first) with six educational researchers, supported by think-aloud protocols, GPT chat logs, and interviews. Results show that the CGCA can approach a moderate accuracy level when prompts are structured in ways aligned with LLM processing constraints, such as simplified reasoning steps and decision-tree-like logic. Beyond technical design, coding outcomes were shaped profoundly by the interaction between the CGCA system features, workflow sequencing, and coder positionality towards AI. Participants who engaged with the…
Read the paper
Related studies
- Regulation of nucleolus assembly by non-coding RNA polymerase II transcripts
- For a Healthy Histone Code, a Little SUMO in the Tail Keeps the Acetyl Away
- Discovering a binary CTCF code with a little help from BORIS
- Article Commentary: Stabilizing the Code–-Methods to Preserve RNA Prove Their Worth
- Foreword to the Special Issue for the RINTC (The Implicit Seismic Risk of Code-Conforming Structures) Project
- Commentary: Promoting Health and Well-being in Pediatric Primary Care Settings: Using Health and Behavior Codes at Routine Well-child Visits
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.