Comparing LLM and Human Expert Coding in Qualitative Research: A Case Study on Emotional Intelligence
Abstract
Large language models (LLMs) are increasingly used for qualitative coding, yet it remains unclear how closely their coding outputs align with frameworks developed by human experts. Existing studies tend to report overall similarity but rarely distinguish thematic granularity from dimension-level semantic alignment or systematically identify which constructs are stably recovered. This study therefore compares human expert coding with three zero-shot LLM-generated qualitative analyses using a structured framework-level mapping method. The method maps human-coded dimensions to LLM-generated qualitative frameworks and classifies each relationship as direct, partial or merged, or not recovered. To quantify alignment, three evaluation metrics are used: Direct Coverage (DC), which measures strict boundary recovery; Broad Coverage (BC), which includes partial or merged mappings; and Frequency-Weighted Mapping (FWM), which weights each dimension by its human coding frequency to reflect construct prominence. The method is demonstrated using a human-derived emotional-intelligence framework developed from 34 Chinese behavioral-event interviews and three zero-shot frameworks generated from the same corpus by DeepSeek V4.0 PRO, Qwen 3.6, and GPT 5.6 under an identical prompt. Two senior experts jointly evaluated 36 dimension-level relationships. The three LLM outputs achieved direct coverage ranging from 75.0% to 91.7%, broad coverage from 83.3% to 100.0%, and frequency-weighted mapping…
Read the paper
Related studies
- Regulation of nucleolus assembly by non-coding RNA polymerase II transcripts
- For a Healthy Histone Code, a Little SUMO in the Tail Keeps the Acetyl Away
- Discovering a binary CTCF code with a little help from BORIS
- Article Commentary: Stabilizing the Code–-Methods to Preserve RNA Prove Their Worth
- Foreword to the Special Issue for the RINTC (The Implicit Seismic Risk of Code-Conforming Structures) Project
- Commentary: Promoting Health and Well-being in Pediatric Primary Care Settings: Using Health and Behavior Codes at Routine Well-child Visits
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.