Toward Ludic-Aware Narrative Generation: A Neuro-Symbolic Framework for Evaluating Playability in LLM-Generated Backstories
Abstract
Although Large Language Models (LLMs) produce fluent prose, they routinely tend to generate narratively sterile content for interactive media—backstories in which every conflict has been resolved before play begins, giving rise to what we call the Closure Paradox. Moreover, current evaluation metrics—from n-gram overlap to retrieval-augmented faithfulness scores—inadvertently reward this exact type of closure in cases in which it is undesired, leaving game designers without tools to measure whether a generated character is actually useful as a foundation for play. We propose a dual-axis evaluation framework that separates Temporal Consistency from Ludic Potential. Two new metrics anchor the framework: the Ludic Potential Index (LPI), a weighted measure of a backstory’s openness to future play, and the Potential Conflict Value (PCV), a per-event measure of playability fuel. The framework is implemented as a three-agent pipeline—Notary, Judge, and Stress-Tester—whose judge layers are stated in G-Eval terms so that they can be driven from standard open-source evaluation harnesses such as DeepEval and TruLens, and is validated through synthetic stress testing in which a separate LLM agent attempts to instantiate a Session 1 encounter from the generated backstory. The paper includes a comparative analysis of a correction mechanism implemented during generation (gated generation—where explicit metrics are applied to filter or constrain outputs) versus direct generation in the…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.