Requirement–State Transition Diagram Dataset for LLM Evaluation
Abstract
This dataset contains 80 software requirement cases paired with UML state transition diagrams represented in PlantUML. It supports research on LLM-based diagram generation, retrieval-augmented generation, automated validation, iterative repair, semantic evaluation, and human and LLM-based quality assessment. Each case includes: an original natural-language requirement; an aligned requirement; a structured requirement specification; a reference PlantUML diagram; and a PNG rendered from the PlantUML source. The cases cover multiple software application domains and varying levels of diagram complexity. The archive also provides a dataset manifest, data dictionary, README, license, and citation metadata. The dataset is intended for software engineering research and education, particularly experiments involving requirement-to-diagram generation, syntactic and structural validation, violation-specific repair, semantic state matching, and LLM-as-a-Judge evaluation.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.