Hierarchical and Stepwise Evaluation of Engineering Project Documents Using Large Language Model Technology
Abstract
Engineering project documents are critical artifacts that govern the planning, execution, and compliance of complex engineering endeavors. However, the evaluation of these documents remains a labor-intensive and error-prone process. This paper proposes a novel Hierarchical and Stepwise Evaluation Framework that leverages Large Language Model (LLM) technology to systematically assess engineering project documents. Our framework operates across three hierarchical levels—document, section, and element—and employs a four-stage stepwise evaluation pipeline comprising structural analysis, content verification, compliance checking, and quality assessment. We implement this framework using state-of-the-art LLMs enhanced with domain-specific prompting strategies and retrieval-augmented generation. Experimental evaluation on a dataset of 847 engineering project documents demonstrates that HSEF achieves an overall evaluation accuracy of 87.3%, with precision of 0.856 and recall of 0.879, significantly outperforming baseline methods. The framework reduces manual evaluation time by approximately 73% while maintaining consistency with expert assessments (Cohen's κ = 0.81). Our results indicate that hierarchical decomposition combined with stepwise processing enables more accurate and interpretable document evaluation, providing a promising direction for automated quality assurance in engineering documentation.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.