Do Large Multimodal Models Understand Construction Drawings? An Evaluation of Visually Grounded Workflows
Abstract
Automated information extraction from construction drawings is a critical yet challenging task. While traditional approaches, from image processing to deep learning, have made great progress, they often suffer from inconsistent performance, extensive task-specific training, and narrow scope. Large Multimodal Models (LMMs) offer a promising alternative due to their vast knowledge bases and instruction-following capabilities, however, they struggle with spatial reasoning and dense high-fidelity images. This preliminary study evaluates the performance of two off-the-shelf Multimodal Generative AI (GenAI) workflows against two specialized architectures: an image-based Retrieval Augmented Generation (RAG) system and an image-based Multi-Agent workflow. By evaluating these workflows against a targeted set of benchmarking questions, this work provides a preliminary assessment of current GenAI capabilities in drawing interpretation and highlights specific areas for improvement in developing more robust automated workflows.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.