Design and Evaluation of a Retrieval-Augmented Generation Architecture for OWASP Security Data
Abstract
Project Description This archive contains a research artifact for the design, implementation, and evaluation of a Retrieval-Augmented Generation (RAG) system in the domain of application security. The artifact provides a complete, self-contained snapshot of the system architecture, curated evaluation data, and automated benchmarking pipelines. RAG System Architecture The archive includes a fully implemented RAG pipeline designed to ground large language model outputs in authoritative security documentation. The system follows a modular architecture with clearly separated components for document ingestion, semantic retrieval, and answer generation. The knowledge base is derived from OWASP Cheat Sheet Series and related security guidance, enabling security-focused question answering with reduced risk of hallucinated responses. Ground Truth Dataset A ground truth dataset is included to support systematic evaluation. The dataset consists of security-relevant questions and corresponding reference information derived from OWASP materials. It is intended exclusively for offline evaluation and benchmarking of RAG behavior and is not meant to serve as training data.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.