Retrieval-Augmented Generation for Internal Documentation Querying and Sequence Code Generation : A Proof-of-Concept Study with Data Confidentiality Considerations
Abstract
As organizational data remains scattered across internal systems, retrieving relevant information from proprietary documentation is time-consuming and inefficient, while conventional large language model applications lack direct access to such data. The thesis is done for Murata Electronics Oy, and the main objective of this work was to build a Retrieval-Augmented Generation enhanced AI assistant to better obtain information from internal documents. Additionally, the assistant was developed as a proof of concept to automatise sequence code generation. The sequence code is used to build test cases to examine the performance of MEMS sensors used in various domains, including health technology applications. The minor objective of this work was to gain insights and perform risk analysis to identify possible pitfalls in LLM applications considering data confidentiality. The dataset used for the AI assistant consisted of codebase function documentation and existing code examples. An iterative approach was used to develop the RAG pipeline, starting with local embedding and language models and progressing to larger, cloud-based models in the main application. The architectural design of the RAG assistant was Naïve, using simple chunking and retrieval mechanisms. Evaluation of the system was based on answer correctness of the local and cloud hosted models in terms of general question-answer format, answer robustness and ultimately, the ability to produce runnable sequence level code.…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.