Local Retrieval-Augmented Generation System for Liquidity Risk Documents
Abstract
Since their emergence, large language models (LLMs) have become a substantial part of personal and professional life. Their use in organisations that handle sensitive and domain-specific data is however constrained, both by confidentiality requirements and by the models' limited knowledge of the internal data. This thesis investigates the feasibility of addressing both constraints by developing a locally deployed Retrieval-Augmented Generation (RAG) chatbot for liquidity-risk documents at a financial institution. An end-to-end RAG-based chatbot was implemented against a corpus of 10 PDF documents, with tables and equations correctly recognised and indexed during parsing. The system was evaluated on two complementary question sets, a set of 75 short factual question–answer pairs and a set of 5 broader questions requiring multi-document reasoning. In total, 20 different pipeline configurations were tested. The best performing configuration achieved an accuracy of 93\% on the factual set evaluated using an LLM-as-a-judge procedure. On the reasoning set, the same configuration received an average grade of 2.5/5 from a panel of liquidity-risk experts. As an external reference the reasoning answers were also compared with answers produced by the Microsoft 365 Copilot system already deployed at the institution. M365 Copilot achieved the higher average score of 3.3/5, but the locally deployed chatbot outperformed it on two of the individual reasoning questions, which is a notable…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.