Developing an Evaluation Framework for Assessing Large-Language Models in the Climate Domain
Abstract
Large-Language Models (LLMs) have emerged as powerful agents for quickly returning relevant information, especially through the use of Retrieval-Augmented Generation (RAG), along with being useful conversationalists through their use of human-readable text.However, LLMs are known to often hallucinate and defend incorrect output.Evaluation, therefore, is a critical component in asserting the trustworthiness of an LLM.Through various, testing a comprehensive evaluation dataset ensures that an LLM is acting in a way that is expected for the end-user.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.