Effectiveness Evaluation of Retrieval-Augmented Generation and Chain-of-Thought Strategies for Large Language Model-Based News Classification
Abstract
Modern Large Language Models (LLMs) often struggle with news categorization due to a lack of contemporary context and flawed deductive processes. This study proposes a decentralized multi-agent architecture that decouples Retrieval-Augmented Generation (RAG) into specialized functional units. Utilizing an original seven-class news dataset, we executed twelve distinct experimental trials using Qwen and DeepSeek. These trials cross-examined Chain-of-Thought (CoT) prompts against varying retrieval depths (no retrieval, example-based retrieval, and discriminative retrieval). Our data reveals that the RAG component is the cornerstone of accuracy gains; notably, DeepSeek's performance surged from 50.95% to 83.81% upon Example-based Retrieval integration. While CoT provides a substantial 18.57% boost in isolation, its impact reaches a plateau once robust external knowledge is present. These results suggest that for specialized tasks like news filtering, refining knowledge retrieval mechanisms offers superior returns compared to escalating computational reasoning complexity, offering a strategic roadmap for engineering efficient multi-agent systems.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.