Evaluating Hybrid Retrieval-Augmented Generation for LLM-Based Python-to-Go Code Translation
Abstract
Python to Go translation requires choices that Python often leaves implicit: types, imports, errors, and APIs. The study asks whether Retrieval-Augmented Generation (RAG) [1] can improve function level translation on HumanEval-X. It builds an agent based pipeline with a semi automatically constructed knowledge base covering grammar mappings, CodeNet examples, API mappings, and Go documentation. MiniMax M2.5 is the main model, evaluated over six runs per condition; OpenAI GPT-5.4 is a supporting single run comparison. All outputs are evaluated without manual repair using Compilation@1 and Pass@1. The main result is negative: no static RAG preset improves MiniMax Pass@1 beyond run-to-run noise, and every OpenAI RAG variant stays at or below its 97.6% Pass@1 baseline. A script-based suitability analysis, run-1 retrieval logs, and a five-case retrieval inspection show that many tasks contain API, type, or idiom signals, and that retrieval often reaches such tasks. The six-run result shows that this coverage does not lead to reliable gains. The practical lesson is that retrieval should be tested on tasks with clear external-knowledge demand and precise retrieved context; adding more context by itself is not enough.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.