AgriMemSynth: a synthetic benchmark for memory and multi-hop reasoning in agricultural question answering
Abstract
Abstract Agricultural advisory systems often need to combine current observations, earlier visits, and crop-disease knowledge, but existing agricultural benchmarks mainly evaluate image classification or short-form question answering. We present AgriMemSynth, a synthetic benchmark framework for evaluating agricultural question answering under two complementary requirements: multi-session conversational memory and structured multi-hop reasoning. AgriMemSynth provides AgriConvMem, a set of 500 generated advisory conversations, and AgriMultiHop, a set of 2,000 generated reasoning questions. Using held-out test splits with 146 conversational-memory questions and 200 multi-hop questions, we compare five retrieval organizations under a shared answer-generation pipeline: retrieval-augmented generation, a neuroscientific memory system, a hybrid retriever, BM25 with semantic reranking, and a graph-based memory system. Semantic evaluation reveals three stable benchmark patterns. First, lexical metrics substantially underestimate answer correctness. Second, conversational memory is harder than generated multi-hop reasoning. Third, retrieval organization is task-dependent: RAG has the highest conversational-memory score (85.6 %, 95 % CI: 79.0-90.4), whereas MemoryGraph has the highest multi-hop score (97.0 %, 95 % CI: 93.6-98.6). Temporal questions remain the weakest conversational category. These results support AgriMemSynth as a controlled benchmark for studying how agricultural AI…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.