Evaluation of Large Language Models for Blockchain Transaction Explanation
Abstract
This repository contains the supplementary materials for the study “Evaluation of Large Language Models for Blockchain Transaction Explanation”. The work evaluates GPT-5 and Mistral 7B, with and without Retrieval-Augmented Generation (RAG), for explaining Ethereum transactions to non-expert users. The evaluation combines a 99-participant user study, LLM-based questionnaire assessment, and an independent expert review of factual reliability. The repository provides the prompts, n8n workflows, and evaluation materials required to reproduce the experimental setup. Repository Structure and Content : 1. Orchestration Workflows (n8n) Multilogue RAG.json & Multilogue Ingest.json: JSON files containing the modular n8n workflow definitions for the document ingestion pipeline and the RAG generation orchestration. 2. Prompts & System Messages system_message_rag & system_message_brut: Text files containing the exact system messages and fixed prompt templates used to generate the explanations in both RAG and non-RAG settings. 3. Evaluation Case Studies (Inputs)These files represent the exploratory controlled sample of four Ethereum cases used in the study : 1_transaction_0x1f754646.json & 2_transaction_0xc15f880c.json: Data for the standard ETH transfer transactions, used as low-complexity baselines. 3_transaction_0x6fac588d.json: Data for the complex DeFi token swap executed on the Uniswap V3 protocol. 4_dashboard.md & dune.PNG: Visual analysis materials based on a Dune Analytics…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.