Scrub Data: A Framework for Reproducible Data Curation with AI Coding Agents
Abstract
Data curation—the process of collecting, cleaning, and joining raw datasets for unified analysis—is a time-consuming yet crucial part of any data science project. With the emergence of powerful agentic coding tools, it is tempting to “vibe curate— i . e ., to prompt AI agents to perform data curation operations and blindly trust their outputs in order to finish work more quickly. However, this can exacerbate two key challenges that already plague manual data curation workflows: reproducibility —the ability to trace the exact set of modifications applied to raw datasets—and verifiability —the ability to audit changes and confirm that data is processed properly. To address these issues, we introduce Scrub Data , a framework that enables the use of AI agents in data curation workflows while transparently maintaining both reproducibility and verifiability. The framework involves an interactive data curation loop between a user and an AI agent, backed by a data provenance graph that tracks and versions all dataset updates. The data graph requires that all updates are formatted as self-contained executable steps, allowing any version of a dataset to be reconstructed from raw data by playing forward the transformations stored in the graph. This workflow takes place within a lightweight web application that is designed to be modified by users and agents in real-time to create custom visualization tools for identifying data curation needs and verifying outcomes. We demonstrate the…
Read the paper
Related studies
- Regulation of nucleolus assembly by non-coding RNA polymerase II transcripts
- For a Healthy Histone Code, a Little SUMO in the Tail Keeps the Acetyl Away
- Discovering a binary CTCF code with a little help from BORIS
- Article Commentary: Stabilizing the Code–-Methods to Preserve RNA Prove Their Worth
- Foreword to the Special Issue for the RINTC (The Implicit Seismic Risk of Code-Conforming Structures) Project
- Commentary: Promoting Health and Well-being in Pediatric Primary Care Settings: Using Health and Behavior Codes at Routine Well-child Visits
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.