The Effects of GitHub Copilot on Computing Students' Programming Effectiveness, Efficiency, and Processes in Brownfield Coding Tasks
Abstract
When graduates of computing degree programs enter the software industry, they will most likely join teams working on legacy code bases developed by people other than themselves. In these so-called brownfield software development settings, generative artificial intelligence (GenAI) coding assistants like GitHub Copilot are rapidly transforming software development practices, yet the impact of GenAI on student programmers performing brownfield development tasks remains underexplored. This paper investigates how GitHub Copilot influences undergraduate students’ programming performance, behaviors, and understanding when completing brownfield programming tasks in which they add new code to an unfamiliar code base. We conducted a controlled experiment in which 10 undergraduate computer science students completed highly similar brownfield development tasks with and without Copilot in a legacy web application. Using a mixed-methods approach combining performance analysis, behavioral analysis, and exit interviews, we found that students completed tasks 34.9% faster (p < 0.05) and made 50.0% more solution progress (p < 0.05) when using Copilot. Moreover, our analysis revealed that, when using Copilot, students spent 10.6% less time manually writing code (p < 0.05), and 11.6% less time conducting web searches (p < 0.05), providing evidence of a fundamental shift in how they engaged in programming. In exit interviews, students reported concerns about not understanding how or why Copilot…
Read the paper
Related studies
- Regulation of nucleolus assembly by non-coding RNA polymerase II transcripts
- For a Healthy Histone Code, a Little SUMO in the Tail Keeps the Acetyl Away
- Discovering a binary CTCF code with a little help from BORIS
- Article Commentary: Stabilizing the Code–-Methods to Preserve RNA Prove Their Worth
- Foreword to the Special Issue for the RINTC (The Implicit Seismic Risk of Code-Conforming Structures) Project
- Commentary: Promoting Health and Well-being in Pediatric Primary Care Settings: Using Health and Behavior Codes at Routine Well-child Visits
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.