Copilot or Culprit? How AI Is Changing Open-Source Bug Reporting and Maintenance
Yinuo Liu, Yunlong Lyu, Zichao Luo, Tian Dong, Zian Luo, Peng Chen, Hao Chen
Abstract
AI-generated bug reports have drawn criticism for unreliable claims and faulty reproducers that burden maintainers. However, as LLM capabilities improve, some maintainers have begun to express more positive views on AI use in bug reporting and maintenance. Understanding this evolution requires examining whether AI-involved reports are becoming more reliable, which uses of AI are associated with valid findings, and how maintainers investigate these reports and act on them. To this end, we conduct the first large-scale study of 120,918 bug reports from 101 open-source projects between January 2024 and September 2026, tracing AI involvement from discovery and reporting through validation and resolution. Our findings show that AI-involved reports become more complete over time, with a growing share of reports confirmed as valid. In bug reporting, AI for reproduction and diagnosis are associated with higher validity, whereas report writing is not. Extends into maintenance, maintainers increasingly use AI to investigate code and recommend fixes. However, they devote more effort to AI-assisted reports of human-discovered bugs than to reports of AI-discovered bugs. These findings highlight both AI's growing participation in bug investigation and the continuing need for verification and follow-up. Drawing on these findings, we provide practical guidance for turning AI-generated findings into verifiable reports: emphasizing reproducibility, verification, and accountability. We followed this guidance when submitting 53 reports on AI-discovered bugs, of which 69.81% were accepted by maintainers as of writing.
Create a lesson
Related papers
Adopting Embedded AI Through an Innovation Team
Viktoria Stray, Kristina Kantima Østvold, Agnes Hunstad Birkemo et al.
From Code Review Comments to Reuse in Production: An Industrial Study of Extracted Rules
Mutsuha Asada, Yuhei Nakasaka, Keito Tabuchi et al.
Characterizing and Detecting Unfaithful Work Reports by Coding Agents
Wenhao Yang, Runzhi He, Minghui Zhou et al.
Technology-Enabled Fuel Supply Chain Resilience: An Integrated Information System for Real-Time Monitoring and Decision Support
Mohamed Loutis, Esma Aïmeur
Behavioral Containment: Source-to-Binary Function Matching for 1-Day Vulnerability Detection
Yifei Yang, Xuejiao Luo, Penghui Li
Ontology of Software Maintenance
Luis Olsina, Pablo Becker, Maria Fernanda Papa