Brittany Reid

Brittany Reid

Auto-generated profile, based on publicly available information

Brittany Reid

Graduate School of Science and Technology Nara Institute of Science and Technology (NAIST) Japan

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 March 2026 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
MWMiku Watanabe
Miku Watanabe
HLHao Li
Hao Li
YKYutaro Kashiwa
Yutaro Kashiwa
BRBrittany Reid
Brittany Reid
HIHajimu Iida
Hajimu Iida
AHAhmed E. Hassan
Ahmed E. Hassan
Large language models (LLMs) are increasingly being integrated into software development processes. The ability to generate code and submit pull requests with minimal human intervention, through the use of autonomous AI agents, is poised to become a standard practice. However, little is known about the practical usefulness of these pull requests and the extent to which their contributions are accepted in real-world projects. In this paper, we empirically study 567 GitHub pull requests (PRs) generated using Claude Code, an agentic coding tool, across 157 diverse open-source projects. Our analysis reveals that developers tend to rely on agents for tasks such as refactoring, documentation, and testing. The results indicate that 83.8% of these agent-assisted PRs are eventually accepted and merged by project maintainers, with 54.9% of the merged PRs being integrated without further modification. The remaining 45.1% require additional changes and benefit from human revisions, especially for bug fixes, documentation, and adherence to project-specific standards. These findings suggest that while agent-assisted PRs are largely acceptable, they still benefit from human oversight and refinement.

23 February 2023 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
BRBrittany Reid
Brittany Reid
MDMarcelo D'AMorim
Marcelo D'AMorim
MWMarkus Wagner
Markus Wagner
CTChristoph Treude
Christoph Treude
Code reuse is an important part of software development. The adoption of code reuse practices is especially common among Node.js developers. The Node.js package manager, NPM, indexes over 1 Million packages and developers often seek out packages to solve programming tasks. Due to the vast number of packages, selecting the right package is difficult and time consuming. With the goal of improving productivity of developers that heavily reuse code through third-party packages, we present Node Code Query (NCQ), a Read-Eval-Print-Loop environment that allows developers to 1) search for NPM packages using natural language queries, 2) search for code snippets related to those packages, 3) automatically correct errors in these code snippets, 4) quickly setup new environments for testing those snippets, and 5) transition between search and editing modes. In two user studies with a total of 20 participants, we find that participants begin programming faster and conclude tasks faster with NCQ than with baseline approaches, and that they like, among other features, the search for code snippets and packages. Our results suggest that NCQ makes Node.js developers more efficient in reusing code.

4 November 2026 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
WCWorawalan Chatlatanagulchai
Worawalan Chatlatanagulchai
HLHao Li
Hao Li
YKYutaro Kashiwa
Yutaro Kashiwa
BRBrittany Reid
Brittany Reid
KTKundjanasith Thonglek
Kundjanasith Thonglek
HIHajimu Iida
Hajimu Iida
Agentic coding tools receive goals written in natural language, break them down into specific tasks, and write or execute code with minimal human intervention. Central to this process are agent context files (e.g., AGENTS.md and CLAUDE.md) that provide persistent, project-level instructions. In this article, we conduct the first large-scale empirical study of 2,303 agent context files from 1,925 repositories to characterize their structure, maintenance, and content. We find that these files are not static documentation but complex, difficult-to-read artifacts that evolve like configuration code through frequent, small additions. Our content analysis of 16 instruction types shows that developers prioritize functional context, such as test procedures (75.9%), implementation details (70.8%), and architecture (68.1%). We also identify a significant gap: non-functional requirements such as security (14.8%) and performance (14.5%) are rarely specified. These findings indicate that while developers use context files to make agents functional, they provide few guardrails to ensure that agent-written code is secure or performant, highlighting the need for improved tools and practices.

12 February 2025 in arXiv
Verified
Preprint
Open access
PDF file
SKSushawapak Kancharoendee
Sushawapak Kancharoendee
TPThanat Phichitphanphong
Thanat Phichitphanphong
CJChanikarn Jongyingyos
Chanikarn Jongyingyos
BRBrittany Reid
Brittany Reid
RKRaula Gaikovina Kula
Raula Gaikovina Kula
TSThanwadee Sunetnanta
Thanwadee Sunetnanta
Open-source projects are essential to software development, but publicly disclosing vulnerabilities without fixes increases the risk of exploitation. The Open Source Security Foundation (OpenSSF) addresses this issue by promoting robust security policies to enhance project security. Current research reveals that many projects perform poorly on OpenSSF criteria, indicating a need for stronger security practices and underscoring the value of SECURITY$.$md files for structured vulnerability reporting. This study aims to provide recommendations for improving security policies. By examining 679 open-source projects, we find that email is still the main source of reporting. Furthermore, we find that projects without SECURITY$.$md files tend to be less secure (lower OpenSSF scores). Our analysis also indicates that, although many maintainers encourage private reporting methods, some contributors continue to disclose vulnerabilities publicly, bypassing established protocols. The results from this preliminary study pave the way for understanding how developers react and communicate a potential security threat. Future challenges include understanding the impact and effectiveness of these mechanisms and what factors may influence how the security threat is addressed.

6 February 2026 in Empirical Software Engineering
Verified
Journal Article
Open access
PDF file
MCMorakot Choetkiertikul
Morakot Choetkiertikul
SKSushawapak Kancharoendee
Sushawapak Kancharoendee
CJChanikarn Jongyingyos
Chanikarn Jongyingyos
TPThanat Phichitphanphong
Thanat Phichitphanphong
CRChaiyong Ragkhitwetsagul
Chaiyong Ragkhitwetsagul
TSThanwadee Sunetnanta
Thanwadee Sunetnanta
With security in open-source software development increasingly becoming crucial, security policies are one way to manage vulnerabilities and guide users toward safe practices. To support secure development, platforms like GitHub provide a dedicated section for security policies within repositories. Existing studies focus on the adoption of security policies. However, the detailed content of the security policies has not been examined. Our study aims to fill this gap by analyzing the security policies of 679 PyPI Python libraries hosted on GitHub. We examine the characteristics and content of existing policies and investigate the relationship with project characteristics and recommended security practices by comparing security practice assessments between projects with and without established security policies. The result indicates that projects with security.md shows stronger recommended security practices. This study highlights the importance of adopting a clear and comprehensive security policy to enhance the overall security practices of open-source projects.

16 June 2025 in arXiv
Verified
Preprint
Open access
PDF file
BABayu Fedra Abdullah
Bayu Fedra Abdullah
YNYusuf Sulistyo Nugroho
Yusuf Sulistyo Nugroho
BRBrittany Reid
Brittany Reid
RKRaula Gaikovina Kula
Raula Gaikovina Kula
KSKazumasa Shimari
Kazumasa Shimari
KMKenichi Matsumoto
Kenichi Matsumoto
Large Language Models (LLMs) are increasingly used in software security, but their trustworthiness in generating accurate vulnerability advisories remains uncertain. This study investigates the ability of ChatGPT to (1) generate plausible security advisories from CVE-IDs, (2) differentiate real from fake CVE-IDs, and (3) extract CVE-IDs from advisory descriptions. Using a curated dataset of 100 real and 100 fake CVE-IDs, we manually analyzed the credibility and consistency of the model's outputs. The results show that ChatGPT generated plausible security advisories for 96% of given input real CVE-IDs and 97% of given input fake CVE-IDs, demonstrating a limitation in differentiating between real and fake IDs. Furthermore, when these generated advisories were reintroduced to ChatGPT to identify their original CVE-ID, the model produced a fake CVE-ID in 6% of cases from real advisories. These findings highlight both the strengths and limitations of ChatGPT in cybersecurity applications. While the model demonstrates potential for automating advisory generation, its inability to reliably authenticate CVE-IDs or maintain consistency upon re-evaluation underscores the risks associated with its deployment in critical security tasks. Our study emphasizes the importance of using LLMs with caution in cybersecurity workflows and suggests the need for further improvements in their design to improve reliability and applicability in security advisory generation.