Toshiki Hirao

Toshiki Hirao

Auto-generated profile, based on publicly available information

Toshiki Hirao

Nara Institute of Science and Technology

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
18 June 2024
Verified
Conference Paper
Open access
MWMiku Watanabe
Miku Watanabe
YKYutaro Kashiwa
Yutaro Kashiwa
BLBin Lin
Bin Lin
THToshiki Hirao
Toshiki Hirao
KYKen'IChi Yamaguchi
Ken'IChi Yamaguchi
HIHajimu Iida
Hajimu Iida
Code review is a critical but time-consuming process for ensuring code quality in modern software engineering. To alleviate the effort of reviewing source code, recent studies have investigated the possibility of automating the review process. Moreover, tools based on large language models such as ChatGPT are playing an increasingly important role in this vision. Understanding how these tools are used during code review can provide valuable insights for code review automation. This study investigates for what purposes developers use ChatGPT during code review and how developers react to the information and suggestions provided by ChatGPT. We manually analyze 229 review comments in 205 pull requests from 179 projects. We find that developers often use ChatGPT for outsourcing their work as frequently as asking for references. Moreover, we observe that only 30.7% of responses to the answers provided by ChatGPT are negative. We further analyze the reasons behind the negative reactions. Our results provide valuable insights for improving the effectiveness of LLMs in code reviews.

3 March 2020 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
THToshiki Hirao
Toshiki Hirao
SMShane McIntosh
Shane McIntosh
AIAkinori Ihara
Akinori Ihara
KMKenichi Matsumoto
Kenichi Matsumoto
Code review is a broadly adopted software quality practice where developers critique each others' patches. In addition to providing constructive feedback, reviewers may provide a score to indicate whether the patch should be integrated. Since reviewer opinions may differ, patches can receive both positive and negative scores. If reviews with divergent scores are not carefully resolved, they may contribute to a tense reviewing culture and may slow down integration. In this paper, we study patches with divergent review scores in the OPENSTACK and QT communities. Quantitative analysis indicates that patches with divergent review scores: (1) account for 15%-37% of patches that receive multiple review scores; (2) are integrated more often than they are abandoned; and (3) receive negative scores after positive ones in 70% of cases. Furthermore, a qualitative analysis indicates that patches with strongly divergent scores that: (4) are abandoned more often suffer from external issues (e.g., integration planning, content duplication) than patches with weakly divergent scores and patches without divergent scores; and (5) are integrated often address reviewer concerns indirectly (i.e., without changing patches). Our results suggest that review tooling should integrate with release schedules and detect concurrent development of similar patches to optimize review discussions with divergent scores. Moreover, patch authors should note that even the most divisive patches are often integrated through discussion, integration timing, and careful revision.

14 May 2016
Verified
Book Chapter
Open access
PDF file
THToshiki Hirao
Toshiki Hirao
AIAkinori Ihara
Akinori Ihara
YUYuki Ueda
Yuki Ueda
PPPassakorn Phannachitta
Passakorn Phannachitta
KMKen-Ichi Matsumoto
Ken-Ichi Matsumoto
Software code review systems are commonly used in software development. In these systems, many patches are submitted to improve the quality. To verify the quality, voting is commonly used by contributors; however, there still exists a major problem, namely, that reviewers do not always simply reach a broad agreement. In our previous study, we found that consensus is not usually reached, implying that an individual reviewer’s final decision usually differs from that of the majority of the other reviewers. In this study, we further investigate the reasons why such situations often occur, and provide suggestions for better handling of these problems. Our analysis of the Qt and OpenStack project datasets allow us to suggest that a patch owner should select more appropriate reviewers who often agree with others’ decisions.

1 December 2019 in IEICE Transactions on Information and Systems
Verified
Journal Article
Open access
PDF file
THToshiki Hirao
Toshiki Hirao
RKRaula Gaikovina Kula
Raula Gaikovina Kula
AIAkinori Ihara
Akinori Ihara
KMKenichi Matsumoto
Kenichi Matsumoto
Modern code review is a well-known practice to assess the quality of software where developers discuss the quality in a web-based review tool. However, this lightweight approach may risk an inefficient review participation, especially when comments becomes either excessive (i.e., too many) or underwhelming (i.e., too few). In this study, we investigate the phenomena of reviewer commenting. Through a large-scale empirical analysis of over 1.1 million reviews from five OSS systems, we conduct an exploratory study to investigate the frequency, size, and evolution of reviewer commenting. Moreover, we also conduct a modeling study to understand the most important features that potentially drive reviewer comments. Our results find that (i) the number of comments and the number of words in the comments tend to vary among reviews and across studied systems; (ii) reviewers change their behaviours in commenting over time; and (iii) human experience and patch property aspects impact the number of comments and the number of words in the comments.