Topic modeling using models such as Latent Dirichlet Allocation (LDA) is a text mining technique to extract human-readable semantic “topics” (i.e., word clusters) from a corpus of textual documents. In software engineering, topic modeling has been used to analyze textual data in empirical studies (e.g., to find out what developers talk about online), but also to build new techniques to support software engineering tasks (e.g., to support source code comprehension). Topic modeling needs to be applied carefully (e.g., depending on the type of textual data analyzed and modeling parameters). Our study aims at describing how topic modeling has been applied in software engineering research with a focus on four aspects: (1) which topic models and modeling techniques have been applied, (2) which textual inputs have been used for topic modeling, (3) how textual data was “prepared” (i.e., pre-processed) for topic modeling, and (4) how generated topics (i.e., word clusters) were named to give them a human-understandable meaning. We analyzed topic modeling as applied in 111 papers from ten highly-ranked software engineering venues (five journals and five conferences) published between 2009 and 2020. We found that (1) LDA and LDA-based techniques are the most frequent topic modeling techniques, (2) developer communication and bug reports have been modelled most, (3) data pre-processing and modeling parameters vary quite a bit and are often vaguely reported, and (4) manual topic naming (such as deducting names based on frequent words in a topic) is common.
Software quality attributes (e.g., security, performance) influence software architecture design decisions, e.g., when choosing technologies, patterns or tactics. As software developers are moving from big upfront design to an evolutionary or emerging design, the architecture of a system evolves as more functionality is added. In agile software development, functional user requirements are often expressed as user stories. Quality attributes might be implicitly referenced in user stories. To support a more systematic analysis and reasoning about quality attributes in agile development projects, this paper explores how to automatically identify quality attributes from user stories. This could help better understand relevant quality attributes (and potential architectural key drivers) before analysing product backlogs and domains in detail and provides the “bigger picture” of potential architectural drivers for early architecture decision making. The goal of this paper is to present our vision and preliminary work towards understanding whether user stories do include information about quality attributes at all, and if so, how we can identify such information in an automated manner.
12 November 2021 in IEEE Transactions on Education
Background: Peer evaluation in software engineering (SE) project courses enhances the learning experience of students. It also helps instructors monitor and assess both teams and individual students. Peer evaluations might influence the way individual students and teams work; therefore, the quality of the peer evaluations should be tracked through the project course. Contribution: In this article, we analyzed the quality and scoring behavior of students in peer evaluation in an undergraduate SE project course over three years. Research Questions: RQ1: What is the quality of peer evaluation of undergraduate students in a SE project course? RQ2: How do undergraduate students in an SE project course score each other? Methodology: The quality of peer evaluation (length, level of detail, etc.) and scoring of peers based on various aspects of peer evaluations of third-year students in a year-long SE project course were studied. Taking into account the grade students received at the end of the course (A, B, C, and F-calibers), peer evaluations were categorized, analyzed over time, and compared between students calibers. Findings: After analyzing 6854 peer evaluations from 193 students, it was found that the quality of peer evaluations across students was mostly consistent throughout the course. Also, it was observed that quantitative aspects of the peer evaluation were scored similarly across student calibers. However, the qualitative aspects of the peer evaluation were impacted by the caliber of students. These findings suggest that weaker students (i.e., C-caliber students) generally receive better quality peer evaluations than stronger students (i.e., A-caliber students). Finally, a preliminary analysis showed a positive connotation of emotions and sentiments found in the textual feedback delivered by students.
User stories are increasingly adopted as the basis of requirement engineering artefacts in Agile Software Development. Surveys have shown that user stories are perceived as being effective at describing the main goals of a system. But the continuous management of a product backlog may be particularly time-consuming and error-prone, especially when assessing the quality or scope of user stories and keeping an eye on the system's big picture. On the other hand, models have been recognised as effective tools for communication and analysis purposes. In this research, we propose a generative approach to create robustness diagrams, i.e. a form of semi-formal use case scenarios, from the automated analysis of user stories. Stories are transformed into diagrams, enabling requirement engineers and users to validate the main concepts and functional steps behind stories and discover malformed or redundant stories. Such models also open the door for automated systematic analysis.