In recent years, numerous Machine Learning (ML) models, including Deep Learning (DL) and classic ML models, have been developed to detect software vulnerabilities. However, there is a notable lack of comprehensive and systematic surveys that summarize, classify, and analyze the applications of these ML models in software vulnerability detection. This absence may lead to critical research areas being overlooked or under-represented, resulting in a skewed understanding of the current state of the art in software vulnerability detection. To close this gap, we propose a comprehensive and systematic literature review that characterizes the different properties of ML-based software vulnerability detection systems using six major Research Questions (RQs). Using a custom web scraper, our systematic approach involves extracting a set of studies from four widely used online digital libraries: ACM Digital Library, IEEE Xplore, ScienceDirect, and Google Scholar. We manually analyzed the extracted studies to filter out irrelevant work unrelated to software vulnerability detection, followed by creating taxonomies and addressing RQs. Our analysis indicates a significant upward trend in applying ML techniques for software vulnerability detection over the past few years, with many studies published in recent years. Prominent conference venues include the International Conference on Software Engineering (ICSE), the International Symposium on Software Reliability Engineering (ISSRE), the Mining Software Repositories (MSR) conference, and the ACM International Conference on the Foundations of Software Engineering (FSE), whereas Information and Software Technology (IST), Computers & Security (C&S), and Journal of Systems and Software (JSS) are the leading journal venues. Our results reveal that 39.1% of the subject studies use hybrid sources, whereas 37.6% of the subject studies utilize benchmark data for software vulnerability detection. Code-based data are the most commonly used data type among subject studies, with source code being the predominant subtype. Graph-based and token-based input representations are the most popular techniques, accounting for 57.2% and 24.6% of the subject studies, respectively. Among the input embedding techniques, graph embedding and token vector embedding are the most frequently used techniques, accounting for 32.6% and 29.7% of the subject studies. Additionally, 88.4% of the subject studies use DL models, with recurrent neural networks and graph neural networks being the most popular subcategories, whereas only 7.2% use classic ML models. Among the vulnerability types covered by the subject studies, CWE-119, CWE-20, and CWE-190 are the most frequent ones. In terms of tools used for software vulnerability detection, Keras with TensorFlow backend and PyTorch libraries are the most frequently used model-building tools, accounting for 42 studies for each. In addition, Joern is the most popular tool used for code representation, accounting for 24 studies. Finally, we summarize the challenges and future directions in the context of software vulnerability detection, providing valuable insights for researchers and practitioners in the field.

This paper conducted a novel study to explore the capabilities of ChatGPT, a state-of-the-art LLM, in static analysis tasks such as static bug detection and false positive warning removal. In our evaluation, we focused on two types of typical and critical bugs targeted by static bug detection, i.e., Null Dereference and Resource Leak, as our subjects. We employ Infer, a well-established static analyzer, to aid the gathering of these two bug types from 10 open-source projects. Consequently, our experiment dataset contains 222 instances of Null Dereference bugs and 46 instances of Resource Leak bugs. Our study demonstrates that ChatGPT can achieve remarkable performance in the mentioned static analysis tasks, including bug detection and false-positive warning removal. In static bug detection, ChatGPT achieves accuracy and precision values of up to 68.37% and 63.76% for detecting Null Dereference bugs and 76.95% and 82.73% for detecting Resource Leak bugs, improving the precision of the current leading bug detector, Infer by 12.86% and 43.13% respectively. For removing false-positive warnings, ChatGPT can reach a precision of up to 93.88% for Null Dereference bugs and 63.33% for Resource Leak bugs, surpassing existing state-of-the-art false-positive warning removal tools.

A systematic review allows synthesizing the state of knowledge related to a clearly formulated research question as well as understanding the correlations between exposures and outcomes. A systematic review usually leverages explicit, reproducible, and systematic methods that allow reducing the potential bias that may arise when conducting a review. When properly conducted, a systematic review yields reliable findings from which conclusions and decisions can be made. Systematic reviews are increasingly popular and have several stakeholders to whom they allow making recommendations on how to act based on the review findings. They also help support future research prioritization. A systematic review usually has several components. The abstract is one of the most important parts of a review because it usually reflects the content of the review. It may be the only part of the review read by most readers when forming an opinion on a given topic. It may help more motivated readers decide whether the review is worth reading or not. But abstracts are sometimes poorly written and may, therefore, give a misleading and even harmful picture of the review’s contents. To assess the extent to which a review’s abstract is well constructed, we used a checklist-based approach to propose a measure that allows quantifying the systematicity of review abstracts i.e., the extent to which they exhibit good reporting quality. Experiments conducted on 151 reviews published in the software engineering field showed that the abstracts of these reviews had suboptimal systematicity.

Background: People who are racialized, gendered, or otherwise minoritized are underrepresented in computing professions in North America. This is reflected in undergraduate computer science (CS) programs, in which students from marginalized backgrounds continue to experience inequities that do not typically affect White cis-men. This is especially true for Black students in general, and Black women in particular, whose experience of systemic, anti-Black racism compromises their ability to persist and thrive in CS education contexts. Objectives: This systematic mapping study endeavours to (1) determine the quantity of existing non-deficit-based studies concerned with the persistence of Black students in undergraduate CS; (2) summarize the findings and recommendations in those studies; and (3) identify areas in which additional studies may be required. We aim to accomplish these objectives by way of two research questions: (RQ1) What factors are associated with Black students’ persistence in undergraduate CS programs?; and (RQ2) What recommendations have been made to further bolster Black students’ persistence in undergraduate CS education programs? Methods: This systematic mapping study was conducted in accordance with PRISMA 2020 and SEGRESS guidelines. Studies were identified by conducting keyword searches in seven databases. Inclusion and exclusion criteria were designed to capture studies illuminating persistence factors for Black students in undergraduate CS programs. To ensure the completeness of our search results, we engaged in snowballing and an expert-based search to identify additional studies of interest. Finally, data were collected from each study to address the research questions outlined above. Results: Using the methods outlined above, we identified 16 empirical studies, including qualitative, quantitative, and mixed-methods studies informed by a range of theoretical frameworks. Based on data collected from the primary studies in our sample, we identified 13 persistence factors across four categories: (I) social capital, networking, & support; (II) career & professional development; (III) pedagogical & programmatic interventions; and (IV) exposure & access. This data-collection process also yielded 26 recommendations across six stakeholder groups: (i) researchers; (ii) colleges and universities; (iii) the computing industry; (iv) K-12 systems and schools; (v) governments; and (vi) parents. Conclusion: This systematic mapping study resulted in the identification of numerous persistence factors for Black students in CS. Crucially, however, these persistence factors allow Black students to persist, but not thrive, in CS. Accordingly, we contend that more needs to be done to address the systemic inequities faced by Black people in general, and Black women in particular, in computing programs and professions. As evidenced by the relatively small number of primary studies captured by this systematic mapping study, there exists an urgent need for additional, asset-based empirical studies involving Black students in CS. In addition to foregrounding the intersectional experiences of Black women in CS, future studies should attend to the currently understudied experiences of Black men.