Refining high-level requirements into low-level ones is a common task, especially in safety-critical systems engineering. The objective is to describe every important aspect of the high-level requirement in a low-level requirement, ensuring a complete and correct implementation of the system's features. To this end, standards and regulations for safety-critical systems require reviewing the coverage of high-level requirements by all its low-level requirements to ensure no missing aspects. The challenge of supporting automatic reviews for requirements coverage originates from the distinct levels of abstraction between high-level and low-level requirements, their reliance on natural language, and the often different vocabulary used. The rise of Large Language Models (LLMs), trained on extensive text corpora and capable of contextualizing both high-level and low-level requirements, opens new avenues for addressing this challenge. This paper presents an initial study to explore the performance of LLMs in assessing requirements coverage. We employed GPT-3.5 and GPT-4 to analyze requirements from five publicly accessible data sets, determining their ability to detect if low-level requirements sufficiently address the corresponding high-level requirement. Our findings reveal that GPT-3.5, utilizing a zero-shot prompting strategy augmented with the prompt of explaining, correctly identifies complete coverage in four out of five evaluation data sets. Additionally, it exhibits an impressive 99.7% recall rate in accurately identifying instances where coverage is incomplete due to removing a single low-level requirement across our entire set of evaluation data.
Efficient software maintenance and evolution rely heavily on effective software traceability, which is crucial for understanding the relationships between code elements and their corresponding requirements. However, ensuring the accuracy of trace links, whether manually or automatically, is a significant challenge due to the labor-intensive and error-prone nature of traceability tasks. The granularity issue in traceability compounds this challenge, as most existing research focuses on class-level traceability, while fine-grained dependencies (e.g., method-level traces) are more pertinent in daily development practices. Our primary aim is to facilitate the checking of requirement-to-method traces. To this end, we investigate an approach that utilizes the method's calling information and textual embeddings of requirement-to-method traces to identify inaccuracies in trace links. Our preliminary results are promising. By leveraging a Random Forest (RF) classifier, we have achieved notable improvements in both precision (≈10%) and recall (≈30%) compared to existing methods. This advancement highlights the potential of our method in enhancing the accuracy and efficiency of traceability processes in software development.