Effective software projects hinge on robust requirements, yet flawed requirements often lead to costly delays and revisions. While tools have been developed to identify defects in Software Requirements Specifications (SRS), the advent of Large Language Models (LLMs) like GPT-4 presents new opportunities for enhancing requirements quality. However, the potential of LLMs in this realm remains largely unexplored, particularly in the context of large-scale industrial documents. To bridge this gap, we investigate the efficacy of zero-shot GPT-4 in various requirements analysis tasks using an industrial software specification document. Our study evaluates LLM performance in detecting defects, such as ambiguities, inconsistencies, and incompleteness, while also analyzing GPT-4’s ability to identify issues across version iterations and support technical experts in requirements analysis. Qualitatively, we identify key limitations of LLMs in defect detection, notably their inability to cross-reference throughout the document and their constrained understanding of specialized contexts. Quantitatively, we find that while LLMs excel in identifying incomplete requirements (precision 0.61), their performance is less impressive in detecting inconsistencies (precision 0.43) and ambiguities (precision 0.39). Although GPT-4 demonstrates promise in automating early defect detection across versions and providing accurate technical answers, our results underscore that they cannot entirely replace human analysts due to their lack of nuanced domain knowledge in a zero-shot setting. Nevertheless, avenues like few-shot learning and complex prompt design offer the potential to enhance LLM precision in defect detection.

Ubiquitous learning environments move learners out of a classroom and into the real world where learners engage in experiential and tangible learning involving instrumented physical things. Learners use peer-to-peer networks connecting learners, teachers, and a host of learning ‘things’ like instrumented pieces of art, flower pots, and even buildings. A key component of such systems is wireless-enabled edge devices augmented with various types of sensors to represent the state of physical things and their context. This paper presents a novel generic architecture for context-aware ubiquitous learning systems based on the Internet of Things (IoT) computing paradigm. A commonly used IoT edge device was used to implement four variants of the proposed architecture. The variants were based on Advanced Message Queuing Protocol (AMQP), Constrained Application Protocol (CoAP), Message Queue Telemetry Transport (MQTT), and Extensible Messaging and Presence Protocol (XMPP). All variants provided more than adequate performance for implementing ubiquitous learning scenarios. As expected, CoAP was generally more resource-efficient. However, practical differences in performance were so small that the choice of which architectural variant to use may depend on non-functional requirements like reliability, security, maximum payload size, and ease of implementation.

Software Requirements Specifications (SRS) are crucial for defining system functionality, constraints, and objectives, particularly in safety-critical applications like autonomous vehicles (AV). Ensuring these requirements are precise, unambiguous, and complete is important for developing reliable self-driving systems. While traditional SRS are predominantly text-based, AV requirements pose unique challenges as they often include visual elements such as images and diagrams. Although Large Language Models (LLMs) have enhanced traditional text-based requirements, Vision-Language Models (VLMs) remain largely unexplored for evaluating defects in visual elements. This paper investigates the few-shot performance of three large-scale VLMs on their ability to detect various types of ambiguities, inconsistencies, and incompleteness in visual traffic scenario images that could be used in AV requirements specifications. Using Soft prompting and Chain-of-Thought (CoT) prompting, we evaluated the effectiveness of GPT-4, GEMINI, and ClaudeAI in detecting six types of visual defects. GPT-4 achieved the highest overall performance among the evaluated models, with the highest F1-score of 0.63 in detecting Spatial Incompleteness. CoT prompting notably enhanced defect detection performance for both GEMINI and ClaudeAI across all defect types. While CoT prompting did not significantly improve GPT-4’s detection capabilities, it significantly enhanced the ability of all models to explain the reasoning behind the identified defects. Despite promising results, none of the models could fully automate the requirements validation process, primarily due to their lack of domain-specific knowledge. Nevertheless, our findings suggest that fine tuning the models and structured prompt design offer the potential to improve VLM precision in visual defect detection.