The rapid evolution of the Internet of Things (IoT) is making way for the development of several IoT applications that require minimal or no human involvement in the data collection, transformation, knowledge extraction, and decision-making (actuation) process. To ensure that such IoT applications (we term them autonomic) function as expected, it is necessary to measure and evaluate their quality, which is challenging in the absence of any human involvement or feedback. Existing Quality of Experience (QoE) literature and most QoE definitions focuses on evaluating application quality from the lens of human receiving application services. However, in autonomic IoT applications, poor quality of decisions and resulting actions can degrade the application quality leading to economic and social losses. In this paper, we present a vision, survey and future directions for QoE research in IoT. We review existing QoE definitions followed by a survey of techniques and approaches in the literature used to evaluate QoE in IoT. We identify and review the role of data from the perspective of IoT architectures, which is a critical factor when evaluating the QoE of IoT applications. We conclude the paper by identifying and presenting our vision for future research in evaluating the QoE of autonomic IoT applications.

The rapid evolution of the Internet of Things (IoT) facilitates the development of IoT applications in domains such as manufacturing, smart cities, retail, agriculture, etc. Such IoT applications collect data, analyze, and extract insightful information to enable decision-making and actuation. There is an unprecedented growth of IoT applications that automate decision-making and actuation without requiring human intervention, which we term autonomic IoT applications. The increasing scale of such applications necessitates holistic measurement and evaluation of application quality. Existing literature has evaluated quality from an end-user perspective, which may be unsuitable when dealing with the complexity of modern IoT applications, especially when they are autonomic. In this paper, we refer to IoT application quality as the aggregate quantitative value of various IoT quality metrics measured at each stage of the autonomic IoT application life cycle. We present an in-depth survey of current state-of-the-art techniques and approaches for evaluating quality of IoT applications. In particular, we survey various definitions to identify the factors that contribute to understanding and evaluating quality in IoT. Furthermore, we present open issues and identify future research directions towards realizing fine-grained quality evaluation of IoT applications. We envision that the identified research directions will, in turn, enable real-time diagnostics of IoT applications and make them quality-aware. This survey can serve as the basis for designing and developing modern, resilient quality-aware autonomic IoT applications.

The unprecedented growth of Internet of Things (IoT) is leading to its increased usage in various domains, such as manufacturing, health, and smart cities. A majority of IoT applications are autonomic, i.e., they operate under minimal/no human intervention, and make decisions/actuations based on machine-to-machine communication and data analytics. A key challenge in the development of such applications is the ability to measure their quality while they are working in a diverse and heterogeneous IoT ecosystem. In this paper, we propose an agent based Internet of Things-Quality Watch (IoT-QWatch) framework that provides the ability to measure IoT quality metrics at each stage of the autonomic IoT application life cycle running in the IoT ecosystem. We envision that IoT-QWatch will enable the development of a new generation of quality-aware autonomic IoT applications that are able to be resilient to the heterogeneous and uncertain nature of IoT ecosystems. We present architectural details and implementation of IoT-QWatch, and corresponding models used to measure IoT quality metrics at different stages. We conduct extensive experiments using a real-world IoT test bed from the domain of manufacturing to validate the efficacy of IoT-QWatch. Experimental outcomes provide promising results in realising IoT-QWatch in real-world deployment, while the framework itself offers significant extensibility to include new models for measuring IoT quality metrics.

The unprecedented growth of Internet of Things (IoT) underpinned by machine-to-machine communication, analytics and actuation is spearheading the development of IoT-based Smart Agriculture Applications (IoTSA). Various factors (e.g., location, communication protocols, weather) impact data availability in such IoT applications, which makes it necessary for the applications to understand metrics that define sensor data quality. Thus, awareness of sensor data quality is crucial to all IoTSAs. This paper proposes a conceptual framework for describing the quality metrics required to formulate a data quality model. To demonstrate the efficacy of the proposed data quality metrics and model, we apply them to a dairy farm application (an instance of IoTSA). Our real-world experimental evaluations conclude that the proposed model: (i) can be employed by IoTSAs to adapt to different factors that may impact the quality of decision making (actuation) and (ii) aids in developing data quality aware IoTSA and beyond.

The amount of protein and fat in raw milk determines its quality, value in the marketplace, and related payment to suppliers. Technicians use expensive specialized laboratory equipment to measure milk quality in specialized laboratories. The continuous quality monitoring of the milk supply in the supplier’s tanks enables the production of higher quality products, better milk supply chain optimization, and reduced milk waste. This paper presents an inexpensive AI-powered IoT sensor that continuously measures the protein and fat in the raw milk in the tanks of dairy farms, pickup trucks, and intermediate storage depots across any milk supply chain. The proposed sensor consists of an in-tank IoT device and related software components that run on any IoT platform. The in-tank IoT device quality incorporates a low-cost spectrometer and a microcontroller that can send milk supply measurements to any IoT platform via NB-IoT. The in-tank IoT device of the milk quality sensor is housed in a food-safe polypropylene container that allows its deployment in any milk tank. The IoT software component of the milk quality sensors uses a specialized machine learning (ML) algorithm to translate the spectrometry measurements into milk fat and protein measurements. The paper presents the design of an in-tank IoT sensor and the corresponding IoT software translation of the spectrometry measurements to protein and fat measurements. Moreover, it includes an experimental milk quality sensor evaluation that shows that sensor accuracy is ±0.14% for fat and ±0.07% for protein.

Producing high-quality product consistently is crucial in manufacturing, as discarding or reprocessing low-quality products increases waste and energy consumption and reduces overall production efficiency. Ensuring high-quality manufactured products is challenging due to relying on human activities for product quality and related consistency assessment, which is often performed postproduction instead of assessing these during each production run and making real-time production adjustments that can mitigate product quality and related consistency issues. In this article, we proposes a novel machine-state learner algorithm that captures the dependencies between product quality and related consistency and machine data (specifically the machine settings and corresponding sensor data). In addition, this article shows how this novel machine-state learner algorithm can be used to predict product quality during the production runs and how such prediction are used to make machine setting recommendations that mitigate product quality and related consistency issues before or during the production runs. These advances in machine state-based data modeling, predictive data analysis and recommendation are incorporated into an inline prediction and decision support system that achieves significant improvement in producing high-quality products consistently by guiding decision-making via recommendations during production in a digital manufacturing environment. In this article, we present an evaluation of the above contributions in a real-world manufacturing plant and yield double digit first pass and nearly perfect second pass product improvements in terms of product quality and related consistency and production efficiency.