Current methodologies in electrical equipment fault diagnosis are not based on an edge-cloud computing architecture; however, the potential applicability of this architecture for realizing real-time autonomous AI-based fault diagnosing is vast. Collaborative cloud and edge computing brings data computation and storage closer to Internet of Things (IoT) data sources; specifically, data are processed at the periphery of the network (the Internet). In this article, an intelligent fault diagnosis methodology based on infrared thermal imaging (infrared thermography (IRT) images) and deep learning (DL) in an edge-cloud collaborative computing architecture is developed for smart electrical equipment fault diagnosis. In the developed methodology, YOLOv8 is used to perform object detection and classification of objects in captured IRT images of electrical components for on-site and on-line condition monitoring. This algorithm is implemented and run on an nVIDIA ^® Jetson Nano™ developer kit to serve as an autonomous edge AI device in the architecture. The Jetson Nano™ kit is integrated with an FLIR Lepton ^® 3.5 thermal imaging module to capture accurate, calibrated and noncontact temperature data in every pixel of each IRT image. The developed architecture and its preliminary implementation are described and tested. Additionally, the collaborativeness and performance evaluation are shown. As reported, via the proposed methodology, a performance improvement of ~20% in the mAP@.5 in object detection and classification is achieved for an addressed task as an illustrative paradigm of the developed architecture. Additionally, a reduction of ~83% of the bandwidth of the backbone network is gained. Ultimately, the developed architecture may be applied to IRT-based intelligent fault diagnosis in other power equipment in the area of power equipment diagnosis.

Three-dimensional (3D) object detection plays a key role in autonomous driving systems. The performance of LiDAR-based models is limited because of the sparsity and inhomogeneity of point clouds. Researchers have developed numerous camera-LiDAR-based fusion methods to address the limitations of LiDAR-based models. Still, because of the major differences in representations between pixel and point cloud, these fusion methods cannot appropriately leverage the benefits of both types of sensors. In this study, we developed a 3D object detection model based on LiDAR-camera fusion. The model first augments raw points with possibility features extracted from a camera stream and subsequently uses possibility-guided farthest point sampling for high-quality feature extraction in the 3D backbone. In addition, we designed a detection head for density-aware artificial point sampling and fusion that can aggregate geometric and semantic features by using high-resolution artificial points and shift the position of an artificial point according to the density of points in a region of interest to obtain more representative point features. We conducted massive experiments by using the public KITTI dataset. Our method outperforms previously developed state-of-the-art 3D object detection methods, especially in detecting small and distant objects.

Multispectral object detection using visible-thermal vision sensors is a crucial method for enabling intelligent vehicles to perceive their environment accurately. However, fusing and complementing multispectral information is challenging because of problems such as miscalibration resulting from disparate camera fields of view and pattern discrepancies in images captured at different wavelengths. To address these challenges, we propose a novel multispectral object detection system named the reinforcement- and alignment-based fusion model (RA-FMDet). This system integrates a cross-modality reinforcement (CMR) module with a hierarchical modality alignment (HMA) strategy. The CMR module uses hybrid transformer-convolutional neural network architecture for the effective extraction of high-quality semantic representations from multispectral data. The HMA strategy addresses miscalibration at the pixel and anchor levels. To evaluate RA-FMDet, we established a comprehensive dataset covering various scenarios and conditions. This dataset was obtained by optimizing the hardware configuration of visible-thermal vision sensors and mitigating the shift between image modalities through the use of a modified calibration board. Extensive experiments conducted on the challenging KAIST dataset and the established dataset indicated that the proposed RA-FMDet outperformed state-of-the-art models in object detection and achieved mean average precision values of 92.60%, 93.37%, and 93.60% in the detection of cars, pedestrians, and motorcyclists, respectively.

In recent years, the development of intelligent and electric vehicles has placed an increased emphasis on autonomous driving systems, which have the potential to improve safety, reduce traffic congestion, and increase accessibility. One of the key technologies in these systems is lane detection. However, existing technology faces many challenges, particularly under complex scenes and adverse lighting conditions, leading to inconsistent detection results and discouraging consumers from using the system. To address this problem, we propose a novel lane detection model based on deep learning, which introduces a multifeature aggregation module to improve the model's ability to extract explicit and implicit spatial features and accurately locate severe occlusion and ambiguous lane lines. We deploy the proposed technology on the vehicle control platform to further implement the lane-keeping assist system and analyze the experimental results using public TuSimple and Taiwan scene datasets. Our analysis demonstrates the high and stable detection performance of the proposed method in different field environments, with improvements over existing models. This work shows the effectiveness and feasibility of our proposed model in addressing the challenges of lane detection in automated driving systems.

For Internet of Vehicles applications, reliable autonomous driving systems usually perform the majority of their computations on the cloud due to the limited computing power of edge devices. The communication delay between cloud platforms and edge devices, however, can cause dangerous consequences, particularly for latency-sensitive object detection tasks. Object detection tasks are also vulnerable to significantly degraded model performance caused by unknown objects, which creates unsafe driving conditions. To address these problems, this study develops an orchestrated system that allows real-time object detection and incrementally learns unknown objects in a complex and dynamic environment. A you-only-look-once–based object detection model in edge computing mode uses thermal images to detect objects accurately in poor lighting conditions. In addition, an attention mechanism improves the system’s performance without significantly increasing model complexity. An unknown object detector automatically classifies and labels unknown objects without direct supervision on edge devices, while a roadside unit (RSU)-based mechanism is developed to update classes and ensure a secure driving experience for autonomous vehicles. Moreover, the interactions between edge devices, RSU servers, and the cloud are designed to allow efficient collaboration. The experimental results indicate that the proposed system learns uncategorized objects dynamically and detects instances accurately.

The use of intelligent vehicle technology is increasing; however, this technology requires further improvement. Semantic segmentation enables intelligent vehicles to understand the environment. Although advances have been achieved in a deep learning model for semantic segmentation, large-scale real-world datasets with manual pixel-level annotations, which are expensive, are required for adequately training these models. Unsupervised domain adaptation (UDA) for semantic segmentation enables training on a synthetic source domain and allows a model to be adapted to the real-world target domain without target domain annotations. Self-training is a highly effective UDA method; however, current UDA methods struggle to shift domains effectively and rely heavily on aggressive image-based perturbations for consistency training, resulting in poor-quality pseudolabels during the early training phase and decreased training efficiency. Therefore, we propose a novel multilevel self-training UDA method for semantic segmentation. First, a feature guidance and resampling strategy is adopted to improve the learning of rare classes in the source domain, and a spatial-aware mixing method is then used for gradually and smoothly eliminating the domain gap through training in various cross-domain intermediate spaces. In addition, we developed an instance and prototype feature consistency technique that explores optimization in the feature space to improve the training process by increasing model consistency. We conducted large-scale experiments on two challenging UDA benchmarks, “GTA5-to-Cityscapes” and “SYNTHIA-to-Cityscapes.” Our method achieved similar or superior performance to that of previously developed UDA methods for semantic segmentation and has advantages in terms of training stability and convergence efficiency.