Traffic sign recognition (TSR) plays an important role in driving assistance system and traffic safety insurance. However, existing methods focus on extracting features of traffic signs and ignore the constraints of spatial positional relationships between traffic signs and other objects in the scene. This way results in incorrectly detecting other similar objects as traffic signs and failing to detect very small traffic signs. A TSR method based on semantic scene understanding and structural traffic sign location is proposed in this study to solve the aforementioned problems. A scene structure model based on the constraints of spatial positional relationships between traffic signs and other objects is proposed to establish trusted search regions. An improved Light-weight RefineNet is used to analyze and understand a scene semantically and accurately and then segment objects in complicated environments precisely. A new network multiscale densely connected object detector (MDCOD) based on densely connected style, multiscale feature fusion, and improved K-means++ algorithms is proposed to recognize very small traffic signs. The trusted traffic signs are found by filtering false candidates outside the scene structure model. The proposed method is tested on Tsinghua-Tencent 100K and German Traffic Sign Detection Benchmark datasets and achieves accuracies of 92.8% and 99.90%, respectively, outperforming the existing methods.

Extracting vehicle information is of great significance to the construction of the Internet of Vehicles (IoV). Vehicle logo detection (VLD) technology can effectively extract vehicle information. Due to the complex traffic environment, existing methods have difficulty to accurately detect the vehicle logo, especially when the vehicle logo has motion blur. To alleviate these problems, a new approach is proposed to detect vehicle logo under motion blur with the combination of Filter-DeblurGAN and VL-YOLO. Filter-DeblurGAN possesses a judgment mechanism, which can determine whether the image needs to be deblurred based on the degree of blur of each image. It can also deblur images with arbitrary resolution. Filter-DeblurGAN solves the defect that DeblurGAN lacks the judgment mechanism and is too harsh on resolution. It can be directly applied to the deblurring process of VLD. We also propose a method using VL-YOLO and Eliminate Outliers Cluster (EOC) algorithm to further improve the detection accuracy. VL-YOLO achieves accurate detection of the vehicle logo by constructing a deeper multi-scale detection network and using the initial candidate boxes provided by EOC algorithm. The EOC algorithm is not affected by outliers. In order to further promote the research of VLD, we have labeled a new vehicle logo dataset named LOGO-17, which contains 17 different categories of vehicle logos. The experimental results demonstrate that the proposed method achieves good detection accuracy in the environment of motion blur, and outperforms existing methods.

The existing deep learning methods for human fall detection have difficulties to distinguish falls from similar daily activities such as lying down because of not using the 3D network. Meanwhile, they are not suitable for mobile devices because they are heavyweight methods and consume a large number of memories. In order to alleviate these problems, a two-stream approach to fall detection with the MobileVGG is proposed in this paper. One stream is based on the motion characteristics of the human body for detection of falls, while the other is an improved lightweight VGG network, named the MobileVGG, put forward in the paper. The MobileVGG is constructed as a lightweight network model through replacing the traditional convolution with a simplified and efficient combination of point convolution, depth convolution and point convolution. The residual connection between layers is designed to overcome the gradient disappeared and the obstruction of gradient reflux in the deep model. The experimental results show that the proposed two-stream lightweight fall classification model outperforms the existing methods in distinguishing falls from similar daily activities such as lying and reducing the occupied memory. Therefore, it is suitable for mobile devices.

Vehicle re-identification (Re-ID) methods often fail to achieve robust performance due to insufficient training data and domain diversities. Although state-of-the-art methods apply image-to-image translation or web data to achieve data augmentation, the construct of new datasets will not only introduce noise, but also undergo a mismatch issue with the source domain. Moreover, the label noise of cross-domain data in existing label distribution technologies cannot be alleviated. In this paper, a multi-domain joint learning with inter-domain adaptation label smoothing regularization (IALSR) is proposed using a semi-supervised learning framework. The overall framework consists of two parts. In one part, a multi-domain joint network (MJNet) is proposed to learn multiple vehicle attributes simultaneously. The output of the training model is employed to group several inter-domain subsets, which are regarded as different domains. To adapt to domain diversities, style transfer models are learned for each pair of subsets to generate free and rich data as a novel data augmentation approach. In the other part, IALSR, which preserves self-similarity and domain-transitivity, is designed to smooth the noise of style-transferred data. Upon our basis, we further introduce the web data to verify the superiority of the IALSR. The results of extensive experimental on two large-scale vehicle Re-ID datasets demonstrate that the proposed approach is superior to other state-of-the-art ones.

Multi-frame human pose estimation has long been an appealing and fundamental issue in visual perception. Owing to the frequent rapid motion and pose occlusion in videos, this task is extremely challenging. Current state-of-the-art methods seek to model spatiotemporal features by equally fusing each frame in the local sequence, which weakens the target frame information. In addition, existing approaches usually emphasize more on deep features while ignoring the detailed information implied in the shallow feature maps, resulting in the dropping of crucial features. To address the above problems, we propose an effective framework, namely spatiotemporal learning transformer for video-based human pose estimation (SLT-Pose), which consists of a Personalized Feature Extraction Module (PFEM), Self-feature Refinement Module (SRM), Cross-frame Temporal Learning Module (CTLM) and Disentangled Keypoint Detector (DKD). To be specific, we propose PFEM which extracts and modulates the individual frame features to adapt to the varying human shape, and integrates single-frame features to obtain the spatiotemporal features. We further present SRM to establish global correlation spatial cues on the target frame to attain the refinement feature. Then, a CTLM is designed to search for the information most closely related to the target frame from the spatiotemporal features to intensify the interaction between the target frame and the local sequence, using both the shallow detailed and the deep semantic representations. Finally, we employ DKD to extract the disentangled characteristics of each joint and encode the articulated joint pairs in the human body, promoting the model to reasonably and accurately predict the keypoint heatmaps. Extensive experiments on three huamn motion benchmarks, including PoseTrack2017, PoseTrack2018, and Sub-JHMDB dataset, demonstrate that SLT-Pose plays favorably against state-of-the-art approaches in terms of both objective evaluation and subjective visual performance.

Behavior analysis through posture recognition is an essential research in robotic systems. Sitting with unhealthy sitting posture for a long time seriously harms human health and may even lead to lumbar disease, cervical disease and myopia. Automatic vision-based detection of unhealthy sitting posture, as an example of posture detection in robotic systems, has become a hot research topic. However, the existing methods only focus on extracting features of human themselves and lack understanding relevancies among objects in the scene, and henceforth fail to recognize some types of unhealthy sitting postures in complicated environments. To alleviate these problems, a scene recognition and semantic analysis approach to unhealthy sitting posture detection in screen-reading is proposed in this paper. The key skeletal points of human body are detected and tracked with a Microsoft Kinect sensor. Meanwhile, a deep learning method, Faster R-CNN, is used in the scene recognition of our method to accurately detect objects and extract relevant features. Then our method performs semantic analysis through Gaussian-Mixture behavioral clustering for scene understanding. The relevant features in the scene and the skeletal features extracted from human are fused into the semantic features to discriminate various types of sitting postures. Experimental results demonstrated that our method accurately and effectively detected various types of unhealthy sitting postures in screen-reading and avoided error detection in complicated environments. Compared with the existing methods, our proposed method detected more types of unhealthy sitting postures including those that the existing methods could not detect. Our method can be potentially applied and integrated as a medical assistance in robotic systems of health care and treatment.