Human action monitoring can be advantageous to remotely monitor the status of patients or elderly person for intelligent healthcare. Human action recognition enables efficient and accurate monitoring of human behaviors, which can exhibit multifaceted complexity attributed to disparities in viewpoints, personality, resolution and motion speed of individuals, etc. The spatial-temporal information plays an important role in the human action recognition. In this paper, we proposed a novel deep learning architecture named as recurrent 3D convolutional neural network (R3D) to extract effective and discriminative spatial-temporal features to be used for action recognition, which enables the capturing of long-range temporal information by aggregating the 3D convolutional network entries to serve as an input to the LSTM (Long Short-Term Memory) architecture. The 3D convolutional network and LSTM are two effective methods for extracting the temporal information. The proposed R3D network integrated these two methods by sharing a shared 3D convolutional network in sliding windows on video streaming to capturing short-term spatial-temporal features into the LSTM. The output features of LSTM encapsulate the long-range spatial-temporal information representing high-level abstraction of the human actions. The proposed algorithm is compared to traditional and the-state-of-the-art and deep learning algorithms. The experimental results demonstrated the effectiveness of the proposed system, which can be used as smart monitoring for remote healthcare.
1 February 2022 in IEEE Transactions on Intelligent Transportation Systems
6G network enables the rapid connection of autonomous vehicles, the generated internet of vehicles establishes a large-scale point cloud, which requires automatic point cloud analysis to build an intelligent transportation system in terms of the 3D object detection and segmentation. Recently, a great variety of deep convolution networks have been proposed for 3D data analysis, making significant progress in the application of deep learning in 3D computer vision. Inspired by the application of transformer network in 2D computer visual tasks, and in order to increase the expression ability of local fine-grained features, we propose an effective local feature transformer network to learn local feature information and correlations between point clouds. Our network is adaptive to the arrangement of set elements through transformer module, so it is suitable for the feature extraction of local point clouds. In addition, experimental results demonstrate that our LFT-network outperforms the state-of-the-art in 3D model classification tasks on ModelNet40 dataset and segmentation tasks on S3DIS dataset.
Snowflakes attached to the camera lens can severely affect the visibility of the background scene and compromise the image quality. In this paper, we solve this problem by visually removing snowflakes to convert the snowy image into a clean one. The problem is troublesome; the information about the background of the occluded regions is completely lost for the most part. For removing snowflakes from a single image, we proposed a composition generative adversarial network. Different from the previous generative adversarial networks, our generator network comprises clean background module and a snow mask estimate module. The clean background module aims to generate a clear image from an input snowy image, and snow mask estimate module is used to produce the snow mask in an input image. During the training step, we put forward a composition loss between the input snowy image and composition of the generated clean image and estimated snow mask. We use a dataset named Snow100K 2 including indoor and outdoor scenes to train and test the proposed method. The extensive experiments on both synthetic and real-world images show that our network has a good effect and it is superior to the other state-of-the-art methods.
3-D vision plays an important role in intelligent perception of robot, while it requires extra 3-D sensors. Depth estimation from monocular videos provides an alternative mechanism to recover the 3-D information. In this paper, we propose an unsupervised learning framework that uses the perceptual loss for depth estimation. Depth and pose networks are first trained to estimate the depth and the camera motion of the video sequence, respectively. With the estimated depth and pose of the original frame, the adjacent frame can be reconstructed. The pixel-wise differences between the constructed frame and the original frame are used as per-pixel loss. Meanwhile, reconstructed views and original views can be used to extract advanced features from a pre-trained network to define and optimize perceptual loss functions to assess the quality of reconstructions. We combine the respective advantages of these two methods and present an approach of generating a depth map by training the feed-forward network with per-pixel loss function and perceptual loss function. The experimental results show that our method can significantly improve the estimation accuracy of depth map.