Semantic segmentation is a process of linking each pixel in an image to a class label, and is widely used in the field of autonomous vehicles and robotics. Although deep learning methods have already made great progress for semantic segmentation, they either achieve great results with numerous parameters or design lightweight models but heavily sacrifice the segmentation accuracy. Because of the strict requirements of real-world applications, it is critical to design an effective real-time model with both competitive segmentation accuracy and small model capacity. In this paper, we propose a lightweight network named DABNet, which employs Depth-wise Asymmetric Bottleneck (DAB) and Point-wise Aggregation Decoder (PAD) module to tackle the challenging real-time semantic segmentation in urban scenes. Specifically, the DAB module creates a sufficient receptive field and densely utilizes the contextual information, and the PAD module aggregates the feature maps of different scales to optimize performance through the attention mechanism. Compared with existing methods, our network substantially reduces the number of parameters but still achieves high accuracy with real-time inference ability. Extensive ablation experiments on two challenging urban scene datasets (Cityscapes and CamVid) have proved the effectiveness of the proposed approach in real-time semantic segmentation.

A timely and accurate damage assessment of buildings after an earthquake is critical for the safety of people and property. Most of the existing methods based on classification and segmentation use two-dimensional information to determine the damage level of the buildings, which cannot provide the multi-view information of the damaged building, resulting in inaccurate assessment results. According to the knowledge of the authors, there is no related research using the deep-learning-based 3D reconstruction method for the evaluation of building damage. In this paper, we first applied the deep-learning-based MVS model to reconstruct the 3D model of the buildings after an earthquake using multi-view UAV images, to assist the building damage assessment task. The method contains three main steps. Firstly, the camera parameters are calculated. Then, 3D reconstruction is conducted based on CasMVSNet. Finally, a building damage assessment is performed based on the 3D reconstruction result. To evaluate the effectiveness of the proposed method, the method was tested in multi-view UAV aerial images of Yangbi County, Yunnan Province. The results indicate that: (1) the time efficiency of CasMVSNet is significantly higher than that of other deep learning models, which can meet the timeliness requirement of post-earthquake rescue and damage assessment. In addition, the memory consumption of CasMVSNet is the lowest; (2) CasMVSNet exhibits the best 3D reconstruction result in both high and small buildings; (3) the proposed method can provide detail and multi-view information of damaged buildings, which can be used to assist the building damage assessment task. The results of the building damage assessment are very similar to the results of the field survey.

Simultaneous localization and mapping (SLAM) is an important function for service robots to self-navigate modernized buildings. However, only a few existing applications allow them to automatically move between stories through elevator. Some approaches have accomplished with the aid of hardware; however, this study shows that computer vision can be a promising alternative for button localization. In this paper, we proposed a real-time multi-story SLAM system which overcomes the problem of detecting elevator buttons using a localization framework that combines tracking and detecting approaches. A two-stage deep neural network initially locates the original positions of the target buttons, and a part-based tracker follows the target buttons in real-time. A positive-negative classifier and deep learning neural network (particular for button shape detection) modify the tracker’s output in every frame. To allow the robot to self-navigate, a 2D grid mapping approach was used for the localization and mapping. Then, when the robot navigates a floor, the A* algorithm generates the shortest path. In the experiment, two dynamic scenes (which include common elevator button localization challenges) were used to evaluate the efficiency of our approach, and compared it with other state-of-the-art methods. Our approach was also tested on a prototype robot system to assesses how well it can navigate a multi-story building. The results show that our method could overcome the common background challenges that occur inside an elevator, and in doing so, it enables the mobile robot to autonomously navigate a multi-story building.

Cervical cancer diagnosis hinges significantly on precise nuclei segmentation at early stages, which however, remains largely elusive due to challenges such as overlapping cells and blurred nuclei boundaries. This paper presents a novel deep neural network (DNN), the Global Context UNet (GC-UNet), designed to adeptly handle intricate environments and deliver accurate cell segmentation.

Dynamic obstacle detection is important for environmental perception in self-driving cars. Instance segmentation using a camera is a major trend in obstacle detection. However, unexpected dynamic obstacles are difficult to detect as their classes are unlabeled in the model. In this study, we combine an understanding of a road scene; optical flow movement tracking; and low-cost online visual tracking to build a system for detecting unexpected dynamic obstacles. To monitor the pixel movement, a mobile recurrent pairwise decoding optical flow deep neural network is employed to rapidly track the pixel flows between two frames. To filter background noises and leave the active region on the road, a mobile DABNet detects the targets (only roads and vehicles) in the scene. To reduce the load on the GPU, a cluster-matching tracker employs multi-tensor CPU resources to follow the estimated unexpected dynamic obstacles extracted by processing based on the road understanding and optical flows and tracks such obstacles one by one in the following frames. A real-time system properly splits the usage of GPU and CPU resources to maximize the performance of the system platform. To evaluate the efficiency, a driver view video dataset is recorded for evaluating real-world obstacles on the urban road scene. Then, animal crash videos are collected from YouTube to evaluate unexpected/rarely labeled objects. Furthermore, a mobile robot platform is used to test the proposed system to avoid obstacles in a complicated indoor scene.