Object detection is an important aspect for autonomous driving vehicles (ADV), which may comprise of a machine learning model that detects a range of classes. As the deployment of ADV widens globally, the variety of objects to be detected may increase beyond the designated range of classes. Continual learning for object detection essentially ensure a robust adaptation of a model to detect additional classes on the fly. This study proposes a novel continual learning method for object detection that learns new object class(es) along with cumulative memory of classes from prior learning rounds to avoid any catastrophic forgetting. The results of PASCAL VOC 2007 have suggested that the proposed ER method obtains 4.3% of mAP drop compared against the all-classes learning, which is the lowest amongst other prior arts.

Deep learning models have revealed outstanding performance on image classification and object detection tasks. However, there is a crucial drop in performance when they are subject to learn new data incrementally in the absence of previous training data. They suffer from catastrophic forgetting—abrupt drop in performance. This phenomenon affects the implementation of artificial intelligence in practical scenarios. To overcome catastrophic forgetting, the previous method has either saved previous data in memory or generated the previous data. However, these methods are computationally complex and infeasible for real-time applications. In this article, we proposed the YOLOv3 as an object detection framework for incremental learning. A knowledge distillation loss is introduced for the prediction of previously learned knowledge without utilizing previous training data. Consequently, these predictions are updated while learning the current model. Experimental results on the Pascal VOC2007 indicate that the proposed method significantly improved the mean average precision up to 74% for two classes in comparison to the state-of-the-art methods.

Audio-based automatic speech recognition as a hearing aid is susceptible to background noise and overlapping speeches. Consequently, audio-visual speech recognition has been developed to complement the audio input with additional visual information. However, the huge improvement of neural networks in the visual task has resulted in a robust and reliable lip reading framework that can recognize speech from visual input alone. In this work, we propose a lip reading recognition model to predict daily Mandarin conversation and collect a new Daily Mandarin Conversation Lip Reading (DMCLR) dataset, consisting of 1,000 videos from 100 daily conversations spoken by ten speakers. Our model consists of a spatiotemporal convolution layer, a SE-ResNet-18 network, and a back-end module consisting of bi-directional gated recurrent unit (Bi-GRU), 1D convolution, and fully-connected layers. This model is able to reach 94.2% of accuracy in the DMCLR dataset. Such performance makes it possible for Mandarin lip reading applications to be practical in real life. Additionally, we are able to achieve 86.6% and 57.2% accuracy on Lip Reading in the Wild (LRW) and LRW-1000 (Mandarin), respectively. The results show that our method achieves state-of-the-art performance on these two challenging datasets.

On-road object detection is a critical component in an autonomous driving system. The safety of the vehicle can only be as good as the reliability of the on-road object detection system. Thus, developing a fast and robust object detection algorithm has been the primary goal of many automotive industries and institutes. In recent years, multi-purpose vision-based driver assistance systems have gained popularity with the emergence of a deep neural network. A monocular camera has been developed to locate an object in the image plane and estimate the distance of the said object in the real world or the vehicle plane. In this work, we present a monocular 3D object detection method that utilizes the discrete depth and orientation representation. Our proposed method strives to predict object locations on 3D space utilizing keypoint detection on the object's center point. To improve the point detection, we employ center regression on the objects segmentation mask, reducing the detection offset significantly. The simplicity of our proposed network architecture and its one-stage approach allows our algorithm to achieve competitive speed compared with prior methods. Our proposed method is able to achieve 26.93% detection score on the Cityscapes 3D object detection dataset, outperforming the preceding monocular method by a margin of 2.8 points.

In autonomous driving systems, the monocular 3D object detection algorithm is a crucial component. The safety of autonomous vehicles heavily depends on a well-designed detection system. Therefore, developing a robust and efficient 3D object detection algorithm is a major goal for institutes and researchers. Having a 3D sense is essential in autonomous vehicles and robotics, as it allows the system to understand its surroundings and react accordingly. Compared with stereo-based and Lidar-based methods, monocular 3D Object detection is a challenging task as it only utilizes 2D information to generate complex 3D features, making it low-cost, less computationally intensive, and with great potential. However, the performance of monocular methods is impaired due to the lack of depth information. In this paper, we propose a simple, end-to-end, and effective network for monocular 3D object detection without the use of external training data. Our work is inspired by auxiliary learning, in which we use a robust feature extractor as our backbone and multiple regression heads to learn auxiliary knowledge. These auxiliary regression heads will be discarded after training for improved inference efficiency, allowing us to take advantage of auxiliary learning and enabling the model to learn critical information more conceptually. The proposed method achieves 17.28% and 20.10% for the moderate level of the Car category on the KITTI benchmark test set and validation set, respectively, which outperforms the previous monocular 3D object detection approaches.

An increasing number of tasks have been developed for autonomous driving and advanced driver assistance systems. However, this gives rise to the problem of incorporating plural functionalities to be ported into a power-constrained computing device. Therefore, the objective of this work is to alleviate the complex learning procedure of the pixel-wise approach for driving scene understanding. In this paper, we go beyond the pixel-wise detection of the semantic segmentation task as a point detection task and implement it to detect free space and lane. Instead of pixel-wise learning, we trained a single deep convolution neural network for point of interest detection in a grid-based level and followed with a computer vision (CV) based post-processing of end branches corresponding to the characteristic of target classes. To achieve the corresponding final result of pixel-wise detection of semantic segmentation and parametric description of lanes, we propose a CV-based post-processing to decode points of output from the neural network. The final results showed that the network could learn the spatial relationship for point of interest, including the representative points on the contour of the free space segmented region and the representative points along the center of the road lane. We verify our method on two publicly available datasets, which achieved 98.2% mIoU on the KITTI dataset for the evaluation of free space and 97.8% accuracy on the TuSimple dataset (with the field of view below the $y=320$ axis) for the evaluation of the lane.