In this paper, we provide a survey on automotive surround-view fisheye optics, with an emphasis on the impact of optical artifacts on computer vision tasks in autonomous driving and ADAS. The automotive industry has advanced in applying state-of-the-art computer vision to enhance road safety and provide automated driving functionality. When using camera systems on vehicles, there is a particular need for a wide field of view to capture the entire vehicle’s surroundings, in areas such as low-speed maneuvering, automated parking, and cocoon sensing. However, one crucial challenge in surround-view cameras is the strong optical aberrations of the fisheye camera, which is an area that has received little attention in the literature. Additionally, a comprehensive dataset is needed for testing safety-critical scenarios in vehicle automation. The industry has turned to simulation as a cost-effective strategy for creating synthetic datasets with surround-view camera imagery. We examine different simulation methods (such as model-driven and data-driven simulations) and discuss the simulators’ ability (or lack thereof) to model real-world optical performance. Overall, this paper highlights the optical aberrations in automotive fisheye datasets, and the limitations of optical reality in simulated fisheye datasets, with a focus on computer vision in surround-view optical systems.

Fully-supervised object detection and instance segmentation models have accomplished notable results on large-scale computer vision benchmark datasets. However, fully-supervised machine learning algorithms’ performances are immensely dependent on the quality of the training data. Preparing computer vision datasets for object detection and instance segmentation is a labor-intensive task requiring each instance in an image to be annotated. In practice, this often results in the quality of bounding box and polygon mask annotations being suboptimal. This paper quantifies empirically the ground truth annotation quality and COCO’s mean average precision (mAP) performance by introducing two separate noise measures, uniform and radial, into the ground truth bounding box and polygon mask annotations for the COCO and Cityscapes datasets. Mask-RCNN models are trained on various levels of noise measures to investigate the performance of each level of noise. The results showed degradation of mAP as the level of both noise measures increased. For object detection and instance segmentation respectively, using the highest level of noise measure resulted in a mAP degradation of 0.185 & 0.208 for uniform noise with reductions of 0.118 & 0.064 for radial noise on the COCO dataset. As for the Cityscapes datasets, reductions of mAP performance of 0.147 & 0.142 for uniform noise and 0.101 & 0.033 for radial noise were recorded. Furthermore, a decrease in average precision is seen across all classes, with the exception of the class motorcycle. The reductions between classes vary, indicating the effects of annotation uncertainty are class-dependent.

As the global production of waste continues to rise, there is a growing demand for more effective waste management strategies to handle this expanding problem. Recycling rates in the United States for recyclable materials are below 35%, resulting in elevated levels of waste being sent to landfills. This situation has alarming consequences, contributing to rising pollution in both soil and aquatic ecosystems, and is a significant source of concern for environmental scientists and the general population alike. The presence of contamination in recycling collection trucks is a root cause impacting recycling rates, leading to the rejection of entire loads from recycling processing sites. This problem can be alleviated by automatically detecting contamination in recyclable waste that is loaded into the collection truck hopper before compaction. In this paper, we have used different state-of-the-art computer vision-based models such as Faster-RCNN, Cascade-RCNN, Retinanet, YOLOv8 and Mask-RCNN to identify contamination within a densely cluttered waste environment. We further investigate the viability of transfer learning, comparing it to the models trained from scratch. The YOLOv8-x model attained a mean-average precision of 0.395 without using transfer learning, whereas with the incorporation of transfer learning, its performance increased to 0.463.

Deep learning-based computer vision models are typically data-hungry, resulting in the rise of dataset sizes. The consensus for computer vision datasets is that larger datasets lead to better model performance. However, the quality of the datasets is often not considered. Annotating datasets for fully supervised object detection and instance segmentation tasks requires a significant investment in time, effort, and cost. In practice, due to the large sample sizes needed, this often leads to inaccuracies in the annotation process. This research aims to understand and quantify the impact of annotation quality and quantity on the performance of object detection and instance segmentation models. Specifically, the research aims to investigate how introducing additional data with varying levels of annotation quality affects mean average precision (mAP) performance. To investigate the relationship between annotation quality and quantity, subsets of the COCO and ADE20K datasets are used. For each of the datasets, three types of annotation uncertainty are added to the annotations, which are localization uncertainty, incorrect class labels, and missing annotations. Mask-RCNN, YOLACT, and Mask2Former models are trained on a variety of sample sizes for varying levels of annotation uncertainties. The results indicate there is utility in adding additional data of lesser annotation quality. The extent of the benefits of the additional data is directly related to how degraded the annotations’ are. Furthermore, the results show that all three annotation uncertainties negatively affect mAP performance, with incorrect class labels degrading mAP performance the most, followed by missing annotations and lastly localization uncertainty.

The automotive industry has made significant strides in enhancing road safety and enabling automated driving features through advanced computer vision techniques. This is particularly true for short-range vehicle automation, where non-linear fisheye cameras are commonly used. However, these cameras are challenged by optical distortions, known as fisheye geometric distortions, which lead to object deformation within the image and significant pixel distortion, particularly at the image periphery. Based on the observation that fisheye and spherical images exhibit at least superficially similar geometric characteristics, we investigate the applicability of spherical models—including Spherical Convolutional Neural Networks (CNNs) and Spherical Vision Transformers (ViTs)—to fisheye images, even though fisheye images are not truly spherical. We perform our comparison using fisheye datasets—Woodscape, SynWoodscape, and SynCityscapes in autonomous driving scenarios, with a specific focus on the ability of spherical methods (Spherical CNNs and ViTs) to manage fisheye distortions and compared them against traditional non-spherical methods. Our findings indicate that spherical methods effectively address fisheye distortions without needing extra data augmentations. This results in better mean Intersection over Union (mIoU) scores, pixel accuracy, and better surround-view perception than other modern approaches for fisheye semantic segmentation. However, we also find that spherical methods have a greater tendency to overfit smaller datasets compared with non-spherical models. These advancements highlight how non-linear camera images can take advantage of spherical approximations through spherical models in autonomous driving.

This paper presents design analysis and insights for a new continuous-time input pipeline (CTIP) analog-to-digital converter (ADC) architecture that has enhanced bandwidth. An all-pass filter-based analog delay in the signal path allows bandwidth extension to Nyquist signal bandwidths. A resetting integrator gain stage provides a signal path delay helping to increase the bandwidth while reducing the power cost. The noise filtering property of the resetting integrator gain stage preserves the medium resistive input benefit of CTIP ADCs. The resetting integrator allows the architecture to be implemented with a feedforward compensated op-amp using low-voltage CMOS processes. This paper has been verified by simulation results of a CTIP ADC with 1.2-V supply voltage designed in TSMC's 65-nm CMOS technology.