Pedestrian detection has never been an easy task for computer vision and the automotive industry. Systems like the advanced driver-assistance system (ADAS) highly rely on far-infrared (FIR) data captured to detect pedestrians at nighttime. The recent development of deep learning-based detectors has proven the excellent results of pedestrian detection in perfect weather conditions. However, it is still unknown what the performance in adverse weather conditions is. In this paper, we introduce a 16-bit thermal data dataset called ZUT (Zachodniopomorski Uniwersytet Technologiczny) as having the widest variety of fine-grained annotated images captured in the four biggest European Union countries captured during severe weather conditions. We also provide a synchronized Controller Area Network (CAN bus) data, including driving speed, brake pedal status, and outside temperature for future ADAS system development. Furthermore, we have tested and provided 16-bit depth modifications for the YOLOv3 deep neural network (DNN) based detector, reaching a mean Average Precision (mAP) up to 89.1%. The ZUT dataset is published and publicly available at IEEE Dataport [1] and Github [2].

Assistive vision-based solutions for the driver extend the capabilities of human vision and support safe travel. Unfortunately, their widespread usage is generally limited to expensive cars. Interestingly, a high price is most likely a derivative of the costs incurred in the research instead of the value of hardware components. In the article we show that a mobile system for pedestrian detection in severe lighting conditions can be build using state of the art algorithms and widely available hardware. The proposed night-vision system for pedestrian detection processes thermal images using a proprietary ODROID XU4 microcomputer under Ubuntu MATE operating system. We applied a cascade object detector for the task of human silhouette detection in context of thermal imagery and contrasted the results with the state of the art deep learning approach. The experiments conducted prove effectiveness of the proposed solution.

Pedestrian detection is an essential task for computer vision and the automotive industry. Complex systems like advanced driver-assistance systems are based on far-infrared data sensors, used to detect pedestrians at nighttime, fog, rain, and direct sun situations. The robust pedestrian detector should work in severe weather conditions. However, only a few datasets include some examples of far-infrared images with distortions caused by atmospheric precipitation and dirt covering sensor optics. This paper proposes the deep learning-based data augmentation technique to enrich far-infrared images collected in good weather conditions by distortions, similar to those caused by bad weather. The six most accurate and fast detectors (TinyV3, TinyL3, You Only Look Once (YOLO)v3, YOLOv4, ResNet50, and ResNext50), performing faster than 15 FPS, were trained on 207,001 annotations and tested on 156,345 annotations, not used for training. The proposed data augmentation technique showed up to a 9.38 mean Average Precision (mAP) increase of pedestrian detection with a maximum of 87.02 mAP (YOLOv4). Proposed in this paper detectors’ Head modifications based on a confidence heat-map gave an additional boost of precision for all six detectors. The most accurate current detector, based on YOLOv4, reached up to 87.20 mAP during our experimental tests.

Touchless interaction with electronic devices using gestures is gaining popularity and along with speech-based communication offers their users natural and intuitive control methods. Now, these interaction modes go beyond the entertainment industry and are successfully applied in real-life scenarios such as a car interior. In the paper, we analyse the potential of hand gesture interaction in the vehicle environment by physically challenged drivers. A survey conducted with potential users shows that the knowledge of gesture-based interaction and its practical use by people with disabilities is low. Based on these results we proposed a gesture-based system for vehicle on-board system. It has been developed on the available state-of-the-art solutions and investigated in terms of usability on a group of people with different physical limitations who drive a car on daily basis mostly using steering aid tools. The obtained results are compared with the performance of users without any disabilities.