For vehicles to navigate autonomously, they need to perceive and understand their immediate surroundings. Currently, cameras are the preferred sensors, due to their high performance and relatively low-cost compared with other sensors like LiDARs and Radars. However, their performance is limited by inherent imaging constraints, a standard RGB camera may perform poorly in extreme conditions, including low illumination, high contrast, bad weather (e.g. fog, rain, snow, etc.), glare, etc. Further, when using monocular cameras, it is more challenging to determine spatial distances than when using active range sensors such as LiDARs or Radars. Over the past years, novel image sensors, namely, infrared cameras, range-gated cameras, polarization cameras, and event cameras, have demonstrated strong potential. Some of them could be game-changers for future autonomous vehicles, they are the result of progress in sensor technology and the development of the accompanying perception algorithms. This paper presents in a systematic manner their principles, comparative advantages, data processing algorithms, and related applications. The purpose is to provide practitioners with an in-depth overview of novel sensing technologies that can contribute to the safe deployment of autonomous vehicles.

Metal nanoparticles have the ability to strongly enhance the local electromagnetic field in their vicinity. Such enhancement is crucial for biomolecular detection and is used by techniques such as surface plasmon resonance detection or surface-enhanced Raman scattering. For these processes, the sensitivity strongly depends on the electromagnetic field intensity confined around such nanoparticles. In this article, we have numerically studied an array of metallic nanocylinders, which can sustain localized surface plasmons (LSP). However, the excitation wavelengths of the LSP are not tunable due to their limited dispersion. We have demonstrated a plasmonic mode, the hybrid lattice plasmon (HLP), which is excited in such a periodic array by adding a uniform thin metallic film below it. This mode is a result of a harmonic coupling of the propagating surface plasmons present in such a metallic film with the Bragg waves of the array. It shows a strong confinement of the electromagnetic field intensity around the nanocylinders, similar to the LSP, but the dispersion of this HLP mode is, however, similar to that of the propagating plasmons and, thus, can be tuned over a wide range of excitation wavelengths. The structure was fabricated using electron beam lithography and characterized by a surface plasmon resonance setup. These experimental results show that the HLP mode can be excited in a classical Kretschmann configuration with a dispersion similar to the prediction of numerical simulations.

We have developed a full-field optical coherence microscopy system providing intensity-based tomographic images and spectroscopic information with ultrahigh spatial resolution. Local measurements of the backscattered light spectrum center of mass are achieved through short-time Fourier analysis of a stack of en face interferometric images acquired with a Linnik-type microscope. Using a halogen lamp as an illumination source enables us to achieve spectroscopic imaging over a wavelength range from 600 to 900 nm with a spatial resolution of ~1 µm. Absorption measurements of a colored gel are reported as a validation of the technique. Enhancement of tissue imaging contrast is demonstrated by imaging a Xenopus Laevis (African frog) tadpole ex vivo.

Ultrahigh-resolution full-field optical coherence tomography (FF-OCT) is demonstrated in the 800 nm and 1200 nm wavelength regions simultaneously using a Silicon-based (Si) CCD camera and an Indium Gallium Arsenide (InGaAs) camera as area detectors and a halogen lamp as illumination source. The FF-OCT setup is optimized to support the two broad spectral bands in parallel, achieving a detection sensitivity of approximately 90 dB and a micrometer-scale resolution in the three directions. Images of ex vivo biological tissues are presented (rabbit trachea and Xenopus laevis tadpole) with an increase in penetration depth at 1200 nm. A color image representation is applied to fuse both images and enhance spectroscopic property visualization.

Event cameras capture changes of illumination in the observed scene rather than accumulating light to create images. Thus, they allow for applications under high-speed motion and complex lighting conditions, where traditional frame-based sensors show their limits with blur and over- or under-exposed pixels. Thanks to these unique properties, they represent nowadays an highly attractive sensor for ITS-related applications. Event-based optical flow (EBOF) has been studied following the rise in popularity of these neuromorphic cameras. The recent arrival of high-definition neuromorphic sensors, however, challenges the existing approaches, because of the increased resolution of the events pixel array and a much higher throughput. As an answer to these points, we propose an optimized framework for computing optical flow in real-time with both low- and high-resolution event cameras. We formulate a novel dense representation for the sparse events flow, in the form of the “inverse exponential distance surface”. It serves as an interim frame, designed for the use of proven, state-of-the-art frame-based optical flow computation methods. We evaluate our approach on both low- and high-resolution driving sequences, and show that it often achieves better results than the current state of the art, while also reaching higher frame rates, 250Hz at 346 $\times260$ pixels and 77Hz at 1280 $\times720$ pixels.

A precise GNSS (Global Navigation Satellite System) localization is vital for autonomous road vehicles, especially in cluttered or urban environments where satellites are occluded, preventing accurate positioning. We propose to fuse GPS (Global Positioning System) data with fisheye stereovision to face this problem independently to additional data, possibly outdated, unavailable, and needing correlation with reality. Our stereoscope is sky-facing with 360° × 180° fisheye cameras to observe surrounding obstacles. We propose a 3D modelling and plane extraction through following steps: stereoscope self-calibration for decalibration robustness, stereo matching considering neighbours epipolar curves to compute 3D, and robust plane fitting based on generated cartography and Hough transform. We use these 3D data with GPS raw data to estimate NLOS (Non Line Of Sight) reflected signals pseudorange delay. We exploit extracted planes to build a visibility mask for NLOS detection. A simplified 3D canyon model allows to compute reflections pseudorange delays. In the end, GPS positioning is computed considering corrected pseudoranges. With experimentations on real fixed scenes, we show generated 3D models reaching metric accuracy and improvement of horizontal GPS positioning accuracy by more than 50%. The proposed procedure is effective, and the proposed NLOS detection outperforms CN0-based methods (Carrier-to-receiver Noise density).