Thanks to some technical progress in interferencefilter design based on different technologies, we can finally successfully implement the concept of multispectral filter array-based sensors. This article provides the relevant state-of-the-art for multispectral imaging systems and presents the characteristics of the elements of our multispectral sensor as a case study. The spectral characteristics are based on two different spatial arrangements that distribute eight different bandpass filters in the visible and near-infrared area of the spectrum. We demonstrate that the system is viable and evaluate its performance through sensor spectral simulation.

Snapshot polarization imaging has gained interest in the last few decades. Recent research and technology achievements defined the polarization Filter Array (PFA). It is dedicated to division-of-focal plane polarimeters, which permits to analyze the direction of light electric field oscillation. Its filters form a mosaicked pattern, in which each pixel only senses a fraction of the total polarization states, so the other missing polarization states have to be interpolated. As for Color or Spectral Filter Arrays (CFA or SFA), several dedicated demosaicking methods exist in the PFA literature. Such methods are mainly based on spatial correlation disregarding inter-channel correlation. We show that polarization channels are strongly correlated in images. We therefore propose to extend some demosaicking methods from CFA/SFA to PFA, and compare them with those that are PFA-oriented. Objective and subjective analysis show that the pseudo panchromatic image difference method provides the best results and can be used as benchmark for PFA demosaicking.

Multispectral acquisition improves machine vision since it permits capturing more information on object surface properties than color imaging. The concept of spectral filter arrays has been developed recently and allows multispectral single shot acquisition with a compact camera design. Due to filter manufacturing difficulties, there was, up to recently, no system available for a large span of spectrum, i.e., visible and Near Infra-Red acquisition. This article presents the achievement of a prototype of camera that captures seven visible and one near infra-red bands on the same sensor chip. A calibration is proposed to characterize the sensor, and images are captured. Data are provided as supplementary material for further analysis and simulations. This opens a new range of applications in security, robotics, automotive and medical fields.

The electric field that describes electromagnetic radiation can be considered as a vector, whose direction of oscillation is perpendicular to the direction of the wave. This geometrical description of waves is known as polarization.¹ Reptiles or birds, for example, are capable of perceiving polarized information, whereas humans, like any other mammals, are only sensitive to two properties of light: intensity and color. The analysis of the polarization scattered by a scene is known as polarimetric imaging; it yields a complementary performance to classical intensity imaging.² A mathematical tool named Stokes vector is used to describe practically and efficiently the modification of the polarization states when the light travels and interacts with different materials. Polarization analysis using polarimeter instruments becomes increasingly popular in imaging applications, such as classification of materials,³ 3D inspection and reconstruction,⁴ image dehazing,⁵ etc.

This review article aims to address common research questions in passive polarized vision for robotics. What kind of polarization sensing can we embed into robots? Can we find our geolocation and true north heading by detecting light scattering from the sky as animals do? How should polarization images be related to the physical properties of reflecting surfaces in the context of scene understanding? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying future directions in passive polarized vision for robotics. After an introduction, three key interconnected areas will be covered in the following sections: embedded polarization imaging; polarized vision for robotics navigation; and polarized vision for scene understanding. We will then discuss how polarized vision, a type of vision commonly used in the animal kingdom, should be implemented in robotics; this type of vision has not yet been exploited in robotics service. Passive polarized vision could be a supplemental perceptive modality of localization techniques to complement and reinforce more conventional ones.