22 December 2017 in IEEE Sensors Journal
This paper presents a novel multi-sensor navigation system for the urban ground vehicle. The integration system can combine the measurements from a polarized skylight sensor, an inertial sensor, and a monocular camera. Utilizing the polarized skylight sensor, we propose a robust orientation algorithm with the total least squares to provide the orientation constraint for the integrating system. In our algorithm, the ambiguity problem of polarized orientation is solved without any other sensor. In order to enhance the algorithm's robustness in the urban environment, we also propose a real-time method that uses the gradient of the degree of the polarization to remove the obstacles. With a monocular camera, we build a metric map and recognize places in the map to provide the position constraint for the integrating system. We develop the Kalman filter to integrate these constraints with the inertial navigation results to estimate the orientation and position for the ground vehicle. The results demonstrate that our proposed system outperforms other vision-based navigation algorithms-the RMSE of the position error is 2.04 m (0.01% of the travelled distance) and the RMSE of the orientation error is 0.84°. Finally, we present interesting insights gained with respect to the further work in sensors and robotics.
29 June 2020 in IEEE Robotics and Automation Letters
The recent development of the on-chip micro-polarizer technology has made it possible to acquire - with the same ease of operation as a conventional camera - spatially aligned and temporally synchronized polarization images simultaneously in four orientations. This development has created new opportunities for interesting applications including those in robotics. In this letter, we investigate the use of this sensor technology in high-dynamic-range (HDR) imaging. Specifically, observing that natural light can be attenuated differently by varying the direction of the polarization filter, we treat the multiple images captured by the polarization camera as a set captured at different exposure times, useful to the reconstruction of an HDR image. In our approach, we first study the radiometric model of the polarization camera, and relate the polarizer direction, degree and angle of polarization of light to the exposure time of a pixel in the polarization images. Subsequently, by applying the standard radiometric calibration procedure of a camera, we recover the camera response function. With multiple polarization images at known pixel-specific exposure times, we can then proceed to estimate the irradiance maps from the images and generate an HDR image. Two datasets are created to evaluate our approach, and experimental results show the dynamic range by our approach can be increased by an amount dependent on light polarization. We also use two robotics experiments on feature matching and visual odometry to demonstrate the potential benefit of this increased dynamic range.
9 April 2018 in Optical Engineering
A robust orientation algorithm based on the skylight polarization patterns for the urban ground vehicle is presented. We present the orientation model with the Rayleigh scattering and propose the robust orientation algorithm with the total least square. The proposed algorithm can utilize the whole sky area polarization patterns for realizing a more robust and accurate orientation. To enhance the algorithm’s robustness in the urban environment, we develop a real-time method that uses the gradient of the degree of the polarization to remove the obstacles in the polarization image. In addition, our algorithm can solve the ambiguity problem of the polarized orientation without any other sensors. We also conduct a static rotating and a dynamic car experiments to evaluate the algorithm. The results demonstrate that our proposed algorithm can provide an accurate orientation estimation for the ground vehicle in the open and urban environments—the root-mean-square error in the static experiment is 0.28 deg and in the dynamic experiment is 0.81 deg. Finally, we discuss insights gained with respect to further work in optics and robotics.
17 March 2021 in IEEE Sensors Journal
In this paper we present a bionic multi-sensor anavigation and control system for unmanned ground vehicles to complete the homing task. The system consists of the pixelated polarized sensor, a Micro Inertial Measurement Unit (MIMU), and a monocular camera. To compensate for the installation error, we provide a joint calibration method for multiple sensors. Utilizing the measurements of pixelated polarized vision sensor, we firstly propose an adaptive integrated method with the Visual-Inertial System. The integrated algorithm can not only solve the ambiguity problem of polarized orientation and reduce the cumulative error of the system, but also increase the navigation output rate and enhance the robustness of the system. We present a homevector-based strategy for the goal-directed navigation and control of the unmanned ground vehicles. When the external data link is interrupted, the unmanned vehicles can return to the starting point autonomously, which benefits for improving the survivability of the system. Finally, we design various experiments to verify the algorithm proposed in this paper. The experimental results of the calibration demonstrate that the RMSE of the orientation after calibration is only 0.014°. In the navigation and homing experiment, the RMSE of the position error is 0.64m, and the minimum homing error is only 0.49m (1.09% of the travelled distance). Finally, we discuss interesting insights gained with respect to future work in multi-sensor integration and robot control strategies.
24 August 2022 in IEEE Sensors Journal
Skylight polarized patterns provide fresh thinking for the human to use the information sources in nature for orientation, but this orientation method is susceptible to weather conditions. Especially, in the case of cloudy weather, the cloud directly leads to the decline of orientation precision. In this article, a robust bionic polarized skylight orientation method based on the model consistency of polarization patterns (MCOPPs) is proposed for cloudy weather. First, we made full use of the DOP information to preprocess the polarization information as an aid to the subsequent polarization light orientation. Then, based on the vertical feature of the E-vector of the polarized light and the scattering plane, we developed the MCOPP error criterion and proposed a two-step optimizing framework for the effective selection of the interior points. Finally, the outdoor static rotating experiment and the dynamic vehicle experiment are conducted for evaluating our algorithm. The experimental results demonstrate that MCOPP can realize an effective and robust polarized orientation under cloudy conditions, and the accuracy of orientation in the outdoor static experiment is within 0.5° and in the dynamic car experiment is within 0.7°. Compared with the other four classic methods, the RMSE of the orientation error can be decreased by 50.6% in the outdoor static experiment and by 46.0% in the dynamic vehicle experiment. Especially, the proposed MCOPP has lower computational complexity than the existing robust polarized skylight method.
4 July 2022 in IEEE Sensors Journal
In the challenging environment, a reliable and autonomous navigation is the key performance factor for robotics. This paper presents a bio-inspired multi-sensor navigation system with the skylight compass and visual place recognition for the UAVs in the GNSS-denied environment. The system can integrate the position and heading constraints outputs from a skylight polarized sensor, a micro-inertia sensor and a monocular camera. An optimal orientation algorithm is proposed with the skylight polarized patterns for obtaining the heading constraint. We also propose a two-dimensional visual place recognition for providing the position constraint in the aerial environment and the metric encoding pattern and place recognition are presented in detail. Based on the position and heading constraints, a biologically multi-sensor integrating system with Kalman Filter has been developed to accurately estimate the motion information for UAVs. Unlike previous approaches having a fragile navigation scheme and path-error accumulation, our system can build a bionic and robust association between the motion metric information and the sensor inputs, and use reliable position and heading constraints for improving the navigation performance. The flying experiments have demonstrated that the proposed bio-inspired navigation system can outperform other vision-based navigation algorithms and the position errors are convergent and do not grow with the flying distance, in which the best performance of position precision is within 5m. Finally, we present interesting insights gained with respect to further work in robotics.