This study develops a novel automatic all-sky imaging system, namely, an all-sky camera (ASC) system, for cloud cover assessment. The proposed system does not require conventional solar occulting devices and can capture complete hemispheric sky images. Cloud detection is performed innovatively using a convolutional neural network model (i.e., the optimized U-Net model). Experiments demonstrate that the optimized U-Net model can effectively detect clouds from sky images. In terms of cloud cover, the estimation results of the ASC system exhibit a high correlation with those obtained via manual observation, thereby indicating the applicability of the ASC system in ground-based cloud observation and analysis.

The formation and evolution of clouds are associated with their thermodynamical and microphysical progress. Previous studies have been conducted to collect images using ground-based cloud observation equipment to provide important cloud characteristics information. However, most of this equipment cannot perform continuous observations during the day and night, and their field of view (FOV) is also limited. To address these issues, this work proposes a day and night clouds detection approach integrated into a self-made thermal-infrared (TIR) all-sky-view camera. The TIR camera consists of a high-resolution thermal microbolometer array and a fish-eye lens with a FOV larger than 160°. In addition, a detection scheme was designed to directly subtract the contamination of the atmospheric TIR emission from the entire infrared image of such a large FOV, which was used for cloud recognition. The performance of this scheme was validated by comparing the cloud fractions retrieved from the infrared channel with those from the visible channel and manual observation. The results indicated that the current instrument could obtain accurate cloud fraction from the observed infrared image, and the TIR all-sky-view camera developed in this work exhibits good feasibility for long-term and continuous cloud observation.

Visual navigation is of vital importance for autonomous mobile robots. Most existing practical perception-aware based visual navigation methods generally require prior-constructed precise metric maps, and learning-based methods rely on large training to improve their generality. To improve the reliability of visual navigation, in this paper, we propose a novel object-level topological visual navigation method. Firstly, a lightweight object-level topological semantic map is constructed to release the dependence on the precise metric map, where the semantic associations between objects are stored via graph memory and topological organization is performed. Then, we propose an object-based heuristic graph search method to select the global topological path with the optimal and shortest characteristics. Furthermore, to reduce the global cumulative error, a global path segmentation strategy is proposed to divide the global topological path on the basis of active visual perception and object guidance. Finally, to achieve adaptive smooth trajectory generation, a Bernstein polynomial-based smooth trajectory refinement method is proposed by transforming trajectory generation into a nonlinear planning problem, achieving smooth multi-segment continuous navigation. Experimental results demonstrate the feasibility and efficiency of our method on both simulation and real-world scenarios. The proposed method also obtains better navigation success rate (SR) and success weighted by inverse path length (SPL) than the state-of-the-art methods.

State estimation is crucial for robot autonomy, visual odometry (VO) has received significant attention in the robotics field because it can provide accurate state estimation. However, the accuracy and robustness of most existing VO methods are degraded in complex conditions, due to the limited field of view (FOV) of the utilized camera. In this paper, we present a novel tightly-coupled multi-keyframe visual-inertial odometry (called VINS-MKF), which can provide an accurate and robust state estimation for robots in an indoor environment. We first modify the monocular ORBSLAM (Oriented FAST and Rotated BRIEF Simultaneous Localization and Mapping) to multiple fisheye cameras alongside an inertial measurement unit (IMU) to provide large FOV visual-inertial information. Then, a novel VO framework is proposed to ensure the efficiency of state estimation, by adopting a GPU (Graphics Processing Unit) based feature extraction method and parallelizing the feature extraction thread that is separated from the tracking thread with the mapping thread. Finally, a nonlinear optimization method is formulated for accurate state estimation, which is characterized as being multi-keyframe, tightly-coupled and visual-inertial. In addition, accurate initialization and a novel MultiCol-IMU camera model are coupled to further improve the performance of VINS-MKF. To the best of our knowledge, it’s the first tightly-coupled multi-keyframe visual-inertial odometry that joins measurements from multiple fisheye cameras and IMU. The performance of the VINS-MKF was validated by extensive experiments using home-made datasets, and it showed improved accuracy and robustness over the state-of-art VINS-Mono.

The atmospheric temperature profile is an important parameter to describe the state of the atmosphere, and it is crucial to climate change research, weather forecasting, and atmospheric parameter retrieval. A machine learning algorithm that incorporates historical observations and ground-based measurements was developed in this study to retrieve the atmospheric temperature profile. Specifically, the deep learning network considered historical observations for the same period and temporally correlated temperature profiles. It combined multi-layer perceptron (MLP) and the convolutional neural network (CNN). MLP derived the features from the ground factors, and CNN captured the essential features associated with the temperature profiles at the current time from latent historical data. Then, the features of the two parts were concatenated to obtain the final network. The construction and parameters of the model were optimized to determine the best model configuration and retrieval performance. The results of the model were evaluated against those of other methods on the same dataset. The model showed a good retrieval precision, which was equivalent to a small retrieval bias, root-mean-square error, and mean absolute error at all altitudes. The analysis of the application of this model to the retrieval of atmospheric temperature profiles indicates that the method is feasible.

Visible and infrared binocular all-sky-view cameras can provide continuous and complementary ground-based cloud observations. Accurate angular calibration for every pixel is an essential premise to further cloud analysis and georeferencing. However, most current calibration methods mainly rely on calibration plates, which still remains difficult for simultaneously calibrating visible and infrared binocular cameras, especially with different imaging resolutions. Thus, in this study, we present a simple and convenient angular calibration method for wide field-of-view visible and infrared binocular cameras. Without any extra instruments, the proposed method only utilizes the relation between the angular information of direct sun lights and the projected sun pixel coordinates to compute the geometric imaging parameters of the two cameras. According to the obtained parameters, the pixel-view-angle for the visible and infrared all-sky images is efficiently computed via back projection. Meanwhile, the projected pixel coordinates for the incident lights at any angle can also be computed via reprojection. Experimental results show the effectiveness and accuracy of the proposed angular calibration through the error estimation of reprojection and back projection. As a novel application, we successfully achieve visible and infrared binocular image registration at the pixel level after finishing angular calibration, which not only verifies the accuracy of calibration results, but also contributes to further cloud parameter analysis under these two different imaging features. The registration results, to our knowledge, also provide a reference for the current blank in visible and infrared binocular cloud image registration.