This letter presents a robust data association method for fusing camera and marine radar measurements in order to automatically detect surface ships and determine their locations with respect to the observing ship. In the preprocessing step for sensor fusion, convolutional neural networks are used to perform feature extraction from camera images and semantic segmentation from radar images. The correspondences between the camera and radar image features are optimized using a pair of geometric parameters, and the positions of all the matched object features are determined. The proposed method enables robust data association even without accurate calibration and alignment between the camera and radar measurements. The feasibility and performance of the proposed method are demonstrated using an experimental dataset obtained in a real coastal environment.
Wheeled mobile robots have a significant role in the mobility industry. When outdoor mobile robots perform given tasks in urban environments, they often face the challenge of climbing and descending curbs. Many commercial mobile robots rely on ramps to navigate curbs. However, for outdoor mobile robots to achieve human-like freedom of movement in urban spaces, they must be capable of climbing and descending curbs. A six-wheeled mobile robot can be a solution to this problem since it can effortlessly climb or descend curbs. When driving on flat terrain, however, driving with six wheels is quite inefficient, wasting energy, and driving with four wheels would be better than using six wheels. Therefore, when operating a six-wheeled mobile robot, we need to determine the appropriate mode to use, specifically the six-wheel mode or the four-wheel mode, depending on the type of the road. In this paper, we present a method that determines the appropriate mode using the video captured from the frontal camera. We employ 3D CNN and the Bayesian fusion to determine the optimal mode for the six-wheeled robot and optimal timing for mode change. We collect a set of video sequences and test our method on these collected video sequences.
22 August 2022 in IEEE Robotics & Automation Magazine
Change detection aims to identify pixel-wise scene changes between observations acquired at different times by leveraging the strong visual representation capabilities of deep neural networks on RGB images. In robotics scenarios such as the ChangeSim dataset, visual observations are naturally collected as temporally sequential image streams rather than isolated image pairs. Our key insight is that these temporal cues can be exploited as a temporal prior for scene change detection, inspired by Bayesian fusion and the Markov assumption in probabilistic robotics. Based on this insight, we propose a pixel-wise temporal fusion framework that integrates information from a reference visual memory and temporally preceding observations. A frozen optical-flow model is employed to align the current-frame change prediction with a temporally accumulated fusion tensor, enabling consistent pixel-wise correspondence across time. This design yields two complementary information pathways: one capturing instantaneous semantic differences from the current image and another progressively integrating historical evidence through temporal fusion. Experiments on the ChangeSim dataset demonstrate that the proposed framework achieves a five-class mean Intersection-over-Union (mIoU) of 29.6%, compared with 26.8% for a recipe-matched model without temporal fusion. The proposed framework also achieves a change-only mIoU of 14.2%, compared with 10.7% for the no-fusion counterpart, indicating improved semantic change recognition under the evaluated conditions.