Cross-view geo-localization (CVGL) aims to determine the locations of ground-view images using corresponding aerial views. However, the inherent differences in the viewpoints and appearances between these cross-view images complicate accurate localization. Existing CVGL approaches have focused on aligning image perspectives; however, these approaches often overlook the information distortion that occurs during the alignment process. To address this challenge, we propose a novel triple-vision transformers framework that fully utilizes the spatially aligned and original images. The triple-vision transformers consists of a Siamese network and two independent networks. The Siamese network takes the polar-transformed ground or aerial images to extract the coarse features from the aligned images. Meanwhile, the two independent networks takes the original images (i.e. aerial or ground images respectively) to compensate for the distortion owing to the alignment process. Furthermore, we introduce geo-aligned triplet learning using a polar warping network for the independent transformers to enhance the correlation between image descriptors within cross-view images. The results of extensive experiments on benchmark datasets CVUSA and CVACT prove that our method exhibits superior performance with respect to existing methods. Furthermore, proposed simple yet effective approach demonstrates scalability across different backbones, including CNNs.

Detecting failure cases is an essential element for ensuring the safety self-driving system. Any fault in the system directly leads to an accident. In this paper, we analyze the failure of semantic segmentation, which is crucial for autonomous driving system, and detect the failure cases of the predicted segmentation map by predicting mean intersection of union (mIoU). Furthermore, we design a deep neural network for predicting mIoU of segmentation map without the ground truth and introduce a new loss function for training imbalance data. The proposed method not only predicts the mIoU, but also detects failure cases using the predicted mIoU value. The experimental results on Cityscapes data show our network gives prediction accuracy of 93.21% and failure detection accuracy of 84.8%. It also performs well on a challenging dataset generated from the vertical vehicle camera of the Hyundai Motor Group with 90.51% mIoU prediction accuracy and 83.33% failure detection accuracy.

Multi-object tracking (MOT) has gained significant attention in computer vision due to its wide range of applications. Specifically, detection-based trackers have shown high performance in MOT, but they tend to fail in occlusive scenarios such as the moment when objects overlap or separate. In this paper, we propose a triplet-based MOT network that integrates the tracking information and the visual features of the object. Using a triplet-based image feature, the network can differentiate similar-looking objects, reducing the number of identity switches over a long period. Furthermore, an attention-based re-identification model that focuses on the appearance of objects was introduced to extract the feature vectors from the images to effectively associate the objects. The extensive experimental results demonstrated that the proposed method outperforms existing methods on the ID switch metric and improves the detection performance of the tracking system.

Goal-conditioned Reinforcement Learning (RL) holds promise for addressing intricate control challenges by enabling agents to learn and execute desired skills through separate decision modules. However, the irregular occurrence of required skills poses a significant challenge to effective learning. In this paper, we demonstrate the detrimental effects of this imbalanced skill (sub-goal) distribution and propose a novel training approach, Classified Experience Replay (CER), designed to mitigate this challenge. We demonstrate that adapting our method to conventional RL methods significantly enhances the performance of the RL agent. Considering the challenges inherent in tasks such as driving, characterized by biased occurrences of required sub-goals, our study demonstrates the improvement in trained outcomes facilitated by the proposed method. In addition, we introduce a specialized framework tailored for self-driving tasks on highways, integrating model predictive control into our RL trajectory optimization training paradigm. Our approach, utilizing CER with the suggested framework, yields remarkable advancements in trajectory optimization for RL agents operating in highway environments.

Removing haze or rain is one of the difficult problems in computer vision applications. On real-world road images, haze and rain often occur together, but traditional methods cannot solve this imaging problem. To address rain and haze problems simultaneously, we present a robust network-based framework consisting of three steps: image decomposition using guided filters, a frequency-based haze and rain removal network (FHRR-Net), and image restoration based on an atmospheric scattering model using predicted transmission maps and predicted rain-removed images. We demonstrate FHRR-Net’s capabilities with synthesized and real-world road images. Experimental results show that our trained framework has superior performance on synthesized and real-world road test images compared with state-of-the-art methods. We use PSNR (peak signal-to-noise) and SSIM (structural similarity index) indicators to evaluate our model quantitatively, showing that our methods have the highest PSNR and SSIM values. Furthermore, we demonstrate through experiments that our method is useful in real-world vision applications.

Palletizing, also known as the 3D bin packing problem, is important for optimizing space utilization and automating packing processes, especially in the logistics industry. In practice, handling mixed palletizing scenarios, where a variety of boxes of different sizes are received in real time, is considerably challenging. Existing methods for solving the mixed palletizing problem often overlook practical constraints encountered in real-world applications, such as those pertaining to stability and robustness. In this paper, we propose a practical mixed palletizing manipulator system designed for structured real-world warehouse environments. Our manipulator system has two main components: a practical mixed palletizing model based on reinforcement learning (PMP-RL), which can facilitate stable and efficient box placing, and a configuration-space motion planning network (CMPNet), which can help achieve robust and efficient collision-free robot movement. The PMP-RL model is designed to maximize the pallet volume utilization while incorporating practical reward functions that enhance stability. CMPNet is used to directly predict motion trajectories in a 3D configuration space, and it facilitates real-time motion generation by effectively imitating expert-level paths. Overall, the manipulator system, comprising an automated conveyor belt, a camera-based recognition system, the PMP-RL model, and CMPNet, provide a robust and practical framework for mixed palletizing. Experiments conducted via simulations and in real-world environments have shown that the manipulator system can handle complex palletizing tasks with high efficiency and high stability.