22 January 2025 in International Journal of Computer Vision
Zero-shot OOD detection is a task that detects OOD images during inference with only in-distribution (ID) class names. Existing methods assume ID images contain a single, centered object, and do not consider the more realistic multi-object scenarios, where both ID and OOD objects are present. To meet the needs of many users, the detection method must have the flexibility to adapt the type of ID images. To this end, we present Global-Local Maximum Concept Matching (GL-MCM), which incorporates local image scores as an auxiliary score to enhance the separability of global and local visual features. Due to the simple ensemble score function design, GL-MCM can control the type of ID images with a single weight parameter. Experiments on ImageNet and multi-object benchmarks demonstrate that GL-MCM outperforms baseline zero-shot methods and is comparable to fully supervised methods. Furthermore, GL-MCM offers strong flexibility in adjusting the target type of ID images. The code is available via https://github.com/AtsuMiyai/GL-MCM.
2 June 2023 in IEEE Transactions on Multimedia
Unsupervised domain adaptation (UDA) is extremely effective for transferring knowledge from a label-rich source domain to a label-scarce target domain. Because the target domain is unlabeled and may contain additional novel classes, open-set domain adaptation (ODA) has been suggested as a possible solution to detect these novel classes in the training phase. However, existing ODA methods rely heavily on abundant fully labeled source data, which are expensive to collect in specific applications and may also contain novel classes. In this study, we propose a novel self-labeling framework with prototypical contrastive learning and mutual information maximization to achieve ODA even when the amount of labeled data is very small, which is a new problem setting named few-shot ODA (FODA). We use self-supervised prototypical contrastive learning to train the network to learn the representations of source and target samples and maximize the mutual information between labels and input data to simultaneously recognize known and novel classes in the source and target domains. We evaluated our strategy in several domain adaptation environments and found that our method performed far better than existing approaches.
4 March 2024 in IEEE Access
Two-dimensional (2D) image registration is a natural choice for simultaneous object pose estimation and object recognition. However, it was not designed to perform object segmentation, which is critical for object-picking applications in warehouse automation scenarios. In this study, we propose a unified 2D image registration framework that simultaneously performs image registration and object segmentation by introducing a deep segmentation network module to the 2D image registration framework. Our method is designed to automatically generate annotations for training the segmentation network module through the process of 2D image registration, that is, no additional manual annotations are required. Specifically, given initial object regions from the 2D image registration results, our method trains the segmentation network module to refine a pseudo-pixel-level object region and remove background pixels based on the pixel-level similarity of an aligned image pair. The experimental results on several picking object datasets demonstrated that the segmentation accuracy of our method was superior to that of existing weakly supervised segmentation methods, and our method simultaneously achieved better performance for object recognition and pose estimation. Furthermore, our segmentation network module smoothly cooperated with many existing 2D image registration techniques.