23 February 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Domain Adaptation aims at adapting the knowledge learned from a domain (source-domain) to another (target-domain). Existing approaches typically require a portion of task-relevant target-domain data a priori. We propose an approach, zero-shot deep domain adaptation (ZDDA), which uses paired dual-domain task-irrelevant data to eliminate the need for task-relevant target-domain training data. ZDDA learns to generate common representations for source and target domains data. Then, either domain representation is used later to train a system that works on both domains or having the ability to eliminate the need to either domain in sensor fusion settings. Two variants of ZDDA have been developed: ZDDA for classification task (ZDDA-C) and ZDDA for metric learning task (ZDDA-ML). Another limitation in Existing approaches is that most of them are designed for the closed-set classification task, i.e., the sets of classes in both the source and target domains are ‘`known." However, ZDDA-C is also applicable to the open-set classification task where not all classes are ``known" during training. Moreover, the effectiveness of ZDDA-ML shows ZDDA’s applicability is not limited to classification tasks. ZDDA-C and ZDDA-ML are tested on classification and metric-learning tasks, respectively. Under most experimental conditions, ZDDA outperforms the baseline without using task-relevant target-domain-training data.
11 September 2023 in arXiv
Recent times have witnessed an increasing number of applications of deep neural networks towards solving tasks that require superior cognitive abilities, e.g., playing Go, generating art, ChatGPT, etc. Such a dramatic progress raises the question: how generalizable are neural networks in solving problems that demand broad skills? To answer this question, we propose SMART: a Simple Multimodal Algorithmic Reasoning Task and the associated SMART-101 dataset, for evaluating the abstraction, deduction, and generalization abilities of neural networks in solving visuo-linguistic puzzles designed specifically for children in the 6--8 age group. Our dataset consists of 101 unique puzzles; each puzzle comprises a picture and a question, and their solution needs a mix of several elementary skills, including arithmetic, algebra, and spatial reasoning, among others. To scale our dataset towards training deep neural networks, we programmatically generate entirely new instances for each puzzle, while retaining their solution algorithm. To benchmark performances on SMART-101, we propose a vision and language meta-learning model using varied state-of-the-art backbones. Our experiments reveal that while powerful deep models offer reasonable performances on puzzles in a supervised setting, they are not better than random accuracy when analyzed for generalization. We also evaluate the recent ChatGPT and other large language models on a subset of SMART-101 and find that while these models show convincing reasoning abilities, the answers are often incorrect.