Artificial intelligence and machine learning (AI/ML) research has aimed to achieve human-level performance in tasks that require understanding and decision making. Although major advances have been made, AI systems still struggle to achieve adaptive learning for generalization. One of the main approaches to generalization in ML is transfer learning, where previously learned knowledge is utilized to solve problems in a different, but related, domain. Another approach, pursued by cognitive scientists for several decades, has investigated the role of analogical reasoning in comparisons aimed at understanding human generalization ability. Analogical reasoning has yielded rich empirical findings and general theoretical principles underlying human analogical inference and generalization across distinctively different domains. Though seemingly similar, there are fundamental differences between the two approaches. To clarify differences and similarities, we review transfer learning algorithms, methods, and applications in comparison with work based on analogical inference. Transfer learning focuses on exploring feature spaces shared across domains through data vectorization while analogical inferences focus on identifying relational structure shared across domains via comparisons. Rather than treating these two learning approaches as synonymous or as independent and mutually irrelevant fields, a better understanding of how they are interconnected can guide a multidisciplinary synthesis of the two approaches.
Recently, generative artificial intelligence (GAI) has impressed the world with its ability to create text, images, and videos. However, there are still areas in which GAI produces undesirable or unintended results due to being “uncertain”. Before wider use of AI-generated content, it is important to identify concepts where GAI is uncertain to ensure the usage thereof is ethical and to direct efforts for improvement. This study proposes a general pipeline to automatically quantify uncertainty within GAI. To measure uncertainty, the textual prompt to a text-to-image model is compared to captions supplied by four image-to-text models (GIT, BLIP, BLIP-2, and InstructBLIP). Its evaluation is based on machine translation metrics (BLEU, ROUGE, METEOR, and SPICE) and word embedding’s cosine similarity (Word2Vec, GloVe, FastText, DistilRoBERTa, MiniLM-6, and MiniLM-12). The generative AI models performed consistently across the metrics; however, the vector space models yielded the highest average similarity, close to 80%, which suggests more ideal and “certain” results. Suggested future work includes identifying metrics that best align with a human baseline to ensure quality and consideration for more GAI models. The work within can be used to automatically identify concepts in which GAI is “uncertain” to drive research aimed at increasing confidence in these areas.
25 March 2024 in Natural Language Processing Journal
As generative artificial intelligence (AI) becomes more common in day-to-day life, AI-generated content (AIGC) needs to be accurate, relevant, and comprehensive. These characteristics typically are determined by subjective, human-based image quality assessment; however, there is limited research on the qualification of AI-generated image quality. Over 9,800 images were generated using Craiyon and OpenAI's DALL-E 2 text-to-image models and evaluated on the three criteria proposed for determining the quality of visual AIGC: (1) the number of objects, (2), resolution (strictly image quality; label/prompt exclusive), and (3) representativeness (consideration for how well the image matches the label/prompt). We observe that the paid, DALL-E 2 model, produced a dataset with fewer objects per image, higher resolution, and higher representativeness compared to Craiyon (free). There is an inverse relationship between the number of objects/images and its resolution and representativeness. This study establishes three subjective metrics for the evaluation of synthetic images to support the creation of more inclusive AIGC.
In recent years, large language models (LLMs) have made substantial strides in mimicking human language and coherently presenting information. However, researchers continue to debate the accuracy and robustness of LLMs’ reasoning abilities. The reasoning abilities of thirteen LLMs were tested on two long-text analogy datasets, named Rattermann and Wharton, which required them to rank a series of stories from most analogous to least analogous compared to a source story. On the Rattermann dataset, GPT-4 obtained the highest accuracy of 70%. As a whole, LLMs seem to struggle with over-emphasizing similar story entities (characters and settings) and a lack of awareness of higher-order relationship(s) between stories. LLMs struggled more with the Wharton dataset, with the highest accuracy achieved being 46.4% by GPT-4o, and all but nine LLMs performing below random chance accuracy. Although LLMs are improving, they still struggle with higher-cognitive tasks such as analogical reasoning.
Size, Weight, and Power (SWaP) concerns are growing as artificial intelligence (AI) use spreads in edge applications. AI algorithms, such as artificial neural networks (ANNs), have revolutionized many fields, e.g. computer vision (CV), but at a large computational/power burden. Biological intelligence is notably more computationally efficient. Neuromorphic edge processors and spiking neural networks (SNNs) aim to follow biology closer with spike-based operations resulting in sparsity and lower-SWaP operations than traditional ANNs with SNNs only "firing/spiking" when needed. Understanding the trade space of SWaP when embracing neuromorphic computing has not been studied heavily. To addresses this, we present a repeatable and scalable apples-to-apples comparison of traditional ANNs and SNNs for edge processing with demonstration on both classical and neuromorphic edge hardware. Results show that SNNs combined with neuromorphic hardware can provide comparable accuracy for CV to ANNs at 1/10 th the power.