16 October 2020 in IEEE Access
Contrastive Learning has recently received interest due to its success in self-supervised representation learning in the computer vision domain. However, the origins of Contrastive Learning date as far back as the 1990s and its development has spanned across many fields and domains including Metric Learning and natural language processing. In this paper, we provide a comprehensive literature review and we propose a general Contrastive Representation Learning framework that simplifies and unifies many different contrastive learning methods. We also provide a taxonomy for each of the components of contrastive learning in order to summarise it and distinguish it from other forms of machine learning. We then discuss the inductive biases which are present in any contrastive learning system and we analyse our framework under different views from various sub-fields of Machine Learning. Examples of how contrastive learning has been applied in computer vision, natural language processing, audio processing, and others, as well as in Reinforcement Learning are also presented. Finally, we discuss the challenges and some of the most promising future research directions ahead.
8 August 2019 in Cognitive Computation
24 November 2015 in Frontiers in Human Neuroscience
The Implicit Association Test (IAT) is a reaction time based categorization task that measures the differential associative strength between bipolar targets and evaluative attribute concepts as an approach to indexing implicit beliefs or biases. An open question exists as to what exactly the IAT measures, and here EEG (Electroencephalography) has been used to investigate the time course of ERPs (Event-related Potential) indices and implicated brain regions in the IAT. IAT-EEG research identifies a number of early (250-450 ms) negative ERPs indexing early-(pre-response) processing stages of the IAT. ERP activity in this time range is known to index processes related to cognitive control and semantic processing. A central focus of these efforts has been to use IAT-ERPs to delineate the implicit and explicit factors contributing to measured IAT effects. Increasing evidence indicates that cognitive control (and related top-down modulation of attention/perceptual processing) may be components in the effective measurement of IAT effects, as factors such as physical setting or task instruction can change an IAT measurement. In this study we further implicate the role of proactive cognitive control and top-down modulation of attention/perceptual processing in the IAT-EEG. We find statistically significant relationships between D-score (a reaction-time based measure of the IAT-effect) and early ERP-time windows, indicating where more rapid word categorisations driving the IAT effect are present, they are in at least partly explainable by neural activity not significantly correlated with the IAT measurement itself. Using LORETA, we identify a number of brain regions driving these ERP-IAT relationships notably involving left-temporal, insular, cingulate, medial frontal and parietal cortex in time regions corresponding to the N2- and P3-related activity. The identified brain regions involved with reduced reaction times on congruent blocks coincide with those of previous studies.
5 April 2022 in IEEE Access
In recent years, computational approaches for automatically extracting the voice of the customer from user generated content have been proposed. These studies have tackled the task of obtaining current customer needs, however, there is a lack of methods that predict future needs (i.e. needs that may become popular in the marketplace). Therefore, this study presents a multi-document keyphrase extraction algorithm which predicts future customer needs from users’ social media posts on Reddit. Key to our approach is a novel document filtering method (discovering potentially relevant social media content) and a keyphrase ranking method, which promotes terms with rising frequency likely to be future product needs. In order to evaluate the approach, a case study of “toothpaste” needs is reviewed and a novel evaluation approach using ground truth automatically extracted from a collection of future specifications of new-to-market products is proposed. In our evaluation, we show that the approach is significantly better than simple baselines at identifying customer needs on social media before they trend in the marketplace. We also show that our approach can capture important customer needs identified by a large multinational company with lead times of up to 25 months ahead of them trending in the marketplace. The findings of this research could provide many benefits to businesses such as gaining early access into markets ahead of their competitors and giving early notice to manufacturers/engineers/developers before a need for a product is in demand.