Stefan Heinrich

Stefan Heinrich

Auto-generated profile, based on publicly available information

Stefan Heinrich

Knowledge Technology Group, Department of Informatics, Universität Hamburg, Hamburg, Germany

Research Focus & Metrics

Academic Career Length

16Years (2011-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
2 September 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Open access
THTobias Hinz
Tobias Hinz
SHStefan Heinrich
Stefan Heinrich
SWStefan Wermter
Stefan Wermter
Generative adversarial networks conditioned on textual image descriptions are capable of generating realistic-looking images. However, current methods still struggle to generate images based on complex image captions from a heterogeneous domain. Furthermore, quantitatively evaluating these text-to-image models is challenging, as most evaluation metrics only judge image quality but not the conformity between the image and its caption. To address these challenges we introduce a new model that explicitly models individual objects within an image and a new evaluation metric called Semantic Object Accuracy (SOA) that specifically evaluates images given an image caption. The SOA uses a pre-trained object detector to evaluate if a generated image contains objects that are mentioned in the image caption, e.g., whether an image generated from “a car driving down the street” contains a car. We perform a user study comparing several text-to-image models and show that our SOA metric ranks the models the same way as humans, whereas other metrics such as the Inception Score do not. Our evaluation also shows that models which explicitly model objects outperform models which only model global image characteristics.

1 June 2020 in IEEE Transactions on Cognitive and Developmental Systems
Verified
Journal Article
Open access
RXRuobing Xie
Ruobing Xie
SHStefan Heinrich
Stefan Heinrich
ZLZhiyuan Liu
Zhiyuan Liu
CWCornelius Weber
Cornelius Weber
YYYuan Yao
Yuan Yao
MSMaosong Sun
Maosong Sun
A variety of brain areas is involved in language understanding and generation, accounting for the scope of language that can refer to many real-world matters. In this work, we investigate how regularities among real-world entities impact on emergent language representations. Specifically, we consider knowledge bases, which represent entities and their relations as structured triples, and image representations, which are obtained via deep convolutional networks. We combine these sources of information to learn representations of an Image-based Knowledge Representation Learning model (IKRL). An attention mechanism lets more informative images contribute more to the image-based representations. Evaluation results show that the model outperforms all baselines on the tasks of knowledge graph completion and triple classification. In analysing the learned models we found that the structure-based and image-based representations integrate different aspects of the entities and the attention mechanism provides robustness during learning.

2 January 2018 in Connection Science
Verified
Journal Article
Open access
SHStefan Heinrich
Stefan Heinrich
SWStefan Wermter
Stefan Wermter
For the complex human brain that enables us to communicate in natural language, we gathered good understandings of principles underlying language acquisition and processing, knowledge about sociocultural conditions, and insights into activity patterns in the brain. However, we were not yet able to understand the behavioural and mechanistic characteristics for natural language and how mechanisms in the brain allow to acquire and process language. In bridging the insights from behavioural psychology and neuroscience, the goal of this paper is to contribute a computational understanding of appropriate characteristics that favour language acquisition. Accordingly, we provide concepts and refinements in cognitive modelling regarding principles and mechanisms in the brain and propose a neurocognitively plausible model for embodied language acquisition from real-world interaction of a humanoid robot with its environment. In particular, the architecture consists of a continuous time recurrent neural network, where parts have different leakage characteristics and thus operate on multiple timescales for every modality and the association of the higher level nodes of all modalities into cell assemblies. The model is capable of learning language production grounded in both, temporal dynamic somatosensation and vision, and features hierarchical concept abstraction, concept decomposition, multi-modal integration, and self-organisation of latent representations.

14 October 2020 in Frontiers in Neurorobotics
Verified
Journal Article
Open access
PDF file
SHStefan Heinrich
Stefan Heinrich
YYYuan Yao
Yuan Yao
THTobias Hinz
Tobias Hinz
ZLZhiyuan Liu
Zhiyuan Liu
THThomas Hummel
Thomas Hummel
SWStefan Wermter
Stefan Wermter
Human infants are able to acquire natural language seemingly easily at an early age. Their language learning seems to occur simultaneously with learning other cognitive functions as well as with playful interactions with the environment and caregivers. From a neuroscientific perspective, natural language is embodied, grounded in most, if not all, sensory and sensorimotor modalities, and acquired by means of crossmodal integration. However, characterizing the underlying mechanisms in the brain is difficult and explaining the grounding of language in crossmodal perception and action remains challenging. In this paper, we present a neurocognitive model for language grounding which reflects bio-inspired mechanisms such as an implicit adaptation of timescales as well as end-to-end multimodal abstraction. It addresses developmental robotic interaction and extends its learning capabilities using larger-scale knowledge-based data. In our scenario, we utilize the humanoid robot NICO in obtaining the EMIL data collection, in which the cognitive robot interacts with objects in a children's playground environment while receiving linguistic labels from a caregiver. The model analysis shows that crossmodally integrated representations are sufficient for acquiring language merely from sensory input through interaction with objects in an environment. The representations self-organize hierarchically and embed temporal and spatial information through composition and decomposition. This model can also provide the basis for further crossmodal integration of perceptually grounded cognitive representations.

10 November 2017 in Behavioral and Brain Sciences
Verified
Journal Article
Closed access
PDF file
SWStefan Wermter
Stefan Wermter
SGSascha Griffiths
Sascha Griffiths
SHStefan Heinrich
Stefan Heinrich
Lake et al. point out that grounding learning in general principles of embodied perception and social cognition is the next step in advancing artificial intelligent machines. We suggest it is necessary to go further and consider lifelong learning, which includes developmental learning, focused on embodiment as applied in developmental robotics and neurorobotics, and crossmodal learning that facilitates integrating multiple senses.

1 May 2019 in Neurocomputing
Verified
Journal Article
Open access
SHStefan Heinrich
Stefan Heinrich
PSPeer Springstübe
Peer Springstübe
TKTobias Knöppler
Tobias Knöppler
MKMatthias Kerzel
Matthias Kerzel
SWStefan Wermter
Stefan Wermter