Caregivers use a range of verbal and nonverbal behaviours when responding to their infants. Previous studies have typically focused on the role of the caregiver in providing verbal responses, while communication is inherently multimodal (involving audio and visual information) and bidirectional (exchange of information between infant and caregiver). In this paper, we present a comprehensive study of caregivers' verbal, nonverbal, and multimodal responses to 10-month-old infants' vocalisations and gestures during free play. A new coding scheme was used to annotate 2036 infant vocalisations and gestures of which 87.1 % received a caregiver response. Most caregiver responses were verbal, but 39.7 % of all responses were multimodal. We also examined whether different infant behaviours elicited different responses from caregivers. Infant bimodal (i.e., vocal-gestural combination) behaviours elicited high rates of verbal responses and high rates of multimodal responses, while infant gestures elicited high rates of nonverbal responses. We also found that the types of verbal and nonverbal responses differed as a function of infant behaviour. The results indicate that infants influence the rates and types of responses they receive from caregivers. When examining caregiver-child interactions, analysing caregivers' verbal responses alone undermines the multimodal richness and bidirectionality of early communication.

There is robust evidence that infants' gestures and vocalisations and caregivers' contingent responses predict later child vocabulary. Recent studies suggest that dyadic combinations of infants' behaviors and caregivers' responses are more robust predictors of children's vocabularies than these behaviors separately. Previous studies have not yet systematically compared different types of dyadic combinations. This study aimed to compare the predictive value of (a) frequencies of infants' behaviors (vocalisations, points, and shows + gives) regardless of caregivers' responses, (b) frequencies of infants' behaviors that elicited verbal responses, (c) frequencies of infants' behaviors that elicited multimodal responses, and (d) frequencies of infants' behaviors that did not elicit any responses from caregivers. We examined 114 caregiver‐infant dyads at 9–11 months and children's concurrent and longitudinal vocabulary outcomes at 2–4 years. We found that infants' points elicited a large proportion of verbal responses from caregivers which were related to children's later receptive vocabularies. We also found that only shows + gives that elicited caregivers' responses related to infants' concurrent gesture repertoires. In contrast, infants' behaviors that did not elicit responses negatively related to child vocabulary. The results highlight the importance of examining dyadic combinations of infants' behaviors and caregivers' responses during interactions when examining relations to children's vocabulary development.

Limited studies have examined demographic differences in children’s vocabulary in longitudinal samples, while there are questions regarding the duration, direction, and magnitude of these effects across development. In this longitudinal study, we included over 400 Dutch children. Caregivers filled out N-CDIs when children were 9–11 months (measuring word comprehension, word production, and gestures) and around 2–5 years of age (measuring word production). At 2–5 years, we also administered a receptive vocabulary task in the lab. We examined demographic effects on vocabulary size across infancy and toddlerhood. We found a disadvantage for males in infants’ gestures and toddlers’ vocabulary production. We found a negative effect of maternal education on infants’ caregiver-reported vocabulary, but a positive effect on toddlers’ lab-administered receptive vocabulary. Lastly, we found a negative effect of multilingualism – but only for the lab-administered task. Examining predictors in large, longitudinal samples ensures their robustness and generalisability across development.

This study assesses the performance of a state-of-the-art automatic speech recognition (ASR) system at extracting target words in two different speech registers: infant-directed speech (IDS) and adult-directed speech (ADS). We used the Kaldi-NL ASR-service, developed by the Dutch Foundation of Open Speech Technology. The results indicate that the accuracy of the tool is much lower in IDS than in ADS. There are differences between IDS and ADS which negatively affect the performance of the existing ASR system. Therefore, new tools need to be developed for the automatic annotation of IDS. Nevertheless, the ASR system can already find more than half of the target words, which is promising.

There is a large interest in the annotation of speech addressed to infants. Infant-directed speech (IDS) has acoustic properties that might pose a challenge to automatic speech recognition (ASR) tools developed for adult-directed speech (ADS). While ASR tools could potentially speed up the annotation process, their effectiveness on this speech register is currently unknown. In this study, we assessed to what extent open-source ASR tools can successfully transcribe IDS. We used speech data from 21 Dutch mothers reading picture books containing target words to their 18- and 24-month-old children (IDS) and the experimenter (ADS). In Experiment 1, we examined how the ASR tool Kaldi-NL performs at annotating target words in IDS vs. ADS. We found that Kaldi-NL only found 55.8% of target words in IDS, while it annotated 66.8% correctly in ADS. In Experiment 2, we aimed to assess the difficulties in annotating IDS more broadly by transcribing all IDS utterances manually and comparing the word error rates (WERs) of two different ASR systems: Kaldi-NL and WhisperX. We found that WhisperX performs significantly better than Kaldi-NL. While there is much room for improvement, the results show that automatic transcriptions provide a promising starting point for researchers who have to transcribe a large amount of speech directed at infants.

Child behaviour is a topic of great scientific interest across a wide range of disciplines, including social sciences and artificial intelligence (AI). Knowledge in these different fields is not yet integrated to its full potential. The aim of this workshop was to bring researchers from these fields together. The first two workshops had a significant impact. In this workshop, we discussed topics such as the use of AI techniques to better examine and model interactions and children’s emotional development, analyzing head movement patterns with respect to child age. This workshop was a successful new step towards the objective of bridging social sciences and AI, attracting contributions from various academic fields on child behaviour analysis. This document summarizes the accepted papers.