In recent years there has been a substantial debate about the need for increasingly spontaneous, conversational corpora of spoken interaction that are not controlled or task directed. In parallel the need has arisen for the recording of multi-modal corpora which are not restricted to the audio domain alone. With a corpus that would fulfill both needs, it would be possible to investigate the natural coupling, not only in turn-taking and voice, but also in the movement of participants. In the following paper we describe the design and recording of such a corpus and we provide some illustrative examples of how such a corpus might be exploited in the study of dynamic interaction. The D64 corpus is a multimodal corpus recorded over two successive days. Each day resulted in approximately 4 h of recordings. In total five participants took part in the recordings of whom two participants were female and three were male. Seven video cameras were used of which at least one was trained on each participant. The Optitrack motion capture kit was used in order to enrich information. The D64 corpus comprises annotations on conversational involvement, speech activity and pauses as well as information of the average degree of change in the movement of participants.

Social signals are integral to conversational interaction and constitute a large part of the social dynamics of multiparty communication. Moreover, social signals may also have a function in discourse structure. We focus on laughter, exploring the extent to which laughter can be shown to signal the structural unfolding of conversation and whether laughter may be used in the signaling of topic changes. Recent research supports this hypothesis. We investigate the relation between laughter and topic changes from two different points of view (temporal distribution and content distribution) as visible in the TableTalk corpus and also in the AMI corpus. Consistent results emerge from studies of these two corpora. Laughter is less likely very soon after a topic change than it is before a topic change. In both studies, we find solo laughter significantly more frequent in times of topic transition than in times of topic continuity. This contradicts previous research about the social dynamics of shared versus solo laughter considering solo laughs as signals of topic continuation. We conclude that laughter has quantifiable discourse functionality concomitant with social signaling capacity.