There is plenty of evidence that speech and gesture form a tightly integrated system, as reflected in parallelisms in language production, comprehension, and development (McNeill, 1992; Kendon, 2004). Yet, it is a common assumption that speakers use gestures to compensate for their expressive difficulties, a notion found in developmental studies of both first and second language acquisition, and in theoretical proposals concerning the gesture-speech relationship. If gestures are compensatory, they should mainly occur in disfluent stretches of speech. However, the evidence is sparse and conflicting. This study extends previous studies and tests the putative compensatory role of gestures by comparing the gestural behavior in fluent vs. disfluent stretches of narratives by competent speakers in two languages (Dutch and Italian), and by language learners (children and adult L2 learners). The results reveal that (1) in all groups speakers overwhelmingly produce gestures during fluent speech and only rarely during disfluencies. However, L2 learners are significantly more likely to gesture in disfluency than the other groups; (2) in all groups gestures during disfluencies tend to be holds; (3) in all groups the rare gestures completed in disfluencies have both referential and pragmatic functions. Overall, the data strongly suggest that when speech stops, so does gesture. The findings constitute an important challenge to both gesture and language acquisition theories assuming a mainly (lexical) compensatory role for (referential) gestures. Instead, the results provide strong support for the notion that speech and gestures form an integrated system.

Across cultures and languages spontaneous speech is often accompanied by gestures. It is a popular belief that people in Italy gesture more than people in Northern Europe, such as in Sweden. Despite this general assumption few studies empirically investigate cultural differences in gesture frequency and gesture function under similar circumstances. This study compares the spoken and gestural behaviours of Italian and Swedish speakers, assumed to represent gesture-rich vs. gesture-sparse cultures. We examine the groups' gestural behaviour for frequency, and in terms of possible differences in rhetorical style probing the distribution of gestural functions (referential vs. pragmatic) across narrative levels (narrative, metanarrative, and paranarrative). The results show that (1) Italians overall do gesture more than Swedes; (2) Italians produce more pragmatic gestures than Swedes who produce more referential gestures; (3) both groups show sensitivity to narrative level: referential gestures mainly occur with narrative clauses, and pragmatic gestures with meta- and paranarrative clauses. However, the overall group preferences for different functions still lead to different styles. These findings indicate that the two groups differ in gesture rate and, more interestingly, in rhetorical styles, one focused on events and actions in speech and gesture (Swedish), the other alternating between events in speech and gesture, and the highlighting of the presentation of new pieces of information in gesture only (Italian). We propose that the findings suggest that the two groups conceptualise narrative production in different ways reflected in two different rhetorical styles revealed by gesture production more than by speech.

Human communication can be either monosemiotic or polysemiotic, depending on whether it combines ensembles of representations from one or more semiotic systems such as language, gesture and depiction. Each semiotic system has its unique storytelling potentials, which makes intersemiotic translation from one system to another challenging. We investigated the influence of the source semiotic system, realised in speech and a sequence of pictures, respectively, on the way the same story was retold using speech and co-speech gestures. The story was the content of the picture book Frog, Where Are You?. A group of Finnish speakers saw the story in pictures, and another group heard it in matched oral narration. Each participant retold the story to an addressee and all narrations were video-recorded and analysed for both speech and gestures. Given the high degree of iconicity in depiction, we expected more iconic gestures (especially enactments) in the narratives translated from pictures than in those translated from speech. Conversely, we expected greater narrative coherence in the narratives translated from speech. The results showed that more iconic gestures were produced in the narratives translated from speech, but these were primarily not from the enactment subtype. As expected, iconic enactments were more frequent in the narratives translated from the story presented in pictures. The narratives produced by participants who had only heard the story did not have a greater variety of connective devices, yet the type of devices differed slightly between the groups. Together with some additional differences between the groups that had not been anticipated, the results indicate that a story presented in different semiotic systems tends to be translated into different polysemiotic narratives. Человеческая коммуникация может быть моносемиотической или полисемиотической, в зависимости от того, комбинируются ли репрезентаций из одной или более семиотических систем, таких как язык, жест и изображение. Каждая знаковая система имеет свои уникальные возможности рассказывания, что делает интерсемиотический перевод из одной системы в другую проблематичным. Мы исследовали влияние исходной семиотической системы, реализованной в речи и последовательности изображений на то, как та же самая история была пересказана с помощью речи и сопровождающими речь жестами. Рассказом служила история в картинках «Лягушка, где ты?» Одной группе показали историю в картинках, другая услышала ее в совпадающем устном рассказе. Каждый участник пересказывал свою историю и все рассказы записывались на видео и анализировались как с точки зрения речи, так и с точки зрения жестов. Учитывая высокую степень иконичности изображения, мы ожидали больше увидеть и иконических жестов в повествованиях, пересказанных с картинок, чем с устной речи. И наоборот, мы ожидали большей согласованности в нарративах, переведенных из речи. Результаты показали, что больше иконических жестов производилось в нарративах в переводе с речи, и, как и ожидалось, в повествованиях, переведенных из рассказа в картинках. Нарративы, произведенные участниками, которые только слышали историю, не имели большего разнообразия соеди- нительных устройств, но тип устройств немного различался между группами. Результаты показывают, что история, представленная в различных семиотических системах, как правило, переводится в различные полисемио тические нарративы.