Recent research in psychology and neuroscience has demonstrated that co-speech gestures are semantically integrated with speech during language comprehension and development. The present study explored whether gestures also play a role in language learning in adults. In Experiment 1, we exposed adults to a brief training session presenting novel Japanese verbs with and without hand gestures. Three sets of memory tests (at five minutes, two days and one week) showed that the greatest word learning occurred when gestures conveyed redundant imagistic information to speech. Experiment 2 was a preliminary investigation into possible neural correlates for such learning. We exposed participants to similar training sessions over three days and then measured event-related potentials (ERPs) to words learned with and without co-speech gestures. The main finding was that words learned with gesture produced a larger Late Positive Complex (indexing recollection) in bi-lateral parietal sites than words learned without gesture. However, there was no significant difference between the two conditions for the N400 component (indexing familiarity). The results have implications for pedagogical practices in foreign language instruction and theories of gesture-speech integration.
It is now widely accepted that hand gestures help people understand and learn language. Here, we provide an exception to this general rule—when phonetic demands are high, gesture actually hurts. Native English-speaking adults were instructed on the meaning of novel Japanese word pairs that were for non-native speakers phonetically hard (/ite/ vs. /itte/, which differ by only a geminate) or easy (/tate/ vs. /butta/, which differ by a geminate and also their segmental composition). The words were presented either with or without congruent iconic gestures, for example, “Ite means stay” (with a STAY gesture). After instruction, participants were given phonetic and vocabulary tests for the words they had learned. Although performance for the phonetic task was above chance for all conditions, gesture played different roles in the semantic task for easy and hard word pairs—it helped word learning for easy pairs, but it hurt for hard pairs. These results suggest that gesture and speech are semantically integrated during word learning, but only when phonetic demands are not too high.
This article investigates the role that nonverbal actions play in language processing over 3 different time frames. First, we speculate that nonverbal actions played a role in how formal language systems emerged from our primate ancestors over evolutionary time. Next, we hypothesize that if nonverbal behaviors played a foundational role in the emergence of language over evolution, these actions should influence how children learn language in the present. Finally, we argue that nonverbal actions continue to play a role for adults in the moment-to-moment processing of language. Throughout, we take an embodied view of language and argue that the neural, cognitive, and social components of language processing are firmly grounded in bodily action.