Narratives are an integral part of human expression. In the graphic form, they range from cave paintings to Egyptian hieroglyphics, from the Bayeux Tapestry to modern day comic books (Kunzle, 1973; McCloud, 1993). Yet not much research has addressed the structure and comprehension of narrative images, for example, how do people create meaning out of sequential images? This piece helps fill the gap by presenting a theory of Narrative Grammar. We describe the basic narrative categories and their relationship to a canonical narrative arc, followed by a discussion of complex structures that extend beyond the canonical schema. This demands that the canonical arc be reconsidered as a generative schema whereby any narrative category can be expanded into a node in a tree structure. Narrative “pacing” is interpreted as a reflection of various patterns of this embedding: conjunction, left‐branching trees, center‐embedded constituencies, and others. Following this, diagnostic methods are proposed for testing narrative categories and constituency. Finally, we outline the applicability of this theory beyond sequential images, such as to film and verbal discourse, and compare this theory with previous approaches to narrative and discourse.
This study examined the electrophysiological correlates of complement coercion. ERPs were measured as participants read and made acceptability judgments about plausible coerced sentences, plausible noncoerced sentences, and highly implausible animacy-violated sentences (“The journalist began/wrote/astonished the article before his coffee break”). Relative to noncoerced complement nouns, the coerced nouns evoked an N400 effect. This effect was not modulated by the number of possible activities implied by the coerced nouns (e.g., began reading the article; began writing the article) and did not differ either in magnitude or scalp distribution from the N400 effect evoked by the animacy-violated complement nouns. We suggest that the N400 modulation to both coerced and animacy-violated complement nouns reflected different types of mismatches between the semantic restrictions of the verb and the semantic properties of the incoming complement noun. This is consistent with models holding that a verb's semantic argument structure is represented and stored at a distinct level from its syntactic argument structure. Unlike the coerced complement noun, the animacy-violated nouns also evoked a robust P600 effect, which may have been triggered by the judgments of the highly implausible (syntactically determined) meanings of the animacy-violated propositions. No additional ERP effects were seen in the coerced sentences until the sentence-final word that, relative to sentence-final words in the noncoerced sentences, evoked a sustained anteriorly distributed positivity. We suggest that this effect reflected delayed attempts to retrieve the specific event(s) implied by coerced complement nouns.
The past decade has seen a rapid growth of cognitive and brain research focused on visual narratives like comics and picture stories. This paper will summarize and integrate this emerging literature into theParallel Interfacing Narrative‐Semantics Model(PINS Model)—a theory of sequential image processing characterized by an interaction between two representational levels: semantics and narrative structure. Ongoing semantic processes build meaning into an evolving mental model of a visual discourse. Updating of spatial, referential, and event information then incurs costs when they are discontinuous with the growing context. In parallel, a narrative structure organizes semantic information into coherent sequences by assigning images to categorical roles, which are then embedded within a hierarchic constituent structure. Narrative constructional schemas allow for specific predictions of structural sequencing, independent of semantics. Together, these interacting levels of representation engage in an iterative process of retrieval of semantic and narrative information, prediction of upcoming information based on those assessments, and subsequent updating based on discontinuity. These core mechanisms are argued to be domain‐general—spanning across expressive systems—as suggested by similar electrophysiological brain responses (N400, P600, anterior negativities) generated in response to manipulation of sequential images, music, and language. Such similarities between visual narratives and other domains thus pose fundamental questions for the linguistic and cognitive sciences.
Both drawing and language are fundamental and unique to humans as a species. Just as language is a representational system that uses systematic sounds (or manual/bodily signs) to express concepts, drawing is a means of graphically expressing concepts. Yet, unlike language, we consider it normal for people not to learn to draw, and consider those who do to be exceptional. Why do we consider drawing to be so different from language? This paper argues that the structure and development of drawing is indeed analogous to that of language. Because drawings express concepts in the visual-graphic modality using patterned schemas stored in a graphic lexicon that combine using ‘syntactic’ rules, development thus requires acquiring a vocabulary of these schemas from the environment. Without sufficient practice and exposure to an external system, a basic system persists despite arguably impoverished developmental conditions. Such a drawing system is parallel to the resilient systems of language that appear when children are not exposed to a linguistic system within a critical developmental period. Overall, this approach draws equivalence between drawing and the cognitive attributes of other domains of human expression.