Pictures, Pantomimes, and a Thousand Words: The Neuroscience of Cross-Modal Narrative Communication in Humans - Yuan, Ye (My University, 2017)
Communication is the exchange of thoughts and ideas from one person to another, often through the form of narratives. People communicate using speech, gesture, and drawing, or some multimodal combination of the three. Although there has been much research on how we understand and produce speech and pantomimes, there is relatively little on drawing, and even less on cross-modal communication. This dissertation presents novel empirical findings that contribute to a better understanding of the brain areas that mediate narrative communication across speech, pantomime, and drawing. Since the neuroscience of drawing was so understudied, I first used functional magnetic resonance imaging (fMRI) to investigate the existence of a basic drawing network in the human brain (Chapter 2). The drawing network was shown to contain three visual-motion areas that process the emanation of the visual image as drawing occurs. Next, to follow up on the poorly-characterized structural connectivity of these areas in the human dorsal visual stream, I used diffusion imaging to explore how these dorsal stream areas are connected (Chapter 3). The tractography results showed structural connectivity for two of the three predicted branches connecting the three visual-motion areas. Finally, I used fMRI to investigate how the basic drawing network is recruited during the more complex task of narrative drawing, and to find common brain areas among narrative speech, pantomime, and drawing (Chapter 4). Results suggest that people approached narratives in an intrinsically mentalistic fashion in terms of the protagonist, rather than as a mere description of action sequences. Together, these studies advance our understanding of the brain areas that comprise a basic drawing network, how these brain areas are interconnected, and how we communicate stories across three modalities of production. I conclude with a general discussion of my findings (Chapter 5).
Keywords: drawing; visual motion; cross-modal communication; fMRI; DTI; tractography