Foundations of Learning from Signing Avatars

ID: 3319
School: TBD
Program: TBD
Status: Completed
Start date: October 2017
End Date: December 2018


The cognitive underpinnings of action perception are well understood, and much progress has been made in how the brain allows for communication using a visual-only modality, as in the case of signed languages. One emerging challenge for the future of this field concerns the use of signing avatars. While interest in avatar-based communication is increasing, we do not know how signing avatars can best convey movement-based visual-spatial language to deaf populations. The proposed study would be the first step toward investigating action and language processing during the perception of signing avatars. In this study, we investigate what features of an avatar are most critical for clear perception of the signed language (e.g., fidelity of hands, fidelity of face). We will use motion capture recordings of deaf fluent signers to create novel motion-capture signing stimuli. We will then use these stimuli in a cognitive behavioral experiment in Gallaudet University's Action & Brain Lab to examine preliminary questions about what features of a signing avatar will be most important for perception. This behavioral experiment will pave the way for a set of cognitive neuroscience experiments that will examine activity in attentional, sensorimotor, and language-related brain networks during perception of signing avatars.

Principal investigators

  • Malzkuhn, Melissa • Science of Learning Center on Visual Language & Visual Learning (VL2)
  • Quandt, Lorna • Science of Learning Center on Visual Language & Visual Learning (VL2)

Additional investigators

Priorities addressed

Funding sources