Foundations of Learning from Signing Avatars
The cognitive underpinnings of action perception are well understood, and much progress has been made in how the brain allows for communication using a visual-only modality, as in the case of signed languages. One emerging challenge for the future of this field concerns the use of signing avatars. While interest in avatar-based communication is increasing, we do not know how signing avatars can best convey movement-based visual-spatial language to deaf populations. The proposed study would be the first step toward investigating action and language processing during the perception of signing avatars. In this study, we investigate what features of an avatar are most critical for clear perception of the signed language (e.g., fidelity of hands, fidelity of face). We will use motion capture recordings of deaf fluent signers to create novel motion-capture signing stimuli. We will then use these stimuli in a cognitive behavioral experiment in Gallaudet University's Action & Brain Lab to examine preliminary questions about what features of a signing avatar will be most important for perception. This behavioral experiment will pave the way for a set of cognitive neuroscience experiments that will examine activity in attentional, sensorimotor, and language-related brain networks during perception of signing avatars.
- Kubicek, Emily (Student) • PhD in Educational Neuroscience (PEN) - Educational Neuroscience (PEN)
- Malzkuhn, Melissa • Science of Learning Center on Visual Language & Visual Learning (VL2)