The Perception of Fingerspelling Using Point Light Displays

ID: 4117
School: Research Center/Lab
Program: Brain and Language Lab (BL2)
Status: Completed
Start date: January 2021
End Date: September 2021


While there is research on how sign language and fingerspelling is generally perceived among signers with different language experiences, perception of fingerspelling in varying visual environments is not well understood. Given that signed language users communicate in a wide array of sub-optimal environments, such as in the dark or from a distance, it is important to know more about how signers read fingerspelling in such circumstances. Signed languages such as American Sign Language (ASL) rely on visuospatial information that combines linguistic features such as hand and bodily movements, facial expressions, and fingerspelling. Linguistic information in ASL is conveyed with movement and spatial patterning, which leads to the possibility of studying the perception of both movement and language by using dynamic Point Light Display (PLD) stimuli to represent the joint movements of sign language. The stimuli used in this study consisted of fingerspelled location names. The location names were either real (e.g., KUWAIT) or made-up (e.g., CLARTAND), and the PLDs were made up of either high or low numbers of markers. We present results from a behavioral study in which deaf, hard-of-hearing, and hearing ASL users (total n = 292) watched the 28 PLD stimulus videos (half real, half made-up; half high # markers, half low # markers). After viewing each video, participants typed the fingerspelled word that was displayed in the video and self-rated on how confident they were about their answer. We predicted that when ASL fingerspelled letter strings are seen in a suboptimal visual environment, language experience in ASL will be positively correlated with accuracy and self-rated confidence scores. We also predicted that real location names would be more readily understood than made-up names. Our preliminary findings show that participants were more confident with their responses when the fingerspelled words were real than when they were fake (f(1,269) = 451.43, p < .001, η2 = 0.55). Participants were also more confident with typing their responses when the fingerspelled videos had the high number of markers than the lower number (F(1,269) = 101.364, p < .001, η2 = 0.019). We will also present analyses of response accuracy and effects of language experience. Studying the relationship between language experience with PLD fingerspelling perception will allow us to explore how hearing status, varying ASL fluency levels, age of language acquisition, and other language experiences affect the core abilities of understanding fingerspelling in varying visual environments.

Principal investigators

Additional investigators

Priorities addressed

Funding sources