Ph.D. in Educational Neuroscience (PEN)


Behavioral and neural responses to American Sign Language avatars

ID: 4035
School: Research Center/Lab
Program: Ph.D. in Educational Neuroscience (PEN)
Status: Ongoing
Start date: February 2020
End Date: October 2021

Description

This is the first neuroscience and Human-Computer Interaction (HCI) study to examine how the biological and synthesized motions of signing avatars will impact neural responses to ASL. Investigating which technology modality of ASL is able to provide users with embodied cognition engagement that comes with fluent biological motions in signing avatars will help us improve the design of signing avatars as embodied interface. The signing avatars can then be used to inform and develop technology design with ASL and novel systems. This will lead to improved ASL resources and HCI design for signing avatars while answering current neuroscience questions on embodied cognition and sign languages. This study will collect user rating and EEG signals as deaf fluent signers and hearing nonsigners imitate signs from human signer video, synthesized motion avatar, and biological motion avatar. Our hypotheses are that in both signing deaf and hearing non-signing groups, both video of human signer and signing avatar with biological motion will elicit significantly better HCI user rating than the signing avatar with synthesized motion. Secondly, there will be significant differences in brain activity for the synthesized motion signing avatar because the signing deaf group will be able to draw on their ASL experience.

Principal investigators

Priorities addressed

Funding sources


Biological Motion Perception in Deaf Native Signers

ID: 4085
School: Research Center/Lab
Program: Ph.D. in Educational Neuroscience (PEN)
Status: Completed
Start date: October 2018
End Date: October 2021

Description

We conducted two studies to test how deaf signed language users perceive biological motions. We created 18 Biological Motion point-light displays (PLDs) depicting everyday human actions, and 18 Scrambled control PLDs. First, we conducted an online behavioral rating survey, in which deaf and hearing raters identified the biological motion PLDs and rated how easy it was for them to identify the actions. Then, we conducted an EEG study in which Deaf Signers and Hearing Non-Signers watched both the Biological Motion PLDs and the Scrambled PLDs, and we computed the time-frequency responses within the theta, alpha, and beta EEG rhythms. From the behavioral rating task, we show that the deaf raters reported significantly less effort required for identifying the Biological motion PLDs, across all stimuli. The EEG results showed that the Deaf Signers showed theta, mu, and beta differentiation between Scrambled and Biological PLDs earlier and more consistently than Hearing Non-Signers. We conclude that native ASL users exhibit experience-dependent neuroplasticity in the domain of biological human motion perception.

Additional investigators

Priorities addressed

Funding sources

Approved Products

2021

Quandt, L. C. & Kubicek, E., Willis, A. S., & Lamberton, J. (2021). Enhanced biological motion perception in deaf native signers. Neuropsychologia, 161, 107996, 1-10.


Cognitive and affective predictors of language and socioemotional outcomes in deaf and hard-of-hearing individuals

ID: 4096
School: Research Center/Lab
Program: Ph.D. in Educational Neuroscience (PEN)
Status: Ongoing
Start date: January 2021
End Date: January 2027

Description

This research will explore how negative emotional experiences in language contexts may be associated with detrimental language outcomes for deaf, DeafBlind and hard of hearing (DDBHH) individuals. Language anxiety, defined here as negative emotional experiences associated with understanding, learning, or expressing language, may reduce language proficiency. The vast majority of deaf children are born to hearing parents. If hearing parents cannot access adequate language-learning resources for themselves and their DDBHH children, such as learning American Sign Language or additional language support, this puts DDBHH children at increased risk for negative experiences with language development, such as language delay or adverse childhood communication experiences (ACCE). This research will explore how language anxiety, secondary to ACCE, is related to a lower level of language proficiency, avoidance, increased negative emotion, and decreased performance and learning in a second language. Addressing language anxiety as a challenge for developing language fluency represents an understudied line of research and may revolutionize our understanding of the role of emotion in language development. This approach is innovative because it considers the important role that a bilingual immersion environment may play in reducing the impact of language anxiety on linguistic and socioemotional development. Establishing language anxiety as a factor predicting language outcomes, neural measures of language representation, and socioemotional outcomes represents a novel way to conceptualize the vital role of emotion in language development.

Principal investigators

Additional investigators

Priorities addressed

Funding sources


Cognitive, Social and Affective Components of Academic Outcomes

ID: 4095
School: Research Center/Lab
Program: Ph.D. in Educational Neuroscience (PEN)
Status: Ongoing
Start date: January 2021
End Date: September 2025

Description

Academic performance is influenced by many cognitive factors, including our own goals and abilities, but also by social and emotional factors, including our emotional experiences and interactions with others in academic environments. In this project, we aim to gain a better understanding of how emotions, social relationships, and cognitive performance are associated with academic outcomes. Specifically, our research will focus on multiple components of academic anxiety, including: math anxiety, the pattern of negative affect and math performance deficits associated with anxious avoidance of math, spatial anxiety, or anxiety associated with thinking about visualization and other spatial tasks, and language anxiety, or patterns of negative affect that may similarly have detrimental influences on language outcomes. This research will explore how negative emotional experiences in academic contexts may have detrimental outcomes for learning and development, including individuals who are hearing, deaf and hard of hearing. Further, this project will explore how implementing different interventions to interrupt or regulate various aspects of academic anxiety is associated with biological, behavioral and educational outcomes. This project explores how math anxiety is reciprocally influenced by a variety of cognitive, emotional, and social factors, and how these components relate to self-reported academic outcomes and experiences.

Principal investigators

Additional investigators

Priorities addressed

Funding sources

Approved Products

2021

Burr, D. A., Pizzie, R. G. , & Kraemer, D. J. M. (2021). Do you know how you regulate your emotions? Comparing self-report versus multivariate psychophysiological measures of emotion regulation. PLOS ONE, 16(3): e0247246. https://doi.org/10.1371/journal.pone.0247246

Pizzie R. (2021) Mind, Brain, and Math Anxiety. In: Danesi M. (eds) Handbook of Cognitive Mathematics. Springer, Cham. https://doi.org/10.1007/978-3-030-44982-7_29-1

Pizzie, R. G., & Kraemer, D. J. M. (2021). The association between emotion regulation, physiological arousal, and performance in math anxiety. Frontiers in Psychology: Emotion Science, 12, 639448, https://doi.org/10.3389/fpsyg.2021.639448

Pizzie, R. G., McDermott, C. L., Salem, T. G., & Kraemer, D. J. M. (2020). Neural Evidence for Cognitive Reappraisal as a Strategy to Alleviate the Effects of Math Anxiety. Social Cognitive and Affective Neuroscience, 15(12), 1271–1287. https://doi.org/10.1093/scan/nsaa161


Do expert signers recruit signed phonology processes while solving single digit multiplication problems?

ID: 3984
School: Research Center/Lab
Program: Ph.D. in Educational Neuroscience (PEN)
Status: Ongoing
Start date: August 2020
End Date: December 2021

Description

The project aims at investigating the role of ASL phonology and the underling neural substrates in solving single digit multiplication problems. In spoken languages, phonology and the left lateralized language areas are recruited when verbally retrieving single digit multiplication problems. The role of ASL phonology in arithmetic in general and specifically in the retrieval of multiplication problems is unknown despite abundant literature addressing reading in ASL users. In this study, we will recruit deaf participants with profound to severe hearing loss who have been exposed to ASL prior to age 2 and have had substantial exposure throughout their educational upbringing. Participants will have no history of neurological or developmental disorders and no know learning disability. Results will inform on the typical network involved in multiplication problem for ASL signers and outline a model to then investigate the impact of late ASL sign language exposure or math learning disability in ASL signers.

Principal investigators

Priorities addressed

Funding sources

Approved Products

2021

Sullivan, S.J.

Sullivan, S.J. (2021) Do expert signers recruit signed phonology while solving single digit multiplication problems? [unpublished dissertation] Gallaudet University


Impact of Language Experience on Early Numerical Cognition

ID: 3742
School: Research Center/Lab
Program: Ph.D. in Educational Neuroscience (PEN)
Status: Ongoing
Start date: July 2019
End Date: December 2021

Description

The objective of the study is to evaluate longitudinally the impact of language modality and early language experience on the core numerical representation and on the acquisition of the concept of exact number. To do this, 180 children aged 3 to 5 will be followed for up to two years. Leveraging the natural variability occurring within the deaf community, 60 children will be native American Sign Language (ASL) users, 60 children will have been exposed to a visual language after 24 months of age (e.g., deaf children with late cochlear implant and no in-home visual language), and the remaining will be English speaking children with no hearing loss and no delay in language exposure. Children will be evaluated at ~8 months intervals, between 2 to 4 times, on basic number skills until they reach proficient understanding of the exact number concept. They will also be assessed for language skills and general IQ. Parents will fill out a comprehensive survey on their child's language use and in-home language. This paradigm will allow to determine the impact of language modality and proficiency on the developmental trajectory of the core numerical representation. It will also allow to determine if the stages for reaching a full understanding of the exact number concept can be delayed or facilitated depending on language modality. Could the use of fingers in ASL to represent numbers facilitate early number concept acquisition? Does a delay in language exposure impact both the core number system and the acquisition of formal number concepts? Are the different stages impermeable to early language experience? What role does language play in the relation between the core numerical representation and the acquisition of exact number concept? These long-standing questions in the field of numerical cognition can be uniquely answered through the perspective of a visual language and time of language exposure.

Principal investigators

Additional investigators

Priorities addressed

Funding sources


Neural investigation on the impact of a visual language on arithmetic processing: an fMRI approach

ID: 3744
School: Research Center/Lab
Program: Ph.D. in Educational Neuroscience (PEN)
Status: Ongoing
Start date: July 2019
End Date: October 2021

Description

One question of relevance is whether the neural networks for solving single-digit arithmetic problems is modality-dependent or -independent. It could be that the different experiences of learning in distinct modalities impacts this processing. Calculation-based strategies, often used for addition and subtraction, call upon the classic number processing areas in the parietal cortex. Conversely, multiplication utilizes the classic left-lateralized language areas, using a verbal retrieval strategy in recalling simple, rote multiplication problems. When considering the modality debate of arithmetic fact retrieval, it calls into question the reliance of multiplication fact retrieval on the language networks. Might using a visual language impact the neural networks for calculating multiplication problems? This has yet to be investigated within the Deaf signing population, rendering the extent of the impact of learning experience on the different operations still unclear. Examining the neural networks in participants native in languages that differ in their modality could be a way to investigate how deeply the learning experience impacts the networks for simple arithmetic. In learning arithmetic, a child using a visual language might rely more on visuo-spatial processes but also rely more on internal representations of manual number signs. Evidence suggests that the brain of adult fluent signers automatically activates areas related to sensorimotor representations when processing linguistic information. Fluent signers have also shown to have increased proficiency in mental representation and mental rotation, indicating a heightened visuo-spatial capability over their hearing non-signing peers. As a result, we expect to find more visuo-spatial and fine motor activations in adult native signers compared to non- signers when solving single-digit arithmetic problems. Additionally, if operations are indeed intrinsically different, we should find distinct neural networks for multiplication and subtraction problems evident in native signers. Because native signers process linguistic information in the same left-lateralized language areas, we can expect to see that multiplication facts are also stored in the language network. The current body of research on arithmetic processing does not take into account a visual modality. Gaining a better understanding of the neural networks involved, and to what extent they are involved, in deaf native signers would deepen our understanding of this learning mechanism, allowing for more nuanced research beyond these foundational findings. If we find that native signers’ learning displays a unique network for computing arithmetic, remediation of the current system for educating native signers may be beneficial to the deaf learner. As deaf students historically lag behind their hearing peers in academic achievement, improving access and quality of education is paramount to ensuring optimal learning and development. Disseminating these findings to policymakers and educators may encourage a closer look at how we can better serve our deaf students. The project’s aim is to compare native adult ASL signers to native adult English speakers using functional Magnetic Resonance Imaging (fMRI) as a lens into the neural networks involved in small and large single-digit arithmetic problems.

Principal investigators

Additional investigators

Priorities addressed

Funding sources


Signing Avatars & Immersive Learning (SAIL)

ID: 3707
School: Research Center/Lab
Program: Ph.D. in Educational Neuroscience (PEN)
Status: Ongoing
Start date: August 2018

Description

The aim of the proposed work is to develop and test a system in which signing avatars (computer-animated virtual humans/characters built from motion capture recordings) help deaf or hearing individuals learn ASL in an immersive virtual environment. The system will be called Signing Avatars & Immersive Learning (SAIL). Interactive speaking avatars have become valuable learning tools, whereas the potential uses of signing avatars have not been adequately explored. Due to the spatial and movement characteristics of natural sign languages, this project leverages the cognitive neuroscience of action perception to test the SAIL system. We will use motion capture recordings of native deaf signers, signing in ASL, to create signing avatars. The avatars will be placed in a virtual reality landscape which can be accessed via head-mounted goggles. Users will enter the virtual reality environment by wearing the goggles, and the user's own movements will be captured via gesture-recognition system (e.g., smart gloves). When using SAIL, users will see a signing avatar from a third person perspective, and they will also see a virtual version of their own arms, from a first person perspective. This first-person perspective can be matched onto their actual movements in the real world. By using gesture recognition systems users will imitate signs and learn through interactive lessons given by avatars. SAIL helps users to visualize and embody a spatial and visual language. This creates an embodied, immersive learning environment which may revolutionize ASL learning. SAIL will provide us the opportunity to understand the cognitive process of visual perception of ASL in a controlled 3d digital environment. Following the development of SAIL, we propose an electroencephalography (EEG) experiment to examine how the sensorimotor systems of the brain are engaged by the embodied experiences provided by SAIL. The action observation network of the human brain is active during the observation of others' movements. The extent of this activity during viewing of another person signing will provide insight into how the observer's own sensorimotor system processes the observed signs within SAIL.

Principal investigators

Additional investigators

Priorities addressed

Funding sources

Approved Products

2019

Quandt, L. C. (2019) Embodying sign language: Using avatars, VR, and EEG to design novel learning tools. Center for Adaptive Systems of Brain-Body Interactions Seminar Series, George Mason University, Fairfax, VA.

Quandt, L. C., Malzkuhn, M. (2019). Participant: NSF STEM for All Video Showcase. "SAIL: Signing Avatars & Immersive Learning" dissemination video.

Quandt, L.C. (2019). Signing avatars and embodied learning in virtual reality. NSF AccessCyberlearning 2.0 Capacity Building Institute, University of Washington, Seattle, WA.

2021

Schwenk, M., Willis, A. S., Weeks, K., Ferster, R., & Quandt, L. C. Attitudes towards sign language avatars in the practice of teletherapy and assessment. (2021). Presented at the 2021 convention of the American Psychological Association, Society of Clinical Psychology (Division 12).

Shao, Q., Sniffen, A., Blanchett, J., Hillis, M. E., Shi, X., Haris, T. K., Liu, J., Lamberton, J., Malzkuhn, M., Quandt, L. C., Mahoney, J., Kraemer, D. J. M., Zhou, X., & Balkcom, D. Teaching American Sign Language in mixed reality. Talk to be given at UbiComp 2021.


The impact of language experience on the neural activations of arithmetical processing (EEG)

ID: 3606
School: Research Center/Lab
Program: Ph.D. in Educational Neuroscience (PEN)
Status: Ongoing
Start date: March 2018
End Date: October 2021

Description

The aim is to investigate the differences and similarities in the neural correlates, through the EEG recordings, of native ASL users and English native speakers while performing single-digit arithmetic problems.

Principal investigators

Additional investigators

Priorities addressed

Funding sources

Approved Products

2021

Berteletti, I., Sullivan, S.J.*, Kimbley, S.*, Berger, L.*, Quandt, L., and Miyakoshi, M. Language modality does not change how our brain processes different mathematical operations. (psyarxiv – to be submitted).


The impact of language modality on number sense and arithmetic processing (adult study)

ID: 4032
School: Research Center/Lab
Program: Ph.D. in Educational Neuroscience (PEN)
Status: Ongoing
Start date: October 2019
End Date: December 2021

Description

In this research project we investigate how using a visual language might support and impact basic number and arithmetic processing abilities. We specifically investigate processing of non symbolic number quantities, symbolic Arabic digit comparison, symbolic ASL number comparison and single digit arithmetic problems in native, early exposed, American Sign Language users.

Principal investigators

Additional investigators

Priorities addressed

Funding sources

Approved Products

2021

Berteletti, I., Sullivan, S.J.*, & Lancaster, L.* (2021). The unexplored role of handshape similarity in processing numbers on the hands. Journal of Numerical Cognition, 7(2), 156-171.