College of Arts & Humanities

Permanent URI for this communityhttp://hdl.handle.net/1903/1611

The collections in this community comprise faculty research works, as well as graduate theses and dissertations.

Browse

Search Results

Now showing 1 - 9 of 9

The Learning and Usage of Second Language Speech Sounds: A Computational and Neural Approach
(2023) Thorburn, Craig Adam; Feldman, Naomi H; Linguistics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Language learners need to map a continuous, multidimensional acoustic signal to discrete abstract speech categories. The complexity of this mapping poses a difficult learning problem, particularly for second language learners who struggle to acquire the speech sounds of a non-native language, and almost never reach native-like ability. A common example used to illustrate this phenomenon is the distinction between /r/ and /l/ (Goto, 1971). While these sounds are distinct in English and native English speakers easily distinguish the two sounds, native Japanese speakers find this difficult, as the sounds are not contrastive in their language. Even with much explicit training, Japanese speakers do not seem to be able to reach native-like ability (Logan, Lively, & Pisoni, 1991; Lively, Logan & Pisoni, 1993) In this dissertation, I closely explore the mechanisms and computations that underlie effective second-language speech sound learning. I study a case of particularly effective learning--- a video game paradigm where non-native speech sounds have functional significance (Lim & Holt, 2011). I discuss the relationship with a Dual Systems Model of auditory category learning and extend this model, bringing it together with the idea of perceptual space learning from infant phonetic learning. In doing this, I describe why different category types are better learned in different experimental paradigms and when different neural circuits are engaged. I propose a novel split where different learning systems are able to update different stages of the acoustic-phonetic mapping from speech to abstract categories. To do this I formalize the video game paradigm computationally and implement a deep reinforcement learning network to map between environmental input and actions. In addition, I study how these categories could be used during online processing through an MEG study where second-language learners of English listen to continuous naturalistic speech. I show that despite the challenges of speech sound learning, second language listeners are able to predict upcoming material integrating different levels of contextual information and show similar responses to native English speakers. I discuss the implications of these findings and how the could be integrated with literature on the nature of speech representation in a second language.
Simply (neuro-)stimulating: The effects of transcutaneous vagus nerve stimulation on phonological and lexical tone learning indexed by behavior and pupillometry
(2021) Pandza, Nicholas Balint; Gor, Kira; Second Language Acquisition and Application; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Mandarin lexical tone learning has repeatedly been identified as a difficult linguistic feature for non-native speakers of tonal languages like English, even for native English learners of Mandarin at high proficiencies (e.g., Pelzl et al., 2019b). Sound perception training has been shown to help native English speakers perceive lexical tone differences, but acquiring lexical tone as a feature still remains difficult, even after as many as 18 training sessions (Bowles et al., 2016; Chandrasekaran et al., 2010; Li & DeKeyser, 2017; 2019; Liu & Chandrasekaran, 2013; Wang et al., 1999; 2003; Wong et al., 2011; Wong & Perrachione, 2007). While much of the tone learning literature has focused on different training interventions to overcome learning plateaus, another type of intervention that could augment learning is non-invasive neurostimulation. Transcutaneous auricular vagus nerve stimulation (taVNS) is a type of safe, non-invasive neurostimulation that delivers electrical current to the ear canal that has been shown to enhance cognition and learning (e.g., Jacobs et al., 2015). This dissertation investigated taVNS and its potential impact as tool to enhance Mandarin tone learning. Participants in three groups, peristim taVNS, priming taVNS, and a sham taVNS control participated in a double-blind two-day Mandarin phonological and lexical tone training study. Behavioral data including accuracy and reaction time were collected, as was physiological data in the form of pupillometry due to its ties both to cognitive effort and the most well-studied taVNS mechanism of action, the production of norepinephrine. Active taVNS groups received stimulation before or during multiple training and testing tasks across the two days. This body of work revealed: (1) priming and peristim administrations of taVNS differentially facilitated vocabulary learning of words with Mandarin tone, (2) priming and peristim administrations of taVNS differentially facilitated learning of new phonological tone categories, and (3) the effects of individual differences were substantially and differentially impacted by priming and peristim administrations taVNS, all results compared to a sham control. The evidence herein supports the potential of taVNS as a practical treatment intervention for enhancing language learning and reveals a number of considerations for its use and implementation to be explored in future research.
THE COMPARISON OF L1 AND L2 CASE PROCESSING: ERP EVIDENCE FROM TURKISH
(2019) KARATAS, NUR BASAK; Gor, Kira; Second Language Acquisition and Application; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
This dissertation investigates the morphological and morphosyntactic processing of case-marking by native and nonnative speakers of Turkish, through behavioral and electrophysiological responses. The study explores the locus of case processing costs during first (L1) and second language (L2) word recognition both in isolation and in sentences. It identifies the factors leading to persistent problems that late L2 learners face in attaining native-like processing of case assignment. To this end, the first experiment (a visual lexical decision task) examines whether different case forms generate differential processing costs, based on four main comparisons that reflect case properties and its status in the inflectional paradigm: 1) structural (genitive, accusative) vs. lexical (dative) case; 2) argument (accusative, dative) vs. non-argument (genitive); 3) higher (genitive) vs. lower type frequency (accusative, dative), and 4) citation form (nominative) vs. oblique cases (genitive, accusative, dative). The behavioral findings show significantly larger processing costs (i.e., longer reaction times and lower accuracy rates) for the genitive than the nominative case (citation form) across both subject groups, and than other oblique cases in L2 group only. ERP findings show significantly larger processing costs for the genitive than the accusative, and for the dative than the accusative only in L2 group. When the same case-inflected nouns were placed in a sentence context, larger N400 effects were found for the genitive, compared to the nominative and accusative in L1 group only. Together, these results suggest that different case forms generate differential processing costs in both subject groups, and L2 learners’ difficulty with the non-argument genitive and lexical dative oblique cases are at the level of form rather than sentence structure. The second (sentence) experiment also examined the processing of case errors (i.e., substitution of the accusative for the dative or vice versa on the object). ERP findings show a qualitative difference between L1 and L2 morphosyntactic patterns: P600 was missing while early negativities (N400 and left anterior negativity, LAN) were present in L2 group. These results suggest that advanced L2 learners evaluate the verb argument structure (LAN) and semantic fit (N400), but do not attempt to reparse the sentence (P600), unlike native speakers.
The Science of Artistic Expression: Intersections of Piano Pedagogy and Human Neuromechanics
(2019) Cunningham, Dion; Sloan, Rita; Shim, Jae; Music; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
This dissertation seeks to illuminate the ways in which the scientific field of neuromechanics can inform and improve piano performance and pedagogy resulting in heightened artistic expression. This work represents a beginning to a long-overdue conversation that needs to happen between various fields of study that intersect in the field of piano performance. Historically, despite over 300 years of treatises and methods, there has been little consensus on the best approaches to the pedagogy of piano performance. Additionally, there seems to be a demonstrable lack of understanding of the impact of physiology, biomechanics and proprioceptive awareness on the pianist as well as the ergonomic considerations of the piano itself. Students having technical or artistic shortcomings are often labeled as untalented without consideration for the efficacy of the pedagogical approaches to which they were exposed. Aspiring and famous pianists continue to accrue injuries. Furthermore, there is the noticeable lack of technological advancement of the instrument itself. In order to point to ways in which these concerns might be addressed in the future, this dissertation explored several significant areas: 1) historical pedagogy that focused on the development of piano technique, which showed not only the lack of consensus but the general lack of scientific authentication within the approaches of respected pedagogues; 2) scientific texts which focused on motor control in instrumental performance and other tasks, revealing the richness of resources potentially available to piano pedagogy; 3) the impacts of piano technology on piano playing, an area shown to be far-reaching in both the advancement of piano technique and injury prevention. Another part of the dissertation included several studies into the current experiences of pianists in the field. One study conveyed a significant gap in knowledge but also a desire for better understanding of and more improvement in piano-playing. Another study, a pilot, proposed a way to study the neuromechanical coordination of the fingers and wrists in piano-playing thereby showing a correlation between lateral wrist movement and individual finger strength, speed and control. Crucially, inquiry into linkages between proprioception and piano-playing showed a missing link which could be explored in the future.
Relating lexical and syntactic processes in language: Bridging research in humans and machines
(2018) Ettinger, Allyson; Phillips, Colin; Resnik, Philip; Linguistics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Potential to bridge research on language in humans and machines is substantial - as linguists and cognitive scientists apply scientific theory and methods to understand how language is processed and represented by humans, computer scientists apply computational methods to determine how to process and represent language in machines. The present work integrates approaches from each of these domains in order to tackle an issue of relevance for both: the nature of the relationship between low-level lexical processes and syntactically-driven interpretation processes. In the first part of the dissertation, this distinction between lexical and syntactic processes focuses on understanding asyntactic lexical effects in online sentence comprehension in humans, and the relationship of those effects to syntactically-driven interpretation processes. I draw on computational methods for simulating these lexical effects and their relationship to interpretation processes. In the latter part of the dissertation, the lexical/syntactic distinction is focused on the application of semantic composition to complex lexical content, for derivation of sentence meaning. For this work I draw on methodology from cognitive neuroscience and linguistics to analyze the capacity of natural language processing systems to do vector-based sentence composition, in order to improve the capacities of models to compose and represent sentence meaning.
The Temporal Dimension of Linguistic Prediction
(2013) Chow, Wing Yee; Phillips, Colin; Linguistics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
This thesis explores how predictions about upcoming language inputs are computed during real-time language comprehension. Previous research has demonstrated humans' ability to use rich contextual information to compute linguistic prediction during real-time language comprehension, and it has been widely assumed that contextual information can impact linguistic prediction as soon as it arises in the input. This thesis questions this key assumption and explores how linguistic predictions develop in real-time. I provide event-related potential (ERP) and reading eye-movement (EM) evidence from studies in Mandarin Chinese and English that even prominent and unambiguous information about preverbal arguments' structural roles cannot immediately impact comprehenders' verb prediction. I demonstrate that the N400, an ERP response that is modulated by a word's predictability, becomes sensitive to argument role-reversals only when the time interval for prediction is widened. Further, I provide initial evidence that different sources of contextual information, namely, information about preverbal arguments' lexical identity vs. their structural roles, may impact linguistic prediction on different time scales. I put forth a research framework that aims to characterize the mental computations underlying linguistic prediction along a temporal dimension.
Windows into Sensory Integration and Rates in Language Processing: Insights from Signed and Spoken Languages
(2011) Hwang, So-One K.; Idsardi, William J.; Linguistics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
This dissertation explores the hypothesis that language processing proceeds in "windows" that correspond to representational units, where sensory signals are integrated according to time-scales that correspond to the rate of the input. To investigate universal mechanisms, a comparison of signed and spoken languages is necessary. Underlying the seemingly effortless process of language comprehension is the perceiver's knowledge about the rate at which linguistic form and meaning unfold in time and the ability to adapt to variations in the input. The vast body of work in this area has focused on speech perception, where the goal is to determine how linguistic information is recovered from acoustic signals. Testing some of these theories in the visual processing of American Sign Language (ASL) provides a unique opportunity to better understand how sign languages are processed and which aspects of speech perception models are in fact about language perception across modalities. The first part of the dissertation presents three psychophysical experiments investigating temporal integration windows in sign language perception by testing the intelligibility of locally time-reversed sentences. The findings demonstrate the contribution of modality for the time-scales of these windows, where signing is successively integrated over longer durations (~ 250-300 ms) than in speech (~ 50-60 ms), while also pointing to modality-independent mechanisms, where integration occurs in durations that correspond to the size of linguistic units. The second part of the dissertation focuses on production rates in sentences taken from natural conversations of English, Korean, and ASL. Data from word, sign, morpheme, and syllable rates suggest that while the rate of words and signs can vary from language to language, the relationship between the rate of syllables and morphemes is relatively consistent among these typologically diverse languages. The results from rates in ASL also complement the findings in perception experiments by confirming that time-scales at which phonological units fluctuate in production match the temporal integration windows in perception. These results are consistent with the hypothesis that there are modality-independent time pressures for language processing, and discussions provide a synthesis of converging findings from other domains of research and propose ideas for future investigations.
Multi-Level Audio-Visual Interactions in Speech and Language Perception
(2011) Rhone, Ariane E.; Idsardi, William J; Linguistics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
That we perceive our environment as a unified scene rather than individual streams of auditory, visual, and other sensory information has recently provided motivation to move past the long-held tradition of studying these systems separately. Although they are each unique in their transduction organs, neural pathways, and cortical primary areas, the senses are ultimately merged in a meaningful way which allows us to navigate the multisensory world. Investigating how the senses are merged has become an increasingly wide field of research in recent decades, with the introduction and increased availability of neuroimaging techniques. Areas of study range from multisensory object perception to cross-modal attention, multisensory interactions, and integration. This thesis focuses on audio-visual speech perception, with special focus on facilitatory effects of visual information on auditory processing. When visual information is concordant with auditory information, it provides an advantage that is measurable in behavioral response times and evoked auditory fields (Chapter 3) and in increased entrainment to multisensory periodic stimuli reflected by steady-state responses (Chapter 4). When the audio-visual information is incongruent, the combination can often, but not always, combine to form a third, non-physically present percept (known as the McGurk effect). This effect is investigated (Chapter 5) using real word stimuli. McGurk percepts were not robustly elicited for a majority of stimulus types, but patterns of responses suggest that the physical and lexical properties of the auditory and visual stimulus may affect the likelihood of obtaining the illusion. Together, these experiments add to the growing body of knowledge that suggests that audio-visual interactions occur at multiple stages of processing.
Self-Monitoring and Feedback in Disordered Speech Production
(2011) Riley-Graham, Joshua; Idsardi, William; Linguistics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
The precise contribution and mechanism of sensory feedback (particularly auditory feedback) in successful speech production is unclear. Some models of speech production, such as DIVA, assert that speech production is based on attempting to produce auditory (and/or somatosensory targets; e.g. Guenther et al. 2006), and thus assign a central role to sensory feedback for successful speech motor control. These models make explicit predictions about the neural basis of speech production and the integration of auditory and somatosensory feedback and predict predict basal ganglia involvement in speech motor control. In order to test the implications of models depending on the integration of sensory feedback for speech, we present neuroimaging studies of two disorders of speech production in the absence of apraxia or dysarthria - one acquired (Foreign Accent Syndrome; FAS) and one developmental (Persistent Developmental Stuttering; PDS). Our results broadly confirm the predictions of the extended DIVA (Bohland et al. 2010) model, and emphasize the importance of the basal ganglia, especially the basal ganglia-thalamic-cortical (BGTC) loops. I argue that FAS should be thought of as a disorder of excessive speech sensory feedback, and that fluency in PDS depends on successful integration of speech sensory feedback with feedforward control commands.

College of Arts & Humanities

Browse

Filters

Settings

Sort By

Results per page

Search Results