TOWARD A THEORY-BASED ACCOUNT OF THE L2 VOCABULARY PROCESSING AND LEARNING BENEFITS OF READING WHILE LISTENING

Abstract

The tantalizing prospects of learning benefits from multimodal conditions on second language (L2) learning in general, and L2 vocabulary development in particular, have important implications. Indeed, opening a language learning app on any device provides the immediate experience of simultaneous input modalities, and a wide range of input types. But how helpful is multimodality to vocabulary learning, especially when the focus of the learner is on the meaning of a text? Researchers have manipulated input to compare a variety of learning conditions and examined vocabulary learning gains. However, relatively few within second language acquisition (SLA) have utilized real-time monitoring of learner behavior to examine how learners encounter new words over multiple exposures during a reading task, and how the quality of these encounters may or may not influence explicit learning outcomes. Even fewer have mapped differences in the developmental trajectory of form-form and form-meaning mapping for new words at the group level, comparing reading only (RO) with reading while listening (RWL). Crucially, to my knowledge, none have made or tested predictions within RWL on possible psycholinguistic source(s) of reported benefits. Our understanding of outcome benefits, along with implications for optimizing input in classroom or individual instructed contexts, is thereby quite limited. My dissertation study was designed to address each of these issues. 119 advanced English learners read or read while listening to a 7,400-word short story under incidental conditions (time pressure, focus on comprehension, and unannounced posttest outcomes). The text was embedded with 25 target pseudoword items 10 times each, with target items replacing real nouns in object positions. Measures of real-time form learning were defined as faster reading times and fewer total visits to the new words across encounters (Godfroid, 2020b), and there were three post-exposure measures of explicit word knowledge (form recognition, meaning recognition, meaning recall). New to this area of vocabulary research, outcome items were presented in randomized item modality (visual or auditory), to ensure congruence between treatment and test items and reducing modality-specific testing bias (Jelani & Boers, 2018). Group-level comparisons examined differences in (1) developmental trajectory of form familiarity and meaning integration for RO and RWL groups, (2) learning outcomes, and (3) effects of multi-componential L2 proficiency and phonological short-term memory (PSTM) skills on processing and learning outcomes. Within-RWL analyses operationalized a theoretical source of benefit (reading slightly ahead of the audio) and its impact on reading time and posttest learning gains. Findings indicated differences between RO and RWL across three measures of eye movements: (1) gaze duration (GD), a measure of form familiarity with new words; (2) total reading time (TRT), a measure of meaning integration; and (3) visit count, or the total number of encounters looking at the words. The overall pattern for RWL indicated longer initial reading times for new words, fewer re-readings, and steadier decrease in GD and TRT across encounters. Additionally, differences in learning outcomes were most clearly revealed through auditory test items, with RWL superior to RO across all three posttest outcome measures, and a group by item modality interaction. In other words, RWL indicated superior overall effects compared with RO across all items in form recognition and meaning recall, across all three posttests in auditory items, and better scores on visual than auditory items in RO (but equal across test item modality in RWL). Within-RWL analyses revealed that reading ahead of the audio was a positive predictor of TRT, as well as the most difficult of the three outcome measures (meaning recall). While PSTM predicted processing of new words, it did not predict outcomes for any of the three measures of vocabulary learning gains for advanced-level L2 readers. In sum, this study provides convergent evidence that process (form-form / form-meaning acquisition) and product (learning gains) are both positively impacted for new words under multimodal incidental conditions for advanced L2 learners, along with an initial indication that audiovisual asynchrony may play a role in RWL benefits in learning new words above and beyond L2 proficiency or memory skills.