SIMULATED REAL-LIFE TASKS AS A TOOL TO INVESTIGATE THE EXTRAPOLATION INFERENCE OF LANGUAGE ASSESSMENTS FOR PROFESSIONAL PURPOSES

Abstract

In language test validity, the extrapolation inference refers to test scores being representative of the target linguistic performance in the target context of language use. This inference is investigated through criterion-related studies that compare test scores/performance with other external evaluation criteria such as actual samples from the target domain. However, obtaining these samples can be difficult in certain professional fields like the medical one. In these cases, simulated real-life tasks could be a suitable alternative. Following the argument-based approach to test validity (Chapelle et al., 2008), this study used a simulated real-life task specifically designed for this project to examine the extrapolation inference of the medical Spanish speaking section of the CanopyCredential exam—a language assessment for the medical domain. The simulated task represents a doctor-patient interaction, and was developed employing the theoretical frameworks of task-based language teaching (Long, 2015), task-based language assessment (Norris, 2016), and the language proficiency interview (Ross, 2017). The medical Spanish speaking section of the CanopyCredential exam measures examinees’ medical Spanish speaking ability to communicate with Spanish speaking patients in the U.S. medical workplace. Upon self-assessing their medical Spanish listening and speaking ability, participants (N = 82) completed the exam and the simulated task in a counterbalanced order, within two weeks from each other. Their performance on each instrument was rated using linguistic-criteria rubrics. The many-facet Rasch model, ordinal regression analyses, and Spearman’s correlation were used to examine the appropriateness of the simulated task as external evaluation criterion, and the extrapolation inference of the medical Spanish speaking section of the exam. The results indicated that the simulated task was a suitable external evaluation criterion. Furthermore, significant associations were found between participants’ scores in the exam and (1) their scores in the simulated task, (2) the linguistic characteristics of their production in the simulated task, and (3) their self-assessment of medical Spanish listening and speaking ability. This suggests that simulated tasks can be an appropriate instrument to examine the extrapolation inference of language assessments for professional purposes, and provide support for the extrapolation inference of the medical Spanish speaking section of the CanopyCredential exam.