METHODS OF INTEGRATING MULTI-MODAL DATA FOR DETECTING ABERRANT TEST-TAKING BEHAVIORS IN LARGE-SCALE ASSESSMENTS

Abstract

Many schools, states, and countries use scores from large-scale assessments in making important high-stakes decisions in such areas as college admissions, academic performance evaluations, and job promotions among others. These decisions rely on accurate, reliable scores from which valid inferences about examinees can be assessed. However, aberrant test-taking behaviors, including copying from other test-takers and practicing with real items ahead of time, undermine the effectiveness of such assessments in yielding accurate, precise information on an examinee’s performance. Also, with the wide adoption of technology-enhanced online learning and testing system, especially as I am writing my thesis while the outbreak of COVID-19 virus, it is critical to address an example question like ”how to make the online-delivered tests more secure?” As a result, investigating ways to identify potential cheaters after these assessments or batteries have been taken and data collected is an important endeavor for the numerous administrators of such assessments. The purpose of this line of research is to create, develop, investigate, and test new approaches that will incorporate bio-information technology, such as eye-tracking, into current machine-learning methods in the detection of cheating and other aberrant testing behaviors in computer-based testing scenarios. In other words, cheating detection for innovative large-scale assessments with big data techniques augmented by bio-information technologies will be explored. The eye-tracking systems, in particular, have the potential to capture cheating and other aberrant test-taking behaviors with visual information gathered through the analysis of eye movement patterns (saccades, fixations, pupil size). This type of data can be subtly gathered in real-time on test-takers as they attempt to answer each assessment item. To assess the visual attention nuances across test-takers, three negative binomial distribution-based visual fixation counts models will be presented. Moreover, a joint-modeling approach of integrating product data (e.g., item responses), process data (e.g., response times), and biometric information (visual fixation counts) will be demonstrated. By joint modeling the three types of information, we can assess test-takers’ performance in a comprehensive way. Finally, selected supervised and unsupervised statistical learning methods will be explored for detecting different types of responding behaviors.