Modeling the Speed-Accuracy-Difficulty Interaction in Joint Modeling of Responses and Response Time

Abstract

With the rapid development of information technology, computer-based tests have become more and more popular in large-scale assessments. Among all the auxiliary data collected during the test-taking process, response times (RTs) seem to be one of the most important and commonly utilized sources of information. A commonly adopted assumption in joint modeling of RTs and item responses is that item responses and RTs are conditionally independent given a person’s speed and ability, and a person has constant speed and ability throughout the test (e.g., Thissen, 1983; van der Linden, 2007).

However, researchers have been investigating more complex scenarios where the conditional independence assumption between item responses and RTs is likely to be violated in various ways (e.g., De Boeck, Chen, & Davison, 2017; Meng, Tao, & Chang, 2015; Ranger & Ortner, 2012b). Empirical evidence suggests that the direction of conditional dependence differs among items in a systematic way (Bolsinova, Tijmstra, & Molenaar, 2017). For difficult items, correct responses are associated with longer RTs; for easier items, however, correct responses are usually associated with shorter RTs (Bolsinova, De Boeck, & Tijmstra, 2017; Goldhammer, Naumann, & Greiff, 2015; Partchev & De Boeck, 2012). This phenomenon reflects a clear pattern that item difficulty affects the direction of conditional dependence between item responses and RTs. However, such an interaction has not been explicitly explored in jointly modeling of RT and response accuracy.

In the present study, various approaches for joint modeling of RT and response accuracy are proposed to account for the conditional dependence between responses and RTs due to the interaction among speed, accuracy, and item difficulty. Three simulation studies are carried out to compare the proposed models with van der Linden’s (2007) hierarchical model that does not take into account the conditional dependence with respect to model fit and parameter recovery. The consequences of ignoring the conditional dependence between RT and item responses on parameter estimation is explored. Further, empirical data analyses are conducted to investigate the potential violations of the conditional independence assumption between item responses and RTs and obtain a more fundamental understanding of examinees’ test-taking behaviors.