PSYCHOMETRIC ANALYSES BASED ON EVIDENCE-CENTERED DESIGN AND COGNITIVE SCIENCE OF LEARNING TO EXPLORE STUDENTS' PROBLEM-SOLVING IN PHYSICS
Mislevy, Robert J
MetadataShow full item record
Most analyses of physics assessment tests have been done within the framework of classical test theory in which only the number of correct answers is considered in the scoring. More sophisticated analyses have been developed recently by physics researchers to further study students' conceptions/misconceptions in physics learning to improve physics instruction. However, they are not connected with the well-developed psychometric machinery. The goal of this dissertation is to use a formal psychometric model to study students' conceptual understanding in physics (in particular, Newtonian mechanics). The perspective is based on the evidence-centered design (ECD) framework, building on previous analyses of the cognitive processes of physics problem-solving and the task design from two physics tests (Force Concept Inventory, FCI and Force Motion Concept Evaluation, FMCE) that are commonly used to measure students' conceptual understanding about force-motion relationships. Within the ECD framework, the little-known Andersen/Rasch (AR) multivariate IRT model that can deal with mixtures of strategies within individuals is then introduced and discussed, including the issue of identification of the model. To demonstrate its usefulness, four data sets (one from FCI and three from FMCE) were used and analyzed with the AR model using a Markov Chain Monte Carlo estimation procedure, carried out with the BUGS computer program. Results from the first three data sets (questions were used to assess students' understanding about force-motion relationships) indicate that most students are in a mixed model state (i.e., in a transition toward understanding Newtonian mechanics) after one semester of physics learning. In particular, they incorrectly tend to believe that there must be a force acting on an object to maintain its movement, one of the common misconceptions indicated in physics literature. Findings from the last data set (which deals with acceleration) indicate that although students have improved their understanding about acceleration after one semester of instruction, they may still find it difficult to represent their understanding in terms of acceleration-time graphs. This is especially so when the object is slowing down or moving toward the left, in which case the sign of acceleration in both task scenarios is negative.