THE MIXTURE DISTRIBUTION POLYTOMOUS RASCH MODEL USED TO ACCOUNT FOR RESPONSE STYLES ON RATING SCALES: A SIMULATION STUDY OF PARAMETER RECOVERY AND CLASSIFICATION ACCURACY

Abstract

Response styles presented in rating scale use have been recognized as an important source of systematic measurement bias in self-report assessment. People with the same amount of a latent trait may be a victim of a biased test score due to the construct's irrelevant effect of response styles. The mixture polytomous Rasch model has been proposed as a tool to deal with the response style problems. This model can be used to classify respondents with different response styles into different latent classes and provides person trait estimates that have been corrected for the effect of a response style.

This study investigated how well the mixture partial credit model (MPCM) recovered model parameters under various testing conditions. Item responses that characterized extreme response style (ERS), middle-category response style (MRS), and acquiescent response style (ARS) on a 5-category Likert scale as well as ordinary response style (ORS), which does not involve distorted rating scale use, were generated.

The study results suggested that ARS respondents could be almost perfectly classified from other response-style respondents while the distinction between MRS and ORS respondents was most difficult followed by the distinction between ERS and ORS respondents. The classifications were more difficult when the distorted response styles were present in small proportions within the sample. Ten-items and a sample size of 3000 appeared to warrant reasonable threshold and person parameter estimation under the simulated conditions in this study. As the structure of mixture of response styles became more complex, increased sample size, test length, and balanced mixing proportion were needed in order to achieve the same level of recovery accuracy. Misclassification impacted the overall accuracy of person trait estimation. BIC was found to be the most effective data-model fit statistic in identifying the correct number of latent classes under this modeling approach.

The model-based correction of score bias was explored with up to four different response-style latent classes. Problems with the estimation of the model including non-convergence, boundary threshold estimates, and label switching were discussed.