RANDOMIZATION-BASED INFERENCE ABOUT LATENT VARIABLES FROM COMPLEX SAMPLES: THE CASE OF TWO-STAGE SAMPLING

Abstract

In large-scale assessments, such as the National Assessment of Educational Progress (NAEP), plausible values based on Multiple Imputations (MI) have been used to estimate population characteristics for latent constructs under complex sample designs. Mislevy (1991) derived a closed-form analytic solution for a fixed-effect model in creating plausible values assuming a classical test theory model and a stratified student sample and proposed an analogous solution for a random-effects model to be applied with a two-stage student sample design. The research reported here extends the discussion of this random-effects model under the classical test theory framework. Under the simplified assumption of known population parameters, analytical solutions are provided for multiple imputations in the case of the classical test theory measurement model and two-stage sampling and their properties are verified in reconstructing population properties for the unobservable latent variables. With the more practical assumptions of unknown population and cluster means, this study empirically examines the reconstruction of population attributes. Next, properties of sample statistics are examined. Specifically, this research explores the impact of the variance components and sample sizes on the sampling variance of the MI-based estimate for the population mean. Findings include significant predictors and influential factors. Last, the relationships between the sampling variance of the estimate of the population mean based on the imputations and those based on observations of the true score and the observed score are discussed. The sampling variance based on the imputed score is expected to be the higher boundary of that based on the observed score, which is expected to be the higher boundary of that based on the true score.