Human Development & Quantitative Methodology
Permanent URI for this communityhttp://hdl.handle.net/1903/2248
The departments within the College of Education were reorganized and renamed as of July 1, 2011. This department incorporates the former departments of Measurement, Statistics & Evaluation; Human Development; and the Institute for Child Study.
Browse
43 results
Search Results
Item THE USE OF RANDOM FORESTS IN PROPENSITY SCORE WEIGHTING(2023) Zheng, Yating; Stapleton, Laura; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)An important problem of social science research is the estimate of causal effects in observationalstudies. Propensity score methods, as effective ways to remove selection bias, have been widely used in estimating causal effects in observational studies. An important step of propensity score methods is to estimate the propensity score. Recently, a machine learning method, random forests, has been proposed as an alternative to the conventional method of logistic regression to estimate the propensity score as it requires less stringent assumptions and provides less biased and more reliable estimate of the treatment effect. However, previous studies only covered limited conditions with a small number of covariates and medium sample sizes, leaving the generalizability of the results in doubt. In addition, previous studies have seldom explored how to choose the hyper-parameters in random forests in the context of propensity score methods. This dissertation, via a simulation study, aims to 1) make a more comprehensive comparison between the use of random forests and logistic regression to determine which model performs better under what conditions, 2) explore the effects of the hyperparameters on the estimate of the treatment effect in propensity score weighting. An empirical study is also used as an illustration about how to choose the hyperparameters in random forests using propensity score weighting in practical settings.Item Construct measurement error in latent social network relationship: An item response theory based latent space model(2023) Ding, Yishan; Sweet, Tracy; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Research on measurement error in social network analysis has primarily focused on proxy measurement error, which refers to inadequate or inaccurate observations of proxy measurements of social relationships. However, construct measurement error, a key concern in modern psychometric studies, has received less attention in social network studies. Construct measurement error is particularly relevant for social network relationships that are difficult or impossible to observe explicitly, such as friendships, which are better conceptualized as latent constructs. Historically, researchers have long advocated to use multi-item scales for social relationships to address construct measurement error (Marsden, 1990). However, there is a lack of methods tailored for multivariate social network analysis using multi-item measurements. Commonly, when data on social network ties is collected from multiple items, prevalent strategies involve either choosing a representative item or analyzing each item as a distinct network. To accommodate construct measurement error in social network analysis, this study proposes a new model, termed as IRT-LSM, that integrates an item response theory (IRT) model into a latent space model (LSM). The proposed method leverages the IRT model to take advantage of a multi-item scale to enhance the measurement of latent social relationships, providing a more comprehensive understanding of social relationships compared to relying on a single item. To evaluate the efficacy of this novel approach, the dissertation comprises three simulation studies: One assessing model feasibility and the impact of construct measurement error, a second exploring various misspecification models, and a third investigating the effects of item parameter distributions. Additionally, an empirical data analysis demonstrates the practical application of the IRT-LSM in real-world settings. The results underscore the effectiveness of the IRT-LSM in addressing construct measurement error. The model consistently yields unbiased estimates and demonstrates robustness against various factors influencing its performance across the simulated conditions. Notably, the IRT-LSM outperforms naive approaches that neglect construct measurement error, leading to divergent conclusions in the empirical data analyses.Item A FINITE MIXTURE MULTILEVEL STRUCTURAL EQUATION MODEL FOR UNOBSERVED HETEROGENEITY IN RANDOM VARIABILITY(2023) Feng, Yi; Hancock, Gregory R; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Variability is often of key interest in various research and applied settings. Important research questions about intraindividual variability (e.g., consistency across repeated measurements) or intragroup variability (e.g., cohesiveness among members within a team) are piquing the interest of researchers from a variety of disciplines. To address the research needs in modeling random variability as the key construct, Feng and Hancock (2020, 2022) proposed a multilevel SEM-based modeling approach where variability can be modeled as a random variable. This modeling framework is a highly flexible analytical tool that can model variability in observed measures or latent constructs, variability as the predictor or the outcome, as well as the between-subject comparison of variability across observed groups. A huge challenge still remains, however, when it comes to modeling the unobserved heterogeneity in random variability. Given that no existing research addresses the methodological considerations of uncovering the unobserved sub-populations that differ in intraindividual variability or intragroup variability, or sub-populations that differ in the various processes and mechanisms involving intraindividual variability or intragroup variability, the current dissertation study aims to fill this gap in literature. In the current study, a finite-mixture MSEM for modeling unobserved heterogeneity in random variability (MMSEM-RV) is introduced. Bayesian estimation via MCMC is proposed for model estimation. The performance of MMSEM-RV with Bayesian estimation is systematically evaluated in a simulation study across varying conditions. An illustrative example with empirical PISA data is also provided to demonstrate the practical application of MMSEM-RV.Item INVESTIGATING MODEL SELECTION AND PARAMETER RECOVERY OF THE LATENT VARIABLE AUTOREGRESIVE LATENT TRAJECTORY (LV-ALT) MODEL FOR REPEATED MEASURES DATA: A MONTE CARLO SIMULATION STUDY(2023) Houser, Ari; Harring, Jeffrey R; Human Development; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Over the past several decades, several highly generalized models have been developed which can reduce, through parameter constraints, to a variety of classical models. One such framework, the Autoregressive Latent Trajectory (ALT) model, is a combination of two classical approaches to longitudinal modeling: the autoregressive or simplex family, in which trait scores at one occasion are regressed on scores at a previous occasion, and latent trajectory or growth curve models, in which individual trajectories are specified by a set of latent factors (typically a slope and an intercept) whose values vary across the population.The Latent Variable-Autoregressive Latent Trajectory (LV-ALT) model has been recently proposed as an extension of the ALT model in which the traits of interest are latent constructs measured by one or more indicator variables. The LV-ALT is presented as a framework by which one may compare the fit of a chosen model to alternative possibilities or use to empirically guide the selection of a model in the absence of theory, prior research, or standard practice. To date, however, there has not been any robust analysis of the efficacy or usefulness of the LV-ALT model for this purpose. This study uses a Monte Carlo simulation study to evaluate the efficacy of the basic formulation of the LV-ALT model (univariate latent growth process, single indicator variable) to identify the true model, model family, and key characteristics of the model under manipulated conditions of true model parameters, sample size, measurement reliability, and missing data. The performance of the LV-ALT model for model selection is mixed. Under most manipulated conditions, the best-fitting of nine candidate models was different than the generating model, and the cost of model misspecification for parameter recovery included significant increases in bias and loss of precision in parameter estimation. As a general rule, the LV-ALT should not be relied upon to empirically select a specific model, or to choose between several theoretical plausible models in the autoregressive or latent growth families. Larger sample size, greater measurement reliability, larger parameter magnitude, and a constant autoregressive parameter are associated with greater likelihood of correct model selection.Item ESTIMATING THE Q-DIFFUSION MODEL PARAMETERS BY APPROXIMATE BAYESIAN COMPUTATION(2023) Tian, Chen; Liu, Yang; Human Development; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The Q-diffusion model is a cognitive process model that considers decision making as an unobservable information accumulation process. Both item and person parameters decide the trace line of the cognitive process, which further decides observed response and response time. Because the likelihood function for the Q-diffusion model is intractable, standard parameter estimation techniques such as the maximum likelihood estimation is difficult to apply. This project applies Approximate Bayesian Computation (ABC) to estimate parameters of the Q-diffusion model. Different from standard Markov chain Monte Carlo samplers that require pointwise evaluation of the likelihood function, ABC builds upon a program for data generation and a metric on the data space to gauge the similarity between imputed and observed data. This project aims to compare the performance of two criteria for gauging the similarity or distance. The limited-information criterion measures the distance in suitable summary statistics (i.e., variances, covariances, and means) between imputed and observed data. The enhanced limited information criterion additionally considers the dependencies among persons’ responses and response times. Bias, rooted mean squared error, and coverage of credible intervals were reported. Results show that when using posterior median as the point estimate, by jointly considering a person’s responses and response time, the enhanced criterion yielded less biased estimation on population scale of person power and slightly better item parameters. This SMC-ABC algorithm informs researchers about key data features that should be captured when determining the stopping rule for the algorithm.Item Bayesian Analysis of a Nonlinear Dynamic Latent Class Structural Equation Model: A Simulation Study(2023) Zou, Jinwang; Harring, Jeffrey R.; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In the past several decades, the so-called ambulatory assessment (AA) of in- tensive longitudinal data (ILD) has gained a substantial amount of attention. Recent advancements in data collection technologies such as smart phones and pedometers have catalysed the creation of richer and denser data sets. Such data sets enable the investigation of momentary dynamic processes underlying the data, but at the same time also pose more challenges in choosing appropriate modeling techniques to an- swer increasingly more complex research questions. Traditional modeling techniques such as structural equation models, latent class analysis, and time series analysis can each be applied to understand the dynamic relations from a particular perspective, but not comprehensively. Recently, Kelava and Brandt (2019) proposed a general nonlinear dynamic latent class structural equation model framework which can be used to examine the intraindividual processes of observed or latent variables using the ILD data set. This general framework allows the decomposition of the process data into individual- and time-specific components so that unobserved heterogeneity of intraindividual processes can be modeled via a latent Markov process which can be predicted by individual- and time-specific variables as random effects. Despite the theoretical advancements in modeling ILD data, little is known about the statistical properties of this general framework. The purpose of this study is to fill this gap by running an extensive Monte Carlo simulation study to investigate the simulation outcomes using various evaluation metrics under a series of conditions using representative submodels from the general framework. Recommendations are given according to the simulation results and findings from the simulation study can serve as useful guidance for both applied and methodological researchers alike.Item TESTING DIFFERENTIAL ITEM FUNCTIONING BY REGULARIZED MODERATED NONLINEAR FACTOR ANALYSIS(2022) Wang, Weimeng; Harring, Jeffrery R; Liu, Yang; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Recent advancements in testing differential item functioning (DIF) have greatly relaxed restrictions made by the conventional multiple group item response theory (IRT) model with respect to the number of grouping variables and the assumption of predefined DIF-free anchor items. The application of the L1 penalty in DIF detection has shown promising results in identifying a DIF item without a priori knowledge on anchor items while allowing the simultaneous investigation of multiple grouping variables. The least absolute shrinkage and selection operator (LASSO) is added directly to the loss function to encourage variable sparsity such that DIF parameters of anchor items are penalized to be zero. Therefore, no predefined anchor items are needed. However, DIF detection using LASSO requires a non-trivial model selection consistency assumption and is difficult to draw statistical inference. Given the importance of identifying DIF items in test development, this study aims to apply the decorrelated score test to test DIF once the penalized method is used. Unlike the existing regularized DIF method which is unable to test the statistical significance of a DIF item selected by LASSO, the decorrelated score test requires weaker assumptions and is able to provide asymptotically valid inference to test DIF. Additionally, the deccorrelated score function can be used to construct asymptotically unbiased normal and efficient DIF parameter estimates via a one-step correction. The performance of the proposed decorrelated score test and the one-step estimator are evaluated by a Monte Carlo simulation study.Item Joint Modeling Of Responses, Response Time, and Answer Changes in Testlet-based Assessment for Cognitive Diagnosis(2022) Yin, Chengbin; Jiao, Hong; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)To address the scenario of testlet-based assessment, this research proposes a joint model of responses, response time, and answer change patterns for testlet-based cognitive diagnostic assessments. A simulation study was conducted to assess the impact of accounting for dual item and item time dependency and of incorporating answer change patterns as an additional data source on model fit, classification accuracy at the attribute and attribute profile level, and parameter estimation. Through manipulating three factors, the simulation study examined the extent to which the manipulated factors impacted the performance of the proposed model and two comparison models in recovering model parameters. Application of the proposed model was demonstrated with an empirical dataset.Item EXPLANATORY COGNITIVE DIAGNOSTIC MODELING INCORPORATING RESPONSE TIMES(2021) Qiao, Xin; Jiao, Hong; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The current study proposes the explanatory cognitive diagnostic models (CDMs) incorporating response times (RTs) with item covariates on both the item response side and the RT side. There are two main contributions of the current study. One appealing usage of this model is that scored item covariates can be used to predict item parameters when item calibration is not feasible in diagnostic assessments while the other is that the cognitive theories underlying the test design can be evaluated. Model parameter estimation is explored using the Bayesian Markov chain Monte Carlo (MCMC) method. A Monte Carlo simulation study is conducted to examine the parameter recovery of the proposed model under different simulated conditions in comparison to a few competing models. The results indicate that model parameter could be well recovered using the MCMC approach. Further, the application of the proposed model is illustrated using the Programme for International Student Assessment (PISA) 2012 problem-solving items using both item response and item RT data.Item A Multilevel Testlet Joint Model of Responses and Response Time(2020) Olson, Evan; Jiao, Hong; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In approaches to joint response and response time (RT) modeling there is an assumption of conditional independence of the responses and the RTs. Further, in IRT modeling of the responses, there is the assumption that the items and the persons have local independence, respectively. In practice, violations of the local item independence results from the bundling of items into testlets. Violation of the person independence are encountered in complex examinee sampling situations. A multilevel testlet joint responses and RT model is proposed and evaluated in this study that accounts for the dual local item and person dependence due to testlets and complex sampling. A simulation study is performed to investigate parameter recovery for the proposed model and provide comparison to models that do not model dual local dependencies. In addition to the simulation study, a study using empirical data is also conducted to evaluate relative model fit indices. Generally, results determined by statistical analyses and inspection of graphs developed from descriptive statistics supported the need to model local item dependency and local person dependency. Parameter recovery outcome measures in the simulation study showed interaction of factors included with the model factor when the comparison models were included. When deviance model fit criterion was applied the proposed model was selected as the best-fitting model. For the Bayesian model fit index DIC the proposed model was not selected as best-fitting in for either the simulation or the empirical data analyses. Limitations of the study and opportunities to refine joint response and RT modeling of this dual dependency were elaborated.