Theses and Dissertations from UMD

Permanent URI for this communityhttp://hdl.handle.net/1903/2

New submissions to the thesis/dissertation collections are added automatically as they are received from the Graduate School. Currently, the Graduate School deposits all theses and dissertations from a given semester after the official graduation date. This means that there may be up to a 4 month delay in the appearance of a give thesis/dissertation in DRUM

More information is available at Theses and Dissertations at University of Maryland Libraries.

Browse

Subject: search.filters.subject.Educational evaluation

Search Results

Now showing 1 - 10 of 28
  • Thumbnail Image
    Item
    Moderating Effects of Difficulty on Individual Differences' Prediction of Intensive Second Language Proficiency Attainment
    (2024) Pulupa, Catherine Maria; Hui, Bronson; Second Language Acquisition and Application; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
    The United States government is perennially in need of employees with proficiency in critical foreign languages to communicate with foreign counterparts and maintain relationships worldwide. In order to fulfill this need, the government devotes significant resources training federal employees to advanced levels of language proficiency through intensive courses aimed at developing communicative language skills that reflect the work that employees will perform in their work advancing the interests of the United States abroad. Notable proportions of employees fail to meet proficiency goals at the end of training, and little is known about what learner individual differences drive whether or not employees will meet their proficiency goals in order to perform their work on behalf of the United States. To this aim, the current investigation utilizes multiple analyses to explore and explain the interrelationships between learner individual differences, language difficulty, and proficiency attainment throughout training. The investigation constitutes two related analyses. First, a path-analytic approach examines associations between a cognitive (aptitude) measure and non-cognitive (motivation, familiarity with curricula, previous advanced second language learning) measures with student proficiency achievement throughout training. A second analysis builds on the first: the path-analytic model incorporates a measure of difficulty of the language studied by the students to determine how difficulty influences language learning and ultimate attainment within the context of individual differences in L2 speaking and reading. Results demonstrated consistent influence of language aptitude on proficiency attainment, and notable influences of previous L2 acquisition and the alignment of training to individuals’ language use goals. L2 difficulty moderated the relationships between individual differences and proficiency assessment scores during several points in training. The findings support an understanding of adult L2 acquisition that more fully considers learners’ goals and previous L2 experiences and consideration of the impact that difficulty can have on individual learners’ abilities to achieve target proficiency goals.
  • Thumbnail Image
    Item
    ACCELERATING RESTORATION THROUGH INFORMATION-SHARING: UNDERSTANDING OPERATOR BEHAVIOR FOR IMPROVED MANAGEMENT OF INTERDEPENDENT INFRASTRUCTURE
    (2024) Yazdisamadi, Mohammadreza; Reilly, Allison C.; Civil Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
    This dissertation examines the roles that organizations and individuals play in restoring interdependent infrastructure following disasters through three studies. In the first study, we focus on how operator heuristics affect the collective restoration speed of three interdependent infrastructure (electric power, chilled water, and IT networks). We do this by developing a novel framework that embeds an interdependent infrastructure network within an agent-based model that mimics the decisions and patterns observed of actual operators. The study sheds light on how coordination and information exchange by separate infrastructure parties affect decisions and thus restoration outcomes. In the second study, we examine recovery times and total unmet demand for the same three interconnected infrastructure systems assuming a variable fraction of node removals. The work is decomposed by the extent to which operators share information and coordinate strategies, enabling us to identify at what fraction of network failure does coordination and information sharing become beneficial. Our study indicates that prioritizing restoration based on node centrality produces the speediest recovery. We also show that communication among organizations may improve collective performance by as much as 50%. Our final research project uses a serious game, Breakdown, focused on restoration of interdependent infrastructure to assess whether engineering graduate students gain a deeper appreciation for the complexity of interdependent infrastructure and socio-technical systems more broadly. This is the first serious game designed to emphasize the value of cooperation, communication, and strategy in times of crisis in the field of interdependent infrastructure. As a result of playing Breakdown, graduate students demonstrated statistically significant improvements in engineering decision-making under uncertainty and sociotechnical systems concepts. As a result of this dissertation, the interdependent infrastructure community gains insight into (1) how individual operators' behavior influences the speed at which interdependent infrastructure systems recover; (2) how policies and procedures, like sharing information and cooperating, can help improve outcomes; and (3) the ways to teach graduate engineering students about socio-technical systems effectively. Using an agent-based model simulation, it quantifies the effects of human behavior, communication, and cooperation on recovery outcomes. By using a serious game, Breakdown, it proposes an innovative way to teach graduate engineering students about socio-technical systems.
  • No Thumbnail Available
    Item
    UNDERSTANDING POLICY ENACTMENT IN DIVERSE INSTITUTIONAL CONTEXTS: AN EXPLORATION OF THE IMPLEMENTATION OF THE SINGLE NATIONAL CURRICULUM POLICY IN PAKISTAN
    (2023) Ali, Fatimah; Klees, Steven J; Education Policy, and Leadership; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
    Education reform is a contested and complicated terrain where multiple stakeholders with competing visions seek to influence the direction of reform in alignment with their own interests. As a result of these competing visions, education policy is not value free or neutral and favors those who hold the political and economic power to maintain the inertia of the status quo. Despite this unequal balance, education reform remains a dialectic process which is affected by several factors including contexts, resources, accountability processes and the agency of those tasked with policy implementation. Since policy is rarely implemented without some refraction from its original formulation, it is important to examine the implementation process to understand its complexities that can aid in better policymaking by decisionmakers. This multi-case qualitative study examines the implementation of the Single National Curriculum (SNC) Policy introduced during the 2021-2022 academic year in Punjab, Pakistan in primary schools (grade 1-5). Using an institutional lens, this study investigates how public and private schools interpreted the Single National Curriculum Policy and implemented it in classrooms. To highlight the complexity of policy implementation and provide a variegated and nuanced view of policy implementation within different institutional and social contexts, the study was conducted in a public school, a low-cost private school, and a high-cost private school in the cities of Lahore and Kamalia. In total, 22 teachers and administrators across the three schools were interviewed for this research. This study found that the alignment of belief systems, the institutional arrangements of schools and perceived costs of deviation all influenced how closely the SNC curriculum was implemented in each school. Schools interpreted the policy in light of mediating factors which included institutional factors such as the learning environment, capacity and agency of the teachers and administrators as well as their hierarchies and support systems. Adherence to policy was also dependent on the perceived risk of being inspected by government officials and monitoring frameworks established to assess policy implementation. Lastly, in all three schools, administrators and teachers adhered to a spectrum of beliefs that were in alignment with or opposed to the SNC policy. These governed how closely the SNC was implemented in classrooms and or if corrective pressure was applied to maintain their desired student outcomes. As a result of this variation, gaps in student learning remained intact as the new policy did not address the roots of quality differences across schools. However, the government was able to successfully coerce all schools into incorporating more religious instruction and nationalistic citizenship education leading to more tight coupling between the schools on this front.
  • Thumbnail Image
    Item
    Characterizing the Adventitious Model Error as a Random Effect in Item-Response-Theory Models
    (2023) Xu, Shuangshuang; Liu, Yang; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
    When drawing conclusions from statistical inferences, researchers are usually concerned about two types of errors: sampling error and model error. The sampling error is caused by the discrepancy between the observed sample and the population from which the sample is drawn from (i.e., operational population). The model error refers to the discrepancy between the fitted model and the data-generating mechanism. Most item response theory (IRT) models assume that models are correctly specified in the population of interest; as a result, only sampling errors are characterized, not model errors. The model error can be treated either as fixed or random. The proposed framework in this study treats the model error as a random effect (i.e., an adventitious error) and provides an alternative explanation for the model errors in IRT models that originate from unknown sources. A random, ideally small amount of discrepancy between the operational population and the fitted model is characterized using a Dirichlet-Multinomial framework. A concentration/dispersion parameter is used in the Dirichlet-Multinomial framework to measure the amount of adventitious error between the operational population probability and the fitted model. In general, the study aims to: 1) build a Dirichlet-Multinomial framework for IRT models, 2) establish asymptotic results for estimating model parameters when the operational population probability is assumed known or unknown, 3) conduct numerical studies to investigate parameter recovery and the relationship between the concentration/dispersion parameter in the proposed framework and the Root Mean Square Error of Approximation (RMSEA), 4) correct bias in parameter estimates of the Dirichlet-Multinomial framework using asymptotic approximation methods, and 5) quantify the amount of model error in the framework and decide whether the model should be retained or rejected.
  • Thumbnail Image
    Item
    THE EFFECT OF DIFFERENT RELATIVE LOGISTIC REGRESSION GENERATED PROPENSITY SCORE DISTRIBUTIONS ON THE PERFORMANCE OF PROPENSITY SCORE METHODS
    (2020) An, Ji; Stapleton, Laura M; Human Development; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
    Much education research involves evaluating the causal effects of interventions. The propensity score (PS) method, which is often used to account for selection bias, has become a popular approach to facilitating causal inference in quasi-experimental designs. Because the success of the application of PS conditioning methods is dependent on the estimated propensity scores, the relative PS distribution between the treated and control groups could be an important yet not well-known factor. The primary goal of this dissertation was to explore, via a simulation study, the relations between the relative PS distributions and the performance of selected PS matching methods. The results indicated that PS weighting (without trimming) tends to be robust to a variety of data conditions and produces more accurate and trustworthy TE and SE estimates. The performance of the methods and conclusions were then illustrated through an empirical data analysis using data selected from the Early Childhood Longitudinal Study Kindergarten Class of 2010-11 study, assessing the effect of having home computers on first grade students’ math achievement.
  • Thumbnail Image
    Item
    The Performance of Balance Diagnostics for Propensity-Score Matched Samples in Multilevel Settings
    (2019) Burnett, Alyson; Stapleton, Laura M; Measurement, Statistics and Evaluation; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
    The purpose of the study was to assess and demonstrate the use of covariate balance diagnostics for samples matched with propensity scores in multilevel settings. A Monte Carlo simulation was conducted that assessed the ability of different balance measures to identify the correctly specified propensity score model and predict bias in treatment effect estimates. The balance diagnostics included absolute standardized bias (ASB) and variance ratios calculated across the pooled sample (pooled balance measures) as well as the same balance measures calculated separately for each cluster and then summarized across the sample (within-cluster balance measures). The results indicated that overall across conditions, the pooled ASB was most effective for predicting treatment effect bias but the within-cluster ASB (summarized as a median across clusters) was most effective for identifying the correctly specified model. However, many of the within-cluster balance measures were not feasible with small cluster sizes. Empirical illustrations from two distinct datasets demonstrated the different approaches to modeling, matching, and assessing balance in a multilevel setting depending on the cluster size. The dissertation concludes with a discussion of limitations, implications, and topics for further research.
  • Thumbnail Image
    Item
    TEACHERS’ PERCEPTIONS OF EDTPA ON THEIR PRACTICE
    (2018) Seelke, John Louis; Brantlinger, Andrew; Education Policy, and Leadership; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
    For centuries, education policymakers have sought to identify the most effective way to assess a potential teacher’s readiness to enter the classroom. These assessments evolved from multiple choice examinations to performance-based assessments focused on teacher actions. The latest iteration of these performance-baed assessments is edTPA. edTPA’s structure mirrors that of the assessment for National Board Certification (NBC) designed for veteran teachers. The NBC assessment has shown to be educative for teachers who complete it, leading to positive changes in their post-assessment practice (Athanases, 1994; Hattie & Clinton, 2010; Sato, Darling-Hammond and Wei, 2008; Steeley, 2003). This study examines whether edTPA has similar educative impacts on early career teachers. Since edTPA is relatively new, little research has been completed on its impact on teacher practice. Most of the current literature on edTPA focuses on its implementation or on pre-service candidate perceptions of completing the assessment. This interview study also examines candidate perceptions but focuses on whether they felt completing edTPA was educative and impacted their current practice. This study includes twenty teachers who participated in two hour-long interviews given roughly six months apart. All of the participants were recent secondary mathematics education graduates from one university. This study is among the first studies of edTPA to include teachers who both completed edTPA and have been teaching for at least two years. Another unique strength of this study is that, prior to the second interview, candidates reviewed their actual edTPA portfolio to help recall components of the assessment and to potentially make clearer connections between edTPA and their current practices. The study results support the notion that edTPA can be educative and influence a teacher’s current practices around planning, instruction, and assessment. The level of influence that completing edTPA has on a teacher’s practices may be impacted by school or district policies that either hinder or support high-scoring edTPA practices. The results demonstrate how edTPA can not only be seen as a summative tool at the end of pre-service teaching, but also a formative tool that impacts the teaching practices of early career teachers.
  • Thumbnail Image
    Item
    An Analysis of a Project Management Oversight Committee as a School Improvement Model in One School District
    (2018) Alston, Anthony Reginal; Imig, David G; Education Policy, and Leadership; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
    There are persistent and significant gaps in performance between various racial and ethnic sub-groups, particularly between African American and White groups of students, on both state assessments and discipline measures (Rothstein, 2014). Effectively closing the gap requires improving the performance of all students while accelerating the achievement of low-performing student groups in order to catch up to their higher-performing peers. Researchers have found that a variety of school, community, economic and familial factors correlate with low student performance and the achievement gap, but views are mixed on how to improve performance for all and narrow the gap (Alliance for Excellence Education, 1999; Cancian & Danziger, 2009; Jacob & Ludwig, 2009; Janet, 1999; Mitra, 2011; Organization for Economic Co-Operation (OECD), 2012). Over the past decade, federal, state and local policies have made the improvement of low-performing schools and the closing of the achievement gap a top priority. This descriptive quantitative study focused on the efforts of one Mid-Atlantic system to address low performance among certain groups of students using a Project Management Oversight Committee (PMOC) model. The model utilizes data and a heightened level of accountability to yield quick, yet sustainable growth and progress. For this study seven schools (three high schools, two middle schools and two elementary schools) were identified as a result of persistent participation in the process. The academic and discipline data were reviewed to determine trends in performance. In addition to the achievement and demographic data, a satisfaction survey was reviewed to understand the impact of the PMOC process in improving low student performance. Analysis of the data did not find any relational impact upon achievement data. Although the PMOC process did not have the promised impact, it does not deminsh the possibilty that there were gains that would not have occured if the process had not been used. Based on these findings,it is recommended that the system consider allocating funding for further studies to examine the impact of the PMOC process.
  • Thumbnail Image
    Item
    Pathways to Proficiency: Examining the Coherence of Initial Second Language Acquisition Patterns within the Language Difficulty Categorization Framework
    (2018) Masters, Megan; Ross, Steven J.; Second Language Acquisition and Application; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
    It has perhaps never been clearer that in order to effectively communicate with global governments and develop reasoned foreign policy, the United States Intelligence Community requires the support of trained linguists. The development of foreign language proficiency is a complex process requiring a significant investment of time and resources. For learners involved in intensive foreign language training within the United States Government (USG), the Department of Defense (DoD) has developed various Language Difficulty Categorization (LDC) frameworks aimed at standardizing the amount of time learners are given to meet established proficiency criteria. Despite the widespread adoption of LDC frameworks over the past 60 years, few empirical studies have examined the systematicity in proficiency patterns for languages grouped within the same difficulty category. By situating the analysis within the framework of a logic model, data-mining techniques were used to statistically model, via path analysis, the relationships between program inputs, activities, and outcomes. Two main studies comprised the investigation. Study 1 employed a contrastive-analytic approach to examine the coherence with which both cognitive (e.g., general aptitude, language-specific aptitude, and average coursework outcomes) and non-cognitive (e.g., language preference self-assessment scores) variables contributed to the development of foreign language achievement and proficiency outcomes for three languages grouped within the same category. For Study 1, only learners who completed the entire foreign language-training program were included in the analysis. Results of Study 1 found a great deal of coherence in the role that language-specific aptitude and 300-level average coursework grades play in predicting end-of-program proficiency outcomes. To examine the potential hidden effects of non-random attrition, Study 2 followed the same methodological procedures as Study 1, but it imputed missing coursework and proficiency test score data for learners who attrited (that is, “dropped out”) during the intensive foreign language-training program. Results of the imputation procedure confirmed that language-specific aptitude plays a robust role in predicting average coursework outcomes across languages. Study 2 also revealed substantial differences in the role that cognitive and non-cognitive variables play in predicting end-of-program proficiency outcomes between the observed and imputed datasets as well as across languages and skills.
  • Thumbnail Image
    Item
    TEACHERS’ PERCEPTIONS OF HOW THE USE OF PEER EVALUATION COULD IMPROVE THEIR TEACHING PRACTICE
    (2017) Farrell, Dale Patrick; Richardson, Patricia M; McLaughlin, Margaret J; Education Policy, and Leadership; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
    As the method, instrument, and reliability of teacher and principal evaluations has come under extreme scrutiny since the Education Reform Act of 2010, school systems across the nation have examined and refined the evaluation process for teachers and principals. Studies have shown the integration of peer evaluation as a model of teacher evaluations can have a positive impact on the teachers and their performance that participate in peer evaluation as well as the potential for an increase in the academic achievement of students. The purpose of this study was to investigate teachers’ overall perceptions of peer evaluation and of how the use of peer evaluation could improve their teaching practice. The goals of this study were to determine the level of sharing of instructional practices among their peers currently, and whether or not they perceived an increase in the amount of time spent sharing of best practices would occur as a result of incorporating peer evaluation into the current model of a formative evaluation. Further, teachers were asked to identify potential benefits as well as any potential challenges they see as a result of implementing peer evaluations. This study used an online survey to gather data from participants. This study was conducted in a suburban school district in Maryland. A total of 34 teachers participated in the survey questionnaire. One elementary, one middle, and one high school were selected to participate in the survey. Data were collected through an online survey conducted in September 2017. This study provides some evidence that teachers do welcome the possible integration of peer evaluation and perceive that peer evaluation may result in an increase in the amount of time spent sharing instructional strategies among other teachers. The findings also reported potential benefits such as increased sharing of best practices of instructional strategies, more timely and relevant feedback, and reduced feelings of isolation could potentially occur as a result of incorporating peer evaluation in the current model of formative evaluations. This study further identified potential challenges such as lack of time to complete peer evaluations, personal bias, and a perceived feeling of needing to add yet another task to the busy professional day of a teacher.